A method, apparatus, equipment and medium for smoke and fire detection

Abstract

This invention provides a method, apparatus, device, and readable medium for smoke and fire recognition. The method includes: constructing a full-scene map of the current scene, obtaining the actual coordinates of each pixel in the full-scene map, and storing the actual coordinates of each pixel in a coordinate library; obtaining an image to be recognized, and processing the image to obtain information on the target detection region and the target segmentation region; in response to no new target entering the full-scene map and the detection of flames in the image, determining whether the flames are located within the full-scene map based on the target detection region and the target segmentation region; in response to the flames being located within the full-scene map, issuing a smoke and fire warning based on the coordinates and duration of the flames and saving the image. By using the solution of this invention, smoke and fire appearing in a scene can be accurately and quickly identified, the false detection rate can be reduced, the efficiency of coordinate comparison during the recognition process can be improved, and the occupation of device memory and storage can be reduced.

Description

Technical Field

[0001] This invention relates to the field of computers, and more specifically to a method, apparatus, device, and readable medium for identifying smoke and fire. Background Art

[0002] Fire prevention is of paramount importance in daily life, especially for hazardous chemical enterprises such as gas stations. A fire near a gas station can easily trigger explosions and other accidents, causing casualties. Therefore, detecting smoke and fire at gas stations in a timely manner to prevent major accidents is crucial. Real-time alarms should be issued upon detection of smoke and fire to inform gas station personnel and allow for timely and necessary measures to minimize losses. Thus, there is an urgent need for an algorithm for identifying smoke and fire at gas stations to address this market problem. Deep learning-based object detection algorithms have received considerable research attention in recent years. With the development of artificial intelligence and object detection algorithms based on deep convolutional neural networks, some algorithms have emerged in the field of computer vision. Currently, object detection algorithms are commonly used to identify whether anti-static chains are connected, such as classic object detection networks like SDD, RCNN, Faster RCNN, and the YOLO series. Deep learning-based detection methods perform reasonably well on normal server sides and indoor environments, but they are poorly implemented in outdoor environments such as gas stations where sunlight and vehicle headlights can cause glare on the ground and walls, resulting in frequent false positives. Furthermore, these algorithms require significant computing power, while current gas station detection equipment is mostly edge terminal devices, which are limited by computing power and difficult to deploy directly on edge terminal devices. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a method, apparatus, device and readable medium for smoke and fire recognition. By using the technical solution of this invention, smoke and fire appearing in a scene can be accurately and quickly identified, the false detection rate can be reduced, the efficiency of coordinate comparison during the recognition process can be improved, and the occupation of device memory and storage can be reduced.

[0004] To achieve the above objectives, one aspect of the present invention provides a method for smoke and fire detection, comprising the following steps:

[0005] Construct a full scene graph of the current scene, obtain the actual coordinates of each pixel in the full scene graph, and store the actual coordinates of each pixel in the coordinate library;

[0006] The image to be identified is acquired, and the image is processed to obtain information on the target detection region and the target segmentation region;

[0007] In response to the absence of new targets entering the full scene map and the detection of flames in the image, it is determined whether the flames are located within the full scene map based on the target detection region and the target segmentation region.

[0008] In response to the flame being located within the entire scene map, issue a smoke warning based on the flame's coordinates and duration, and save the image.

[0009] According to one embodiment of the present invention, constructing a full-scene graph of the current scene, obtaining the actual coordinates of each pixel in the full-scene graph, and storing the actual coordinates of each pixel in a coordinate library includes:

[0010] Install several cameras at the corresponding locations in the current scene, and ensure that every point in the current scene is within the field of view of at least two cameras. Based on the matching effect of the re-attachment field of view of different cameras, construct a full scene map of the current scene.

[0011] Images are acquired from the video stream captured by the camera and preprocessed.

[0012] The actual coordinates of each pixel in the whole scene map are calculated based on the preprocessed image, and the actual coordinates of each pixel are stored in the coordinate library.

[0013] According to one embodiment of the present invention, acquiring images from a video stream captured by a camera and performing image preprocessing includes:

[0014] Convert the image to grayscale.

[0015] Gaussian blurring is applied to the grayscale image to reduce noise interference and decrease the calculation error of pixel coordinates in the highlight area.

[0016] According to one embodiment of the present invention, obtaining information on the target detection region and the target segmentation region after processing the image includes:

[0017] Images are captured from the video stream captured by the camera, and the images are preprocessed to remove noise and resized to a preset size.

[0018] Image data augmentation is performed using Mosaic (a data augmentation technique) and / or MixUp (a data augmentation technique);

[0019] The augmented image is then fed into a ResNet50 network (a 50-layer residual network) for processing.

[0020] The processed image is then fed into the FPN (Feature Pyramid Network) to extract feature maps;

[0021] The extracted feature maps are input into the RPN (Region Production Network) to obtain the target recommendation regions of the image;

[0022] The obtained feature map and target recommendation region are simultaneously input into the ROI Align network (regions of interest arrangement) to obtain a feature map of the required size;

[0023] The feature map of the required size is passed through the head layer (the network that obtains the network output and makes predictions) to obtain the target detection region and the target segmentation region;

[0024] The target segmentation region is processed through two fully connected layers to obtain the segmentation result;

[0025] The object detection region is processed through a fully connected layer to obtain the bounding box, classification, and score.

[0026] According to one embodiment of the present invention, inputting the obtained feature map and the target recommendation region simultaneously into the ROIAlign network to obtain a feature map of the desired size includes:

[0027] Divide the bounding box area into equal parts according to the required output size;

[0028] Within each equally divided block, select four fixed points;

[0029] The value of each point is obtained by weighting the values ​​of the four nearest real pixels to each point.

[0030] The maximum value among the four new values ​​calculated in each block is taken as the output value of the block, resulting in a 2x2 output.

[0031] According to one embodiment of the present invention, in response to a flame being located within the entire scene map, issuing a smoke warning and saving an image based on the flame's coordinates and duration includes:

[0032] Extract the target area where the flame is located, calculate the actual coordinate position of each pixel in the target area, and extract the coordinate position of the corresponding pixel in the target area from the coordinate library;

[0033] Determine whether the difference between the actual coordinates of a pixel in the target area and the coordinates of the corresponding pixel in the coordinate library is less than a set threshold.

[0034] If the difference is not less than a set threshold, the KCF algorithm is used to track the target.

[0035] Determine whether the tracked target has reached the set time;

[0036] When the tracked target reaches the set time, a smoke warning notification is issued and the image is saved.

[0037] According to one embodiment of the present invention, it further includes:

[0038] In response to a new target entering the full scene image, the region containing the new target and the region after removing the new target are separated from the image.

[0039] In response to the detection of flames in the image within the region after removing the new target, the system determines whether the flames are located within the entire scene image based on the target detection region and the target segmentation region.

[0040] In response to the flame being located within the entire scene map, issue a smoke warning based on the flame's coordinates and duration, and save the image.

[0041] Another aspect of the present invention provides a smoke and fire detection device, the device comprising:

[0042] The module is configured to construct a full scene graph of the current scene, obtain the actual coordinates of each pixel in the full scene graph, and store the actual coordinates of each pixel in the coordinate library.

[0043] The processing module is configured to acquire the image to be identified and process the image to obtain information about the target detection region and the target segmentation region.

[0044] The judgment module is configured to determine whether the flame is located within the full scene map based on the target detection area and the target segmentation area when no new target enters the full scene map and the image contains flames.

[0045] The alarm module is configured to issue a smoke and fire warning and save an image in response to a flame being located within the entire scene map, based on the flame's coordinates and duration.

[0046] Another aspect of the embodiments of the present invention also provides a computer device, the computer device comprising:

[0047] At least one processor; and

[0048] The memory stores computer instructions that can be executed by a processor, which, when executed by the processor, implement the steps of any of the methods described above.

[0049] In another aspect, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0050] The present invention has the following beneficial technical effects: The smoke and fire recognition method provided in the embodiments of the present invention constructs a full scene map of the current scene, obtains the actual coordinates of each pixel in the full scene map, and stores the actual coordinates of each pixel in a coordinate library; obtains the image to be recognized, and processes the image to obtain information on the target detection area and the target segmentation area; in response to no new target entering the full scene map and the detection of flames in the image, it determines whether the flames are located within the full scene map based on the target detection area and the target segmentation area; in response to the flames being located within the full scene map, it issues a smoke and fire warning based on the coordinates and duration of the flames and saves the image. This technical solution can accurately and quickly identify smoke and fire appearing in the scene, reduce the false detection rate, improve the efficiency of coordinate comparison during the recognition process, and reduce the occupation of device memory and storage. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic flowchart of a method for identifying fireworks according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the entire scene area of ​​a gas station according to an embodiment of the present invention;

[0054] Figure 3 A schematic diagram of a smoke detection device according to an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of a computer device according to an embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0058] Based on the above objectives, a first aspect of the embodiments of the present invention provides an embodiment of a method for firework recognition. Figure 1 The diagram shown is a schematic flowchart of the method.

[0059] like Figure 1 As shown in , the method may include the following steps:

[0060] S1 constructs a full-scene graph of the current scene, obtains the actual coordinates of each pixel in the full-scene graph, and stores the actual coordinates of each pixel in a coordinate library. Taking a gas station as an example, the current scene is as follows: Figure 2 As shown, 1 represents the entire gas station scene, 2 represents the camera, 3 represents the gas pump, and 4 represents the car. First, a panoramic map of the gas station is constructed, obtaining the actual coordinates of each pixel. The main steps are: accessing the video stream, extracting images from the video stream, performing image preprocessing (converting the images to grayscale), applying Gaussian blur to the grayscale images to further reduce noise interference and minimize calculation errors for pixel coordinates in highlight areas, ensuring that every point in the gas station is within the field of view of at least two cameras, matching overlapping fields of view from different cameras, and constructing a full-scene map of the gas station based on the matching effect of overlapping fields of view from different cameras. The actual coordinates of each pixel in the full-scene map are calculated and stored in a coordinate library, completing the construction of the full-scene map of the gas station.

[0061] S2 acquires the image to be identified and processes the image to obtain information about the target detection region and the target segmentation region. First, set the training batch size, extract images, preprocess them to remove noise, and scale the images to a specific size (preferably 640*640). During scaling, maintain the aspect ratio, scaling the largest side to 640 and filling the smaller sides with grayscale. Then, perform feature augmentation on the training data, using Mosaic and MixUp. The augmented images are then input into a ResNet50 network, followed by an FPN network to extract feature maps. These feature maps are then input into an RPN network to obtain the target recommendation region. The obtained feature maps and target recommendation regions are simultaneously input into an ROI Align network to obtain a feature map of the required size. This feature map is then processed through a head layer to obtain the target detection region and the target segmentation region. Preferably, the head layer has two branches: one for target segmentation and the other for target detection. The target segmentation region is processed through two fully connected layers to obtain the segmentation result, and the target detection region is processed through fully connected layers to obtain the bounding box, classification, and score.

[0062] S3 responds when no new target enters the full scene image and flames are detected in the image. Based on the target detection area and target segmentation area, it determines whether the flame is located within the full scene image. It then determines whether a new target (such as a vehicle or person) has entered the field of view. If not, it determines whether the image contains flames. If it does contain flames, it determines whether the flame target is located within the full scene area of ​​the gas station based on the detected target area and segmented target area obtained in the previous step.

[0063] S4 responds when the flame is located within the entire scene map, issuing a smoke warning and saving the image based on the flame's coordinates and duration. If the flame target is within the entire scene area, the target area is extracted, and the target's ID is added to the target tracking library. The actual coordinates of each pixel in the target area are calculated, and the corresponding pixel coordinates are extracted from the coordinate library. It is then determined whether the difference between the actual calculated coordinates of the target area pixel and the extracted coordinates of the corresponding pixel in the coordinate library is less than a set threshold. If it is less, the ID is deleted from the target tracking library. If it is not less, the KCF algorithm is used to track the target. It is then determined whether the tracked target has reached a set time threshold. If the threshold is reached, a smoke warning notification is issued, and the detection result image is saved.

[0064] By using the technical solution of this invention, it is possible to accurately and quickly identify smoke and fire in a scene, reduce the false detection rate, improve the efficiency of coordinate comparison during the identification process, and reduce the occupation of device memory and storage.

[0065] In a preferred embodiment of the present invention, constructing a full-scene map of the current scene, obtaining the actual coordinates of each pixel in the full-scene map, and storing the actual coordinates of each pixel in a coordinate library includes:

[0066] Install several cameras at the corresponding locations in the current scene, and ensure that every point in the current scene is within the field of view of at least two cameras. Based on the matching effect of the re-attachment field of view of different cameras, construct a full scene map of the current scene.

[0067] Images are acquired from the video stream captured by the camera and preprocessed.

[0068] The actual coordinates of each pixel in the whole scene map are calculated based on the preprocessed image, and the actual coordinates of each pixel are stored in the coordinate library.

[0069] In a preferred embodiment of the present invention, acquiring images from a video stream captured by a camera and performing image preprocessing includes:

[0070] Convert the image to grayscale.

[0071] Gaussian blurring is applied to the grayscale image to reduce noise interference and decrease the calculation error of pixel coordinates in the highlight area.

[0072] In a preferred embodiment of the present invention, obtaining information about the target detection region and the target segmentation region after processing the image to be identified includes:

[0073] Images are captured from the video stream captured by the camera, and the images are preprocessed to remove noise and resized to a preset size.

[0074] Image data augmentation is performed using Mosaic and / or MixUp methods;

[0075] The augmented image is then fed into a ResNet50 network for processing.

[0076] The processed image is then input into the FPN network to extract feature maps.

[0077] The extracted feature maps are input into the RPN network to obtain the target recommendation region of the image;

[0078] The obtained feature map and the target recommendation region are simultaneously input into the ROI Align network to obtain a feature map of the required size.

[0079] The feature map of the required size is passed through the head layer to obtain the target detection region and the target segmentation region;

[0080] The target segmentation region is processed through two fully connected layers to obtain the segmentation result;

[0081] The object detection region is processed through a fully connected layer to obtain the bounding box, classification, and score.

[0082] In a preferred embodiment of the present invention, the obtained feature map and the target recommendation region are simultaneously input into the ROIAlign network to obtain a feature map of the required size, including:

[0083] Divide the bounding box area into equal parts according to the required output size;

[0084] Within each equally divided block, select four fixed points;

[0085] The value of each point is obtained by weighting the values ​​of the four nearest real pixels (bilinear interpolation);

[0086] The maximum value among the four new values ​​calculated in each block is taken as the output value of the block, resulting in a 2x2 output.

[0087] In a preferred embodiment of the present invention, in response to the flame being located within the entire scene map, issuing a smoke warning and saving the image based on the flame's coordinates and duration includes:

[0088] Extract the target area where the flame is located, calculate the actual coordinate position of each pixel in the target area, and extract the coordinate position of the corresponding pixel in the target area from the coordinate library;

[0089] Determine whether the difference between the actual coordinates of a pixel in the target area and the coordinates of the corresponding pixel in the coordinate library is less than a set threshold.

[0090] If the difference is not less than a set threshold, the KCF algorithm is used to track the target.

[0091] Determine whether the tracked target has reached the set time;

[0092] When the tracked target reaches the set time, a smoke warning notification is issued and the image is saved.

[0093] In a preferred embodiment of the present invention, it further includes:

[0094] In response to a new target entering the full scene image, the region containing the new target and the region after removing the new target are separated from the image.

[0095] In response to the detection of flames in the image within the region after removing the new target, the system determines whether the flames are located within the entire scene image based on the target detection region and the target segmentation region.

[0096] In response to a flame being located within the entire scene image, a smoke warning is issued and the image is saved based on the flame's coordinates and duration. If a new target enters the scene image, the target region and the entire view area after removing the target region are separated from the image. The entire view area after removing the target region is then checked for flames; if found, the above method is executed. Additionally, the new target region undergoes a new target detection and segmentation process to determine if it contains flames. If so, the flame target region is extracted, and the vertex pixel coordinates of the edge frame of the flame target region are extracted. The polygon formed by the vertex pixels of the edge frame is determined to be a convex polygon. The Simpson area method is used to determine if it is a polygon. If it is, this ID is added to the target tracking library, and the KCF algorithm is used to track the target. It is determined whether the tracked target reaches a set time threshold. If the threshold is reached, a smoke warning notification is issued, and the detection result image is saved.

[0097] Compared with existing algorithms, the present invention has the following advantages:

[0098] 1. This invention uses Mosaic and MixUp for data augmentation during the training process, which greatly increases the generalization ability of the model;

[0099] 2. This invention employs a pre-constructed panoramic map of the gas station area and the actual coordinate values ​​corresponding to each pixel on the map, thereby improving the efficiency of coordinate comparison during the recognition process;

[0100] 3. The network model architecture set in this invention is small and has few model parameters, which reduces the occupation of device memory and storage;

[0101] 4. This invention uses the actual coordinates of the target area to calculate and compares them with the actual coordinates of the project points in the coordinate library. If the coordinate values ​​of both are less than a set threshold, it is considered a false detection because when light shines on the object, the actual pixel coordinates of the object do not change, thus effectively reducing the false detection rate of the model.

[0102] 5. This invention uses KCF target tracking. When the target appears continuously, an alarm will only be triggered when a set time threshold is reached, further reducing the false detection rate.

[0103] 6. Track new objects that enter the system and determine the smoke and fire by comparing the shapes of flames and headlights to improve the accuracy of the model. Generally, flames are irregular in shape, while headlights or their illuminated areas are convex polygons.

[0104] It should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The embodiments of the computer program described above can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0105] Furthermore, the method disclosed in the embodiments of the present invention can also be implemented as a computer program executed by a CPU, which may be stored in a computer-readable storage medium. When the computer program is executed by the CPU, it performs the functions defined in the method disclosed in the embodiments of the present invention.

[0106] In view of the above objectives, a second aspect of the embodiments of the present invention provides a smoke and fire detection device, such as... Figure 3 As shown, the device 200 includes:

[0107] The module is configured to construct a full scene graph of the current scene, obtain the actual coordinates of each pixel in the full scene graph, and store the actual coordinates of each pixel in the coordinate library.

[0108] The processing module is configured to acquire the image to be identified and process the image to obtain information about the target detection region and the target segmentation region.

[0109] The judgment module is configured to determine whether the flame is located within the full scene map based on the target detection area and the target segmentation area when no new target enters the full scene map and the image contains flames.

[0110] The alarm module is configured to issue a smoke and fire warning and save an image in response to a flame being located within the entire scene map, based on the flame's coordinates and duration.

[0111] In view of the above objectives, a third aspect of the present invention provides a computer device. Figure 4 The diagram shown is a schematic representation of an embodiment of the computer device provided by the present invention. Figure 4 As shown, embodiments of the present invention include the following apparatus: at least one processor 21; and a memory 22, the memory 22 storing computer instructions 23 that can be executed on the processor, the instructions implementing the above method when executed by the processor.

[0112] In view of the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium. Figure 5 The diagram shown is a schematic representation of an embodiment of the computer-readable storage medium provided by the present invention. Figure 5 As shown, the computer-readable storage medium 31 stores a computer program 32 that, when executed by a processor, performs the methods described above.

[0113] Furthermore, the method disclosed in the embodiments of the present invention can also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. When the computer program is executed by the processor, it performs the functions defined in the method disclosed in the embodiments of the present invention.

[0114] Furthermore, the above-described method steps and system units can also be implemented using a controller and a computer-readable storage medium for storing a computer program that enables the controller to perform the functions of the above-described steps or units.

[0115] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0116] In one or more exemplary designs, functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, functionality may be stored as one or more instructions or code on or transmitted via a computer-readable medium. Computer-readable media include computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one location to another. Storage media may be any available medium accessible to a general-purpose or special-purpose computer. By way of example, and not limitation, computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, or any other medium that may be used to carry or store the required program code in the form of instructions or data structures and is accessible to a general-purpose or special-purpose computer or a general-purpose or special-purpose processor. Furthermore, any connection may be appropriately referred to as computer-readable media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the aforementioned coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are all included in the definition of media. As used herein, disks and optical discs include compact discs (CDs), laser discs, optical discs, digital multifunction discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0117] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0118] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0119] The embodiment numbers disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0120] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0121] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for identifying smoke and fire, characterized in that, Includes the following steps: Construct a full scene graph of the current scene, obtain the actual coordinates of each pixel in the full scene graph, and store the actual coordinates of each pixel in the coordinate library; The system acquires the image to be identified and processes it to obtain information about the target detection region and the target segmentation region. Specifically, it acquires images from the video stream captured by the camera, preprocesses the images to remove noise, and scales the image size to a preset size. Image data augmentation is performed using Mosaic and / or MixUp methods; The augmented image is then fed into a ResNet50 network for processing. The processed image is then input into the FPN network to extract feature maps. The extracted feature maps are input into the RPN network to obtain the target recommendation region of the image; The obtained feature map and target recommendation region are simultaneously input into the ROI Align network to obtain a feature map of the required size. The bounding box region is divided equally according to the required output size. Within each equally divided block, four fixed points are selected. The values ​​of the four nearest ground truth pixels to each point are weighted to obtain the value of each point. The maximum value among the four calculated new values ​​in each block is taken as the block's output value, resulting in a 2x2 output. In response to no new target entering the full scene image and the detection of flames, the network determines whether the flame is within the full scene image based on the target detection region and the target segmentation region. In response to a new target entering the full scene image, the network separates the region containing the new target and the region after removing the new target from the image. In response to the detection of flames in the image within the region after removing the new target, the system determines whether the flames are located within the entire scene image based on the target detection region and the target segmentation region. In response to the flame being located within the entire scene map, a smoke warning is issued based on the flame's coordinates and duration, and the image is saved. In response to the flame being located within the entire scene map, issue a smoke warning based on the flame's coordinates and duration, and save the image.

2. The method according to claim 1, characterized in that, Construct a full scene graph of the current scene, obtain the actual coordinates of each pixel in the full scene graph, and store the actual coordinates of each pixel in a coordinate library, including: Install several cameras at the corresponding locations in the current scene, and ensure that every point in the current scene is within the field of view of at least two cameras. Based on the matching effect of the re-attachment field of view of different cameras, construct a full scene map of the current scene. Images are acquired from the video stream captured by the camera and preprocessed. The actual coordinates of each pixel in the whole scene map are calculated based on the preprocessed image, and the actual coordinates of each pixel are stored in the coordinate library.

3. The method according to claim 2, characterized in that, Acquiring images from the video stream captured by the camera and performing image preprocessing includes: Convert the image to grayscale. Gaussian blurring is applied to the grayscale image to reduce noise interference and decrease the calculation error of pixel coordinates in the highlight area.

4. The method according to claim 1, characterized in that, The process of acquiring the image to be identified and processing it yields information on the target detection region and the target segmentation region, including: The feature map of the required size is passed through the head layer to obtain the target detection region and the target segmentation region; The target segmentation region is processed through two fully connected layers to obtain the segmentation result; The object detection region is processed through a fully connected layer to obtain the bounding box, classification, and score.

5. The method according to claim 1, characterized in that, In response to a fire located within the entire scene map, a smoke warning is issued based on the fire's coordinates and duration, and an image is saved, including: Extract the target area where the flame is located, calculate the actual coordinate position of each pixel in the target area, and extract the coordinate position of the corresponding pixel in the target area from the coordinate library; Determine whether the difference between the actual coordinates of a pixel in the target area and the coordinates of the corresponding pixel in the coordinate library is less than a set threshold. If the difference is not less than a set threshold, the KCF algorithm is used to track the target. Determine whether the tracked target has reached the set time; When the tracked target reaches the set time, a smoke warning notification is issued and the image is saved.

6. A device for detecting smoke and fire, characterized in that, The device includes: The acquisition module is configured to construct a full scene map of the current scene, acquire the actual coordinates of each pixel in the full scene map, and store the actual coordinates of each pixel in a coordinate library. The processing module is configured to acquire an image to be identified and process the image to obtain information about the target detection region and the target segmentation region; wherein, the image is acquired from the video stream captured by the camera, and the image is preprocessed to remove noise from the image and the image size is scaled to a preset size; Image data augmentation is performed using Mosaic and / or MixUp methods; The augmented image is then fed into a ResNet50 network for processing. The processed image is then input into the FPN network to extract feature maps. The extracted feature maps are input into the RPN network to obtain the target recommendation region of the image; The obtained feature map and the target recommendation region are simultaneously input into the ROI Align network to obtain a feature map of the required size. The bounding box region is divided into equal parts according to the required output size. In each block, four fixed points are selected. The values ​​of the four nearest real pixels to each point are weighted to obtain the value of each point. The maximum value among the four calculated values ​​in each block is taken as the output value of the block, and finally a 2x2 output is obtained. The judgment module is configured to, in response to the absence of a new target entering the full scene image and the detection of flames in the image, determine whether the flames are located within the full scene image based on the target detection region and the target segmentation region; and in response to the presence of a new target entering the full scene image, separate the region of the new target and the region after removing the new target from the image. In response to the detection of flames in the image within the region after removing the new target, the system determines whether the flames are located within the entire scene image based on the target detection region and the target segmentation region. An alarm module is configured to issue a smoke and fire warning and save an image in response to a flame being located within the entire scene map, based on the flame's coordinates and duration. In response to the flame being located within the entire scene map, issue a smoke warning based on the flame's coordinates and duration, and save the image.

7. A computer device, characterized in that, include: At least one processor; as well as A memory storing computer instructions executable on the processor, which, when executed by the processor, implement the steps of the method according to any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-5.

Images