Flame detection method and device

Abstract

The invention belongs to the technical field of computer image target detection, and particularly relates to a flame detection method and device. The flame detection method provided by the scheme comprises the following steps: acquiring flame related picture data, marking flame and smoke, and obtaining flame information in the image data; marking animals in the image information, and obtaining animal behavior information in the image information; labeling the environment in the image information to obtain illumination information in the image information; a YOLOv8 model is constructed, and ACmix is added to a convolution module of a backbone of the YOLOv8 model; and training a YOLOv8 model by using the flame information, the animal behavior information and the illumination information to obtain a flame model for judging flame and smoke forms, an animal behavior model for judging animal types and animal behaviors, and an illumination model for judging the number of light sources and light source objects. According to the scheme, the problems that a traditional YOLO algorithm is poor in stability and low in accuracy in an actual application scene of flame capture are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image target detection, and specifically relates to a flame detection method and device. Background Art

[0002] Flame detection is of great significance in the fields of industrial safety, forest fire prevention, smart home, etc. Traditional flame detection methods mainly rely on photoelectric sensors, infrared sensors and ultraviolet sensors, which are effective under certain conditions, but are easily interfered in complex environments and cannot process large amounts of video data in real time.

[0003] In recent years, with the development of deep learning technology, object detection algorithms based on convolutional neural networks (CNN) have been widely used. Among them, the YOLO (You Only Look Once) series of algorithms have become the first choice for many applications due to their high efficiency and real-time characteristics. However, the traditional YOLO algorithm has limitations in capturing global information and long-distance dependencies, resulting in reduced detection accuracy in complex backgrounds and changing environments; and the anchor frame mechanism of the traditional YOLO algorithm performs poorly when facing flames with large changes in shape and size, and cannot flexibly adapt to all flame forms, especially in the case of large environmental interference (such as ambient light interference, smoke interference), it is even more difficult to effectively capture flames. These problems have caused the traditional YOLO algorithm to have poor stability and low accuracy in the actual application scenarios of flame capture. Summary of the invention

[0004] The purpose of this solution is to provide a flame detection method and device to solve the problems of poor stability and low accuracy of the traditional YOLO algorithm in the actual application scenarios of flame capture.

[0005] In order to achieve the above object, the present invention provides a flame detection method, comprising the following steps: S10: Collect flame-related image data, and mark the flame and smoke to obtain flame information in the image data; mark the animals in the image information to obtain animal behavior information in the image information; mark the environment in the image information to obtain lighting information in the image information; S20: Build the YOLOv8 model and add ACmix to the convolution module of the backbone of the YOLOv8 model; S30: training a YOLOv8 model with flame information, animal behavior information, and illumination information to obtain a flame model for determining flame and smoke forms, an animal behavior model for determining animal species and animal behaviors, and an illumination model for determining the number of light sources and light source objects; S40: Combining the results generated by the flame model, the animal behavior model and the illumination model to determine the fire condition.

[0006] A flame detection device, comprising: Information collection unit: collects flame-related image data, and marks flames and smoke to obtain flame information in the image data; marks animals in the image information to obtain animal behavior information in the image information; marks the environment in the image information to obtain lighting information in the image information; Model building unit: build a YOLOv8 model, add ACmix to the convolution module of the backbone of the YOLOv8 model, train the YOLOv8 model with flame information, animal behavior information, and illumination information, and obtain a flame model for judging the form of flames and smoke, an animal behavior model for judging the type and behavior of animals, and an illumination model for judging the number of light sources and light source objects; Comprehensive judgment unit: Combines the results produced by the flame model, animal behavior model and illumination model to judge the fire situation.

[0007] The principle and technical effect of this scheme are: by introducing the ACmix self-attention mechanism, the present invention can effectively capture global information and long-distance dependencies in the feature extraction process, better identify and locate flames, and improve the accuracy and flexibility of detection even in complex backgrounds, overcoming the limitations of the fixed nature of the traditional anchor frame mechanism. At the same time, the ACmix self-attention mechanism combines multi-scale features and global information to make the model more accurate when processing small objects, especially small-scale flames or long-distance flames. The improved YOLOv8 algorithm can capture these targets more effectively and improve the comprehensiveness and reliability of detection. Finally, ACmix optimizes the attention calculation process to efficiently capture global information while reducing additional calculation and memory overhead as much as possible. This optimization ensures that the model can maintain a high reasoning speed while achieving high-precision flame detection, meeting the needs of real-time detection. The improved YOLOv8 algorithm can better filter interference factors in the environment and capture fine-grained features and edge information of flames, thereby improving the stability and accuracy of detection, and is suitable for more diverse application scenarios.

[0008] This solution also uses the optimized YOLOv8 model to establish animal behavior models and lighting models. By judging the animal species and animal behavior, the number and objects of light sources, and combining the flame and smoke forms, it can verify the actual situation of the flame and the impact of the fire. It can exclude special situations where the flame is real but the fire is not serious, and reduce the system's misjudgment of the fire situation.

[0009] In summary, this solution solves the problems of poor stability and low accuracy of the traditional YOLO algorithm in the actual application scenarios of flame capture.

[0010] Furthermore, when the flame model detects flames, the object superimposed on the flame is taken as the light source object through the relationship between the edge contour of the object and the flame contour, and it is judged whether the flame contour and the edge contour of the light source object change within a period of time; if the range covered by the flame contour becomes larger and the range covered by the edge contour of the light source object becomes smaller, it is judged that the fire is spreading; if the flame contour and the edge contour of the light source object remain unchanged, it is judged that there is no flame; if the range covered by the flame contour becomes smaller, it is judged that the fire is getting smaller; when the fire becomes larger, it is determined whether the fire continues to grow by multiple detections of the flame size. If it continues to grow, it is determined to be a dangerous situation.

[0011] By comparing the changes in the flame outline and the edge outline of the light source object, the spread of the fire can be discovered in time. For example, in a forest, when the flame overlaps with objects such as trees, the system can quickly identify that the coverage of the flame outline becomes larger and the coverage of the edge outline of the light source object (tree) becomes smaller, thereby determining that the fire is spreading. When the flame outline and the edge outline of the light source object remain unchanged, it can be accurately judged that there is no flame to avoid false alarms. In some environments where there may be similar flame features, such as reflected light, strong light exposure, etc., this method can effectively eliminate interference and accurately judge whether there is actually a flame. For example, in some industrial sites, there may be strong light reflections or high-temperature objects. This judgment method can avoid misjudging these non-flame situations as fires, reducing unnecessary panic and waste of resources.

[0012] Furthermore, the ACmix self-attention mechanism is added on the basis of YOLOv8 to integrate self-attention and convolution technology.

[0013] Furthermore, the ACmix self-attention mechanism is added on the basis of YOLOv8, and the operations in self-attention and convolution are decomposed and reconstructed into a 1x1 convolution form.

[0014] ACmix is ​​a hybrid model that combines the advantages of self-attention mechanism and convolution operation. Its core idea is that most of the computations of traditional convolution operation and self-attention module can be achieved through 1x1 convolution. ACmix first uses 1x1 convolution to project the input feature map to generate a set of intermediate features, and then reuses and aggregates these intermediate features based on different paradigms, namely self-attention and convolution. In this way, ACmix can not only take advantage of the global perception ability of self-attention, but also capture local features through convolution, thereby improving the performance of the model while keeping the computational cost low.

[0015] Furthermore, when the flame model recognizes that there is flame or smoke in the image data, the image data is put into the illumination model to identify the light source object, and the relationship between the light source object and the flame is used to determine whether the fire is safe.

[0016] Through the relationship between the light source object and the flame and smoke, special scenes where there is flame but no fire can be ruled out, reducing the misjudgment of the fire situation.

[0017] Furthermore, when marking the animals in the image information, the edge contours and pixel positions of each object in the image information are extracted, and the animal species and flame information in the image information are identified through the edge contours. When the animal is identified in the image information, the frequency of image data acquisition is increased, and the animal species in the image information and the pixel position of the flame information are combined to determine the distance between the animal and the flame. The position change of the animal and the flame is obtained by combining the image information acquisition interval with the distance between the animal and the flame, and the animal behavior is inferred by combining the position change with the flame intensity. If the fire intensity increases while the animal accelerates away from the flame, the animal behavior is judged as fire avoidance behavior. If the animal approaches the flame or maintains a fixed distance from the flame, the shooting angle of the image data is adjusted to shoot the image data, and it is determined whether the position of the flame in the image information shot after the shooting angle is changed. If it has not changed, the flame is continuously photographed. If the flame position and the positions of other objects in the animal image change, the flame is judged to be a misjudgment caused by light and shadow. When the flame is continuously photographed, if the animal accelerates away from the flame, the animal behavior is judged to be fire avoidance behavior.

[0018] When the flame is continuously photographed, if the animal accelerates away from the flame, the animal behavior is judged as fire avoidance behavior. This further verifies the authenticity of the flame and determines that the animal's behavior is a response to the fire. By continuously photographing and observing the behavior of animals, the fire situation and the safety of the animals can be judged more accurately. For example, in forest fire monitoring, if an animal is found to be accelerating away from the flame area, it can be determined that the fire is spreading and fire extinguishing and rescue measures need to be taken in time. When it is recognized that the animal and the flame exist at the same time and the animal does not avoid the fire, the animal's behavior verifies that the flame is not a real flame. The flame collected at this time may be caused by the light and shadow of the shooting angle. For example, in an environment with strong sunlight or interference from other light sources, changing the shooting angle can avoid mistaking the light and shadow for flames (the flame cannot be photographed and the flame cannot be identified in the image data), reducing the occurrence of false alarms. If the flame is still recognized after changing the angle, there may be an object similar to flame in the environment, or there may be an abnormality in the device collecting image data. At this time, if the position of the flame does not change in the picture taken after adjusting the angle (the shape and position of the flame will not change with the shooting angle), the flame is an object similar to flame. If the position of the flame is always fixed at a certain position in the image data, there is an abnormality in the device collecting image data.

[0019] Furthermore, if the presence of a human is identified in the image information, the human body structure is identified to determine whether the human is another object; when the human is holding another object, the relationship between the object and the light source object is determined. If the light source object and the object held by the human are the same object, the judgment time is extended to determine whether the light source object is in contact with the human's head. If so, the human's behavior is determined to be smoking. If the light source object is not an object held by the human, the judgment time is extended to determine the change in the fire after the object held by the human comes into contact with the flame. If the fire becomes smaller or the flame disappears, the human's behavior is determined to be fire extinguishing. If the light source object and the object held by the human are the same object, after the judgment is extended, a gradually increasing flame appears on the other object after the light source object comes into contact with the other object, and the human's behavior is determined to be arson.

[0020] When humans smoke, smoke (light source object) not only contacts with human hands, but also with human heads. Through this connection, human smoking behavior can be judged. When humans put out fires, the fire should be reduced or even extinguished after humans, objects used to put out fires and light source objects (or humans and light source objects) come into contact. When humans set fires, the flames will be transferred from objects that humans touch to light source objects, or objects that come into contact with humans will become light source objects. By extending the judgment time, judging the positional relationship between humans and light source objects (burning objects), the changes in the positional relationship over time and the changes in the flame over time, human behavior can be accurately judged, and a more accurate and detailed understanding of the flame can be obtained through human behavior.

[0021] Furthermore, when humans commit arson, the flame is taken as the center to identify and determine whether there are blocking materials or flammable items around the ground. If there are blocking materials or no flammable items, the human's arson behavior is judged to be a safe behavior; otherwise, the number of pixels of the human in the image information at the time of arson is compared with the number of pixels collected subsequently, and the distance of the human's position change is calculated. The ratio of the number of pixels after the human's position change to the number of pixels of the light source object and the flame is used as the long-distance ratio. If the distance of the human's change exceeds the preset flame control distance, the human behavior is regarded as malicious arson, and the burning range of the flame is inferred based on the long-distance ratio and the human body structure. When the burning range exceeds the preset maximum burning range, an alarm is issued, and then it is identified whether the human has extinguished the fire. If there is no fire extinguishing behavior, the image information of the human arson behavior is reported.

[0022] When humans set fires, it may be for the sake of keeping warm, or it may be a malicious act of intentional arson. If there are no flammable items around the flame when the flame is burning, or there are blocking materials on the ground around the flame, it means that humans have controlled the excessive spread of the fire before setting the fire. At this time, the human behavior is safe. If the burning range of the flame is not controlled in advance before the flame burns, but the human is not too far away from the flame (humans can still control the flame), the judgment time is extended. Through the ratio of the flame and the human in the picture, the distance between the human and the flame and the size of the flame can be inferred. If the human is far away from the flame, it means that the human knows the danger of the flame and still chooses to set the fire. If the human is not far away from the flame after setting the fire, it means that the human may lack the awareness of fire safety. At this time, it is necessary to remind the human to put out the fire in time when the flame spreads excessively. If the human does not choose to put out the fire after hearing the fire extinguishing reminder, it means that the human is deliberately setting the fire.

[0023] Furthermore, it also includes an alarm unit, which is used to issue a fire alarm prompt according to the fire situation and display specific image information. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is the YOLOv8 model diagram in this embodiment.

[0025] Figure 2 This is a flow chart related to a flame detection method in an embodiment of this scheme.

[0026] Figure 3 This is a conceptual diagram of ACmix in the embodiment of this solution.

[0027] Figure 4 Schematic diagram of the structure of the mixing module of ACmix in the embodiment of this scheme. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the concept and technical effects of the present invention in combination with the embodiments, so as to fully understand the purpose, features and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention: YOLOv8 is the latest model in the YOLO series. The basic structure of YOLOv8 consists of three parts: Backbone, Neck and Head. Its principle is to extract, fuse and predict features of input images through neural networks to achieve target detection. The following is a detailed introduction: The backbone network is mainly responsible for extracting the features of the input image and providing a basis for subsequent target detection. The backbone network converts the original image into a series of feature maps with different levels of abstraction. These feature maps contain information of different scales and semantic levels in the image. The backbone network usually adopts a structure similar to CSPDarknet, which uses residual connections and CSP (Cross Stage Partial) structures. Residual connections can effectively solve the gradient vanishing problem in deep neural networks, making the network easier to train; the CSP structure can divide the feature map into two parts, one part passes directly, and the other part undergoes a series of convolution operations and then merges with the part that passes directly, which can reduce the amount of calculation and enhance the expressiveness of the features. In addition, some attention mechanisms may be introduced into the backbone network of YOLOv8, such as Efficient Channel Attention (ECA) and Global Attention Mechanism (GAM). These attention mechanisms help the model to better focus on the key information in the image and improve the effect of feature extraction.

[0029] The neck network (Neck) is located between the backbone network and the detection head. It is responsible for fusing and processing the multi-scale features extracted by the backbone network to enhance the model's ability to detect objects of different scales. Multi-scale feature fusion is very important for object detection because objects of different sizes have different feature scales in the image. By fusing feature maps of different scales, the model can better capture information about objects of various sizes. The neck network of YOLOv8 often uses a structure similar to PANet (Path Aggregation Network). PANet first uses an upsampling operation to restore the low-resolution feature map to a higher resolution, and then fuses it with the feature map of the corresponding scale from the backbone network. This allows the high-level semantic information to be transferred to the low-level feature map, enhancing the low-level feature map's ability to recognize the target. Next, a downsampling operation is performed to further process the fused feature map to obtain richer multi-scale information. In the neck network of YOLOv8, a new module called CSPlayer_2conv (c2f) is added. It refers to the design concept of ELAN of YOLOv7, applies the CSP method and introduces multiple gradient flows. Compared with the previous structure, it has richer gradient information and can improve the transmission and fusion effects of features.

[0030] The detection head (Head) predicts the category and position of the target based on the feature map output by the neck network. It generates prediction results on each grid unit of the feature map, including the category probability of the target and the location information of the bounding box. The detection head of YOLOv8 adopts the decoupled head approach to separate the classification and regression tasks for prediction. Compared with the previous coupled head structure, the decoupled head can better handle the differences between classification and regression tasks and improve the detection accuracy of the model. In addition, YOLOv8 has changed from an anchor-based method to an anchor-free method. It no longer relies on pre-defined anchor boxes to predict the position and size of the target, but directly predicts the coordinates of the center point of the target's bounding box and the offset of the width and height relative to the grid unit. This method reduces the dependence on anchor boxes and reduces the amount of calculation, while also improving the model's adaptability to targets of different shapes and sizes. The YOLOv8 algorithm shown above is a real-time object detection technology based on convolutional neural network (CNN), which can detect multiple objects in a single network forward propagation. This solution improves the YOLOv8 algorithm and adds the ACmix self-attention mechanism on the basis of YOLOv8, so that the model constructed by YOLOv8 can effectively capture global information and long-distance dependencies in the feature extraction process. By calculating the similarity between each position and other positions on the feature map and combining the content features for weighted averaging, it realizes the effective fusion of global information and local details, thus making up for the shortcomings of traditional convolution operations in processing global information. At the same time, it can enhance the model's adaptability to the diversity of flame shapes and sizes. The self-attention mechanism can better identify and locate flames, improve the accuracy and flexibility of detection even in complex backgrounds, and overcome the limitations of the fixed nature of the traditional anchor frame mechanism.

[0031] The ACmix self-attention mechanism combines multi-scale features and global information to make the model more accurate when processing small objects. For flame detection, especially small-scale flames or long-distance flames, the improved YOLOv8 algorithm can capture these targets more effectively, improving the comprehensiveness and reliability of detection. Although the self-attention mechanism usually increases computational complexity and memory consumption, ACmix optimizes the attention calculation process to efficiently capture global information while minimizing additional computational and memory overhead. This optimization ensures that the model can maintain a high inference speed while achieving high-precision flame detection to meet the needs of real-time detection.

[0032] This solution also improves the adaptability of flame detection technology to changes in ambient lighting and interference factors by introducing the ACmix self-attention mechanism. The improved YOLOv8 algorithm can better filter interference factors in the environment and capture the fine-grained features and edge information of the flame, thereby improving the stability and accuracy of detection and being suitable for more diverse application scenarios.

[0033] A flame detection method provided by this scheme is as follows Figure 2 shown.

[0034] In step S10, flame-related image data is collected, and the flame and smoke are marked to obtain flame information in the image data; animals in the image information are marked to obtain animal behavior information in the image information; the environment in the image information is marked to obtain lighting information in the image information.

[0035] Specifically, this solution collects images and videos of flames in various situations through online media, extracts images frame by frame from the collected videos, and annotates and interprets the flames, smoke, lighting references, and animals in the images.

[0036] Among them, when the flame model detects flames, the object superimposed with the flame is taken as the light source object through the relationship between the edge contour of the object and the flame contour, and it is judged whether the flame contour and the edge contour of the light source object change within a period of time; if the range covered by the flame contour becomes larger and the range covered by the edge contour of the light source object becomes smaller, it is judged that the fire is spreading; if the flame contour and the edge contour of the light source object remain unchanged, it is judged that there is no flame; if the range covered by the flame contour becomes smaller, it is judged that the fire is getting smaller; when the fire becomes larger, it is determined whether the fire continues to grow by multiple detections of the flame size. If it continues to grow, it is judged to be a dangerous situation.

[0037] More specifically, when annotating flames and smoke (when obtaining flame information in image data), use anchor frames to annotate flames, use marker points to draw the shape and outer contour of smoke, and annotate the spatial connection between flames and smoke. When annotating the environment, select reference objects with light and dark contrast in the image, and mark the light conditions of the reference objects in various directions; mark the light source objects that appear in the image, and record the number of light sources in the image. When annotating animals, annotate the animals that appear in the image and explain the behavior of the animals in the image.

[0038] More specifically, after collecting a large amount of data, when performing light and dark contrast annotation (obtaining lighting information in image information), the human annotator first anchors a reference object in the image data for collecting lighting conditions, selects lighting receiving points from the surface of the reference object that receives lighting, and selects backlight points from the surface of the reference object that is away from lighting. In a more specific embodiment, the human annotator marks multiple lighting receiving points and multiple backlight points on the same reference object.

[0039] More specifically, when annotating animals in image data (when obtaining animal behavior information in image information), the animals are selected by means of anchor boxes, and then labels are added to distinguish the animal species, thereby distinguishing the animals into humans and other animals.

[0040] In an embodiment of the present scheme, when annotating a light source object (i.e., a light object), a human annotator anchors the light source object in the picture, and simultaneously marks whether the light source object is in a burning state, and inputs the number of light sources present in the picture after anchoring all the light source objects in the picture.

[0041] When the manual annotator draws the shape and outer wheel of the smoke, he selects the turning point on the outer contour curve of the smoke as the marking point, outlines the outer contour curve of the flame as the flame curve, associates the flame curve with the marking point, and marks the positional relationship between the flame curve and the marking point.

[0042] In another embodiment of the present solution, when a manual annotator annotates image data of different angles in the same scene and time, the image data are associated together using reference objects as anchor points, and the association form of the image data is determined to be spatial association.

[0043] In step S20, a YOLOv8 model is constructed, and ACmix is ​​added to the convolution module of the backbone of the YOLOv8 model.

[0044] Specifically, if Figure 1 As shown in the figure, this scheme integrates self-attention and convolution technology to achieve a combination of the advantages of both, and then decomposes the operations in self-attention and convolution and reconstructs them into a 1x1 convolution form, thereby improving the computational efficiency.

[0045] More specifically, the integration of self-attention and convolution is achieved in the following way: S21: Feature decomposition, the query, key, value and convolution operation of the self-attention mechanism are feature decomposed through 1x1 convolution.

[0046] S22: Operation sharing, convolution and self-attention share the same 1x1 convolution operation, reducing the amount of repeated calculations.

[0047] S23: Feature fusion,In the ACmix model, the features generated by convolution and self-attention are fused through a summation operation,,which enhances the feature extraction capability of the model.

[0048] S24: Modular design. Through modular design, ACmix can be flexibly embedded into different network structures to enhance the representation capability of the network.

[0049] In the embodiment of this scheme, if Figure 3 As shown, Figure 3 The main concepts in ACmix are demonstrated, which compares the structures and computational complexity of convolution, self-attention, and ACmix. Figure 3 middle: (a) Convolution: shows the standard convolution operation, including a 1x1 convolution, indicating the convolution kernel size and the aggregation of convolution operations.

[0050] (b) Self-Attention: The self-attention mechanism is shown, which consists of a 1x1 convolution with three heads, a linear transformation of each head in the multi-head attention mechanism, and self-attention aggregation.

[0051] (c) ACmix: combines convolution and self-attention aggregation, where 1x1 convolution is shared between the two, aiming to reduce computational overhead and integrate lightweight aggregation operations. Overall, ACmix first optimizes the computational complexity on the feature channel by sharing computational resources (1x1 convolution) and combining two different aggregation operations.

[0052] More specifically, operation decomposition and reconstruction is to split the traditional convolution operation and self-attention operation and reconstruct them into a more efficient form. It is mainly achieved through the following steps: S25: Decompose convolution and self-attention, decompose the standard convolution kernel into multiple 1x1 convolution kernels, each of which processes a different subset of features, and convert the generation of query, key, and value in the self-attention mechanism into 1x1 convolution operations.

[0053] S26: Reconstructed into a hybrid module, the decomposed convolution and self-attention operations are reconstructed into a unified hybrid module, which includes both the spatial feature extraction capability of convolution and the global information aggregation function of self-attention.

[0054] S27: Improve computing efficiency. This decomposition and reconstruction method reduces redundant calculations, improves computing efficiency, and reduces the complexity of the model.

[0055] In the embodiment of this scheme, if Figure 4 As shown, Figure 4 The structure of the hybrid module proposed by ACmix is ​​shown. Figure 4 Contains: (a) Convolution: The 3x3 convolution is decomposed into 1x1 convolution, showing the transformation process of the feature map.

[0056] (b) Self-attention: The input features are first converted into query, key and value, implemented using x1 product, and the attention weight is calculated by similarity matching.

[0057] (c) ACmix: It combines the features of (a) and (b). In the first stage, three 1x1 convolutions are used to project the input feature map. In the second stage, the features obtained from the two paths are added as the final output.

[0058] Figure 1 The flow of the ACmix module is shown in Figure 5, emphasizing the fusion of the two mechanisms and providing the computational complexity of each operation block.

[0059] In step S30, the flame information, animal behavior information, and lighting information are used to train the YOLOv8 model to obtain a flame model for determining the form of flames and smoke, an animal behavior model for determining the type of animal and animal behavior, and a lighting model for determining the number of light sources and light source objects.

[0060] Specifically, the ACmix self-attention mechanism calculates the similarity of each position on the feature map and combines the content features (flame information, animal behavior information, and lighting information) for weighted averaging.

[0061] When marking the animals in the image information at step S30, the edge contours and pixel positions of each object in the image information are extracted, and the animal species and flame information in the image information are identified through the edge contours. When the animal is identified in the image information, the acquisition frequency of the picture data is increased, and the animal species in the image information and the pixel position of the flame information are combined to determine the distance between the animal and the flame. The position change of the animal and the flame is obtained by combining the acquisition interval of the image information with the distance between the animal and the flame, and the animal behavior is inferred by combining the position change with the flame intensity; if the fire intensity increases while the animal accelerates away from the flame, the animal behavior is determined as fire avoidance behavior; if the animal approaches the flame or maintains a fixed distance from the flame, the shooting angle of the picture data is adjusted to shoot the picture data, and it is determined whether the position of the flame in the image information shot after the shooting angle is changed. If it has not changed, the flame is continuously photographed. If the flame position and the positions of other objects in the animal image are changed, the flame is determined to be a misjudgment caused by light and shadow; when the flame is continuously photographed, if the animal accelerates away from the flame, the animal behavior is determined as fire avoidance behavior.

[0062] In step S40, the results generated by the flame model, the animal behavior model and the illumination model are combined to determine the fire condition.

[0063] Specifically, if the presence of a human is recognized in the image information, the human body structure is identified to determine whether the human is another object; when the human is holding another object, the relationship between the object and the light source object is determined. If the light source object and the object held by the human are the same object, the judgment time is extended to determine whether the light source object is in contact with the human's head. If so, the human's behavior is determined to be smoking. If the light source object is not an object held by the human, the judgment time is extended to determine the change in the fire after the object held by the human comes into contact with the flame. If the fire becomes smaller or the flame disappears, the human's behavior is determined to be fire extinguishing. If the light source object and the object held by the human are the same object, after the judgment is extended, a gradually increasing flame appears on the other object after the light source object comes into contact with the other object, and the human's behavior is determined to be arson. More specifically, when humans commit arson, the system uses the flame as the center to identify whether there are blocking materials or flammable items on the ground around the flame. If there are blocking materials or no flammable items, the human arson behavior is judged to be safe. Otherwise, the ratio of the number of pixels of the human in the image information to the number of pixels of the flame is calculated. If the ratio is less than the preset minimum ratio, it is considered a dangerous flame.

[0064] In an embodiment of the present scheme, the image data is a display device that is playing a fire image, the flame model recognizes the flame curve and multiple marking points corresponding to the smoke in the image data, the illumination model determines that the display device is a light source object, the flame curve and the marking points are both located within the outer contour of the light source object, and this scene is determined to be a temporarily safe flame. In another embodiment, the image data is a display device that is burning, the flame recognition model recognizes that the flame curve is located within the outer contour of the light source object, but there are marking points located outside the outer contour of the light source object, and this scene determines that the light source object, that is, the display device, is a burning object.

[0065] Specifically, when the animal behavior model identifies that the animal species is non-human, the actual situation of the flame is judged by the behavior of the animal in this scene (scene with flames).

[0066] When the species of the animal matches that of humans, the upper and lower limb behaviors of humans are separated, and the angle between the lower limbs and the ground, the relative position of the two legs, and the overlapping angle are used to judge the human's movement state. The human's movement state is then supplemented and corrected through the relative position and angle of the upper body and the torso.

[0067] When the human torso comes into contact with other objects, the human behavior is identified through the relationship between the human and the burning object; if the burning object is in contact with the human hand and the human is in a non-moving state, the human behavior is judged as smoking behavior; if the burning object is separated from the human and the human is in a moving state, the human is in a fire extinguishing state; if the burning object is separated from the human and the human is in a non-moving state, the human behavior is judged as safe behavior, otherwise the human behavior is judged as safe behavior; the fire conditions under safe behavior and smoking behavior are negligible fire conditions, and the fire conditions under fire extinguishing behavior and other behaviors are dangerous fire conditions.

[0068] If animals and humans exist in the same scene (in image data) at the same time, and the animal's behavior determines that the flame is a dangerous flame, while the human's behavior determines that the flame is a safe flame, then by tracing the image data in the same scene, the changes in the flame and fire conditions, as well as the connection between humans and the flame can be determined.

[0069] When the flame model detects flames, the object superimposed with the flame is taken as the light source object through the relationship between the edge contour of the object and the flame contour, and it is judged whether the edge contour of the flame contour and the edge contour of the light source object change within a period of time; if the range surrounded by the flame contour becomes larger, the edge contour of the light source object is changed, it is judged that the fire is spreading; if the flame contour and the edge contour of the light source object remain unchanged, it is judged that there is no flame; if the range surrounded by the flame contour becomes smaller, it is judged that the fire is getting smaller; when the fire becomes larger, it is determined whether the fire continues to grow by multiple detections of the flame size. If it continues to grow, it is judged to be a dangerous situation.

[0070] In the embodiments of the present scheme, most non-human animals will not set fires. These animals react to flames very realistically and will not pretend not to feel the presence of flames. By combining the behavior of animals moving away from or approaching flames after seeing them with the species of the animals, it is possible to determine whether the flames in the image information are real.

[0071] Specifically, when the animal behavior model identifies that the animal species is human, a partially safe fire situation can be determined through human behavior; by tracing the human behavior in the image data of the same scene, the actual fire situation and the relationship between the fire and humans can be further determined.

[0072] In an embodiment of the present scheme, the image data is of a person lighting a lighter and preparing to smoke. Since the human smoking behavior is a very characteristic behavior, in a scene where smoking is allowed, the flame produced by smoking proves the real existence of the flame, but this scene is not judged as a fire scene (that is, it is determined to be a safe fire).

[0073] In another embodiment, the image data is of a person sleeping by a fire in front of a fireplace. The fireplace is determined by the illumination model to be a temporarily safe flame. Although the person is in a sleeping state (safe behavior), the scene is determined to be a temporarily safe scene because the fire is safe, and the actual situation of the flame can be temporarily ignored.

[0074] In another embodiment, the image data shows that a person is putting out a fire. The person's behavior of putting out the fire proves the real existence of the flame, and the scene is determined to be a scene where a dangerous fire exists.

[0075] In another embodiment, the image data is of a human being leaving a burning object. By tracing the human's behavior in this scene, it is determined whether there is a direct or indirect connection between the human being and the generation of the flame. At the same time, by comparing the changes in the flame in the traced image data, the changes in the fire intensity are judged, and thus the generation of the flame is judged.

[0076] A flame detection method also includes a safety verification step. When there is a temporarily safe flame, the flame is detected again at a certain interval (generally 1 minute, or a fixed interval is set). If the flame size is always within a controllable range or the flame disappears, the flame in this scene is determined to be a safe flame.

[0077] A flame detection device, comprising: Information collection unit: collects flame-related image data, and marks flames and smoke to obtain flame information in the image data; marks animals in the image information to obtain animal behavior information in the image information; marks the environment in the image information to obtain lighting information in the image information; Model building unit: build a YOLOv8 model, add ACmix to the convolution module of the backbone of the YOLOv8 model, train the YOLOv8 model with flame information, animal behavior information, and lighting information, and obtain a flame model for judging the form of flames and smoke, an animal behavior model for judging the type and behavior of animals, and an animal behavior model for judging the number of light sources and light source objects; Comprehensive judgment unit: Combines the results produced by the flame model, animal behavior model and illumination model to judge the fire situation.

[0078] An alarm unit is used to issue a fire alarm prompt according to the fire situation and display specific image information.

[0079] Specifically, the information acquisition unit is communicated with the comprehensive judgment unit and the model building unit. The image data is first processed and labeled by the information acquisition unit, and then the flame model, animal behavior model and illumination model are constructed and trained by the model building unit. The information acquisition unit obtains the image data again and labels it for processing. After labeling, the image data is identified by the flame model, animal behavior model and illumination model to obtain flame information, animal behavior information and illumination information, and the flame information, animal behavior information and illumination information are transmitted to the comprehensive judgment unit. The comprehensive judgment unit combines the results generated by the flame model, animal behavior model and illumination model to judge the fire situation. The alarm unit issues a fire alarm prompt according to the fire situation and displays specific image information.

[0080] The above is only an embodiment of the present invention, and the common knowledge such as the known specific structure and characteristics in the scheme is not described in detail here. It should be pointed out that for those skilled in the art, several deformations and improvements can be made without departing from the structure of the present invention, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A flame detection method, characterized in that: The following steps are involved: S10: Collect flame-related image data, and mark the flame and smoke to obtain flame information in the image data; mark the animals in the image information to obtain animal behavior information in the image information; mark the environment in the image information to obtain lighting information in the image information; S20: Build the YOLOv8 model and add ACmix to the convolution module of the backbone of the YOLOv8 model; S30: training a YOLOv8 model with flame information, animal behavior information, and illumination information to obtain a flame model for determining flame and smoke forms, an animal behavior model for determining animal species and animal behaviors, and an illumination model for determining the number of light sources and light source objects; S40: Combining the results generated by the flame model, the animal behavior model and the illumination model to determine the fire condition.

2. A flame detection method according to claim 1, characterized in that: When the flame model detects flames, the object superimposed with the flame is taken as the light source object through the relationship between the edge contour of the object and the flame contour, and it is judged whether the flame contour and the edge contour of the light source object change within a period of time; if the range covered by the flame contour becomes larger and the range covered by the edge contour of the light source object becomes smaller, it is judged that the fire is spreading; if the flame contour and the edge contour of the light source object remain unchanged, it is judged that there is no flame; if the range covered by the flame contour becomes smaller, it is judged that the fire is getting smaller; when the fire becomes larger, it is determined whether the fire continues to grow by multiple detections of the flame size. If it continues to grow, it is judged to be a dangerous situation.

3. A flame detection method according to claim 2, characterized in that: Based on YOLOv8, the ACmix self-attention mechanism is added to integrate self-attention and convolution techniques.

4. A flame detection method according to claim 3, characterized in that: The ACmix self-attention mechanism is added on the basis of YOLOv8, and the operations in self-attention and convolution are decomposed and reconstructed into a 1x1 convolution form.

5. A flame detection method according to claim 4, characterized in that: When the flame model recognizes that there is flame or smoke in the image data, the image data is put into the illumination model to identify the light source object, and the relationship between the light source object and the flame is used to determine whether the fire is safe.

6. A flame detection method according to claim 5, characterized in that: When marking animals in image information, edge contours and pixel positions of various objects in the image information are extracted, and the animal species and flame information in the image information are identified through the edge contours. When an animal is identified in the image information, the frequency of image data acquisition is increased, and the animal species in the image information and the pixel position of the flame information are combined to determine the distance between the animal and the flame. The position change of the animal and the flame is obtained by combining the image information acquisition interval with the distance between the animal and the flame, and the animal behavior is inferred by combining the position change with the flame intensity. If the fire intensity increases while the animal accelerates away from the flame, the animal behavior is determined as fire avoidance behavior. If the animal approaches the flame or maintains a fixed distance from the flame, the shooting angle of the image data is adjusted to shoot the image data, and it is determined whether the position of the flame in the image information shot after the shooting angle is changed. If it has not changed, the flame is continuously photographed. If the flame position and the positions of other objects in the animal image are changed, the flame is determined to be a misjudgment caused by light and shadow. When the flame is continuously photographed, if the animal accelerates away from the flame, the animal behavior is determined to be fire avoidance behavior.

7. A flame detection method according to claim 6, characterized in that: If the presence of a human is recognized in the image information, the human body structure is identified to determine whether the human is another object; when the human is holding another object, the relationship between the object and the light source object is determined. If the light source object and the object held by the human are the same object, the judgment time is extended to determine whether the light source object is in contact with the human's head. If so, the human's behavior is determined to be smoking. If the light source object is not the object held by the human, the judgment time is extended to determine the change in the fire after the object held by the human comes into contact with the flame. If the fire becomes smaller or the flame disappears, the human's behavior is determined to be fire extinguishing. If the light source object and the object held by the human are the same object, after the judgment is extended, a gradually increasing flame appears on the other object after the light source object comes into contact with the other object, and the human's behavior is determined to be arson.

8. A flame detection method according to claim 7, characterized in that: When humans commit arson, the system uses the flame as the center to identify whether there are blocking materials or flammable items around the ground. If there are blocking materials or no flammable items, the arson behavior is judged to be safe. Otherwise, the ratio of the number of pixels of the human in the image information to the number of pixels of the flame is calculated. If the ratio is less than the preset minimum ratio, it is considered a dangerous flame.

9. A flame detection device, characterized in that: include: Information collection unit: collects flame-related image data, annotates flames and smoke, and obtains flame information from image data; Label the animals in the image information to obtain the animal behavior information in the image information; label the environment in the image information to obtain the lighting information in the image information; Model building unit: build a YOLOv8 model, add ACmix to the convolution module of the backbone of the YOLOv8 model, train the YOLOv8 model with flame information, animal behavior information, and illumination information, and obtain a flame model for judging the form of flames and smoke, an animal behavior model for judging the type and behavior of animals, and an illumination model for judging the number of light sources and light source objects; Comprehensive judgment unit: Combines the results produced by the flame model, animal behavior model and illumination model to judge the fire situation.

10. A flame detection device according to claim 9, characterized in that: It also includes an alarm unit, which is used to issue a fire alarm prompt according to the fire situation and display specific image information.

Images