Flame detection method and device based on image recognition, electronic equipment and medium

Abstract

The embodiment of the invention provides a flame detection method and device based on image recognition, electronic equipment and a medium, and relates to the technical field of computer vision. The method comprises the steps of obtaining a target video; performing video frame extraction on the target video to obtain at least two target video frames; according to the color feature of each target video frame and a preset target flame color rule, performing flame color region detection on each target video frame to obtain a flame color region; performing moving area detection on any two adjacent target video frames to obtain a moving area of each target video frame; wherein the adjacent target video frames are used for representing the adjacent video frames in the time sequence; and performing overlapped region screening on the flame color region and the moving region of each target video frame to obtain a target flame region. According to the embodiment of the invention, the accuracy and reliability of flame detection can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular, to a flame detection method and device based on image recognition, an electronic device, and a medium. Background Art

[0002] Fire detection is a technology for detecting fires in the environment by sensing the environment. For example, in the field of computer vision, images or videos can be collected through an image sensor (such as a camera), and image recognition technology can be used to detect flames from the images or videos. However, the current flame detection methods based on image recognition are easily affected by environmental interference. For example, static objects with colors similar to that of flames (such as red fire hydrants, red buildings, etc.) are likely to interfere with flame detection, making it difficult for flame detection to accurately identify the flames in the video and prone to false alarms. Especially in complex scenarios with the above interference factors, the accuracy of flame detection is relatively low.

[0003] Therefore, how to improve the accuracy and reliability of flame detection has become a technical problem to be solved urgently. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose a flame detection method and device based on image recognition, an electronic device, and a medium, aiming to improve the accuracy and reliability of flame detection.

[0005] To achieve the above object, a first aspect of the embodiments of the present application proposes a flame detection method based on image recognition, the method comprising:

[0006] Obtain a target video;

[0007] Extract video frames from the target video to obtain at least two target video frames;

[0008] According to the color feature of each target video frame and a preset target flame color rule, perform flame color region detection on each target video frame to obtain a flame color region; wherein, the color feature of each target video frame includes a red value, a green value, and a blue value, and the target flame color rule includes that the red value is greater than the green value and the green value is greater than the blue value:

[0009] Perform moving region detection on any two adjacent target video frames to obtain the moving region of each target video frame; wherein, the adjacent target video frames are used to represent video frames adjacent in time sequence;

[0010] Perform overlapping region screening on the flame color region and the moving region of each target video frame to obtain a target flame region.

[0011] To achieve the above object, a second aspect of the embodiments of the present application provides a flame detection device based on image recognition, the device comprising:

[0012] A video acquisition module, configured to acquire a target video;

[0013] A video frame extraction module, configured to extract video frames from the target video to obtain at least two target video frames;

[0014] A flame color detection module, configured to perform flame color region detection on each target video frame according to the color feature of each target video frame and a preset target flame color rule to obtain a flame color region;

[0015] A moving region detection module, configured to perform moving region detection on any two adjacent target video frames to obtain the moving region of each target video frame; wherein, the adjacent target video frames are used to represent video frames adjacent in time sequence;

[0016] A coincidence region screening module, configured to screen the coincidence region of the flame color region and the moving region of each target video frame to obtain a target flame region.

[0017] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the method described in the first aspect when executing the computer program.

[0018] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer-readable storage medium, the computer-readable storage medium storing a computer program, and the computer program implementing the method described in the first aspect when executed by a processor.

[0019] The flame detection method and device, electronic device, and medium based on image recognition proposed in this application extract target video frames from a target video to perform flame detection on each target video frame. According to the color features of each target video frame and the target flame color rule, the flame color region is detected, so that the region in each target video frame whose color features conform to the target flame color rule can be detected. For example, the region where the color features satisfy that the red value is greater than the green value and the green value is greater than the blue value. Moreover, by performing moving region detection on adjacent video frames, the moving region of each target video frame is obtained, so that the region where there are moving objects in the video frame can be detected. For each target video frame, the overlapping region of the flame color region and the moving region is screened, so that the target flame region satisfies both the color matching and motion characteristics conditions, significantly reducing the interference of red static objects or non-flame dynamic objects on flame detection, and effectively improving the accuracy and reliability of flame detection in complex scenarios. By combining the dual verification methods of flame color features and dynamic moving region detection, the reliability of flame detection in complex scenarios is effectively improved, and the accuracy of flame detection is also improved. Description of the Drawings

[0020] Figure 1 is the flowchart of the flame detection method based on image recognition provided by an embodiment of this application;

[0021] Figure 2 is Figure 1 the flowchart of step 103 in

[0022] Figure 3 is Figure 1 the flowchart of step 104 in

[0023] Figure 4 is Figure 3 the flowchart of step 302 in

[0024] Figure 5 is Figure 4 the flowchart of step 402 in

[0025] Figure 6 is Figure 1 the flowchart of step 105 in

[0026] Figure 7 is the schematic diagram of a target video frame provided by an embodiment of this application;

[0027] Figure 8 is provided by an embodiment of this application Figure 7 the binary image of the moving region in

[0028] Figure 9 is provided by an embodiment of this application Figure 7 the binary image of the flame color region in

[0029] Figure 10 It is a flowchart for analyzing the overlapping rate of flame regions in an application example provided by an embodiment of the present application;

[0030] Figure 11 It is a flowchart of a flame detection method based on image recognition provided by another embodiment of the present application;

[0031] Figure 12 It is a flowchart of a flame detection method based on image recognition in an application example provided by an embodiment of the present application;

[0032] Figure 13 It is a schematic structural diagram of a flame detection device based on image recognition provided by an embodiment of the present application;

[0033] Figure 14 It is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0034] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] It should be noted that although functional module division is performed in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different module division in the device or a different order in the flowchart. Terms such as "first" and "second" in the description, claims and the above-mentioned drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0037] First, several nouns involved in the present application are analyzed:

[0038] Artificial Intelligence (AI): It is a technical science that studies, develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, robotics, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning. The present application can acquire and process relevant data based on artificial intelligence technology.

[0039] Computer vision: It is a technology that simulates the human visual system through algorithms to achieve the perception, analysis, and understanding of digital images or video sequences. Computer vision includes image processing, etc.

[0040] Image recognition: It refers to the technology of using a computer to process, analyze, and understand images to identify various different objects. Image recognition is a practical application of deep learning algorithms.

[0041] Color space: Also known as color model, color space, or color system. A color space is a mathematical model used to represent colors. For example, color spaces include RGB color space, CMY color space, etc. For example, the RGB (red, green, blue) color space is a color space that describes colors through the three primary colors of red, green, and blue. The RGB color space is mainly defined based on the colors recognized by the human eye and can represent most colors.

[0042] The flame detection method, device, electronic device, and medium based on image recognition provided in the embodiments of the present application will be specifically described through the following embodiments. First, the flame detection method based on image recognition in the embodiments of the present application will be described.

[0043] The flame detection method based on image recognition provided in the embodiments of the present application can be applied to a terminal, can also be applied to a server side, or can also be software running on a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, can also be configured as a server cluster or a distributed system composed of multiple physical servers, or can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, middleware services, domain name services, and big data and artificial intelligence platforms; the software can be an application that implements the flame detection method based on image recognition, etc., but is not limited to the above forms.

[0044] This application can be used in many general or specific computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This application can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0045] It should be noted that in each specific embodiment of this application, when it comes to relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first. Moreover, the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when this application embodiment needs to obtain the user's sensitive personal information, the user's separate permission or separate consent will be obtained through methods such as pop-up windows or redirecting to a confirmation page. After clearly obtaining the user's separate permission or separate consent, the necessary user-related data for the normal operation of this application embodiment will be obtained.

[0046] Figure 1 It is an optional flowchart of the flame detection method based on image recognition provided by an embodiment of this application. Figure 1 The method in [description] can include but is not limited to steps 101 to 105.

[0047] Step 101, obtain a target video;

[0048] Step 102, extract video frames from the target video to obtain at least two target video frames;

[0049] Step 103, according to the color characteristics of each target video frame and a preset target flame color rule, detect the flame color area for each target video frame to obtain the flame color area; wherein, the color characteristics of each target video frame include a red value, a green value, and a blue value, and the target flame color rule includes that the red value is greater than the green value and the green value is greater than the blue value:

[0050] Step 104: Detect the moving regions for any two adjacent target video frames to obtain the moving region of each target video frame. Herein, adjacent target video frames are used to represent video frames adjacent in chronological order.

[0051] Step 105: Screen the overlapping regions of the flame color region and the moving region of each target video frame to obtain the target flame region.

[0052] The beneficial effects of the embodiments of the present application include but are not limited to: extracting target video frames from the target video to perform flame detection on each target video frame. Detecting the flame color region according to the color feature of each target video frame and the target flame color rule, so that the region with color features conforming to the target flame color rule in each target video frame can be detected, such as the region where the color feature satisfies that the red value is greater than the green value and the green value is greater than the blue value. Moreover, detecting the moving regions for adjacent video frames to obtain the moving region of each target video frame, so that the region with moving objects in the video frame can be detected. For each target video frame, screening the overlapping region of the flame color region and the moving region, so that the target flame region satisfies both the color matching and motion characteristics conditions, significantly reducing the interference of red static objects or non-flame dynamic objects on flame detection, and effectively improving the accuracy and reliability of flame detection in complex scenarios. By combining the dual verification method of flame color feature and dynamic detection of moving regions, the reliability of flame detection in complex scenarios is effectively improved, and the accuracy of flame detection is also improved.

[0053] In step 101 of some embodiments, the target video is a video containing flame image information. For example, the target video can be obtained by collecting a video of the flame through an image sensor (such as a camera, a camera, etc.), so as to perform flame detection on the target video. For another example, the target video can be obtained by selecting a video from a preset fire video database. The target video can also be obtained by other means, which is not limited thereto.

[0054] In step 102 of some embodiments, the target video frame refers to the video frame of the target video. The video frame can be extracted from the target video by a preset video frame extraction tool. The video frame can also be extracted by other means, which is not limited thereto.

[0055] In some embodiments, before step 103, the target video frame can be subjected to image enhancement processing to obtain an enhanced video frame. Then, the enhanced video frame is determined as the target video frame to perform flame color region detection on the target video frame (see step 103), which can improve the accuracy of flame color region detection. Specifically, the image enhancement processing includes enhancing the contrast, denoising, etc. For example, in the image enhancement processing, the image enhancement formula is as shown in the following formula:

[0056] l c = R a ·G b ·B c , formula (1);

[0057] In the formula, l c represents the enhanced video frame, that is, the target video frame after image enhancement; R a represents the red channel value (i.e., the red value) of the enhanced video frame; G b represents the green channel value (i.e., the green value) of the enhanced video frame; B c represents the blue channel value (i.e., the blue value) of the enhanced video frame; R represents the red value of the target video frame; G represents the green value of the target video frame; B represents the blue value of the target video frame; a represents the red index parameter, b represents the green index parameter, and c represents the blue index parameter.

[0058] It should be noted that the above three index parameters a, b, and c can be obtained through experimental optimization, and the index parameters a, b, and c are used to enhance the flame characteristics dominated by the red channel. The image enhancement formula can improve the contrast of the flame area and suppress background noise.

[0059] In step 103 of some embodiments, the color feature of the target video frame may include the pixel value of each pixel in the target video frame. Specifically, the color feature of the target video frame includes the color value of each pixel in the target video frame, such as the RGB value (i.e., the numerical values of the three color channels of red, green, and blue). For example, assume that the target video frame includes pixel p1, pixel p2, and pixel p3, where the RGB value of pixel p1 is [200, 50, 30], the RGB value of pixel p2 is [0, 255, 0], and the RGB value of pixel p3 is [203, 40, 20]. Then, the color feature of the target video frame includes {[200, 50, 30], [0, 255, 0], [203, 40, 20]}.

[0060] It should be noted that the flame color area refers to the area of the video frame in the target video frame where the color feature conforms to the target flame color rule. The target flame color rule is used to characterize the constraint conditions that the color feature of the flame should satisfy. If the color feature of a pixel conforms to the target flame color rule, it can be considered that the pixel contains flame information.

[0061] Similar to the above example, assume that the target flame color rule includes that the red value of a pixel is greater than 150. Then, the red value of pixel p1 (200) is greater than 150, and pixel p1 conforms to the target flame color rule. The red value of pixel p2 (0) is less than 150, and pixel p2 does not conform to the target flame color rule. The red value of pixel p3 (203) is greater than 150, and pixel p3 conforms to the target flame color rule. The video frame area composed of pixel p1 and pixel p3 that conform to the target flame color rule is used as the flame color area.

[0062] In step 104 of some embodiments, the moving area of the target video frame can be detected by a model for detecting moving objects (including mathematical models and neural network models). For example, the Gaussian Mixture Model (GMM) can be used to distinguish the background (static part) and foreground (moving object) of the target video frame to detect the moving object in the video frame.

[0063] It should be noted that adjacent target video frames refer to the two adjacent frames in the front and back in the time sequence in the target video. For example, assume that the frame interval of the target video is 1 second, the time of the first frame is t = 0, and the time of the second frame is t = 1. Then, the first frame and the second frame of the target video belong to adjacent target video frames.

[0064] In step 105 of some embodiments, the target flame area refers to the moving area with flame color characteristics, that is, the overlapping area of the flame color area and the moving area in the same target video frame. Specifically, the set of pixels that belong to both the flame color area and the moving area in the same target video frame can be determined as the target flame area.

[0065] In some embodiments, it should be noted that current fire detection technologies mainly rely on physical devices such as smoke sensors and temperature sensors to achieve fire warning by detecting parameters such as smoke concentration and temperature changes. Since smoke sensors or temperature sensors are easily affected by the environment (such as high-temperature environments or non-fire heat sources, including sunlight, heating, etc.), the false alarm rate is relatively high, thereby reducing the reliability of the fire detection system. Moreover, in order to detect the fire source, smoke sensors and temperature sensors need to be in close contact with the fire source and need to be densely deployed, which results in a limited coverage area for fire detection and it is difficult to detect large-scale scenarios in real time. In addition, physical sensing devices such as smoke sensors and temperature sensors all rely on changes in physical signals to detect flames, which leads to a significant signal response delay in the detection process and it is difficult to achieve rapid identification in the early stage of a fire.

[0066] In view of the defects of limited coverage and signal response delay, computer vision technology can be used to quickly detect fires in a large-scale scene. However, there is still a problem of high false alarm rate in the current flame detection method based on computer vision. For example, the current flame detection method is easily interfered by static objects or moving objects with colors similar to flames. In response to this, the embodiments of the present application improve the reliability of flame detection in complex scenes and reduce the false alarm rate by combining the dual verification methods of flame color features and dynamic detection of moving areas.

[0067] Please refer to Figure 2 , in some embodiments, the target video frame includes at least two target pixels; the target flame color rules include a first flame color rule and a second flame color rule;

[0068] Step 103 may include but is not limited to steps 201 to 205:

[0069] Step 201, extract first color features from each target pixel through a preset first color space to obtain first color features; wherein, the first color features include a red value, a green value, and a blue value;

[0070] Step 202, select target pixels whose first color features conform to the first flame color rule from at least two target pixels to obtain first flame color pixels; wherein, the first flame color rule is used to represent that the red value is greater than the green value and the green value is greater than the blue value;

[0071] Step 203, extract second color features from each target pixel through a preset second color space to obtain second color features; wherein, the second color features include a red-green axis chromaticity and a yellow-blue axis chromaticity;

[0072] Step 204, select target pixels whose second color features conform to the second flame color rule from at least two target pixels to obtain second flame color pixels; wherein, the second flame color rule is used to represent that the yellow-blue axis chromaticity is greater than or equal to the red-green axis chromaticity;

[0073] Step 205, obtain at least one flame color pixel according to the union of the first flame color pixels and the second flame color pixels, and determine the combination of at least one flame color pixel as the flame color area.

[0074] The advantages of this embodiment are as follows. By jointly analyzing the color features of each target pixel in the target video frame in the first color space and the second color space, it is possible to comprehensively determine whether the target pixel represents a flame. First, the red value, green value, and blue value of the target pixel are extracted in the first color space, and the target pixels that conform to the flame color sequence characteristics are screened out using the first flame color rule (the red value is greater than the green value, and the green value is greater than the blue value), which can effectively reduce the interference of non-flame color sequences such as blue light sources. Moreover, the red-green axis chromaticity and yellow-blue axis chromaticity of the target pixel are extracted in the second color space, and the target pixels with a color bias towards the yellow spectrum are identified through the second flame color rule (the yellow-blue axis chromaticity is greater than or equal to the red-green axis chromaticity) to select the pixels that conform to the color characteristics of natural flames and reduce false positives caused by red objects or red artificial light sources. Then, based on the union of the first flame color pixels and the second flame color pixels, at least one flame color pixel is obtained, which not only retains the pixel detection results in the two color models but also reduces the limitations of single-color space analysis through complementary rules (i.e., the above-mentioned first flame color rule and second flame color rule). This can improve the accuracy and robustness of flame color recognition, making the detection results more in line with the spectral characteristics of natural flames, thereby improving the accuracy of flame detection and reducing the false alarm rate of fire alarms.

[0075] In step 201 of some embodiments, the first color space may include the RGB color space. The first color feature of the target pixel is used to characterize the color value of the target pixel, that is, the RGB value (including the red value, green value, and blue value). For example, assume that the red value of the target pixel p1 is 200, the green value is 50, and the blue value is 30. Then the color feature of the target pixel p1 is [200, 50, 30].

[0076] In step 202 of some embodiments, specifically, the first flame color rule may include three sub-rules. For example, the first flame color rule includes: (1) The first flame color sub-rule, which characterizes that the red channel value of the target pixel is greater than the mean red channel value of the target video frame; (2) The second flame color sub-rule, which characterizes that the red channel value of the target pixel is greater than the green channel value and the green channel value is greater than the blue channel value; (3) The third flame color sub-rule, which characterizes that the saturation of the target pixel is greater than a preset saturation threshold.

[0077] For example, the formula group corresponding to the first flame color rule is as follows:

[0078]

[0079] In the formula, R represents the red channel value (i.e., the red value) of the target pixel; G represents the green channel value (i.e., the green value) of the target pixel; B represents the blue channel value (i.e., the blue value) of the target pixel; R meanrepresents the mean value of the red channel of the target video frame; S represents the saturation of the target pixel; S T represents the saturation channel threshold; R T represents the red channel threshold;

[0080] In some embodiments, the first formula of the above formula group corresponds to the first flame color sub - rule, the second formula corresponds to the second flame color sub - rule, and the third formula corresponds to the third flame color sub - rule. Specifically, the first flame color sub - rule is used to screen out the pixels with red channel values higher than the mean value to filter out the dark background. The second flame color sub - rule is used to screen out the pixels whose color sequence satisfies red > green > blue to exclude non - flame color sequences (such as blue light sources). The third flame color sub - rule is used to reduce the interference of low - saturation objects (such as gray smoke).

[0081] In some embodiments, the maximum value of the color channel (such as the red channel, green channel or blue channel) is 255. It should be noted that in the third flame color sub - rule, the judgment threshold of saturation is dynamically adjusted according to the red value of the target pixel, which can reduce the interference caused by reflection. For example, if the red value R is small, the judgment threshold of saturation is large, so that it can effectively distinguish the reflection area with low red value - low saturation (such as the reflection of red clothes) from the flame edge area with low red value - high saturation (such as the dark part outside the flame), thereby filtering out the false red signals caused by reflection and accurately identifying the color characteristics of the flame edge area at the same time.

[0082] It should be noted that the saturation channel threshold and / or the red channel threshold can be set according to experimental experience. In some embodiments, the saturation channel threshold S T = 50. In some embodiments, the red channel threshold R T has a value range including [150, 250]. For example, the red channel threshold R T can take any value among 150, 200, 250. It can also dynamically adjust the saturation channel threshold and / or the red channel threshold according to the scene or other requirements. The embodiments of the present application do not limit the specific values of the above thresholds.

[0083] In some embodiments, the mean value R of the red channel of the target video frame mean is defined as shown in the following formula:

[0084]

[0085] In the formula, R mean represents the mean value of the red channel of the target video frame; N represents the total number of pixels of the target video frame I, that is, the number of target pixels x; x represents the target pixel; I R(x) represents the red channel value of the target pixel x.

[0086] In step 203 of some embodiments, the second color space may include a Lab color space. It should be noted that the Lab color space is a mathematical model that describes color using brightness, red-green axis chromaticity, and yellow-blue axis chromaticity. In some embodiments, the second color feature includes brightness, red-green axis chromaticity, and yellow-blue axis chromaticity.

[0087] In step 204 of some embodiments, specifically, the second flame color rule may include four sub-rules. For example, the second flame color rule includes: (1) a fourth flame color sub-rule, indicating that the luminance of the target pixel is greater than or equal to the average luminance of the target video frame; (2) a fifth flame color sub-rule, indicating that the red-green axis chromaticity of the target pixel is greater than or equal to the average red-green axis chromaticity of the target video frame; (3) a sixth flame color sub-rule, indicating that the yellow-blue axis chromaticity of the target pixel is greater than or equal to the average yellow-blue axis chromaticity of the target video frame; and (4) a seventh flame color sub-rule, indicating that the yellow-blue axis chromaticity is greater than or equal to the red-green axis chromaticity.

[0088] For example, the formula group corresponding to the second flame color rule is as follows:

[0089]

[0090] Where, L * Indicates the brightness of the target pixel; Represents the mean brightness of the target video frame; a * Indicates the red-green axis chromaticity of the target pixel; represents the mean chromaticity of the red and green axes of the target video frame; b * Indicates the yellow-blue axis chromaticity of the target pixel; Indicates the mean chromaticity of the yellow-blue axis of the target video frame.

[0091] It should be noted that the target video frame's mean luminance value refers to the average luminance value of each pixel in the target video frame. The target video frame's red-green axis chromaticity value refers to the average chromaticity value of each pixel in the target video frame. The target video frame's yellow-blue axis chromaticity value refers to the average chromaticity value of each pixel in the target video frame.

[0092] Specifically, in Lab color space, the greater the chromaticity of the red-green axis, the more the color leans toward the red spectrum. For example, positive values on the red-green axis indicate a reddish color, while negative values indicate a greenish color. Furthermore, the greater the chromaticity of the yellow-blue axis, the more the pixel color leans toward the yellow spectrum. For example, positive values on the yellow-blue axis indicate a yellowish color, while negative values indicate a bluish color.

[0093] In some embodiments, the first formula of the above formula group corresponds to the fourth flame color sub-rule, the second formula corresponds to the fifth flame color sub-rule, the third formula corresponds to the sixth flame color sub-rule, and the fourth formula corresponds to the seventh flame color sub-rule. Specifically, the fourth flame color sub-rule is used to screen pixels with higher brightness. Among them, brightness is independent of chromaticity, so the brightness of pixels can be judged separately, reducing the influence of brightness changes caused by light on flame detection. The fifth flame color sub-rule is used to screen pixels biased towards the red spectrum. The sixth flame color sub-rule is used to screen pixels biased towards the yellow spectrum. The seventh flame color sub-rule is used to screen pixels biased towards the yellow-red spectrum, thereby identifying pixels that conform to the natural flame color characteristics and reducing the interference of red artificial light sources.

[0094] In some embodiments, it should be noted that the current flame color detection method cannot accurately distinguish flames from objects with similar colors (such as red clothes, red lights, red autumn leaves, etc.). Considering the above problems, in the second color space, the embodiments of the present application screen out pixels whose chromaticity on the yellow-blue axis is greater than or equal to the chromaticity on the red-green axis through the second flame color rule, so as to identify pixels whose color is biased towards the yellow-red spectrum rather than the pure red spectrum, thereby reducing false judgments caused by red artificial light sources (such as fire truck lights, neon lights, car tail lights).

[0095] In step 205 of some embodiments, at least one flame color pixel is the union of the first flame color pixel and the second flame color pixel, which can improve the sensitivity of detecting the flame area and more comprehensively identify potential flames. For example, if the target pixel belongs to the first flame color pixel or the second flame color pixel, the target pixel is determined as a flame color pixel. In some embodiments, the flame color area is a connected area composed of at least one flame color pixel. The flame color area can also be a non-connected area, which is not limited herein.

[0096] In some embodiments, the flame color pixel can be determined by the following formula:

[0097] R color =R RGB ∪R Lab , formula (5);

[0098] In the formula, R color represents the flame color pixel; R RGB represents the first flame color pixel; R Lab represents the second flame color pixel.

[0099] Please refer to Figure 3 , in some embodiments, at least two target video frames include the first video frame and the second video frame, and the first video frame is the previous frame of the second video frame;

[0100] Step 104 may include but is not limited to steps 301 to 303:

[0101] Step 301, performing first moving region detection on the first video frame through a pre-trained initial motion detection model to obtain a first moving region;

[0102] Step 302, updating the parameters of the initial motion detection model according to the region proportion of the first moving region in the first video frame and the color features of the second video frame to obtain a target motion detection model;

[0103] Step 303, performing second moving region detection on the second video frame through the target motion detection model to obtain a second moving region.

[0104] The advantage of this embodiment is that by dynamically adjusting the parameters of the initial motion detection model and jointly adjusting the model parameters using the region proportion of the moving region in the previous frame (such as the first video frame) and the color features of the current frame (such as the second video frame), the model can adjust the detection strategy in real time according to the dynamic characteristics of the scene. Specifically, the region proportion of the moving region in the first video frame reflects the dynamic degree of the scene. For example, when the region proportion is relatively large, it indicates the presence of significant moving objects or a large number of moving objects. Updating the model parameters by combining the region proportion of the moving region in the first video frame and the color features of the second video frame enables the target motion detection model to not only capture fast-changing moving targets (such as the flame spreading region) in the dynamic scene but also enhance the recognition ability for specific targets through color features. That is to say, the embodiments of the present application do not perform frame-by-frame independent detection but update the model parameters by analyzing the correlation of the information of the previous and current frames and adopting a scene-adaptive detection method, so that the target motion detection model can adapt to the changes in the scene, thereby improving the flexibility and reliability of the moving region detection of the target motion detection model and further improving the reliability of flame detection.

[0105] In step 301 of some embodiments, the initial motion detection model may be a Gaussian mixture model (GMM). The Gaussian mixture model is a model used to detect moving objects in a video sequence. The Gaussian mixture model can be used for background modeling, foreground detection, and moving object detection. It should be noted that the moving region refers to the pixel region containing moving objects identified in the video frame through a motion detection model (such as the initial motion detection model, the target motion detection model). The first moving region refers to the moving region of the first video frame.

[0106] In some embodiments, it should be noted that current moving object detection methods, such as frame difference method, optical flow method, etc., are easily affected by environmental interferences (such as illumination changes, noise interferences, etc.), resulting in a relatively high false alarm rate. For example, the frame difference method is sensitive to noise interference and has a high sensitivity in threshold selection. If the threshold is set too low, there will be a large amount of noise interference in the detection result; if the threshold is set too high, slow-moving objects in the image may be ignored. Especially in a complex environment where the speed of moving objects may change rapidly, current moving detection methods are difficult to apply to such a complex environment. Considering the above problems, the embodiments of the present application update the model parameters based on the motion state of the current frame (i.e., the ratio of the moving area of the first video frame in the first video frame), so that the target moving detection model adapts to the changes in the motion state of the scene, thereby improving the reliability of flame detection.

[0107] In some embodiments, before step 301, the flame detection method may further include: constructing a model for the color value of each pixel in the first video frame according to a preset number of target distributions to obtain an initial moving detection model; wherein, the initial moving detection model includes at least one Gaussian distribution (also known as normal distribution), and the number of Gaussian distributions is the number of target distributions. For example, a mixture of K Gaussian distributions can be obtained by modeling according to the color value of each pixel in the first video frame (such as RGB value or Lab value). The value range of K (i.e., the number of target distributions) may include 3 to 5. For example, the value of K can be 3, 4, or 5.

[0108] Then, at least one video frame can be selected from the target video, and Gaussian distribution fitting is performed on the at least one video frame through the Gaussian distribution of the initial moving detection model to determine the initial values of the parameters of the Gaussian distribution. For example, the first N video frames in the target video can be selected and input into the initial moving detection model, so that the model learns the color distribution of the pixels in the video frame, and the mean μ, variance σσ, and weight ω of each Gaussian distribution are determined, thereby obtaining the initial moving detection model after training.

[0109] In some embodiments, step 301 may include: performing matching analysis on the pixel value of each target pixel in the first video frame and each Gaussian distribution through the pre-trained initial moving detection model to obtain a matching result; if the matching result indicates a matching failure, the target pixel is determined as a moving pixel; and a combination of at least one moving pixel is determined as the first moving area. For example, during the process of moving area detection, the distribution matching formula is as shown in the following formula:

[0110] |X t -μ i |≤2.5σ i (i = 1, 2,..., K), Formula (6);

[0111] Wherein, X t represents the color value of the pixels in the t-th frame; μ i represents the mean of the i-th Gaussian distribution, and σ i represents the standard deviation of the i-th Gaussian distribution; K represents the number of Gaussian distributions.

[0112] In some embodiments, if the pixel value of the target pixel satisfies the distribution matching formula corresponding to at least one Gaussian distribution in the initial motion detection model, the matching result indicates a successful match. Conversely, if the pixel value of the target pixel does not satisfy any distribution matching formula corresponding to the Gaussian distributions in the initial motion detection model, the matching result indicates a failed match. The matching result can also be determined by other means, not limited to this. It should be noted that if the match is successful, it is considered that the pixel may belong to the background (i.e., static pixel), otherwise it belongs to the foreground (i.e., moving pixel).

[0113] In step 302 of some embodiments, the area ratio of the moving area of the first video frame in the first video frame can be the ratio of the moving pixels of the first video frame. As for the meaning and function of the ratio of moving pixels, reference can be made to the specific description of step 401 below, and details will not be elaborated here.

[0114] In some embodiments, the color feature of the second video frame can be the color value of the pixels in the second video frame, such as the RGB value. The target motion detection model refers to the initial motion detection model after updating the parameters.

[0115] In step 303 of some embodiments, it should be noted that the second moving area refers to the moving area of the second video frame.

[0116] Please refer to Figure 4 , in some embodiments, the initial motion detection model has initial distribution parameters;

[0117] Step 302 may include but is not limited to steps 401 to 403:

[0118] Step 401, calculate the ratio according to the number of pixels in the first moving area and the number of pixels in the first video frame to obtain the ratio of moving pixels;

[0119] Step 402, calculate the distribution parameters according to the ratio of moving pixels, the color value of each pixel in the second video frame, and the initial distribution parameters to obtain the target distribution parameters;

[0120] Step 403, update the distribution parameters of the initial motion detection model according to the target distribution parameters to obtain the target motion detection model.

[0121] The advantage of this embodiment is that by calculating the ratio of the number of pixels in the first moving area to the number of pixels in the first video frame, the proportion of moving pixels is obtained, and based on the proportion of moving pixels, the color value of each pixel in the second video frame, and the initial distribution parameters, the distribution parameter calculation is performed to obtain the target distribution parameter. In this way, the initial distribution parameter of the initial motion detection model can be updated according to the proportion of moving pixels in the previous frame (i.e., the first video frame), so that the initial motion detection model can adapt to the scene motion state of the first video frame, thereby improving the flexibility and reliability of the target motion detection model for performing motion detection on subsequent video frames (such as the second video frame), and further improving the reliability of flame detection.

[0122] In step 401 of some embodiments, the proportion of moving pixels refers to the ratio of the number of pixels in the first moving area in the first video frame to the total number of pixels in the first video frame. Specifically, the number of pixels in the first moving area can be divided by the total number of pixels in the first video frame to obtain the proportion of moving pixels. For example, assuming that the number of pixels in the first moving area is 50 and the total number of pixels in the first video frame is 100, then the proportion of moving pixels is 50%.

[0123] In step 402 of some embodiments, the initial distribution parameter refers to the distribution parameter of the initial motion detection model. The target distribution parameter refers to the updated initial distribution parameter. Specifically, the initial motion detection model can be a Gaussian mixture model, which is composed of multiple Gaussian distributions, and the distribution parameters (such as the initial distribution parameter or the target distribution parameter) include the mean and variance of the Gaussian distribution.

[0124] In step 403 of some embodiments, the distribution parameter of the target motion detection model is the target distribution parameter.

[0125] Please refer to Figure 5 , in some embodiments, the initial distribution parameter includes an initial first distribution parameter and an initial second distribution parameter;

[0126] Step 402 may include but is not limited to steps 501 to 507:

[0127] Step 501, multiply the proportion of moving pixels by a preset attenuation coefficient to obtain a target product;

[0128] Step 502, perform cumulative distribution calculation on the target product through a preset exponential cumulative distribution function to obtain a target learning rate;

[0129] Step 503, determine the color value of each pixel in the second video frame as the target pixel color vector;

[0130] Step 504: Perform weighted summation based on the initial first distribution parameter, the target learning rate, and the target pixel color vector to obtain the target first distribution parameter;

[0131] Step 505: Calculate the difference between the target pixel color vector and the target first distribution parameter to obtain the deviation vector;

[0132] Step 506: Perform weighted summation based on the initial second distribution parameter, the target learning rate, the deviation vector, and the transposed vector of the deviation vector to obtain the target second distribution parameter;

[0133] Step 507: Use the combination of the target first distribution parameter and the target second distribution parameter as the target distribution parameter.

[0134] The advantage of this embodiment is that the target product is obtained by multiplying the moving pixel ratio by the preset attenuation coefficient, and the target learning rate is calculated based on the exponential cumulative distribution function for the target product; in this way, the learning rate of the model (i.e., the target learning rate) can be dynamically adjusted according to the moving pixel ratio to update the distribution parameters of the model. Then, weighted summation is performed based on the initial first distribution parameter, the target learning rate, and the target pixel color vector to obtain the target first distribution parameter, enabling the model to retain the stability of historical parameters and quickly respond to the changes in the current frame during the parameter update process. Also, based on the difference between the target pixel color vector and the target first distribution parameter, the deviation vector is obtained; weighted summation is performed based on the initial second distribution parameter, the target learning rate, the deviation vector, and the transposed vector of the deviation vector to obtain the target second distribution parameter to update the initial second distribution parameter, thereby obtaining the model after updating the distribution parameters, that is, the target motion detection model, which can improve the adaptability of the target motion detection model to complex motion patterns and enhance the flexibility and reliability of the target motion detection model for motion detection.

[0135] In step 501 of some embodiments, the target product refers to the product of the moving pixel ratio and the attenuation coefficient, that is, λ·MPR in the definition formula of the target learning rate (see the specific description of step 502 below).

[0136] In some embodiments, it should be noted that the exponential cumulative distribution function is the cumulative distribution function of the exponential distribution. The formula for the exponential cumulative distribution function is 1 - e -z , where z represents the independent variable of the exponential cumulative distribution function. For example, the independent variable of the exponential cumulative distribution function can be the target product λ·MPR.

[0137] In step 502 of some embodiments, the target learning rate is defined as follows:

[0138] α = 1 - e -λ·MPR , formula (7);

[0139] In the formula, α represents the target learning rate; λ represents the decay coefficient; MPR represents the Moving Pixel Ratio.

[0140] Specifically, the value of the decay coefficient λ can be 0.1. The decay coefficient λ can also be set or updated to other values, not limited to this.

[0141] It should be noted that the value range of the Moving Pixel Ratio MPR includes [0, 1]. The relationship between the Moving Pixel Ratio and the target learning rate is negatively correlated. The target learning rate is used to control the speed of parameter update. For example, if the Moving Pixel Ratio is close to 0, it indicates that the scene of the video frame is a static scene. In this case, the target learning rate is small, so as to reduce false alarms. If the Moving Pixel Ratio is close to 1, it indicates that the scene of the video frame is a dynamic scene. In this case, the target learning rate is large, so as to update the parameters more quickly to adapt to the scene change.

[0142] In some embodiments, for example, when the initial motion detection model is a Gaussian mixture model, the initial motion detection model includes at least one Gaussian distribution, and each Gaussian distribution has a weight, a mean, and a variance. The update formula of the weight is as shown in the following formula:

[0143] ω i * =(1 - α)ω i +α, Formula (8);

[0144] In the formula, ω i * represents the updated weight of the i-th Gaussian distribution; ω i represents the weight of the i-th Gaussian distribution; α represents the target learning rate.

[0145] In step 503 of some embodiments, the target pixel color vector can be a vector composed of the red value, green value, and blue value of the target pixel. For example, assume that the red value (R) of the target pixel is 200, the green value (G) is 50, and the blue value (B) is 30, then the target pixel color vector X t is [200, 50, 30].

[0146] In step 504 of some embodiments, performing weighted summation according to the initial first distribution parameter, the target learning rate, and the target pixel color vector may include: obtaining a first updated weight according to the ratio of the target learning rate to the weight of the Gaussian distribution in the initial motion detection model; obtaining a second updated weight according to the difference between 1 and the first updated weight; and performing a summation calculation according to the product of the first updated weight and the target pixel color vector, and the product of the second updated weight and the initial first distribution parameter, to obtain the target first distribution parameter.

[0147] It should be noted that in the above process of weighted summation, the first update weight is equivalent to the weight of the target pixel color vector, and the second update weight is equivalent to the weight of the initial first distribution parameter. As for the definition of the weight of the Gaussian distribution in the initial motion detection model, reference can be made to the specific description of the weight update formula above, which will not be elaborated here.

[0148] In some embodiments, the initial first distribution parameter may be the mean. Specifically, the definition of the target first distribution parameter is shown in the following formula (also known as the mean update formula):

[0149]

[0150] In the formula, μ i * represents the updated mean of the i-th Gaussian distribution, that is, the target first distribution parameter; μ i represents the mean of the i-th Gaussian distribution, that is, the initial first distribution parameter; α represents the target learning rate; ω i * represents the updated weight of the i-th Gaussian distribution; x t represents the color value of the pixel in the target video frame of the t-th frame, that is, the target pixel color vector.

[0151] In some embodiments, it should be noted that represents the first update weight; represents the second update weight.

[0152] In step 505 of some embodiments, the target first distribution parameter may specifically be a vector, and the deviation vector is the vector obtained by subtracting the target first distribution parameter from the target pixel color vector. For example, assume that the target first distribution parameter μ i * is [5, 30, 80], and the target pixel color vector x t is [20, 50, 90], then the deviation vector (x t - μ i * ) is [15, 20, 10].

[0153] In step 506 of some embodiments, performing weighted summation according to the initial second distribution parameter, the target learning rate, the deviation vector, and the transposed vector of the deviation vector may include: obtaining the first update weight according to the ratio of the target learning rate to the weight of the Gaussian distribution in the initial motion detection model; obtaining the second update weight according to the difference between 1 and the first update weight; and performing a summation calculation according to the product of the first update weight, the transposed vector of the deviation vector, and the deviation vector, and the product of the second update weight and the initial second distribution parameter to obtain the target second distribution parameter.

[0154] It should be noted that in the above process of weighted summation, the first update weight is equivalent to the weight of the product of the transposed vector of the deviation vector and the deviation vector, and the second update weight is equivalent to the weight of the initial second distribution parameter.

[0155] In some embodiments, the initial second distribution parameter may be variance. Specifically, the definition of the target second distribution parameter is shown in the following formula (also referred to as the variance update formula):

[0156]

[0157] In the formula, represents the variance after the update of the i-th Gaussian distribution, that is, the target second distribution parameter; represents the variance of the i-th Gaussian distribution, that is, the initial second distribution parameter; α represents the target learning rate; ω i * represents the weight after the update of the i-th Gaussian distribution; (X t -μ i * ) represents the difference between each component in the target pixel color vector and the updated mean value (that is, the target first distribution parameter), that is, the deviation vector; (X t -μ i * ) T represents the transposed vector of the deviation vector.

[0158] In step 507 of some embodiments, the target distribution parameter includes the target first distribution parameter and the target second distribution parameter. For example, the target distribution parameter includes the updated mean μ i * , and the updated variance

[0159] In some embodiments, for example, assume that the target pixel color vector X t (that is, the RGB value) of the pixel at time t is [200, 50, 30]; the current weight ω i * of the Gaussian distribution is 1, the mean μ i is [180, 60, 40], and the variance is 100; the target learning rate α is 0.01.

[0160] The updated mean can be calculated through the mean update formula: μ i * = 0.99×[180, 60, 40] + 0.01×[200, 50, 30] = [180.2, 59.9, 39.9].

[0161] The updated variance can be calculated through the variance update formula:

[0162] It should be noted that in the embodiments of the present application, by iteratively updating the distribution parameters of the initial motion detection model, the obtained target motion detection model can accurately distinguish the background (such as static objects) and the foreground (such as dynamically changing fires or moving objects), thereby improving the accuracy of motion detection.

[0163] Please refer to Figure 6 , in some embodiments, at least two target video frames include a first video frame and a second video frame, and the first video frame is the previous frame of the second video frame;

[0164] Step 105 may include but is not limited to steps 601 to 604:

[0165] Step 601, screening the overlapping area between the flame color area and the moving area of each target video frame to obtain an initial flame area;

[0166] Step 602, calculating the area overlap rate between the initial flame area of the first video frame and the initial flame area of the second video frame to obtain a flame area overlap rate;

[0167] Step 603, comparing the flame area overlap rate with a preset overlap rate threshold;

[0168] Step 604, if the flame area overlap rate is greater than or equal to the overlap rate threshold, confirm the initial flame area as the target flame area.

[0169] The advantage of this embodiment is that by analyzing the overlap rate of the initial flame areas in consecutive video frames (such as the first video frame and the second video frame), the flame area overlap rate is obtained and compared with the overlap rate threshold to verify the stability of the flame area in consecutive frames. For example, if the flame area overlap rate is greater than or equal to the overlap rate threshold, it indicates that the initial flame area is a relatively stable flame, so the initial flame area is confirmed as the target flame area. This can exclude the interference caused by short-term moving objects (such as birds, fluttering clothes), further improve the robustness and accuracy of flame detection, and reduce the false detection rate.

[0170] In step 601 of some embodiments, the initial flame area refers to the overlapping pixel area between the flame color area and the moving area. For example, the initial flame area can be obtained through the following formula:

[0171] F(x, y, t) = O(x, y) ∩ S(x, y), formula (11);

[0172] Wherein, F(x, y, t) represents the initial flame area, O(x, y) represents the moving area, and S(x, y) represents the flame color area; ∩ represents the union operation, that is, the operation of screening the above overlapping areas.

[0173] It should be noted that x and y jointly represent the coordinates of the pixel (x, y). For example, x represents the row coordinate of the pixel, and y represents the column coordinate of the pixel; t represents the t-th frame of the target video to which the pixel (x, y) belongs.

[0174] In some embodiments, for example, the target video frame is as Figure 7 shown. The moving detection of the target video frame can be performed through the Gaussian mixture model, and the flame color feature detection of the target video frame can be performed through the color space. As Figure 8 shown, the specific type of the moving area can be a binary image (also called a foreground mask) output by the Gaussian mixture model. If the value of a pixel in the binary image is 1, it means that the pixel is a moving pixel, that is, it belongs to the moving area. As Figure 9 shown, the specific type of the flame color area can be a binary image after color space fusion. If the value of a pixel in the binary image is 1, it means that the pixel conforms to the flame color feature, that is, it belongs to the flame color area.

[0175] In step 602 of some embodiments, the flame area overlap rate is the area overlap rate between the initial flame areas of two adjacent frames (that is, the first video frame and the second video frame), and the flame area overlap rate can reflect the growth rate of the flame. For example, the definition of the flame area overlap rate is shown in the following formula:

[0176]

[0177] Wherein, OV represents the flame area overlap rate; Area(·) represents the number of pixels in the area; F t-1 represents the initial flame area of the first video frame; F t represents the initial flame area of the second video frame; min(Area(F t ), Area(F t-1 )) represents the minimum value of the number of pixels in the initial flame area of the first video frame and the number of pixels in the initial flame area of the second video frame.

[0178] In step 603 of some embodiments, the overlap rate threshold can be 0.5. It can also be set or updated to other values, not limited to this.

[0179] In step 604 of some embodiments, if the flame area overlap rate is greater than or equal to the overlap rate threshold, it indicates that the stability of the initial flame area between adjacent frames is relatively high, so it can be determined as a stable fire situation. If the flame area overlap rate is less than the overlap rate threshold, it indicates that the stability of the flame area is poor, and this area may be a misjudged area caused by transient moving objects (such as birds, fluttering clothes), so this area is not marked as the final flame area and can be marked as moving interference and discarded.

[0180] In some embodiments, confirming the initial flame area as the target flame area means confirming the initial flame area of the first video frame as the target flame area of the first video frame, and confirming the initial flame area of the second video frame as the target flame area of the second video frame.

[0181] Please refer to Figure 10 , in an application example, the overlap rate of the initial flame area of the current frame and the initial flame area of the previous frame can be calculated to obtain the flame area overlap rate OV. If the flame area overlap rate OV is greater than 0.5 and lasts for 15 frames, the initial flame area can be confirmed as the fire area. If the flame area overlap rate OV is less than or equal to 0.5, the initial flame area can be confirmed as a non-fire area.

[0182] Please refer to Figure 11 , in some embodiments, after step 105, the flame detection method based on image recognition may include but is not limited to steps 701 to 706:

[0183] Step 701, select two video frames from at least two target video frames according to a preset time interval threshold to obtain a third video frame and a fourth video frame; wherein, the time interval between the third video frame and the fourth video frame is less than or equal to the time interval threshold;

[0184] Step 702, determine the number of pixels in the target flame area of the third video frame as the first flame pixel number, and determine the number of pixels in the target flame area of the fourth video frame as the second flame pixel number;

[0185] Step 703, determine the time interval between the third video frame and the fourth video frame as the target time interval;

[0186] Step 704, perform a difference calculation on the first flame pixel number and the second flame pixel number to obtain the flame pixel increment;

[0187] Step 705, perform a ratio calculation according to the flame pixel increment and the target time interval to obtain the flame growth rate;

[0188] Step 706, in response to the flame growth rate being greater than or equal to a preset growth rate threshold, send a fire prompt message.

[0189] The advantage of this embodiment is that by selecting two video frames with a fixed time interval (i.e., the third video frame and the fourth video frame) through the time interval threshold, calculating the ratio of the flame pixel increment between the third video frame and the fourth video frame to the target time interval, the flame growth rate is analyzed, that is, the growth rate of the fire is analyzed, and then the intensity of the fire is judged. If the flame growth rate is greater than or equal to the preset growth rate threshold, it indicates that the intensity of the fire is relatively large, and the alarm mechanism is triggered in a timely manner, such as sending a fire prompt message, to improve the reliability and accuracy of fire detection.

[0190] In step 701 of some embodiments, both the third video frame and the fourth video frame are video frames in the target video (i.e., target video frames), and the time interval between the third video frame and the fourth video frame is less than or equal to the time interval threshold. For example, assuming that the time interval threshold is 10 seconds, the frame interval of the target video is 1 second, and the first frame is selected as the third video frame from the target video, then another frame whose time interval from the first frame does not exceed 10 seconds should be selected as the fourth video frame from the video frames of the target video. For example, the third frame can be selected as the fourth video frame, and the time interval between the first frame and the third frame is 2 seconds, which is less than the time interval threshold (10 seconds).

[0191] In step 702 of some embodiments, the first flame pixel quantity is the number of pixels in the target flame area of the third video frame. The second flame pixel quantity is the number of pixels in the target flame area of the fourth video frame.

[0192] In step 703 of some embodiments, the target time interval refers to the time interval between the third video frame and the fourth video frame. Using the same example above, the target time interval (i.e., the time interval between the first frame and the third frame) is 2 seconds.

[0193] In step 704 of some embodiments, the flame pixel increment is the difference between the first flame pixel quantity and the second flame pixel quantity. The formula definition of the flame pixel increment can be referred to the specific explanation of step 705 below, and will not be elaborated here.

[0194] In some embodiments, it should be noted that when analyzing the fire intensity of a fire, it is crucial to accurately obtain the growth rate in the start stage and spread stage of the fire. To improve the accuracy of fire detection, the flame growth rate can also be calculated to analyze the fire intensity. The fire intensity can be reflected by the flame growth rate.

[0195] In step 705 of some embodiments, the definition of the flame growth rate is shown in the following formula:

[0196]

[0197] Wherein, Ity represents the flame growth rate; FP n represents the number of flame pixels in the fourth video frame, that is, the second number of flame pixels; FP n-1 represents the number of flame pixels in the third video frame, that is, the first number of flame pixels; ΔFP represents the flame pixel increment; t n represents the time of the fourth video frame; t n-1 represents the time of the third video frame; Δt represents the time interval between the third video frame and the fourth video frame, that is, the target time interval.

[0198] In step 706 of some embodiments, the growth rate threshold can be 10%, or other values within the range of [0, 1], which is not limited thereto. For example, when the flame growth rate is less than or equal to 10%, it indicates that the flame spreads slowly and the fire risk is low. At this time, a local alarm can be triggered. When the flame growth rate is greater than 10%, it indicates that the flame spreads rapidly and the fire risk is high. At this time, an emergency response can be triggered, such as sending a fire prompt message to the fire protection system.

[0199] Please refer to Figure 12 , in an application example, the flame detection method includes: acquiring a video; performing image preprocessing on video frames in the video; jointly performing flame color feature detection on the video frames by using a color space fusion method through the RGB color space and the Lab color space to obtain a flame color region; performing dynamic detection on the video frames through a Gaussian mixture model (GMM) to obtain a moving region; using the overlapping region of the flame color region and the moving region as the flame region, so as to reduce the interference of static red objects. Then, fire trend analysis and verification can be performed based on the flame region to determine whether there is a fire. For example, the fire trend can be analyzed by calculating the flame growth rate, or the accuracy of the fire detection result can be verified by calculating the flame region overlap rate. If there is a fire, a fire alarm decision can be made. If there is no fire, no subsequent processing is performed.

[0200] Please refer to Figure 13 , an embodiment of the present application further provides a flame detection device based on image recognition, which can implement the above-mentioned flame detection method based on image recognition. The device includes:

[0201] A video acquisition module 801, configured to acquire a target video;

[0202] A video frame extraction module 802, configured to extract video frames from the target video to obtain at least two target video frames;

[0203] A flame color detection module 803, configured to detect a flame color region for each target video frame according to the color feature of each target video frame and a preset target flame color rule to obtain a flame color region;

[0204] A moving area detection module 804 is configured to perform moving area detection on any two adjacent target video frames to obtain the moving area of each target video frame; wherein, the adjacent target video frames are used to represent video frames adjacent in chronological order.

[0205] A coincidence area screening module 805 is configured to screen the coincidence area between the flame color area and the moving area of each target video frame to obtain the target flame area.

[0206] The specific implementation manner of the flame detection device based on image recognition is basically the same as the specific embodiments of the above-mentioned flame detection method based on image recognition, and will not be elaborated here.

[0207] An embodiment of the present application further provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the above-mentioned flame detection method based on image recognition is implemented. The electronic device may include any intelligent terminal such as a tablet computer or an in-vehicle computer.

[0208] Please refer to Figure 14 , Figure 14 which schematically shows the hardware structure of an electronic device in another embodiment. The electronic device includes:

[0209] A processor 901, which can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present application;

[0210] A memory 902, which can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 902 can store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 902 and are called by the processor 901 to execute the flame detection method based on image recognition of the embodiments of the present application;

[0211] An input / output interface 903, which is configured to implement information input and output;

[0212] A communication interface 904, which is used to implement the communication interaction between this device and other devices. The communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WI-FI, Bluetooth, etc.);

[0213] A bus 905, which transmits information between various components of the device (such as a processor 901, a memory 902, an input / output interface 903, and a communication interface 904);

[0214] Among them, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904 achieve the communication connection among themselves inside the device through the bus 905.

[0215] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned flame detection method based on image recognition is implemented.

[0216] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory optionally includes a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and their combinations.

[0217] It should be noted that the non-company software tools or components appearing in the embodiments of the present application are only introduced by way of example and do not represent actual use.

[0218] The embodiments described in the present application are for more clearly explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0219] Those skilled in the art can understand that the technical solutions shown in the figure do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figure, or combine certain steps, or different steps.

[0220] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0221] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and appropriate combinations thereof.

[0222] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of this application and the above-mentioned drawings are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of this application described here can be implemented in an order different from those illustrated or described here. It should be understood that in this application, "at least one (item)" means one or more, and "a plurality" means two or more. "And / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist at the same time. Here, A and B can be singular or plural.

[0223] In several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the above-mentioned unit division is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0224] In addition, in each embodiment of the present application, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store programs.

[0225] The preferred embodiments of the embodiments of the present application have been described above with reference to the accompanying drawings. This does not limit the scope of the rights of the embodiments of the present application. Any modification, equivalent replacement, and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. A flame detection method based on image recognition, characterized in that, The method includes: Obtaining a target video; Performing video frame extraction on the target video to obtain at least two target video frames; According to the color feature of each target video frame and a preset target flame color rule, performing flame color region detection on each target video frame to obtain a flame color region; wherein, the color feature of each target video frame includes a red value, a green value, and a blue value, and the target flame color rule includes that the red value is greater than the green value and the green value is greater than the blue value: Performing moving region detection on any two adjacent target video frames to obtain the moving region of each target video frame; wherein, the adjacent target video frames are used to represent video frames adjacent in chronological order; Performing overlapping region screening on the flame color region and the moving region of each target video frame to obtain a target flame region.

2. The method according to claim 1, wherein The at least two target video frames include a first video frame and a second video frame, and the first video frame is the previous frame of the second video frame; The performing moving region detection on any two adjacent target video frames to obtain the moving region of each target video frame includes: Performing first moving region detection on the first video frame through a pre-trained initial moving detection model to obtain a first moving region; Updating the parameters of the initial moving detection model according to the region ratio of the first moving region in the first video frame and the color feature of the second video frame to obtain a target moving detection model; Performing second moving region detection on the second video frame through the target moving detection model to obtain a second moving region.

3. The method according to claim 2, wherein The initial moving detection model has initial distribution parameters; The updating the parameters of the initial moving detection model according to the region ratio of the first moving region in the first video frame and the color feature of the second video frame to obtain a target moving detection model includes: Calculating a ratio according to the number of pixels of the first moving region and the number of pixels of the first video frame to obtain a moving pixel ratio; Calculating distribution parameters according to the moving pixel ratio, the color value of each pixel in the second video frame, and the initial distribution parameters to obtain target distribution parameters; Updating the distribution parameters of the initial moving detection model according to the target distribution parameters to obtain the target moving detection model.

4. The method according to claim 3, characterized in that The initial distribution parameters include an initial first distribution parameter and an initial second distribution parameter; The calculating distribution parameters according to the moving pixel ratio, the color value of each pixel in the second video frame, and the initial distribution parameters to obtain target distribution parameters includes: Multiplying the moving pixel ratio by a preset attenuation coefficient to obtain a target product; Performing cumulative distribution calculation on the target product through a preset exponential cumulative distribution function to obtain a target learning rate; Determining the color value of each pixel in the second video frame as a target pixel color vector; Performing weighted summation according to the initial first distribution parameter, the target learning rate, and the target pixel color vector to obtain a target first distribution parameter; Calculate the difference between the target pixel color vector and the target first distribution parameter to obtain a deviation vector; Perform weighted summation based on the initial second distribution parameter, the target learning rate, the deviation vector, and the transposed vector of the deviation vector to obtain the target second distribution parameter; Use the combination of the target first distribution parameter and the target second distribution parameter as the target distribution parameter.

5. The method according to any one of claims 1 to 4, characterized in that, The target video frame includes at least two target pixels; the target flame color rule includes a first flame color rule and a second flame color rule; The flame color region detection for each target video frame according to the color feature of each target video frame and the preset target flame color rule to obtain a flame color region includes: Extract the first color feature for each target pixel through a preset first color space to obtain the first color feature; wherein, the first color feature includes a red value, a green value, and a blue value; Select the target pixels whose first color feature conforms to the first flame color rule from at least two target pixels to obtain first flame color pixels; wherein, the first flame color rule is used to represent that the red value is greater than the green value and the green value is greater than the blue value; Extract the second color feature for each target pixel through a preset second color space to obtain the second color feature; wherein, the second color feature includes a red-green axis chromaticity and a yellow-blue axis chromaticity; Select the target pixels whose second color feature conforms to the second flame color rule from at least two target pixels to obtain second flame color pixels; wherein, the second flame color rule is used to represent that the yellow-blue axis chromaticity is greater than or equal to the red-green axis chromaticity; Obtain at least one flame color pixel according to the union of the first flame color pixels and the second flame color pixels, and determine the combination of at least one flame color pixel as the flame color region.

6. The method according to any one of claims 1 to 4, characterized in that At least two target video frames include a first video frame and a second video frame, and the first video frame is the previous frame of the second video frame; The screening of the overlapping region between the flame color region and the moving region for each target video frame to obtain the target flame region includes: Screen the overlapping region between the flame color region and the moving region for each target video frame to obtain an initial flame region; Calculate the region overlap rate of the initial flame region of the first video frame and the initial flame region of the second video frame to obtain the flame region overlap rate; Compare the flame region overlap rate with a preset overlap rate threshold; If the flame region overlap rate is greater than or equal to the overlap rate threshold, confirm the initial flame region as the target flame region.

7. The method according to any one of claims 1 to 4, characterized in that, After the screening of the overlapping region between the flame color region and the moving region for each target video frame to obtain the target flame region, the method further includes: Select two video frames from at least two of the target video frames according to a preset time interval threshold to obtain a third video frame and a fourth video frame; wherein, the time interval between the third video frame and the fourth video frame is less than or equal to the time interval threshold; Determine the number of pixels in the target flame area of the third video frame as the first flame pixel number, and determine the number of pixels in the target flame area of the fourth video frame as the second flame pixel number; Determine the time interval between the third video frame and the fourth video frame as the target time interval; Perform a difference calculation on the first flame pixel number and the second flame pixel number to obtain a flame pixel increment; Perform a ratio calculation according to the flame pixel increment and the target time interval to obtain a flame growth rate; In response to the flame growth rate being greater than or equal to a preset growth rate threshold, send a fire prompt message.

8. A flame detection device based on image recognition, characterized in that, The device includes: A video acquisition module for acquiring a target video; A video frame extraction module for extracting video frames from the target video to obtain at least two target video frames; A flame color detection module for detecting a flame color area of each target video frame according to the color feature of each target video frame and a preset target flame color rule; A moving area detection module for detecting a moving area of any two adjacent target video frames to obtain a moving area of each target video frame; wherein, the adjacent target video frames are used to represent video frames adjacent in time sequence; A coincidence area screening module for screening a coincidence area between the flame color area and the moving area of each target video frame to obtain a target flame area.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that, The computer program implements the method according to any one of claims 1 to 7 when executed by the processor.

Images