Flame detection method and device based on image recognition, electronic device, and medium

Abstract

Embodiments of the present application provide a flame detection method and device based on image recognition, electronic equipment and medium, and relate to the technical field of computer vision. The method comprises: acquiring a target video; performing video frame extraction on the target video to obtain at least two target video frames; performing flame color region detection on each target video frame according to the color feature of each target video frame and a preset target flame color rule to obtain a flame color region; performing moving region detection on any two adjacent target video frames to obtain a moving region of each target video frame; wherein the adjacent target video frames are used to represent video frames adjacent in time sequence; and performing overlapping region screening on the flame color region and the moving region of each target video frame to obtain a target flame region. The embodiments of the present application can improve the accuracy and reliability of flame detection.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a flame detection method and apparatus, electronic device, and medium based on image recognition. Background Technology

[0002] Fire detection is a technology that detects fires from their environment by sensing the surroundings. For example, in computer vision, image sensors (such as cameras) can capture images or videos, and image recognition technology can be used to detect flames from these images or videos. However, current image recognition-based flame detection methods are susceptible to environmental interference. For instance, static objects with colors similar to flames (such as red fire hydrants or red buildings) can easily interfere with flame detection, making it difficult to accurately identify flames in videos and leading to false fire alarms. Especially in complex scenarios with these interfering factors, the accuracy of flame detection is low.

[0003] Therefore, improving the accuracy and reliability of flame detection has become an urgent technical problem to be solved. Summary of the Invention

[0004] The main objective of this application is to propose a flame detection method, device, electronic device, and medium based on image recognition, aiming to improve the accuracy and reliability of flame detection.

[0005] To achieve the above objectives, a first aspect of this application proposes a flame detection method based on image recognition, the method comprising:

[0006] Acquire the target video;

[0007] The target video is subjected to video frame extraction to obtain at least two target video frames;

[0008] Based on the color features of each target video frame and a preset target flame color rule, flame color region detection is performed on each target video frame to obtain the flame color region; wherein, the color features of each target video frame include a red value, a green value, and a blue value, and the target flame color rule includes that the red value is greater than the green value, and the green value is greater than the blue value.

[0009] Motion region detection is performed on any two adjacent target video frames to obtain the motion region of each target video frame; wherein, adjacent target video frames are used to represent video frames that are adjacent in time order.

[0010] For each target video frame, the overlapping areas of the flame color region and the moving region are filtered to obtain the target flame region.

[0011] To achieve the above objectives, a second aspect of this application provides a flame detection device based on image recognition, the device comprising:

[0012] The video acquisition module is used to acquire the target video;

[0013] The video frame extraction module is used to extract video frames from the target video to obtain at least two target video frames;

[0014] The flame color detection module is used to detect the flame color region of each target video frame according to the color features of each target video frame and the preset target flame color rules, so as to obtain the flame color region.

[0015] The moving region detection module is used to perform moving region detection on any two adjacent target video frames to obtain the moving region of each target video frame; wherein, adjacent target video frames are used to represent video frames that are adjacent in time sequence.

[0016] The overlapping region filtering module is used to filter the overlapping regions of the flame color region and the moving region in each target video frame to obtain the target flame region.

[0017] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0018] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0019] This application proposes an image recognition-based flame detection method, apparatus, electronic device, and medium. It extracts target video frames from a target video and performs flame detection on each frame. Based on the color features and target flame color rules of each target video frame, flame color regions are detected. This allows for the detection of regions within each target video frame whose color features conform to the target flame color rules, such as regions where red values ​​are greater than green values ​​and green values ​​are greater than blue values. Furthermore, moving region detection is performed on adjacent video frames to obtain the moving regions of each target video frame, thus detecting areas where moving objects exist within the video frames. For each target video frame, overlapping areas between the flame color region and the moving region are filtered, ensuring that the target flame region simultaneously satisfies both color matching and motion characteristics. This significantly reduces interference from static red objects or non-flame dynamic objects, effectively improving the accuracy and reliability of flame detection in complex scenes. By combining a dual verification method of flame color features and dynamic moving region detection, the reliability and accuracy of flame detection in complex scenes are effectively improved. Attached Figure Description

[0020] Figure 1 This is a flowchart of the flame detection method based on image recognition provided in the embodiments of this application;

[0021] Figure 2 yes Figure 1 The flowchart for step 103 in the text;

[0022] Figure 3 yes Figure 1 The flowchart for step 104 in the document;

[0023] Figure 4 yes Figure 3 The flowchart for step 302 in the document;

[0024] Figure 5 yes Figure 4 The flowchart for step 402 in the document;

[0025] Figure 6 yes Figure 1 The flowchart for step 105 in the document;

[0026] Figure 7 This is a schematic diagram of a target video frame provided in an embodiment of this application;

[0027] Figure 8 This is an embodiment provided by this application. Figure 7 A binary map of the moving region in the image;

[0028] Figure 9 This is an embodiment provided by this application. Figure 7 A binary image of the flame color region in the image;

[0029] Figure 10 This is a flowchart illustrating the analysis of flame region overlap rate, as provided in an embodiment of this application.

[0030] Figure 11 This is a flowchart of a flame detection method based on image recognition provided in another embodiment of this application;

[0031] Figure 12 This is a flowchart illustrating an image recognition-based flame detection method, as provided in an embodiment of this application.

[0032] Figure 13 This is a schematic diagram of the structure of the flame detection device based on image recognition provided in the embodiments of this application;

[0033] Figure 14 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0037] First, let's analyze some of the terms used in this application:

[0038] Artificial Intelligence (AI) is a technological science that studies and develops theories, methods, technologies, and application systems to simulate, extend, and expand human intelligence. AI software technologies mainly include computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning. This application utilizes AI technology to acquire and process relevant data.

[0039] Computer vision is a technology that uses algorithms to simulate the human visual system to perceive, analyze, and understand digital images or video sequences. Computer vision includes image processing, among other things.

[0040] Image recognition refers to the technology of using computers to process, analyze, and understand images in order to identify various objects. Image recognition is a practical application of deep learning algorithms.

[0041] Color space: Also known as a color model, color space, or color system. A color space is a mathematical model used to represent colors. For example, color spaces include the RGB color space and the CMY color space. The RGB (red, green, blue) color space, for instance, describes colors using the three primary colors: red, green, and blue. The RGB color space is primarily based on the color definition recognized by the human eye and can represent most colors.

[0042] The flame detection method, apparatus, electronic device, and medium based on image recognition provided in this application are specifically described through the following embodiments. First, the flame detection method based on image recognition in this application is described.

[0043] The image recognition-based flame detection method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, middleware services, domain name services, and big data and artificial intelligence platforms; the software can be an application implementing the image recognition-based flame detection method, but is not limited to the above forms.

[0044] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0045] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0046] Figure 1 This is an optional flowchart of the image recognition-based flame detection method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps 101 to 105.

[0047] Step 101, acquire the target video;

[0048] Step 102: Extract video frames from the target video to obtain at least two target video frames;

[0049] Step 103: Based on the color features of each target video frame and the preset target flame color rules, perform flame color region detection on each target video frame to obtain the flame color region; wherein, the color features of each target video frame include red value, green value, and blue value, and the target flame color rules include red value being greater than green value, and green value being greater than blue value:

[0050] Step 104: Perform motion region detection on any two adjacent target video frames to obtain the motion region of each target video frame; wherein, adjacent target video frames are used to represent video frames that are adjacent in time sequence.

[0051] Step 105: For each target video frame, the overlapping areas of the flame color region and the moving region are filtered to obtain the target flame region.

[0052] The beneficial effects of this application's embodiments include, but are not limited to: extracting target video frames from the target video to perform flame detection on each target video frame. Flame color regions are detected based on the color features and target flame color rules of each target video frame, thus detecting regions in each target video frame whose color features conform to the target flame color rules, such as regions where the red value is greater than the green value and the green value is greater than the blue value. Furthermore, moving region detection is performed on adjacent video frames to obtain the moving region of each target video frame, thus detecting regions in the video frame where moving objects exist. For each target video frame, overlapping areas of flame color regions and moving regions are filtered, ensuring that the target flame region simultaneously satisfies both color matching and motion characteristics, significantly reducing the interference of red static objects or non-flame dynamic objects on flame detection, and effectively improving the accuracy and reliability of flame detection in complex scenes. By combining the dual verification method of flame color features and moving region dynamic detection, the reliability and accuracy of flame detection in complex scenes are effectively improved.

[0053] In step 101 of some embodiments, the target video is a video containing flame image information. For example, the target video can be obtained by capturing video of the flame using an image sensor (such as a camera), and then flame detection can be performed on the target video. Alternatively, the target video can be obtained by selecting a video from a preset fire video database. Other methods can also be used to obtain the target video, and are not limited to these.

[0054] In step 102 of some embodiments, the target video frame refers to a video frame of the target video. Video frames can be extracted from the target video using a preset video frame extraction tool. Other methods can also be used to extract video frames, and are not limited to these.

[0055] In some embodiments, before step 103, the target video frame may undergo image enhancement processing to obtain an enhanced video frame. Then, the enhanced video frame is determined as the target video frame for flame color region detection (see step 103), which improves the accuracy of flame color region detection. Specifically, the image enhancement processing includes contrast enhancement, noise reduction, etc. For example, in the image enhancement processing, the image enhancement formula is as follows:

[0056] l c =R a ·G b ·B c , formula (1);

[0057] In the formula, l c This represents the enhanced video frame, that is, the target video frame after image enhancement; R a This represents the red channel value (i.e., the red value) of the enhanced video frame; G b B represents the green channel value (i.e., the green value) of the enhanced video frame; c R represents the blue channel value (i.e., blue value) of the enhanced video frame; G represents the red value of the target video frame; B represents the blue value of the target video frame; a represents the red index parameter, b represents the green index parameter, and c represents the blue index parameter.

[0058] It should be noted that the three exponential parameters a, b, and c mentioned above can be obtained through experimental optimization. These parameters are used to enhance the flame features dominated by the red channel. The image enhancement formula can improve the contrast of the flame area and suppress background noise.

[0059] In step 103 of some embodiments, the color features of the target video frame may include the pixel value of each pixel in the target video frame. Specifically, the color features of the target video frame include the color value of each pixel in the target video frame, such as RGB values ​​(i.e., the values ​​of the red, green, and blue color channels). For example, suppose the target video frame includes pixels p1, p2, and p3, where the RGB value of pixel p1 is [200, 50, 30], the RGB value of pixel p2 is [0, 255, 0], and the RGB value of pixel p3 is [203, 40, 20]. Then, the color features of the target video frame include {[200, 50, 30], [0, 255, 0], [203, 40, 20]}.

[0060] It should be noted that the flame color region refers to the video frame region whose color features conform to the target flame color rule. The target flame color rule is used to characterize the constraints that the color features of a flame must satisfy. If the color features of a pixel conform to the target flame color rule, then that pixel can be considered to contain flame information.

[0061] Similar to the previous example, assuming the target flame color rule includes a red value greater than 150 for pixels, then pixel p1 has a red value (200) greater than 150, and pixel p1 conforms to the target flame color rule. Pixel p2 has a red value (0) less than 150, and pixel p2 does not conform to the target flame color rule. Pixel p3 has a red value (203) greater than 150, and pixel p3 conforms to the target flame color rule. The video frame region consisting of pixels p1 and p3 that conform to the target flame color rule is taken as the flame color region.

[0062] In step 104 of some embodiments, moving region detection can be performed on the target video frame using a model (including mathematical models and neural network models) for detecting moving objects. For example, a Gaussian Mixture Model (GMM) can be used to distinguish between the background (static portion) and foreground (moving object) of the target video frame to detect moving objects in the video frame.

[0063] It should be noted that adjacent target video frames refer to two frames that are sequentially adjacent in time within the target video. For example, assuming the frame interval of the target video is 1 second, the time of the first frame is t=0, and the time of the second frame is t=1. Then, the first and second frames of the target video are considered adjacent target video frames.

[0064] In step 105 of some embodiments, the target flame region refers to a moving region with flame color characteristics, that is, the overlapping area of ​​the flame color region and the moving region in the same target video frame. Specifically, the set of pixels in the same target video frame that belong to both the flame color region and the moving region can be determined as the target flame region.

[0065] In some embodiments, it should be noted that current fire detection technologies primarily rely on physical devices such as smoke sensors and temperature sensors to detect parameters like smoke concentration and temperature changes to provide fire warnings. Because smoke sensors and temperature sensors are susceptible to environmental interference (such as high temperatures or non-fire heat sources, including sunlight and heating), they suffer from high false alarm rates, thus reducing the reliability of fire detection systems. Furthermore, to detect fire sources, smoke and temperature sensors need to be in close proximity to the fire source and require dense deployment, which limits the coverage of fire detection and makes it difficult to detect large areas in real time. In addition, physical sensing devices such as smoke and temperature sensors rely on changes in physical signals to detect flames, resulting in significant signal response delays in the detection process, making rapid identification difficult in the early stages of a fire.

[0066] To address the limitations of limited coverage and signal response delays, computer vision technology can be used for rapid fire detection in large-scale scenes. However, current computer vision-based flame detection methods still suffer from a high false alarm rate. For example, current flame detection methods are easily interfered with by static objects or moving objects that are similar in color to the flame. To address this, this application's embodiments improve the reliability of flame detection in complex scenes and reduce the false alarm rate by combining flame color features with dynamic detection of moving areas as a dual verification method.

[0067] Please see Figure 2 In some embodiments, the target video frame includes at least two target pixels; the target flame color rule includes a first flame color rule and a second flame color rule;

[0068] Step 103 may include, but is not limited to, steps 201 through 205:

[0069] Step 201: Extract first color features from each target pixel using a preset first color space to obtain first color features; wherein, the first color features include red value, green value and blue value;

[0070] Step 202: Select a target pixel whose first color feature conforms to the first flame color rule from at least two target pixels to obtain the first flame color pixel; wherein, the first flame color rule is used to indicate that the red value is greater than the green value and the green value is greater than the blue value;

[0071] Step 203: Extract second color features from each target pixel using a preset second color space to obtain second color features; wherein, the second color features include red-green axis chromaticity and yellow-blue axis chromaticity;

[0072] Step 204: Select target pixels whose second color features conform to the second flame color rule from at least two target pixels to obtain the second flame color pixel; wherein, the second flame color rule is used to indicate that the yellow-blue axis chromaticity is greater than or equal to the red-green axis chromaticity;

[0073] Step 205: Obtain at least one flame color pixel based on the union of the first flame color pixel and the second flame color pixel, and determine the combination of at least one flame color pixel as the flame color region.

[0074] The advantage of this embodiment lies in its collaborative analysis of the color features of each target pixel in the target video frame using both a first and a second color space to determine whether a target pixel represents a flame from multiple perspectives. First, the red, green, and blue values ​​of the target pixels are extracted in the first color space. The first flame color rule (red value greater than green value, and green value greater than blue value) is used to filter out target pixels that conform to the flame color sequence characteristics, effectively reducing interference from non-flame color sequences such as blue light sources. Then, the red-green axis chromaticity and yellow-blue axis chromaticity of the target pixels are extracted in the second color space. The second flame color rule (yellow-blue axis chromaticity greater than or equal to red-green axis chromaticity) is used to identify target pixels with a yellow spectral bias, selecting pixels that conform to the color characteristics of natural flames and reducing misjudgments caused by red objects or red artificial light sources. Finally, at least one flame color pixel is obtained based on the union of the first and second flame color pixels. This preserves the pixel detection results under both color models and reduces the limitations of single color space analysis through complementary rules (i.e., the aforementioned first and second flame color rules). This improves the accuracy and robustness of flame color recognition, making the detection results more consistent with the spectral characteristics of natural flames, thereby improving the accuracy of flame detection and reducing the false alarm rate of fire alarms.

[0075] In step 201 of some embodiments, the first color space may include the RGB color space. The first color feature of the target pixel is used to characterize the color value of the target pixel, that is, the RGB value (including red value, green value and blue value). For example, assuming that the red value of the target pixel p1 is 200, the green value is 50 and the blue value is 30, then the color feature of the target pixel p1 is [200, 50, 30].

[0076] In step 202 of some embodiments, specifically, the first flame color rule may include three sub-rules. For example, the first flame color rule includes: (1) a first flame color sub-rule, which indicates that the red channel value of the target pixel is greater than the average red channel value of the target video frame; (2) a second flame color sub-rule, which indicates that the red channel value of the target pixel is greater than the green channel value and the green channel value is greater than the blue channel value; and (3) a third flame color sub-rule, which indicates that the saturation of the target pixel is greater than a preset saturation threshold.

[0077] For example, the formula group corresponding to the first flame color rule is shown below:

[0078]

[0079] In the formula, R represents the red channel value (i.e., red value) of the target pixel; G represents the green channel value (i.e., green value) of the target pixel; B represents the blue channel value (i.e., blue value) of the target pixel; R meanS represents the mean value of the red channel of the target video frame; S represents the saturation of the target pixel; S T R represents the saturation channel threshold; T Indicates the threshold value for the red channel;

[0080] In some embodiments, the first formula in the above formula group corresponds to the first flame color sub-rule, the second formula corresponds to the second flame color sub-rule, and the third formula corresponds to the third flame color sub-rule. Specifically, the first flame color sub-rule is used to filter out pixels with red channel values ​​higher than the average value to filter out dark backgrounds. The second flame color sub-rule is used to filter out pixels with a color order that satisfies red > green > blue to exclude non-flame color orders (such as blue light sources). The third flame color sub-rule is used to reduce interference from low-saturation objects (such as gray smoke).

[0081] In some embodiments, the maximum value of a color channel (such as a red channel, green channel, or blue channel) is 255. It should be noted that in the third flame color sub-rule, dynamically adjusting the saturation judgment threshold based on the red value of the target pixel can reduce interference caused by reflections. For example, if the red value R is small, the saturation judgment threshold... The larger size of the flame effectively distinguishes between reflective areas with low red values ​​and low saturation (such as reflections from red clothing) and flame edge areas with low red values ​​and high saturation (such as the dark areas around the flame), thus filtering out false red signals caused by reflections and accurately identifying the color characteristics of the flame edge area.

[0082] It should be noted that the saturation channel threshold and / or red channel threshold can be set based on experimental experience. In some embodiments, the saturation channel threshold S T =50. In some embodiments, the red channel threshold R T The value range includes [150, 250]. For example, the red channel threshold R... T The value can be any of 150, 200, or 250. The saturation channel threshold and / or the red channel threshold can also be dynamically adjusted according to the scenario or other requirements. This application embodiment does not limit the specific values ​​of the above thresholds.

[0083] In some embodiments, the mean R of the red channel of the target video frame mean The definition is shown in the following formula:

[0084]

[0085] In the formula, R mean N represents the mean value of the red channel of the target video frame; N represents the total number of pixels in the target video frame I, which is also the number of target pixels x; x represents the target pixel; I R(x) represents the red channel value of the target pixel x.

[0086] In step 203 of some embodiments, the second color space may include the Lab color space. It should be noted that the Lab color space is a mathematical model that uses luminance, red-green axis chromaticity, and yellow-blue axis chromaticity to describe color. In some embodiments, the second color features include luminance, red-green axis chromaticity, and yellow-blue axis chromaticity.

[0087] In step 204 of some embodiments, specifically, the second flame color rule may include four sub-rules. For example, the second flame color rule includes: (1) a fourth flame color sub-rule, which indicates that the brightness of the target pixel is greater than or equal to the average brightness of the target video frame; (2) a fifth flame color sub-rule, which indicates that the red-green axis chromaticity of the target pixel is greater than or equal to the average red-green axis chromaticity of the target video frame; (3) a sixth flame color sub-rule, which indicates that the yellow-blue axis chromaticity of the target pixel is greater than or equal to the average yellow-blue axis chromaticity of the target video frame; and (4) a seventh flame color sub-rule, which indicates that the yellow-blue axis chromaticity is greater than or equal to the red-green axis chromaticity.

[0088] For example, the formula group corresponding to the second flame color rule is shown below:

[0089]

[0090] In the formula, L * Indicates the brightness of the target pixel; This represents the average luminance of the target video frame; a * Represents the red-green axis chromaticity of the target pixel; b represents the mean chromaticity value along the red and green axes of the target video frame; * Represents the yellow-blue axis chromaticity of the target pixel; This represents the average chromaticity value along the yellow and blue axes of the target video frame.

[0091] It should be noted that the average luminance of the target video frame refers to the average luminance of each pixel in the target video frame. The average red-green axis chromaticity of the target video frame refers to the average red-green axis chromaticity of each pixel in the target video frame. The average yellow-blue axis chromaticity of the target video frame refers to the average yellow-blue axis chromaticity of each pixel in the target video frame.

[0092] Specifically, in the Lab color space, the higher the chromaticity of the red-green axis, the more the color leans towards the red spectrum. For example, a positive value on the red-green axis indicates a reddish tint, and a negative value indicates a greenish tint. Similarly, the higher the chromaticity of the yellow-blue axis, the more the pixel's color leans towards the yellow spectrum. For example, a positive value on the yellow-blue axis indicates a yellowish tint, and a negative value indicates a bluish tint.

[0093] In some embodiments, the first formula in the above formula group corresponds to the fourth flame color sub-rule, the second formula corresponds to the fifth flame color sub-rule, the third formula corresponds to the sixth flame color sub-rule, and the fourth formula corresponds to the seventh flame color sub-rule. Specifically, the fourth flame color sub-rule is used to filter pixels with higher brightness. Since brightness is independent of chromaticity, it allows for independent judgment of pixel brightness, reducing the impact of brightness variations caused by illumination on flame detection. The fifth flame color sub-rule is used to filter pixels biased towards the red spectrum. The sixth flame color sub-rule is used to filter pixels biased towards the yellow spectrum. The seventh flame color sub-rule is used to filter pixels biased towards the yellow-red spectrum, thereby identifying pixels that conform to the characteristics of natural flame colors and reducing interference from red artificial light sources.

[0094] In some embodiments, it should be noted that current flame color detection methods cannot accurately distinguish flames from objects of similar colors (such as red clothing, red lights, red autumn leaves, etc.). Considering the above problems, embodiments of this application use a second flame color rule in a second color space to filter pixels whose yellow-blue axis chromaticity is greater than or equal to their red-green axis chromaticity. This enables the identification of pixels whose colors are biased towards the yellow-red spectrum rather than the pure red spectrum, thereby reducing misjudgments caused by red artificial light sources (such as fire truck lights, neon lights, and vehicle taillights).

[0095] In step 205 of some embodiments, at least one flame-colored pixel is the union of a first flame-colored pixel and a second flame-colored pixel. This improves the sensitivity of flame region detection and allows for more comprehensive identification of potential flames. For example, if a target pixel belongs to either the first or the second flame-colored pixel, then the target pixel is identified as a flame-colored pixel. In some embodiments, the flame-colored region is a connected region composed of at least one flame-colored pixel. The flame-colored region can also be a non-connected region, which is not limited here.

[0096] In some embodiments, the flame color pixel can be determined using the following formula:

[0097] R color =R RGB ∪R Lab , formula (5);

[0098] In the formula, R color Represents the flame color pixel; R RGB Represents the first flame color pixel; R Lab This represents the second flame color pixel.

[0099] Please see Figure 3 In some embodiments, at least two target video frames include a first video frame and a second video frame, wherein the first video frame is the frame preceding the second video frame.

[0100] Step 104 may include, but is not limited to, steps 301 to 303:

[0101] Step 301: Detect the first moving region on the first video frame using a pre-trained initial motion detection model to obtain the first moving region;

[0102] Step 302: Based on the region proportion of the first moving region in the first video frame and the color features of the second video frame, update the parameters of the initial motion detection model to obtain the target motion detection model.

[0103] Step 303: Detect the second moving region by using the target motion detection model on the second video frame to obtain the second moving region.

[0104] The advantage of this embodiment lies in that by dynamically adjusting the parameters of the initial motion detection model, and jointly adjusting the model parameters using the proportion of the moving region in the previous frame (e.g., the first video frame) and the color features of the current frame (e.g., the second video frame), the model can adjust the detection strategy in real time according to the dynamic characteristics of the scene. Specifically, the proportion of the moving region in the first video frame reflects the dynamic level of the scene. For example, a larger proportion indicates the presence of significant moving objects or a large number of moving objects. By updating the model parameters using the proportion of the moving region in the first video frame and the color features of the second video frame, the target motion detection model can capture rapidly changing moving targets in dynamic scenes (such as flame spread areas) and enhance the recognition ability of specific targets through color features. In other words, this embodiment does not detect independently frame by frame, but updates the model parameters by analyzing the correlation between information in consecutive frames and using a scene-adaptive detection method. This allows the target motion detection model to adapt to changes in the scene, thereby improving the flexibility and reliability of the target motion detection model's moving region detection, and ultimately improving the reliability of flame detection.

[0105] In step 301 of some embodiments, the initial motion detection model may be a Gaussian Mixture Model (GMM). A Gaussian Mixture Model is a model used to detect moving objects in a video sequence. Gaussian Mixture Models can be used for background modeling, foreground detection, and moving object detection. It should be noted that a moving region refers to the pixel region containing a moving object identified in a video frame by a motion detection model (such as an initial motion detection model or a target motion detection model). A first moving region refers to the moving region of the first video frame.

[0106] In some embodiments, it should be noted that current moving target detection methods, such as frame difference and optical flow methods, are easily affected by environmental interference (such as changes in illumination and noise interference), leading to a high false alarm rate. For example, the frame difference method is sensitive to noise interference, and the threshold selection is also highly sensitive. If the threshold is set too low, the detection results will contain a large amount of noise interference; if the threshold is set too high, slowly moving targets in the image may be ignored. Especially in complex environments, the speed of moving objects may change rapidly, making current motion detection methods difficult to apply to such complex environments. Considering the above problems, embodiments of this application update the model parameters based on the motion state of the current frame (i.e., the proportion of the moving area in the first video frame within the first video frame), so that the target motion detection model adapts to changes in the motion state of the scene, thereby improving the reliability of flame detection.

[0107] In some embodiments, prior to step 301, the flame detection method may further include: constructing a model for the color value of each pixel in the first video frame according to a preset number of target distributions, to obtain an initial motion detection model; wherein the initial motion detection model includes at least one Gaussian distribution (also known as a normal distribution), and the number of Gaussian distributions is the number of target distributions. For example, a model can be constructed based on the color value (such as RGB value or Lab value) of each pixel in the first video frame to obtain a mixture of K Gaussian distributions. The value range of K (i.e., the number of target distributions) may include 3 to 5. For example, the value of K may be 3, 4, or 5.

[0108] Then, at least one video frame can be selected from the target video, and a Gaussian distribution can be fitted to the at least one video frame using the Gaussian distribution of the initial motion detection model to determine the initial values ​​of the Gaussian distribution parameters. For example, the first N video frames in the target video can be selected and input into the initial motion detection model so that the model learns the color distribution of pixels in the video frames, determines the mean μ, variance σ, and weight ω of each Gaussian distribution, and thus obtains the trained initial motion detection model.

[0109] In some embodiments, step 301 may include: performing matching analysis on the pixel value of each target pixel in the first video frame and each Gaussian distribution using a pre-trained initial motion detection model to obtain a matching result; if the matching result indicates a matching failure, then the target pixel is identified as a moving pixel; and a combination of at least one moving pixel is identified as a first moving region. For example, in the process of moving region detection, the distribution matching formula is as follows:

[0110] |X t -μ i |≤2.5σ i (i = 1, 2, ..., K), Formula (6);

[0111] In the formula, X t μ represents the color value of the pixel in frame t. i Let σ represent the mean of the i-th Gaussian distribution. i Let represent the standard deviation of the i-th Gaussian distribution; K represents the number of Gaussian distributions.

[0112] In some embodiments, if the pixel value of the target pixel satisfies the distribution matching formula corresponding to at least one Gaussian distribution in the initial motion detection model, the matching result indicates a successful match. Conversely, if the pixel value of the target pixel does not satisfy the distribution matching formula corresponding to any Gaussian distribution in the initial motion detection model, the matching result indicates a failed match. Matching results can also be determined in other ways, not limited to these. It should be noted that if the match is successful, the pixel is considered to likely belong to the background (i.e., a static pixel); otherwise, it belongs to the foreground (i.e., a moving pixel).

[0113] In step 302 of some embodiments, the proportion of the moving region of the first video frame in the total area of ​​the first video frame can be the proportion of moving pixels in the first video frame. The meaning and function of the moving pixel proportion can be found in the detailed description of step 401 below, and will not be repeated here.

[0114] In some embodiments, the color features of the second video frame can be the color values ​​of pixels in the second video frame, such as RGB values. The target motion detection model refers to the initial motion detection model after parameter updates.

[0115] In step 303 of some embodiments, it should be noted that the second moving region refers to the moving region of the second video frame.

[0116] Please see Figure 4 In some embodiments, the initial motion detection model has initial distribution parameters;

[0117] Step 302 may include, but is not limited to, steps 401 to 403:

[0118] Step 401: Calculate the ratio of the number of pixels in the first moving region to the number of pixels in the first video frame to obtain the proportion of moving pixels;

[0119] Step 402: Calculate the distribution parameters based on the proportion of moving pixels, the color value of each pixel in the second video frame, and the initial distribution parameters to obtain the target distribution parameters;

[0120] Step 403: Update the distribution parameters of the initial motion detection model according to the target distribution parameters to obtain the target motion detection model.

[0121] The advantage of this embodiment is that the moving pixel ratio is calculated by comparing the number of pixels in the first moving region with the number of pixels in the first video frame. Then, the target distribution parameters are calculated based on the moving pixel ratio, the color value of each pixel in the second video frame, and the initial distribution parameters. This allows the initial distribution parameters of the initial motion detection model to be updated based on the moving pixel ratio of the previous frame (i.e., the first video frame), so that the initial motion detection model can adapt to the scene motion state of the first video frame. This improves the flexibility and reliability of the target motion detection model in motion detection of subsequent video frames (such as the second video frame), thereby improving the reliability of flame detection.

[0122] In step 401 of some embodiments, the moving pixel percentage refers to the ratio of the number of pixels in the first moving region of the first video frame to the total number of pixels in the first video frame. Specifically, the moving pixel percentage can be obtained by dividing the number of pixels in the first moving region by the total number of pixels in the first video frame. For example, assuming the number of pixels in the first moving region is 50 and the total number of pixels in the first video frame is 100, then the moving pixel percentage is 50%.

[0123] In step 402 of some embodiments, the initial distribution parameters refer to the distribution parameters of the initial motion detection model. The target distribution parameters refer to the updated initial distribution parameters. Specifically, the initial motion detection model can be a Gaussian mixture model, which consists of multiple Gaussian distributions, and the distribution parameters (such as the initial distribution parameters or the target distribution parameters) include the mean and variance of the Gaussian distributions.

[0124] In step 403 of some embodiments, the distribution parameters of the target movement detection model are the target distribution parameters.

[0125] Please see Figure 5 In some embodiments, the initial distribution parameters include an initial first distribution parameter and an initial second distribution parameter;

[0126] Step 402 may include, but is not limited to, steps 501 through 507:

[0127] Step 501: Multiply the percentage of moving pixels by the preset attenuation coefficient to obtain the target product;

[0128] Step 502: Calculate the cumulative distribution of the target product using a preset exponential cumulative distribution function to obtain the target learning rate;

[0129] Step 503: Determine the color value of each pixel in the second video frame as the target pixel color vector;

[0130] Step 504: Perform a weighted summation based on the initial first distribution parameters, the target learning rate, and the target pixel color vector to obtain the target first distribution parameters;

[0131] Step 505: Calculate the difference between the target pixel color vector and the target first distribution parameter to obtain the deviation vector;

[0132] Step 506: Perform a weighted summation based on the initial second distribution parameters, the target learning rate, the bias vector, and the transpose of the bias vector to obtain the target second distribution parameters;

[0133] Step 507: Use the combination of the first distribution parameter and the second distribution parameter of the target as the target distribution parameter.

[0134] The advantage of this embodiment lies in that it obtains the target product by multiplying the proportion of moving pixels by a preset attenuation coefficient, and calculates the target learning rate based on the exponential cumulative distribution function. This allows for dynamic adjustment of the model's learning rate (i.e., the target learning rate) to update the model's distribution parameters based on the proportion of moving pixels. Then, a weighted sum is performed based on the initial first distribution parameters, the target learning rate, and the target pixel color vector to obtain the target first distribution parameters. This ensures that the model retains the stability of historical parameters while rapidly responding to changes in the current frame during parameter updates. Furthermore, a deviation vector is obtained based on the difference between the target pixel color vector and the target first distribution parameters. A weighted sum is then performed based on the initial second distribution parameters, the target learning rate, the deviation vector, and the transpose of the deviation vector to obtain the target second distribution parameters, which update the initial second distribution parameters. This results in a model with updated distribution parameters, i.e., a target motion detection model. This improves the adaptability of the target motion detection model to complex motion patterns and enhances its flexibility and reliability in motion detection.

[0135] In step 501 of some embodiments, the target product refers to the product of the percentage of moving pixels and the attenuation coefficient, which is λ·MPR in the definition formula of the target learning rate (see the detailed description of step 502 below).

[0136] In some embodiments, it should be noted that the exponential cumulative distribution function is the cumulative distribution function of the exponential distribution. The formula for the exponential cumulative distribution function is 1 - e^(-1 / 2) * ( ... -z , where z represents the independent variable of the exponential cumulative distribution function. For example, the independent variable of the exponential cumulative distribution function can be the target product λ·MPR.

[0137] In step 502 of some embodiments, the target learning rate is defined as shown in the following formula:

[0138] α = 1 - e -λ·MPR , formula (7);

[0139] In the formula, α represents the target learning rate; λ represents the decay coefficient; and MPR represents the moving pixel ratio.

[0140] Specifically, the attenuation coefficient λ can be 0.1. It can also be set or updated to other values, and is not limited to these.

[0141] It should be noted that the moving pixel ratio (MPR) ranges from [0,1]. The moving pixel ratio is negatively correlated with the target learning rate. The target learning rate controls the speed of parameter updates. For example, if the moving pixel ratio is close to 0, it indicates a static scene in the video frame. In this case, a smaller target learning rate reduces false alarms. If the moving pixel ratio is close to 1, it indicates a dynamic scene in the video frame. In this case, a larger target learning rate allows for faster parameter updates to adapt to scene changes.

[0142] In some embodiments, for example, when the initial motion detection model is a Gaussian mixture model, the initial motion detection model includes at least one Gaussian distribution, each Gaussian distribution having weights, a mean, and a variance. The weight update formula is shown in the following formula:

[0143] ω i * =(1-α)ω i +α, Formula (8);

[0144] In the formula, ω i * ω represents the weights updated by the i-th Gaussian distribution; i α represents the weight of the i-th Gaussian distribution; α represents the target learning rate.

[0145] In step 503 of some embodiments, the target pixel color vector can be a vector composed of the red, green, and blue values ​​of the target pixel. For example, assuming the red value (R) of the target pixel is 200, the green value (G) is 50, and the blue value (B) is 30, then the target pixel color vector X t [200,50,30].

[0146] In step 504 of some embodiments, the weighted summation based on the initial first distribution parameter, the target learning rate, and the target pixel color vector may include: obtaining a first update weight based on the ratio of the target learning rate to the weights of the Gaussian distribution in the initial motion detection model; obtaining a second update weight based on the difference between 1 and the first update weight; and summing the product of the first update weight and the target pixel color vector, and the product of the second update weight and the initial first distribution parameter, to obtain the target first distribution parameter.

[0147] It should be noted that in the above weighted summation process, the first updated weight is equivalent to the weight of the target pixel color vector, and the second updated weight is equivalent to the weight of the initial first distribution parameter. As for the definition of the Gaussian distribution weights in the initial motion detection model, please refer to the detailed description of the weight update formula above; it will not be repeated here.

[0148] In some embodiments, the initial first distribution parameter can be the mean. Specifically, the target first distribution parameter is defined as follows (also known as the mean update formula):

[0149]

[0150] In the formula, μ i * μ represents the mean of the i-th Gaussian distribution after updating, which is also the parameter of the target first distribution; i Let represent the mean of the i-th Gaussian distribution, i.e., the initial first distribution parameter; α represents the target learning rate; ω i * x represents the weights updated by the i-th Gaussian distribution; t This represents the color value of a pixel in the target video frame at frame t, which is also the target pixel color vector.

[0151] In some embodiments, it should be noted that, Indicates the first updated weight; This indicates the second update weight.

[0152] In step 505 of some embodiments, the target first distribution parameter can specifically be a vector, and the deviation vector is the vector obtained by subtracting the target pixel color vector from the target first distribution parameter. For example, suppose the target first distribution parameter μ i * The values ​​are [5, 30, 80], and the target pixel color vector x t If the value is [20, 50, 90], then the deviation vector (x) t -μ i * The value is [15, 20, 10].

[0153] In step 506 of some embodiments, the weighted summation based on the initial second distribution parameter, the target learning rate, the bias vector, and the transpose of the bias vector may include: obtaining a first update weight based on the ratio of the target learning rate to the weights of the Gaussian distribution in the initial motion detection model; obtaining a second update weight based on the difference between 1 and the first update weight; and summing the first update weight, the transpose of the bias vector, the product of the bias vector, and the second update weight and the initial second distribution parameter to obtain the target second distribution parameter.

[0154] It should be noted that in the above weighted summation process, the first updated weight is equivalent to the weight of the product of the transpose of the deviation vector and the deviation vector, and the second updated weight is equivalent to the weight of the initial second distribution parameter.

[0155] In some embodiments, the initial second distribution parameter can be the variance. Specifically, the target second distribution parameter is defined as follows (also known as the variance update formula):

[0156]

[0157] In the formula, This represents the variance after the i-th Gaussian distribution is updated, which is also the target second distribution parameter; Let represent the variance of the i-th Gaussian distribution, which is also the initial second distribution parameter; α represents the target learning rate; ω i * Represents the weights updated by the i-th Gaussian distribution; (X t -μ i * (X) represents the difference between each component of the target pixel color vector and the updated mean (i.e., the target first distribution parameter), which is also the bias vector; t -μ i * ) T This represents the transpose of the deviation vector.

[0158] In step 507 of some embodiments, the target distribution parameters include a first target distribution parameter and a second target distribution parameter. For example, the target distribution parameters include the updated mean μ. i * and the updated variance

[0159] In some embodiments, for example, assuming the target pixel color vector X of the pixel at time t t (That is, the RGB values) are [200, 50, 30]; the current Gaussian distribution weights ω i * =1, mean μ i The variance is [180, 60, 40]. The learning rate is 100; the target learning rate α is 0.01.

[0160] The updated mean can be calculated using the formula for updating the mean: μ i * =0.99×[180,60,40]+0.01×[200,50,30]=[180.2,59.9,39.9].

[0161] The updated variance can be calculated using the variance update formula:

[0162] It should be noted that, in this embodiment of the application, by iteratively updating the distribution parameters of the initial motion detection model, the updated target motion detection model can accurately distinguish between the background (such as static objects) and the foreground (such as dynamically changing fires or moving objects), thereby improving the accuracy of motion detection.

[0163] Please see Figure 6 In some embodiments, at least two target video frames include a first video frame and a second video frame, wherein the first video frame is the frame preceding the second video frame.

[0164] Step 105 may include, but is not limited to, steps 601 to 604:

[0165] Step 601: For each target video frame, the overlapping areas of the flame color region and the moving region are filtered to obtain the initial flame region.

[0166] Step 602: Calculate the overlap rate of the initial flame region in the first video frame and the initial flame region in the second video frame to obtain the flame region overlap rate.

[0167] Step 603: Compare the flame area overlap rate with a preset overlap rate threshold;

[0168] Step 604: If the overlap rate of the flame regions is greater than or equal to the overlap rate threshold, the initial flame region is identified as the target flame region.

[0169] The advantage of this embodiment lies in that it obtains the flame region overlap rate by analyzing the overlap rate of the initial flame region in consecutive video frames (e.g., the first video frame and the second video frame), and compares it with an overlap rate threshold to verify the stability of the flame region in consecutive frames. For example, if the flame region overlap rate is greater than or equal to the overlap rate threshold, it indicates that the initial flame region is a relatively stable flame, and therefore the initial flame region is identified as the target flame region. This can eliminate interference caused by transiently moving objects (such as birds or fluttering clothing), further improving the robustness and accuracy of flame detection and reducing the false detection rate.

[0170] In step 601 of some embodiments, the initial flame region refers to the pixel region where the flame color region and the moving region overlap. For example, the initial flame region can be obtained using the following formula:

[0171] F(x, y, t)=O(x, y)∩S(x, y), formula (11);

[0172] In the formula, F(x,y,t) represents the initial flame region, O(x,y) represents the moving region, and S(x,y) represents the flame color region; ∩ represents the union operation, which is the operation of filtering overlapping regions mentioned above.

[0173] It should be noted that x and y together represent the coordinates of the pixel (x, y). For example, x represents the row coordinate of the pixel, and y represents the column coordinate of the pixel; t indicates that the pixel (x, y) belongs to the t-th frame in the target video.

[0174] In some embodiments, for example, the target video frame such as Figure 7 As shown, motion detection can be performed on the target video frame using a Gaussian mixture model, and flame color feature detection can be performed on the target video frame using a color space. Figure 8 As shown, the specific type of the moving region can be a binary image output by a Gaussian mixture model (also known as a foreground mask). If the value of a pixel in the binary image is 1, it indicates that the pixel is a moving pixel, that is, it belongs to the moving region. Figure 9 As shown, the specific type of the flame color region can be a binary image after color space fusion. If the value of a pixel in the binary image is 1, it means that the pixel conforms to the flame color characteristics, that is, it belongs to the flame color region.

[0175] In step 602 of some embodiments, the flame region overlap rate is the region overlap rate between the initial flame regions of two adjacent frames (i.e., the first video frame and the second video frame), and the flame region overlap rate can reflect the flame growth rate. For example, the flame region overlap rate is defined by the following formula:

[0176]

[0177] In the formula, OV represents the overlap rate of the flame region; Area(·) represents the number of pixels in the region; F t-1 Indicates the initial flame region of the first video frame; F t This represents the initial flame region in the second video frame; min(Area(F) t Area(F) t-1 )) represents the minimum number of pixels in the initial flame region of the first video frame and the minimum number of pixels in the initial flame region of the second video frame.

[0178] In step 603 of some embodiments, the overlap rate threshold may be 0.5. The overlap rate threshold may also be set or updated to other values, and is not limited thereto.

[0179] In step 604 of some embodiments, if the overlap rate of the flame region is greater than or equal to the overlap rate threshold, it indicates that the stability of the initial flame region between adjacent frames is high, and therefore it can be judged as a stable fire. If the overlap rate of the flame region is less than the overlap rate threshold, it indicates that the stability of the flame region is poor. This region may be a misjudged region caused by a short-lived moving object (such as a bird or fluttering clothing). Therefore, this region is not marked as the final flame region, but can be marked as motion interference and discarded.

[0180] In some embodiments, identifying the initial flame region as the target flame region means identifying the initial flame region of the first video frame as the target flame region of the first video frame, and identifying the initial flame region of the second video frame as the target flame region of the second video frame.

[0181] Please see Figure 10 In one application example, the overlap rate between the initial flame region of the current frame and the initial flame region of the previous frame can be calculated to obtain the flame region overlap rate (OV). If the flame region overlap rate OV is greater than 0.5 and lasts for 15 frames, the initial flame region can be identified as a fire region. If the flame region overlap rate OV is less than or equal to 0.5, the initial flame region can be identified as a non-fire region.

[0182] Please see Figure 11 In some embodiments, after step 105, the image recognition-based flame detection method may include, but is not limited to, steps 701 to 706:

[0183] Step 701: Select two video frames from at least two target video frames according to a preset time interval threshold to obtain a third video frame and a fourth video frame; wherein the time interval between the third video frame and the fourth video frame is less than or equal to the time interval threshold.

[0184] Step 702: Determine the number of pixels in the target flame region of the third video frame as the number of first flame pixels, and determine the number of pixels in the target flame region of the fourth video frame as the number of second flame pixels.

[0185] Step 703: Determine the time interval between the third video frame and the fourth video frame as the target time interval;

[0186] Step 704: Calculate the difference between the number of first flame pixels and the number of second flame pixels to obtain the flame pixel increment;

[0187] Step 705: Calculate the flame growth rate by comparing the ratio of the flame pixel increment to the target time interval;

[0188] Step 706: In response to the flame growth rate being greater than or equal to a preset growth rate threshold, a fire warning message is sent.

[0189] The advantage of this embodiment lies in selecting two video frames with a fixed time interval (i.e., the third and fourth video frames) using a time interval threshold. The ratio of the flame pixel increment between the third and fourth video frames to the target time interval is calculated to analyze the flame growth rate, i.e., the fire growth rate, and thus the fire intensity. If the flame growth rate is greater than or equal to a preset growth rate threshold, it indicates a relatively strong fire, triggering an alarm mechanism promptly, such as sending a fire alert message, to improve the reliability and accuracy of fire detection.

[0190] In step 701 of some embodiments, both the third and fourth video frames are video frames in the target video (i.e., target video frames), and the time interval between the third and fourth video frames is less than or equal to a time interval threshold. For example, assuming the time interval threshold is 10 seconds, the frame interval of the target video is 1 second, and the first frame is selected from the target video as the third video frame, then another frame from the target video whose time interval with the first frame does not exceed 10 seconds should be selected as the fourth video frame. For example, the third frame can be selected as the fourth video frame, and the time interval between the first and third frames is 2 seconds, which is less than the time interval threshold (10 seconds).

[0191] In step 702 of some embodiments, the first flame pixel count is the number of pixels in the target flame region of the third video frame. The second flame pixel count is the number of pixels in the target flame region of the fourth video frame.

[0192] In step 703 of some embodiments, the target time interval refers to the time interval between the third video frame and the fourth video frame. Similarly, in the example above, the target time interval (i.e., the time interval between the first frame and the third frame) is 2 seconds.

[0193] In step 704 of some embodiments, the flame pixel increment is the difference between the first flame pixel count and the second flame pixel count. The formula for the flame pixel increment is defined in the detailed explanation of step 705 below, and will not be repeated here.

[0194] In some embodiments, it should be noted that accurately obtaining the growth rates of the fire's initiation and spread stages is crucial when performing fire intensity analysis. To improve the accuracy of fire detection, the flame growth rate can also be calculated to analyze the fire's intensity. The flame growth rate can reflect the fire intensity.

[0195] In step 705 of some embodiments, the flame growth rate is defined by the following formula:

[0196]

[0197] In the formula, Ity represents the flame growth rate; FP n This indicates the number of flame pixels in the fourth video frame, which is also the number of flame pixels in the second frame; FP n-1 The value represents the number of flame pixels in the third video frame, which is also the number of flame pixels in the first frame; ΔFP represents the flame pixel increment; t n Indicates the time of the fourth video frame; t n-1 Δt represents the time of the third video frame; Δt represents the time interval between the third and fourth video frames, which is also the target time interval.

[0198] In step 706 of some embodiments, the growth rate threshold can be 10%, or other values ​​within the range [0,1], and is not limited thereto. For example, when the flame growth rate is less than or equal to 10%, it indicates that the flame is spreading slowly and the risk of fire is low, at which point a local alarm can be triggered. When the flame growth rate is greater than 10%, it indicates that the flame is spreading rapidly and the risk of fire is high, at which point an emergency response can be triggered, such as sending a fire warning message to the fire protection system.

[0199] Please see Figure 12 In one application example, the flame detection method includes: acquiring video; preprocessing video frames; performing flame color feature detection on the video frames using RGB and Lab color spaces through color space fusion to obtain flame color regions; performing dynamic detection on the video frames using a Gaussian mixture model (GMM) to obtain moving regions; and using the overlapping area between the flame color region and the moving region as the flame region, thereby reducing interference from static red objects. Then, fire intensity analysis and verification can be performed based on the flame region to determine if a fire exists. For example, the fire intensity can be analyzed by calculating the flame growth rate, and the accuracy of the fire detection results can be verified by calculating the flame region overlap rate. If a fire exists, a fire alarm decision can be made. If no fire exists, no further processing is performed.

[0200] Please see Figure 13 This application also provides an image recognition-based flame detection device that can implement the above-described image recognition-based flame detection method. The device includes:

[0201] Video acquisition module 801 is used to acquire the target video;

[0202] The video frame extraction module 802 is used to extract video frames from the target video to obtain at least two target video frames.

[0203] The flame color detection module 803 is used to detect the flame color region of each target video frame based on the color features of each target video frame and the preset target flame color rules, so as to obtain the flame color region.

[0204] The moving region detection module 804 is used to perform moving region detection on any two adjacent target video frames to obtain the moving region of each target video frame; wherein, adjacent target video frames are used to represent video frames that are adjacent in time sequence.

[0205] The overlapping area filtering module 805 is used to filter the overlapping areas of the flame color area and the moving area of ​​each target video frame to obtain the target flame area.

[0206] The specific implementation of the image recognition-based flame detection device is basically the same as the specific implementation of the image recognition-based flame detection method described above, and will not be repeated here.

[0207] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described image recognition-based flame detection method. This electronic device can include any smart terminal such as a tablet computer or an in-vehicle computer.

[0208] Please see Figure 14 , Figure 14 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0209] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0210] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and called and executed by the processor 901 to implement the image recognition-based flame detection method of the embodiments of this application.

[0211] The input / output interface 903 is used to implement information input and output;

[0212] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0213] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0214] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0215] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image recognition-based flame detection method.

[0216] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0217] It should be noted that the software tools or components not belonging to our company that appear in the embodiments of this application are merely examples and do not represent actual use.

[0218] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0219] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0220] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0221] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0222] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural.

[0223] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of the above units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0224] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0225] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A flame detection method based on image recognition, characterized in that, The method includes: Acquire the target video; The target video is subjected to video frame extraction to obtain at least two target video frames; the at least two target video frames include a first video frame and a second video frame, wherein the first video frame is the previous frame of the second video frame; Based on the color features of each target video frame and a preset target flame color rule, flame color region detection is performed on each target video frame to obtain the flame color region; wherein, the color features of each target video frame include a red value, a green value, and a blue value, and the target flame color rule includes that the red value is greater than the green value, and the green value is greater than the blue value. The first moving region is obtained by detecting the first moving region in the first video frame using a pre-trained initial motion detection model. Based on the region proportion of the first moving region in the first video frame and the color features of the second video frame, the parameters of the initial motion detection model are updated to obtain the target motion detection model; The second video frame is subjected to second motion region detection using the target motion detection model to obtain the second motion region; wherein, adjacent target video frames are used to represent video frames that are adjacent in time sequence; the first motion region refers to the motion region of the first video frame, and the second motion region refers to the motion region of the second video frame. For each target video frame, the overlapping areas of the flame color region and the moving region are filtered to obtain the target flame region.

2. The method according to claim 1, characterized in that, The initial motion detection model has initial distribution parameters; The step of updating the parameters of the initial motion detection model based on the region proportion of the first moving region in the first video frame and the color features of the second video frame to obtain the target motion detection model includes: The percentage of moving pixels is calculated by comparing the number of pixels in the first moving region with the number of pixels in the first video frame. The target distribution parameters are obtained by calculating the distribution parameters based on the moving pixel ratio, the color value of each pixel in the second video frame, and the initial distribution parameters. The distribution parameters of the initial motion detection model are updated based on the target distribution parameters to obtain the target motion detection model.

3. The method according to claim 2, characterized in that, The initial distribution parameters include an initial first distribution parameter and an initial second distribution parameter; The step of calculating the target distribution parameters based on the moving pixel ratio, the color value of each pixel in the second video frame, and the initial distribution parameters includes: Multiply the percentage of moving pixels by the preset attenuation coefficient to obtain the target product; The target learning rate is obtained by performing cumulative distribution calculation on the target product using a preset exponential cumulative distribution function; The color value of each pixel in the second video frame is determined as the target pixel color vector; The target first distribution parameters are obtained by weighted summation of the initial first distribution parameters, the target learning rate, and the target pixel color vector; The deviation vector is obtained by calculating the difference between the target pixel color vector and the target first distribution parameter; The target second distribution parameters are obtained by weighted summation of the initial second distribution parameters, the target learning rate, the bias vector, and the transpose of the bias vector. The combination of the first target distribution parameter and the second target distribution parameter is used as the target distribution parameter.

4. The method according to any one of claims 1 to 3, characterized in that, The target video frame includes at least two target pixels; the target flame color rule includes a first flame color rule and a second flame color rule; The step of detecting the flame color region for each target video frame based on its color features and a preset target flame color rule, to obtain the flame color region, includes: By using a preset first color space, a first color feature is extracted from each target pixel to obtain a first color feature; wherein, the first color feature includes a red value, a green value, and a blue value; A first flame color pixel is obtained by selecting a target pixel whose first color feature conforms to the first flame color rule from at least two target pixels; wherein, the first flame color rule is used to indicate that the red value is greater than the green value and the green value is greater than the blue value; By using a preset second color space, a second color feature is extracted from each target pixel to obtain a second color feature; wherein, the second color feature includes red-green axis chromaticity and yellow-blue axis chromaticity; Select a target pixel whose second color feature conforms to the second flame color rule from at least two target pixels to obtain a second flame color pixel; wherein, the second flame color rule is used to indicate that the yellow-blue axis chromaticity is greater than or equal to the red-green axis chromaticity; At least one flame color pixel is obtained based on the union of the first flame color pixel and the second flame color pixel, and the combination of at least one flame color pixel is determined as the flame color region.

5. The method according to any one of claims 1 to 3, characterized in that, At least two of the target video frames include a first video frame and a second video frame, wherein the first video frame is the frame preceding the second video frame; The step of filtering the overlapping areas of the flame color region and the movement region in each target video frame to obtain the target flame region includes: For each target video frame, the overlapping area between the flame color region and the moving region is filtered to obtain the initial flame region; The overlap rate of the initial flame region in the first video frame and the initial flame region in the second video frame is calculated to obtain the flame region overlap rate. The overlap rate of the flame regions is compared with a preset overlap rate threshold. If the overlap rate of the flame regions is greater than or equal to the overlap rate threshold, the initial flame region is identified as the target flame region.

6. The method according to any one of claims 1 to 3, characterized in that, After performing overlapping region filtering on the flame color region and the movement region of each target video frame to obtain the target flame region, the method further includes: According to a preset time interval threshold, two video frames are selected from at least two target video frames to obtain a third video frame and a fourth video frame; wherein the time interval between the third video frame and the fourth video frame is less than or equal to the time interval threshold. The number of pixels in the target flame region of the third video frame is determined as the first flame pixel count, and the number of pixels in the target flame region of the fourth video frame is determined as the second flame pixel count. The time interval between the third video frame and the fourth video frame is determined as the target time interval; The difference between the number of first flame pixels and the number of second flame pixels is calculated to obtain the flame pixel increment. The flame growth rate is calculated by comparing the flame pixel increment with the target time interval. In response to the flame growth rate being greater than or equal to a preset growth rate threshold, a fire warning message is sent.

7. A flame detection device based on image recognition, characterized in that, The device includes: The video acquisition module is used to acquire the target video; The video frame extraction module is used to extract video frames from the target video to obtain at least two target video frames; the at least two target video frames include a first video frame and a second video frame, wherein the first video frame is the previous frame of the second video frame; The flame color detection module is used to detect the flame color region of each target video frame according to the color features of each target video frame and the preset target flame color rules, so as to obtain the flame color region. The moving region detection module is used to perform a first moving region detection on the first video frame using a pre-trained initial moving region detection model to obtain a first moving region; update the parameters of the initial moving region based on the region proportion of the first moving region in the first video frame and the color features of the second video frame to obtain a target moving region detection model; and perform a second moving region detection on the second video frame using the target moving region detection model to obtain a second moving region; wherein, adjacent target video frames are used to represent video frames that are adjacent in time sequence; the first moving region refers to the moving region of the first video frame, and the second moving region refers to the moving region of the second video frame; The overlapping region filtering module is used to filter the overlapping regions of the flame color region and the moving region in each target video frame to obtain the target flame region.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

Images