Fire warning method, system, program product, electronic device and storage medium

Abstract

The application provides a fire warning method, system, program product, electronic equipment and storage medium. The method comprises the following steps: acquiring an image sequence; the image sequence comprises a left-eye image sequence and a right-eye image sequence; extracting dynamic features from continuous multiple frames of images in the image sequence, and determining whether a model calling trigger condition is met according to the dynamic features; in the case where the model calling trigger condition is met, confirming a fire target in the image by using a fire detection model, and outputting a detection result; the detection result comprises whether the fire target exists, and in the case where it is confirmed that the fire target exists, determining a left-eye center point of the fire target in a left-eye image and a right-eye center point of the fire target in a right-eye image; calculating a parallax according to the left-eye center point and the right-eye center point, and solving three-dimensional space coordinates of a fire source; and generating warning information according to the fire target confirmation result and the three-dimensional space coordinates of the fire source. The number of invalid reasoning times of the fire detection model is reduced, and the overall processing efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of fire early warning technology, and more specifically, to a fire early warning method, system, program product, electronic device and storage medium. Background Technology

[0002] Currently, fire early warning solutions include sensor-based solutions, such as those based on smoke, temperature, or gas sensors. These solutions determine whether to trigger an alarm using a single threshold or a combination of multiple sensors. However, sensors are easily affected by factors such as installation location, airflow, dust, and moisture, which can lead to delayed responses or false alarms. Summary of the Invention

[0003] The purpose of this application is to provide a fire early warning method, system, program product, electronic device and storage medium to improve the above-mentioned technical problems.

[0004] In a first aspect, embodiments of this application provide a fire early warning method, comprising: acquiring an image sequence; the image sequence including a left-eye image sequence acquired by a first image acquisition node and a right-eye image sequence acquired by a second image acquisition node; extracting dynamic features from multiple consecutive frames in the image sequence, and determining whether a model call triggering condition is met based on the dynamic features; the dynamic features include at least one of the following: area features of candidate regions in the image that meet preset flame color conditions, inter-frame grayscale change features within the candidate regions, area fluctuation features of the candidate regions within a preset frame number window, and frequency domain features; wherein, the frequency domain features are obtained by measuring the area fluctuation of the candidate regions within a preset frame number window. The area value sequence is obtained by performing a Fast Fourier Transform. The frequency domain features include the dominant frequency component and the proportion of spectral energy of the area value sequence within the preset flame flicker frequency band. Under the condition of meeting the model call triggering conditions, the flame target is confirmed in the image through the flame detection model, and the detection result is output. The detection result includes whether there is a flame target, and if the flame target is confirmed, the left center point of the flame target in the left eye image and the right center point in the right eye image are determined. The disparity is calculated based on the left and right center points, and the three-dimensional spatial coordinates of the fire source are solved. Warning information is generated based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source.

[0005] In the above implementation process, by extracting dynamic features from multiple consecutive frames of images and making rapid judgments before calling the flame detection model, and only calling the model for confirmation when the triggering conditions are met, the number of invalid model inferences in flameless scenarios can be reduced, thereby reducing the computational load and power consumption of the edge device and improving processing efficiency. Simultaneously, after confirming the flame target, this method directly uses the principle of binocular vision to calculate the three-dimensional spatial coordinates of the fire source, so that the warning information not only includes the judgment of whether a flame exists, but also provides the specific direction and distance of the fire source relative to the monitoring device, providing richer reference information for emergency response. Furthermore, all the above steps are completed on the edge, eliminating the need to continuously upload the original images to the cloud, reducing network bandwidth consumption and data transmission latency, and making the warning response more timely.

[0006] Optionally, in this embodiment, the preset flame color condition includes at least one of the following: the intensity of the red channel of the pixel is higher than the intensity of the green channel and the intensity of the blue channel, the brightness of the pixel is greater than a preset brightness threshold, and the hue of the pixel is within the preset flame hue range; the model call trigger condition includes at least one of the following: the area of ​​the candidate region exceeds a first threshold, the inter-frame grayscale change feature value of the candidate region exceeds a second threshold, the area fluctuation feature value of the candidate region exceeds a third threshold, and a frequency domain feature condition; the frequency domain feature condition is that the main frequency component falls within the preset flame flicker frequency band range, and the spectral energy ratio exceeds a preset energy threshold, and the main frequency component does not fall within the preset interference frequency band range.

[0007] In the above implementation process, by setting preset flame color conditions and model call trigger conditions, three dynamic features can be used to quickly filter each frame of image before calling the flame detection model. Since the extraction of these three features involves only low-overhead operations such as color comparison, inter-frame differencing, and area statistics, and does not involve complex calculations such as convolution and activation of deep learning models, the computational load of the filtering process is very small, and the processing speed is very fast. In actual operation, the vast majority of image sequences without flames can be excluded at this filtering stage, and only a small number of candidate frames with flame features will trigger model calling. This pre-screening mechanism effectively reduces the number of invalid inferences by the flame detection model, allowing the computing resources of the edge device to be more concentrated on image frames that truly pose a flame risk, thereby improving overall processing efficiency.

[0008] Optionally, in this embodiment, the inter-frame grayscale change feature is determined based on the grayscale difference between the corresponding pixels in the candidate region between the current frame and the previous frame; the area fluctuation feature is determined based on the area variance or standard deviation of the candidate region within a preset frame number window.

[0009] In the above implementation process, the gray-level difference between adjacent frames and the area statistics within consecutive frame windows are used to determine the inter-frame gray-level change features and area fluctuation features, respectively. The extraction of these two types of features does not involve complex mathematical models or a large number of multiplication and addition operations; the calculation process is simple and the processing speed is fast, meeting the real-time image sequence processing requirements of the edge microcontroller. By combining the inter-frame gray-level change features with the area fluctuation features, the behavior of candidate regions with flame-specific patterns can be comprehensively judged from two different dimensions: dynamic change and morphological pulsation. Compared with the screening method that relies solely on a single static color feature, this method can effectively eliminate interference from static objects and slowly changing light sources, making the pre-screening results more reliable, reducing invalid calls to the subsequent flame detection model, and thus improving overall processing efficiency.

[0010] Optionally, in this embodiment of the application, determining the left center point of the flame target in the left eye image and the right center point in the right eye image includes: obtaining the coordinates of the upper left corner, width, and height of the detection box output by the flame detection model; calculating the center point coordinates of the detection box based on the upper left corner coordinates, width, and height; and mapping the center point coordinates to the image coordinates at the calibration scale according to the ratio between the input scale of the flame detection model and the camera calibration scale.

[0011] In the above implementation process, by extracting the center point coordinates from the detection box output by the flame detection model, the position of the flame target in the image is quantified into a pair of specific coordinate values, providing a clear target matching point for subsequent binocular disparity calculation. Simultaneously, by mapping the center point coordinates at the model input scale to the camera calibration scale, the coordinate deviation caused by the inconsistency between the model input size and the calibration image size is reduced. This improves the scale matching between the coordinate data used in the binocular localization calculation and calibration parameters such as camera intrinsics and baselines, thereby improving the accuracy and reliability of subsequent three-dimensional coordinate calculation of the fire source.

[0012] Optionally, in this embodiment of the application, calculating the parallax based on the left and right eye center points and solving the three-dimensional spatial coordinates of the fire source includes: calculating the parallax based on the horizontal coordinates of the left and right eye center points; determining the depth based on the parallax and preset camera calibration parameters; and solving the horizontal and vertical coordinates of the fire source in three-dimensional space based on the coordinates of the left eye center point, the depth, and the camera calibration parameters.

[0013] In the aforementioned implementation process, parallax is calculated by the difference in the horizontal coordinates of the center points of the left and right eyes, and depth is determined based on the parallax and camera calibration parameters. This allows for the calculation of the horizontal and vertical coordinates of the fire source in three-dimensional space. This process directly transforms the flame detection results into specific spatial location information, enabling early warning information to go beyond simple qualitative judgments of "fire" or "no fire," providing the specific direction and distance of the fire source, thus providing a reference for on-site personnel to quickly determine the fire source's location. Simultaneously, the binocular positioning calculation is completed on the device side, eliminating the need to upload image data to the cloud for processing, reducing data transmission latency, and allowing the positioning results to be obtained in real time, thereby improving the overall response speed.

[0014] Optionally, in this embodiment, the flame detection model is a lightweight target detection model or image classification model deployed on the edge.

[0015] In the above implementation process, the flame detection model adopts a lightweight target detection model or image classification model deployed on the edge, enabling the model inference task to be completed locally at the image acquisition node without uploading images to the cloud for processing, effectively reducing the time and bandwidth consumed by data transmission. Simultaneously, due to the lightweight model processing, the number of parameters and computational load are reduced, allowing it to run on microcontrollers with limited memory and computing power. This avoids the problems of deployment limitations or excessively high inference latency caused by overly large models, enabling the entire detection process to be completed in real time on the edge, improving the timeliness of early warnings and the practicality of the system.

[0016] Optionally, in this embodiment of the application, before generating the warning information, the method further includes: acquiring environmental parameters collected by an environmental sensor group, the environmental parameters including one or more of temperature, humidity, gas concentration, and infrared flame response; performing multi-source joint judgment based on the environmental parameters, the flame target confirmation result, and the three-dimensional spatial coordinates of the fire source to determine the warning level; and generating the warning information based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source, including: generating the warning information based on the flame target confirmation result, the three-dimensional spatial coordinates of the fire source, and the warning level.

[0017] In the aforementioned implementation process, a multi-source joint judgment is performed by integrating the flame target confirmation results output by the flame detection model, the three-dimensional spatial coordinates of the fire source calculated by binocular positioning, and environmental parameters such as temperature, humidity, gas concentration, and infrared flame response collected by the environmental sensor group. Compared with judgment methods that rely solely on a single sensor or a single frame image, this approach can more comprehensively reflect the actual situation on site and effectively reduce the possibility of false alarms triggered by false alarms from a single sensor or false detections from a single frame image. Simultaneously, the warning information includes the warning level and the three-dimensional spatial coordinates of the fire source, allowing the receiver to intuitively understand the specific location and urgency of the fire source, facilitating rapid response decisions. Furthermore, the warning information is generated and uploaded only after the judgment is completed at the edge, eliminating the need for continuous transmission of original image data, reducing network bandwidth consumption, and making warning notifications more timely.

[0018] Secondly, this application also provides a fire early warning system, comprising: a first image acquisition node for acquiring a left-eye image sequence; a second image acquisition node, fixed relative to the first image acquisition node with a preset binocular baseline, for acquiring a right-eye image sequence; a processing module for extracting dynamic features from multiple consecutive frames of images in the left-eye and right-eye image sequences, and determining whether the model call triggering conditions are met based on the dynamic features; the dynamic features include at least one of the following: area features of candidate regions in the image that meet preset flame color conditions, inter-frame grayscale change features within the candidate regions, and area fluctuation features of the candidate regions within a preset frame number window; a flame confirmation module for confirming flame targets in the image through a flame detection model when the model call triggering conditions are met, and outputting detection results; the detection results include whether a flame target exists, and if a flame target is confirmed to exist, determining the left-eye center point of the flame target in the left-eye image and the right-eye center point in the right-eye image; a binocular positioning module for calculating disparity based on the left-eye center point and the right-eye center point, and solving the three-dimensional spatial coordinates of the fire source; and an early warning output module for generating early warning information based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source.

[0019] In the above implementation process, before performing flame detection model inference on the images, the processing module first extracts dynamic features from multiple consecutive frames for pre-screening. This ensures that most image frames without flame signs are excluded before model invocation, reducing the number of invalid inferences in the flame detection model and lowering the computational load and power consumption of the edge device. After confirming the presence of a flame target, the flame confirmation module directly outputs the center point of the flame target in the left and right eye images. The binocular positioning module calculates the disparity based on this center point and solves the three-dimensional spatial coordinates of the fire source, so that the warning information not only includes the flame target confirmation result but also provides the specific spatial position of the fire source relative to the monitoring equipment. The division of labor among the modules is clear, and the entire process is completed in a closed loop at the edge, without relying on cloud computing power, reducing data transmission latency and improving the timeliness of the warning response and the independence of the system.

[0020] Optionally, in this embodiment, the environmental sensor group is used to collect environmental parameters, including one or more of temperature, humidity, gas concentration and infrared flame response; the processing module is also used to perform multi-source joint judgment based on the environmental parameters, flame target confirmation results and the three-dimensional spatial coordinates of the fire source to determine the warning status or warning level.

[0021] In the aforementioned implementation process, in addition to visual detection, an environmental sensor array collects environmental parameters such as temperature, humidity, gas concentration, and infrared flame response. The processing module comprehensively judges these environmental parameters, along with the flame target confirmation results and the three-dimensional spatial coordinates of the fire source, to determine the warning status or warning level. Since environmental sensors and visual detection perceive fire signs from different physical dimensions, they complement each other. When one detection method is interfered with, the other may still provide valid information, thus reducing the risk of false alarms or missed alarms from a single detection method. Simultaneously, the multi-source joint judgment mechanism makes the determination of the warning status or warning level more comprehensive, helping to reduce false warnings triggered by momentary anomalies in a single sensor or misdetection of a single frame image, thereby improving the system's reliability.

[0022] Thirdly, embodiments of this application also provide a computer program product, including computer program instructions, which are executed by a processor to perform the method provided in the first aspect or any implementation thereof.

[0023] Fourthly, embodiments of this application also provide an electronic device, including: a processor and a memory, the memory storing computer program instructions, which are executed by the processor to perform the method provided in the first aspect or any implementation thereof.

[0024] Fifthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, perform the method provided in the first aspect or any implementation thereof.

[0025] The fire early warning method, system, program product, electronic device, and storage medium provided in this application extract dynamic features from multiple consecutive frames of images and make rapid judgments before calling the flame detection model. The model is only invoked for confirmation when triggering conditions are met. This reduces the number of invalid model inferences in flameless scenarios, thereby reducing the computational load and power consumption of the edge device and improving processing efficiency. Simultaneously, after confirming the flame target, the method directly calculates the three-dimensional spatial coordinates of the fire source using binocular vision principles. This ensures that the early warning information not only includes a judgment on the presence of a flame but also provides the specific direction and distance of the fire source relative to the monitoring equipment, providing richer reference information for emergency response. Furthermore, all the above steps are completed on the edge, eliminating the need to continuously upload the original images to the cloud, reducing network bandwidth usage and data transmission latency, and making the early warning response more timely. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A schematic flowchart illustrating a fire early warning method provided in an embodiment of this application; Figure 2 This is a geometrical schematic diagram of binocular fire source positioning provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a fire early warning system provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application.

[0030] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0031] Currently, fire early warning systems are mainly divided into two categories.

[0032] The first type relies on smoke, temperature, or gas sensors, using a single threshold or a combination of sensors to determine whether to trigger an alarm. This type of solution is simple in structure and low in cost, and is widely used in homes, warehouses, and other similar settings. However, in practical applications, sensors are easily affected by factors such as installation location, airflow, dust, and moisture. When a fire is in its early stages, combustion products have not yet spread to the vicinity of the sensor, and the system often cannot respond in time; while the presence of fumes, water vapor, or temperature fluctuations in the environment can easily lead to false alarms.

[0033] The second type of fire detection method, based on machine vision, uses cameras to capture images and identifies flames through image processing or deep learning models. Existing vision solutions typically rely on high-performance processors or cloud servers for inference calculations, resulting in high equipment costs and power consumption. Deploying the model directly on low-power microcontrollers for frame-by-frame execution faces challenges such as limited computing resources and high inference latency. Furthermore, traditional vision solutions can only determine the presence of flames in the image, but cannot provide the specific direction or distance of the fire source relative to the monitoring equipment, which is detrimental to subsequent emergency response.

[0034] To address the aforementioned issues, this application proposes a fire early warning method. By extracting dynamic features from multiple consecutive frames of images and performing rapid judgment before invoking the flame detection model, the method only invokes the model for confirmation when triggering conditions are met. This reduces the number of invalid model inferences in flameless scenarios, thereby lowering the computational load and power consumption of the edge device and improving processing efficiency. Furthermore, after confirming the flame target, this method directly calculates the three-dimensional spatial coordinates of the fire source using binocular vision principles. This ensures that the early warning information not only includes the presence of a flame but also provides the specific direction and distance of the fire source relative to the monitoring equipment, offering richer reference information for emergency response. Moreover, all the above steps are completed on the edge device, eliminating the need to continuously upload the original images to the cloud, reducing network bandwidth usage and data transmission latency, and enabling more timely early warning responses.

[0035] Please see Figure 1 The illustrated diagram shows a flowchart of a fire early warning method provided in an embodiment of this application. The fire early warning method provided in this application can be applied to electronic devices, which may include physical devices such as servers, PCs, tablets, or smartphones, or virtual devices such as virtual machines or containers. The electronic device can be a single device, a combination of multiple devices, or a cluster of a large number of devices. The fire early warning method may include: Step S110: Obtain the image sequence; the image sequence includes the left eye image sequence acquired by the first image acquisition node and the right eye image sequence acquired by the second image acquisition node.

[0036] Step S120: Extract dynamic features from multiple consecutive frames of images in the image sequence, and determine whether the model call triggering conditions are met based on the dynamic features; the dynamic features include at least one of the following: area features of candidate regions in the image that meet the preset flame color conditions, inter-frame grayscale change features within the candidate regions, area fluctuation features of the candidate regions within a preset frame number window, and frequency domain features; wherein, the frequency domain features are obtained by performing a fast Fourier transform on the area value sequence of the candidate regions within the preset frame number window, and the frequency domain features include the main frequency component and the spectral energy ratio of the area value sequence within the preset flame flicker frequency band.

[0037] Step S130: If the model call triggering condition is met, the flame detection model is used to confirm the flame target in the image and output the detection result. The detection result includes whether the flame target exists, and if the flame target exists, the left center point of the flame target in the left eye image and the right center point of the flame target in the right eye image are determined.

[0038] Step S140: Calculate the parallax based on the center point of the left eye and the center point of the right eye, and solve the three-dimensional spatial coordinates of the fire source.

[0039] Step S150: Generate early warning information based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source.

[0040] In step S110, the image sequence includes a left-eye image sequence acquired by the first image acquisition node and a right-eye image sequence acquired by the second image acquisition node. The first and second image acquisition nodes are fixed relative to each other with a preset binocular baseline distance. The two image acquisition nodes can use the same camera module, for example, both using OV2640 cameras, and the left and right images can be kept consistent in terms of imaging quality, field of view, resolution, and frame rate. As one implementation, during actual installation, the optical axes of the two cameras should be kept parallel, and the length of the binocular baseline can be preset according to the application scenario. For example, it can be set to 120 mm in a fixed monitoring scenario, and can be appropriately adjusted according to the platform size in a mobile inspection scenario.

[0041] The image acquisition nodes can utilize modules such as the ESP32-CAM, which integrate microcontrollers and camera functions. These modules incorporate image sensors, processors, and wireless communication interfaces, enabling them to independently perform image acquisition and preliminary processing tasks. After system startup, the two image acquisition nodes continuously acquire images at a preset frame rate (e.g., 10 to 30 frames per second) and perform necessary preprocessing on the acquired raw images, including resizing, color space conversion, and format encapsulation. The preprocessed images will then be used in subsequent dynamic feature extraction steps.

[0042] To facilitate the differentiation between the left and right images, the first image acquisition node can add identification information to each frame it acquires, indicating whether the frame comes from the left or right view. A data communication link can be established between the first and second image acquisition nodes, with communication methods including but not limited to TCP, UDP, serial port, I2C, SPI, Bluetooth, or direct WiFi connection. In one implementation, the second image acquisition node acts as the master node, and the first image acquisition node acts as the slave node. The first image acquisition node sends its acquired images or image processing results to the second image acquisition node for aggregation processing. In another implementation, both image acquisition nodes send image data to a separately configured processing module for centralized processing. Regardless of the communication method used, the left and right images need to be synchronized in time; that is, the left and right view images participating in the same round of processing are acquired at the same time or within an allowed time difference to improve the accuracy of subsequent parallax calculations.

[0043] In addition, the two image acquisition nodes can also achieve synchronous acquisition through external trigger signals. For example, the master node can send acquisition trigger commands to the two cameras at fixed time intervals, so that the two cameras can be exposed and output image frames at the same time.

[0044] In step S120, after acquiring the image sequence, dynamic features need to be extracted from multiple consecutive frames in the image sequence, and the current image is determined to meet the model call triggering conditions based on the extracted dynamic features. This step serves to perform a rapid screening before calling the computationally intensive flame detection model, avoiding wasting computational resources in most scenarios where there are no signs of flame. For each frame, the system first extracts candidate color regions based on preset flame color conditions. Preset flame color conditions refer to pre-defined discrimination rules used to determine whether the color of a pixel falls within the common color range of flames. Since flames typically exhibit hues ranging from red to yellow in the visible light band and have high brightness, candidate regions can be filtered by setting thresholds for the RGB or HSV components of the pixel.

[0045] For example, in the RGB color space, flame pixels typically satisfy the condition that the intensity value of the red channel is higher than the intensity values ​​of the green and blue channels, and the intensity value of the red channel is greater than a certain brightness threshold. In the HSV color space, the hue of flame pixels is typically within the red-yellow range between 0° and 60°, and the saturation and value are also within specific ranges. These two judgment methods can be used individually or in combination to reduce misjudgments caused by changes in lighting or similar object colors. The system traverses the current frame image pixel by pixel, marking pixels that meet the preset color conditions as 1 and those that do not as 0, thereby generating a binary candidate region mask. The regions with a value of 1 in this mask are the candidate regions.

[0046] After obtaining the candidate regions, the system extracts several dynamic features from them. The first is the area feature of the candidate region, which is the total number of pixels in the candidate region. As a flame burns, its shape and size constantly change, so the area of ​​the candidate region also fluctuates. This feature can be used as one of the criteria for determining whether a real flame exists.

[0047] The second method is the inter-frame grayscale variation feature within the candidate region. This is calculated by converting the current and previous frames into grayscale images, and then, within the area covered by the candidate region, counting the number or proportion of pixels whose absolute value of the grayscale difference between corresponding pixels in adjacent frames exceeds a preset difference threshold. Flames exhibit flickering characteristics, and the brightness changes at their edges between adjacent frames are typically drastic, while the inter-frame grayscale changes of static objects or stable light sources are relatively gradual. Therefore, this feature can effectively distinguish flames from static interference.

[0048] The third type is the area fluctuation feature of the candidate region within a preset frame window. This feature is extracted by saving the area values ​​of the candidate regions in the most recent K frames, forming a time series of length K, and then calculating the variance or standard deviation of this series as the area fluctuation feature. The value of K can be set according to actual needs, for example, K=5 or K=10. When a flame is present, its area will fluctuate up and down over several consecutive frames, with a large variance; while when there are objects of a fixed color or slowly changing lighting in the image, the area fluctuation is smaller.

[0049] The fourth type is frequency domain features. Frequency domain features are frequency parameters obtained by performing a Fast Fourier Transform (FFT) on the area value sequence of candidate regions within a preset frame window. These parameters describe the periodic fluctuations in the area of ​​the candidate regions. In a specific implementation, the system saves the area values ​​of the candidate regions corresponding to the most recent K frames, forming a time series A1, A2, ..., A_K of length K. The value of K can be set according to the actual application scenario and the flickering frequency of the flame, and is generally an integer between 5 and 15. When the time series reaches the preset length, the system calls the Fast Fourier Transform algorithm deployed on the edge to transform the area value sequence from the time domain to the frequency domain.

[0050] In an optional embodiment, the FFT operation employs fixed-point arithmetic and table lookup to simplify the calculation process. The rotation factor is pre-calculated and stored in a table to avoid redundant calculations of trigonometric functions. Simultaneously, floating-point operations are converted to integer fixed-point operations to reduce processor overhead. On a microcontroller, the transformation of hundreds of sampling points can be completed within hundreds of milliseconds, meeting real-time requirements. After transformation, a spectral distribution is obtained. Based on this spectral distribution, the system extracts two key parameters as frequency domain features: first, the dominant frequency component, i.e., the frequency value with the largest amplitude in the spectrum, which reflects the most significant oscillation frequency of the area change in the candidate region; second, the proportion of spectral energy of the area value sequence within the preset flame flicker frequency band. The system calculates the sum of the energy of all frequency components of the area value sequence within the preset flame flicker frequency band and divides it by the total energy of the entire spectrum to obtain this ratio. This ratio reflects the concentration of energy of the area change in the candidate region within the flame characteristic frequency band. The preset flame flicker frequency band is a frequency range pre-defined based on the physical characteristics of the flame. Real flames produce unique flickering phenomena due to turbulent combustion effects, and their flicker frequencies are typically distributed in the range of, for example, 1~20Hz. The system uses this range as the preset flame flicker frequency band.

[0051] After extracting the aforementioned dynamic features, the system determines whether the model call triggering conditions are met based on these features. The specific method for determining the triggering conditions can be preset by those skilled in the art according to the actual scenario. For example, the triggering conditions can be considered met when the area of ​​the candidate region is greater than a preset area threshold, the inter-frame grayscale change feature value within the candidate region is greater than a preset change threshold, and the area fluctuation feature value of the candidate region is greater than a preset fluctuation threshold. These thresholds can be adjusted based on factors such as the resolution of the image acquisition node, the distance between the camera and the monitored target, and ambient lighting conditions. In one implementation, all three conditions must be met simultaneously to trigger a model call; in another implementation, a combination of multiple conditions can be set, which can be flexibly adjusted by those skilled in the art according to actual application needs. It should be noted that the triggering condition determination in step S120 is based on multiple consecutive frames of images; that is, a trigger is only determined when the preset conditions are met in several consecutive frames, in order to reduce unnecessary model calls caused by accidental noise in a single frame.

[0052] In step S130, after determining that the model invocation trigger condition is met, step S130 invokes the flame detection model deployed on the edge to confirm the flame target in the current image. The flame detection model is a pre-trained lightweight deep learning model whose function is to analyze the input image and determine whether a flame target exists in the image. This model can be a target detection model or an image classification model. The target detection model can not only output whether a flame exists in the image, but also output the position and range of the flame target in the image, presented in the form of a detection box. The image classification model only outputs the probability of whether the overall image contains a flame target, and its computational load is smaller. In implementations that require subsequent binocular localization, the target detection model is preferred to extract the center point coordinates of the flame target from the detection box. To adapt to the limited computing resources of the edge device, the model undergoes lightweight processing such as quantization, pruning, or knowledge distillation, and can be deployed on microcontrollers or low-power edge devices. For example, the model can be converted into a format suitable for microcontroller platforms using the TensorFlow Lite Micro framework.

[0053] After the flame detection model completes inference calculations on the current frame image, it outputs the detection results. The results include information determining the presence of a flame target, and, if confirmed, further determining the center point of the flame target in the image coordinate system. Specifically, if the model outputs a bounding box, the output detection results include the coordinates (x, y) of the top-left corner of the bounding box, as well as its width w and height h. The system calculates the center point coordinates based on this bounding box: the horizontal coordinate is x + w / 2, and the vertical coordinate is y + h / 2. This center point is the representative point of the flame target on the image plane, used for subsequent binocular localization calculations. To improve accuracy, when multiple bounding boxes exist, the system can select the bounding box with the highest confidence level based on the detection confidence, or perform non-maximum suppression on multiple bounding boxes and select the optimal box.

[0054] The coordinates of the detection box output by the flame detection model are based on the scale of the model's input image, while the camera calibration parameters used in the subsequent binocular localization steps are based on the original image scale or a specific calibration scale. Therefore, scale mapping is required when transferring the center point coordinates. For example, the system obtains the width and height of the model's input image, as well as the width and height of the image used for camera calibration, calculates the width scaling factor and height scaling factor respectively, and then multiplies the x-coordinate of the center point at the model scale by the width scaling factor and the y-coordinate by the height scaling factor to obtain the center point coordinates at the calibration scale. After the above processing, the left and right eye images respectively yield the left eye center point in the left eye image and the right eye center point in the right eye image.

[0055] In one implementation, the left-eye image is processed by a first image acquisition node, and the right-eye image is processed by a second image acquisition node. After each node completes the flame detection model inference, they send their respective center point coordinates to the master node for aggregation. In another implementation, the left and right images are sent to a single processing module, which then completes the model inference for the two images sequentially or in parallel.

[0056] In step S140, after obtaining the center points of the left and right eyes, step S140 calculates the parallax using the principle of binocular vision, and then solves the three-dimensional spatial coordinates of the fire source relative to the monitoring equipment. The principle of binocular vision is similar to human stereoscopic perception. Because there is a fixed physical distance between the left and right cameras (i.e., the binocular baseline), the imaging position of the same spatial target point in the left and right images differs in the horizontal direction. This difference is called parallax. The magnitude of parallax is inversely proportional to the distance from the target point to the camera: the closer the target, the greater the parallax; the farther the target, the smaller the parallax.

[0057] In the specific calculation, the system first obtains the x-coordinate uL of the left eye center point and the x-coordinate uR of the right eye center point, and then calculates the disparity d = uL - uR. It should be noted that in a standard binocular vision configuration, the optical axes of the left and right eye cameras are parallel and their imaging planes are coplanar; therefore, the y-coordinates of the same target point in the left and right images should be approximately equal. In practical applications, due to installation errors or lens distortion, there may be slight deviations in the y-coordinates of the left and right center points. Therefore, before calculating the disparity, the left and right center points can be vertically epipolar corrected to ensure they are on the same horizontal line, or disparity calculation can be performed only when the difference in the y-coordinates of the left and right center points is less than a preset threshold, to improve the reliability of the matching.

[0058] After obtaining the parallax, the three-dimensional spatial coordinates of the fire source can be calculated by combining the intrinsic parameters and baseline parameters obtained through pre-calibration of the cameras. Camera calibration is an operation performed after system installation or periodically. By taking images of the calibration board and using the Zhang Zhengyou calibration method or other calibration algorithms, the intrinsic parameter matrices, distortion parameters, and extrinsic parameters (rotation matrix and translation vector) between the left and right cameras can be obtained. For binocular localization, at least the following parameters are required: equivalent focal length fx in the horizontal direction, equivalent focal length fy in the vertical direction, principal point coordinates (cx, cy), and binocular baseline length B.

[0059] Based on the above parameters, the formula for calculating depth Z (i.e., the vertical distance from the fire source to the camera plane) is: Z = (fx × B) / d. The larger the parallax d, the smaller the depth Z, indicating the closer the fire source; conversely, the smaller the parallax d, the farther away the fire source. After obtaining depth Z, the offset of the fire source in the horizontal and vertical directions is calculated based on the horizontal coordinate uL and vertical coordinate vL of the left eye's center point, combined with the principal point coordinates (cx, cy) and focal length parameters. Specifically, taking the optical center of the left eye camera as the origin, the direction of the optical axis as the positive Z-axis, the horizontal direction to the right as the positive X-axis, and the vertical direction downwards as the positive Y-axis (or vertically upwards, depending on the coordinate system definition), the three-dimensional coordinates (X, Y, Z) of the fire source can be calculated using similar triangle relationships. In engineering, these coordinate values ​​can be transformed according to the actual scenario, for example, converted to absolute coordinates with the ground as a reference or relative coordinates with the inspection platform as the origin. The calculated three-dimensional coordinates will be output as an important component of subsequent early warning information.

[0060] In step S150, after confirming the flame target and calculating the three-dimensional coordinates of the fire source, step S150 generates an early warning message based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source. The flame target confirmation result includes the determination information on the existence of a flame target output by the model in step S130, as well as auxiliary information such as detection confidence. The three-dimensional spatial coordinates of the fire source provide the specific location and distance of the fire source relative to the monitoring equipment. The early warning message is a structured message generated by integrating the above confirmation results and coordinate information, used to notify relevant personnel or systems.

[0061] In one implementation, if the flame target confirmation result indicates that no flame target exists, a warning message indicating no anomaly is generated. This message may include a flag indicating that the current status is normal, for the cloud or local monitoring system to record the status. If the flame target confirmation result indicates that a flame target exists, the generated warning message includes at least the following: the three-dimensional spatial coordinates of the fire source, the detection confidence level, and the warning level or warning status indicator. The warning level can be graded according to the confidence level and the distance to the fire source reflected by the three-dimensional coordinates. For example, a closer distance and higher confidence level can be set as an emergency warning, while a farther distance or lower confidence level can be set as a general warning.

[0062] In the implementation of the above embodiments: by extracting dynamic features from multiple consecutive frames of images and making rapid judgments before calling the flame detection model, and only calling the model for confirmation when the triggering conditions are met, the number of invalid model inferences in flameless scenarios can be reduced, thereby reducing the computational load and power consumption of the edge device and improving processing efficiency. Simultaneously, after confirming the flame target, this method directly uses the principle of binocular vision to calculate the three-dimensional spatial coordinates of the fire source, so that the warning information not only includes the judgment of whether a flame exists, but also provides the specific direction and distance of the fire source relative to the monitoring device, providing richer reference information for emergency response. Furthermore, all the above steps are completed on the edge, eliminating the need to continuously upload the original images to the cloud, reducing network bandwidth usage and data transmission latency, and making the warning response more timely.

[0063] Optionally, in this embodiment of the application, the preset flame color condition includes at least one of the following: the intensity of the red channel of the pixel is higher than the intensity of the green channel and the intensity of the blue channel, the brightness of the pixel is greater than a preset brightness threshold, and the hue of the pixel is within the preset flame hue range.

[0064] A preset flame color condition is used to filter pixels in an image that are similar in color to flame features, forming candidate regions. In one implementation, the preset flame color condition includes a pixel whose red channel intensity is higher than its green and blue channel intensity. This condition is based on the fact that flames are typically dominated by red and orange in the visible light spectrum, and the response value of the red channel is often higher than that of the green and blue channels. By comparing the values ​​of the three RGB channels, a large number of objects and background areas with cool tones such as blue or cyan can be quickly eliminated, retaining pixels that may be flames. In a specific implementation, this condition can be expressed as R>G and R>B, where R, G, and B represent the grayscale values ​​of the pixel in the red, green, and blue channels, respectively. To further improve the accuracy of the filtering, additional constraints can be added to the R>G and R>B condition, such as requiring the difference between R and G or the difference between R and B to be greater than a certain minimum difference threshold, to avoid misjudgments caused by noise or similar colors.

[0065] The preset flame color condition also includes ensuring that pixel brightness exceeds a preset brightness threshold. Flames typically have high brightness, while dark red objects or red light in low-light environments, although potentially dominant in the red channel, have overall low brightness. Therefore, setting a brightness threshold can exclude these low-brightness non-flame pixels. In the RGB color space, brightness can be represented by the weighted sum or average of the R, G, and B channels. For example, the standard grayscale conversion formula Gray = 0.299 × R + 0.587 × G + 0.114 × B can be used to calculate the pixel brightness value. This value is then compared to the preset brightness threshold, retaining only pixels whose brightness exceeds the threshold. The preset brightness threshold can be adjusted according to actual ambient lighting conditions. The threshold can be increased during the day or in strong light, and decreased at night or in low light, to effectively extract flame candidate regions under different conditions.

[0066] The preset flame color condition also includes ensuring that the pixel's hue falls within the preset flame hue range. This condition is based on the HSV (Hue, Saturation, Brightness) color space because the hue components in the HSV space are more consistent with human visual perception of color, allowing for more accurate differentiation of different colors. In the HSV space, the hue of a flame typically lies between 0° and 60°, representing the transition from red to yellow. A preset lower and upper hue limit is set to constitute the preset flame hue range; for example, the lower limit could be set to 0° and the upper limit to 60°. Then, each pixel's hue value is checked individually to see if it falls within this range. These three color conditions can be used individually or in any combination. In practical applications, using all three conditions together achieves the best filtering effect: a pixel is only marked as a candidate pixel if it simultaneously satisfies the following conditions: red channel dominance, brightness above the threshold, and hue within the flame range. This minimizes false positives caused by similar object colors.

[0067] The model call triggering conditions include at least one of the following: the area of ​​the candidate region exceeds a first threshold, the inter-frame grayscale change feature value of the candidate region exceeds a second threshold, the area fluctuation feature value of the candidate region exceeds a third threshold, and frequency domain feature conditions; the frequency domain feature conditions are that the main frequency component falls into the preset flame flashing frequency band range, and the spectral energy ratio exceeds the preset energy threshold, and the main frequency component does not fall into the preset interference frequency band range.

[0068] The model invocation trigger condition is used to determine whether the current image needs to be confirmed by invoking the flame detection model. The trigger condition includes the candidate region area exceeding a first threshold. The candidate region area refers to the total number of pixels marked as candidate regions in the binary mask generated in step S120. When the candidate region area is small, it may correspond to tiny bright spots or noise in the image, insufficient to constitute a flame target; therefore, model invocation is only considered when the area exceeds the first threshold. The size of the first threshold can be set according to the image resolution. For example, for a 320×240 pixel image, the first threshold can be set to a value between 50 and 200, indicating that the candidate region must contain at least 50 pixels to be considered a valid candidate.

[0069] The model call triggering condition also includes the inter-frame grayscale change feature value within the candidate region exceeding a second threshold. The inter-frame grayscale change feature value reflects the degree of grayscale change of pixels within the candidate region between adjacent frames. When this feature value exceeds the second threshold, it indicates significant motion within the candidate region, consistent with the characteristics of flickering flames; conversely, a low feature value indicates a relatively static scene within the candidate region, possibly a static red or yellow object (such as a red wall or orange sign) or a stable light source. The setting of the second threshold is related to the frame rate of image acquisition and the dynamic range of the scene. At a higher frame rate, the changes between adjacent frames are smaller, so the second threshold can be set at a lower level; at a lower frame rate, the changes between adjacent frames are larger, requiring an appropriate increase in the second threshold.

[0070] The model invocation trigger condition also includes the area fluctuation feature value of the candidate region exceeding the third threshold. The area fluctuation feature reflects the degree of area change of the candidate region across multiple consecutive frames. During actual combustion, the shape and size of a flame fluctuate irregularly, resulting in a large variance or standard deviation of its area; static or slowly moving objects do not exhibit this characteristic. The third threshold is used to determine whether this fluctuation meets the condition; only when the area fluctuation feature value exceeds the third threshold is the candidate region in the current image considered to possess the dynamic behavior unique to flames.

[0071] The model call triggering conditions also include frequency domain feature conditions, which include three sub-conditions: the main frequency component falls within the preset flame flashing frequency band, the spectral energy ratio exceeds the preset energy threshold, and the main frequency component does not fall within the preset interference frequency band. All three sub-conditions must be satisfied simultaneously for the frequency domain feature conditions to be valid.

[0072] Among these criteria, the dominance frequency component falling within the preset flame flicker frequency band is a positive confirmation that the candidate region possesses a flame flicker frequency; the spectral energy proportion exceeding a preset energy threshold verifies the reliability of this frequency domain characteristic, preventing spurious frequency peaks caused by random noise from triggering model calls; this condition is only valid when the area change energy is sufficiently concentrated within the flame frequency band; and the dominance frequency component not falling within the preset interference frequency band is a reverse exclusion of common electric light source interference in the environment. The preset interference frequency band is a frequency range pre-defined based on the frequency characteristics of typical interference sources in the environment. The dominance frequency components of interference sources such as lights and reflections either approach 0Hz to represent static or slow changes, or are located near the power frequency of 50Hz or its harmonics to represent periodic flickering of electric light sources, which are clearly distinguishable from the flame flicker frequency band. The system sets these frequency ranges as the preset interference frequency band. Through the coordinated determination of the above three sub-conditions, the frequency domain characteristic condition can effectively distinguish between real flames and interference sources with different frequency domain characteristics, such as electric light sources.

[0073] The four triggering conditions mentioned above are determined using a logical "AND" relationship. That is, the model invocation triggering condition is met only when all four conditions are simultaneously satisfied: the area of ​​the candidate region exceeds the first threshold, the inter-frame grayscale change feature value exceeds the second threshold, and the area fluctuation feature value exceeds the third threshold. This simultaneous satisfaction of all four conditions effectively eliminates interference from static objects, stable light sources, and random noise. Only when the image sequence consistently exhibits the unique color, dynamic, and fluctuating characteristics of a flame will the subsequent computationally intensive flame detection model be triggered, thereby reducing unnecessary computational overhead on the edge device. Alternatively, the four triggering conditions can also be determined using a logical "OR" relationship. That is, the model invocation triggering condition is met only when any one of the following conditions is satisfied: the area of ​​the candidate region exceeds the first threshold, the inter-frame grayscale change feature value exceeds the second threshold, the area fluctuation feature value exceeds the third threshold, or the frequency domain feature condition is satisfied, thus triggering the subsequent invocation of the flame detection model.

[0074] It should be noted that within the frequency domain characteristic conditions, the three sub-conditions (the main frequency component falls within the preset flame flicker frequency band, the spectral energy ratio exceeds the preset energy threshold, and the main frequency component does not fall within the preset interference frequency band) adopt a logical 'AND' relationship, that is, the frequency domain characteristic conditions are only valid if all three conditions are met simultaneously, in order to eliminate random noise and electric light source interference and ensure the reliability of frequency domain determination.

[0075] In the implementation of the above embodiments: by setting preset flame color conditions and model call trigger conditions as described above, each frame of image can be quickly filtered using three dynamic features before the flame detection model is invoked. Since the extraction of these three features involves only low-overhead operations such as color comparison, inter-frame differencing, and area statistics, and does not involve complex calculations such as convolution and activation of deep learning models, the computational load of the filtering process is very small, and the processing speed is very fast. In actual operation, the vast majority of image sequences without flames can be excluded in this filtering stage, and only a small number of candidate frames with flame features will trigger model invocation. This pre-screening mechanism effectively reduces the number of invalid inferences by the flame detection model, allowing the computing resources of the edge device to be more concentrated on image frames that truly pose a flame risk, thereby improving overall processing efficiency.

[0076] In one implementation, the dynamic characteristics also include frequency domain characteristics. During combustion, flames exhibit a unique flickering phenomenon due to turbulence, with their radiation intensity displaying a fluctuating, periodic variation. The flickering frequency of a flame typically ranges from 1 to 20 Hz, with the main energy concentrated between 3 and 25 Hz, and this frequency characteristic is independent of flame size and observation distance. Based on this physical characteristic, during the dynamic pre-screening stage, the system saves the area value sequence of candidate regions within a preset frame window as a time series. When this time series reaches a preset length, the system calls the Fast Fourier Transform (FFT) algorithm deployed on the edge to transform the area value sequence of candidate regions within the preset frame window from the time domain to the frequency domain. The FFT operation can use fixed-point arithmetic and lookup table methods to simplify the calculation process, and on a microcontroller, the transformation of hundreds of sampling points can be completed within hundreds of milliseconds, meeting real-time requirements. After the transformation, a spectral distribution is obtained. The system extracts the dominant frequency component, i.e., the frequency value with the largest amplitude in the spectrum, and determines whether this dominant frequency component falls within the preset flame flickering frequency band. The dominant frequency component of a real flame typically lies within the 1-20Hz range, while the dominant frequency components of interference sources such as lights and reflections are either close to 0Hz (static or slowly changing) or located near the 50Hz power frequency or its harmonics (electric light sources), making them clearly distinguishable from the flickering frequency band of flames. Furthermore, the system can extract the spectral energy percentage of the area value sequence within a preset frequency band. When the spectral energy percentage exceeds a preset energy threshold, it serves as a trigger condition for model invocation, further improving the accuracy of the screening. By introducing frequency domain features, dynamic pre-screening comprehensively judges whether candidate regions possess the unique dynamic behavior patterns of flames from both time and frequency domain dimensions, effectively distinguishing flames from interference sources with similar time-domain variation characteristics but different frequency-domain characteristics.

[0077] Optionally, in this embodiment, the inter-frame grayscale change feature is determined based on the grayscale difference between the corresponding pixels in the candidate region between the current frame and the previous frame; the area fluctuation feature is determined based on the area variance or standard deviation of the candidate region within a preset frame window.

[0078] Inter-frame grayscale variation features are used to measure the degree of dynamic change of candidate regions in an image sequence. The calculation method is as follows: First, convert the current frame and the previous frame into grayscale images. Then, for the pixel positions covered by the candidate region in the current frame, calculate the absolute value of the grayscale difference of the corresponding pixels in adjacent frames one by one. Finally, compare the absolute values ​​of these grayscale differences with a preset difference threshold and count the number or proportion of pixels that exceed the threshold.

[0079] In its implementation, the system first acquires the current frame image It and the previous frame image It-1, and uses the standard grayscale conversion formula to convert the two color images into grayscale images Gt and Gt-1. Then, it acquires the binary mask Ct of the candidate region in the current frame; the pixel positions marked as 1 in this mask represent the range covered by the candidate region. The system iterates through each pixel (x, y) within the candidate region, calculating the absolute value of the grayscale difference Dt(x, y) = |Gt(x, y) - Gt-1(x, y)|, and then compares Dt(x, y) with a preset difference threshold TD. The value of the difference threshold TD is related to the frame rate of image acquisition, the amplitude of ambient light variation, and the noise level of the image sensor. At a high frame rate (e.g., 30 frames per second), the normal illumination variation between adjacent frames is small, and TD can be set in the range of 10 to 30 (grayscale value range of 0 to 255). At a low frame rate, the variation between adjacent frames is relatively large, and the value of TD can be appropriately increased. The number of pixels with Dt(x,y) greater than TD within the candidate region is counted to obtain the total number of changed pixels. This total number is then divided by the area of ​​the candidate region (i.e., the total number of pixels within the candidate region) to obtain the proportion of changed pixels, Mt. This proportion is the inter-frame grayscale change feature value. When the grayscale values ​​of pixels within the candidate region change drastically, it indicates that there is obvious motion or flicker in that region, which is consistent with the dynamic characteristics of a flame. Conversely, when the proportion of change is low, it indicates that the image within the candidate region is relatively static, possibly a static red or yellow object or a region with stable lighting, and is not considered a valid flame candidate. Since the inter-frame grayscale difference calculation only involves subtraction and comparison operations, it does not require complex motion estimation or optical flow calculations, resulting in minimal computational overhead, making it suitable for operation on resource-constrained edge devices.

[0080] Area fluctuation features are used to measure the degree of area change of candidate regions across multiple consecutive frames of images. In practice, the system maintains a queue of area values ​​of length K, which stores the area values ​​of candidate regions corresponding to the most recent K frames, denoted as A1, A2, ..., AK, where K is a preset frame window size. The value of K can be set according to the actual application scenario and the flickering frequency of the flame, generally taking an integer between 5 and 15. When the value of K is small, the area fluctuation feature is more sensitive to short-term changes; when the value of K is large, the area fluctuation feature is smoother and more stable, with stronger anti-interference capabilities. In one implementation, K is set to 10, meaning it stores the area values ​​of candidate regions from the most recent 10 frames. After each frame is processed, the system updates the queue with the area values ​​of the candidate regions in the current frame, removing the oldest frame and adding the latest frame, then recalculates the variance or standard deviation of the area values ​​in the queue. The variance is calculated as σ² = (1 / K)×Σ(Ai - μ)², where μ is the arithmetic mean of the area values ​​of K frames; the standard deviation is the positive square root of the variance. Both variance and standard deviation effectively reflect the dispersion of a set of data. The larger the value, the more significant the fluctuation of the candidate region area between consecutive frames; the smaller the value, the more stable the area tends to be. During combustion, the shape of a flame constantly changes, resulting in a continuous pulsation of its area, thus exhibiting a large area fluctuation characteristic value. In contrast, while lights, reflective areas, or static red-yellow objects may resemble flames in color, their area remains essentially unchanged across multiple frames, resulting in a smaller area fluctuation characteristic value. The area fluctuation characteristic can effectively distinguish between dynamic flame targets and static or quasi-static interference. In step S120, the area fluctuation characteristic value is compared with an area threshold. When the area fluctuation is sufficiently large, the current frame is considered a candidate frame with dynamic flame characteristics, further triggering model invocation for confirmation.

[0081] The first, second, and third thresholds mentioned above can be adaptively adjusted and set according to factors such as camera exposure parameters, distance between the camera and the monitoring target, minimum size of the flame to be detected, and ambient lighting conditions in the actual monitoring scenario.

[0082] In the implementation of the above embodiments: the gray-level difference between adjacent frames and the area statistics within consecutive frame windows are used to determine the inter-frame gray-level change features and area fluctuation features, respectively. The extraction of these two types of features does not involve complex mathematical models or a large number of multiplication and addition operations. The calculation process is simple and fast, meeting the real-time image sequence processing requirements of the edge microcontroller. By combining the inter-frame gray-level change features with the area fluctuation features, it is possible to comprehensively judge whether the candidate region possesses the unique behavior pattern of a flame from two different dimensions: dynamic change and morphological pulsation. Compared with the screening method that relies solely on a single static color feature, it can effectively eliminate interference from static objects and slowly changing light sources, making the pre-screening judgment results more reliable, reducing invalid calls to the subsequent flame detection model, and thus improving overall processing efficiency.

[0083] Optionally, in this embodiment of the application, determining the left center point of the flame target in the left eye image and the right center point in the right eye image includes: Obtain the coordinates of the top left corner, width, and height of the detection box output by the flame detection model.

[0084] After the flame detection model completes inference on the current frame image and confirms the presence of a flame target, it outputs the position and size information of the detection box. The detection box is a rectangular bounding box that the model marks the area where the flame target is located in the image. It is represented by the coordinates of its upper-left corner (x, y), width w, and height h. The upper-left corner coordinates are taken as the origin of the upper-left corner of the image, with the horizontal axis pointing to the right and the vertical axis pointing downwards. When the model outputs multiple candidate detection boxes, the system first filters out detection boxes belonging to the flame category based on the category label. Then, it sorts them from highest to lowest confidence score. For multiple overlapping detection boxes that may exist in the same image, non-maximum suppression is performed, i.e., the detection box with the highest confidence score is retained, and other detection boxes with an intersection-union ratio (IoU) greater than a preset threshold are removed. Finally, the detection box with the highest confidence score is selected as the final output from the remaining detection boxes, and its upper-left corner coordinates (x, y), width w, and height h are obtained.

[0085] In one implementation, if multiple detection boxes with high confidence are retained after non-maximum suppression, the center point coordinates of each detection box can be calculated separately, and then a weighted average can be performed using confidence as the weight to obtain the overall center point position. The system calculates the center point coordinates of the detection box based on the coordinates of its upper left corner (x, y), width w, and height h. The x-coordinate of the center point is u = x + w / 2, and the y-coordinate is v = y + h / 2. This center point serves as the representative point of the flame target on the image plane and is used for subsequent binocular localization calculations.

[0086] The center point coordinates of the detection box are calculated based on the coordinates of the top-left corner, width, and height. The center point coordinates are then mapped to image coordinates at the calibration scale, according to the ratio between the input scale of the flame detection model and the camera calibration scale.

[0087] Because flame detection models typically scale the original acquired images during inference to fit the model's fixed input size, and the camera calibration parameters used in subsequent binocular localization steps are based on the original image resolution or a certain calibration scale, it is necessary to map the center point coordinates from the model input scale to the calibration scale. For example, the system acquires the width W_model and height H_model of the model input image, and the width W_calib and height H_calib of the calibration image used for camera calibration. It then calculates the horizontal scaling factor scale_x = W_calib / W_model and the vertical scaling factor scale_y = H_calib / H_model, respectively. Finally, it multiplies the center point's x-coordinate u by scale_x and its y-coordinate v by scale_y to obtain the center point coordinates (u_calib, v_calib) mapped to the calibration scale. After this scale mapping, the image coordinate system containing the center point coordinates is consistent with the image coordinate system corresponding to the camera calibration parameters, ensuring that the camera intrinsic parameters used in subsequent disparity calculations and 3D coordinate solutions can match it. The left and right eye images undergo the same processing described above to obtain the coordinates of the left and right eye center points at the calibration scale, respectively, providing accurate target location input for subsequent binocular localization steps.

[0088] In the implementation of the above embodiments: by extracting the center point coordinates from the detection box output by the flame detection model, the position of the flame target in the image is quantified into a pair of specific coordinate values, providing a clear target matching point for subsequent binocular disparity calculation. Simultaneously, by mapping the center point coordinates at the model input scale to the camera calibration scale, the coordinate deviation caused by the inconsistency between the model input size and the calibration image size is reduced. This improves the scale matching between the coordinate data used in binocular positioning calculation and calibration parameters such as camera intrinsics and baselines, thereby improving the accuracy and reliability of subsequent three-dimensional coordinate calculation of the fire source.

[0089] Figure 2 This is a geometrical schematic diagram of binocular fire source positioning provided in an embodiment of this application.

[0090] Optionally, in this embodiment of the application, calculating the parallax based on the left and right eye center points and solving for the three-dimensional spatial coordinates of the fire source includes: Calculate the disparity based on the x-coordinates of the left and right eye center points.

[0091] After obtaining the center points of the left and right eyes, the disparity is calculated using the principle of binocular vision, thereby resolving the three-dimensional spatial coordinates of the fire source. First, the disparity is calculated based on the x-coordinates of the left and right eye center points. The principle of binocular vision is that, due to the fixed physical distance between the left and right cameras, the imaging position of the same spatial target point differs horizontally in the two images; this difference is called disparity. After epipolar correction of the binocular cameras, the y-coordinates of the same target point are the same in the left and right images, and the disparity is only expressed as the difference in the x-coordinates. The system obtains the x-coordinates uL of the left eye center point and uR of the right eye center point determined in step S130, and calculates the disparity d = uL - uR. In the ideal case of epipolar correction, the y-coordinates of the left and right center points are equal. If there is a slight deviation, the left and right center points can be vertically aligned before calculating the disparity, or the disparity can be calculated only when the difference in y-coordinates is less than a preset threshold. The larger the disparity, the closer the target is to the camera; the smaller the disparity, the farther the target is from the camera.

[0092] As one implementation method, before calculating disparity, the validity of the left and right center points needs to be determined. The determination criteria may include that both the left and right center points are within the valid range of the image, the confidence scores of both left and right detections exceed a preset confidence threshold, the data sent by the first image acquisition node has not timed out, the difference in the ordinates of the left and right center points does not exceed a preset range, and the absolute value of the disparity is not less than a preset minimum disparity threshold. If one or more of the above conditions are not met, the binocular matching of the current frame is deemed invalid, and three-dimensional coordinate calculation is not performed.

[0093] Depth is determined based on parallax and preset camera calibration parameters.

[0094] After obtaining the parallax, the depth is determined based on the parallax and preset camera calibration parameters. The camera calibration parameters are obtained through a calibration process after system installation and include the camera's intrinsic and extrinsic parameters. Parameters directly related to depth calculation include the equivalent focal length fx in the horizontal direction and the binocular baseline length B. The equivalent focal length fx reflects the lens's magnification in the horizontal direction; its value can be obtained through calibration, and the unit is usually pixels. The binocular baseline length B is the physical distance between the optical centers of the left and right cameras, usually in millimeters. The formula for calculating depth Z is Z = (fx × B) / d, which physically represents the vertical distance from the fire source to the binocular camera baseline plane, with the same unit as B. When the parallax d is 0, it indicates that the target is at infinity, and the depth approaches infinity; this situation usually does not occur in real-world scenarios. If the parallax is negative, it indicates an error in matching the left and right center points; in this case, the positioning result of the current frame can be determined to be invalid, and subsequent spatial coordinate calculations should not be performed.

[0095] The horizontal and vertical coordinates of the fire source in three-dimensional space are calculated based on the coordinates of the left eye center point, depth, and camera calibration parameters.

[0096] After obtaining the depth, the horizontal and vertical coordinates of the fire source in three-dimensional space are calculated based on the coordinates of the left eye center point, the depth, and the camera calibration parameters. Here, a three-dimensional coordinate system is established with the optical center of the left eye camera as the origin, the Z-axis pointing towards the target, the X-axis horizontally to the right, and the Y-axis vertically downwards. The coordinates of the left eye center point include the horizontal coordinate uL and the vertical coordinate vL. The camera calibration parameters also include the principal point coordinates (cx, cy) and the equivalent focal length fy in the vertical direction, where the principal point coordinates are the intersection of the optical axis and the imaging plane.

[0097] The formula for calculating the horizontal coordinate X is X = (uL - cx) × Z / fx. This formula utilizes the principle of similar triangles, calculating the physical offset of the fire source in the horizontal direction based on the offset of the center point's x-coordinate relative to the principal point's x-coordinate and the depth value. The formula for calculating the vertical coordinate Y is Y = (vL - cy) × Z / fy, and the physical offset of the fire source in the vertical direction is calculated similarly.

[0098] In one implementation, the coordinate system can be defined with the vertically upward direction as positive. In this case, the formula for calculating the vertical coordinates needs to be adjusted accordingly to Y = (cy - vL) × Z / fy. After the above calculation, the three-dimensional spatial coordinates (X, Y, Z) of the fire source relative to the optical center of the left eye camera can be obtained. If other positions are needed as reference points, such as the midpoint of the binocular camera baseline or the optical center of the first image acquisition node, the three-dimensional coordinates can be further transformed according to the translation vector between the left eye camera and the reference point. In addition, in one implementation, in order to improve the robustness of positioning, the three-dimensional coordinates of the fire source can be calculated separately in multiple consecutive frames, and then the coordinates of multiple frames can be filtered, for example, by using mean filtering or Kalman filtering, to smooth the coordinate values ​​and reduce the impact of single-frame positioning errors.

[0099] In the implementation of the above embodiment: disparity is calculated by the difference in the horizontal coordinates of the center points of the left and right eyes, and depth is determined based on the disparity and camera calibration parameters. This allows for the calculation of the horizontal and vertical coordinates of the fire source in three-dimensional space. This process directly transforms the flame detection results into specific spatial location information, enabling early warning information to go beyond simple qualitative judgments of "fire" or "no fire," and instead provide the specific direction and distance of the fire source, providing a reference for on-site personnel to quickly determine its location. Simultaneously, the binocular positioning calculation is completed on the device side, eliminating the need to upload image data to the cloud for processing, reducing data transmission latency, and allowing the positioning results to be obtained in real time, thus improving the overall response speed.

[0100] In one implementation, after calculating the three-dimensional spatial coordinates of the fire source, a step of predicting the fire source's movement trend is included: a time series of fire source coordinates is constructed by continuously recording the three-dimensional spatial coordinates of the fire source output by the binocular positioning module over multiple consecutive frames, and the sequence is updated each time a new positioning result is obtained. Based on this time series of fire source coordinates, the system predicts the predicted three-dimensional spatial coordinates of the fire source at the next moment or several subsequent moments using a preset temporal prediction model. The temporal prediction model can be a lightweight model deployed on the edge, such as a Kalman filter or a lightweight long short-term memory network (LSTM). The Kalman filter optimally estimates the motion state of the fire source through a recursive approach, extracting the true trajectory of the fire source from the noisy positioning results and predicting its position at the next moment. The lightweight LSTM model is suitable for capturing the nonlinear change patterns of the fire source's motion; after quantization, it can be deployed on the edge microcontroller, controlling inference latency and memory usage while ensuring prediction accuracy. After obtaining the predicted three-dimensional spatial coordinates, the system determines the direction and speed of the fire source's movement based on the predicted coordinates at multiple consecutive time points. The direction of movement can be determined by the displacement vector of the predicted coordinates relative to the current coordinates, while the speed is calculated by dividing the displacement by the time interval. The generated warning information accordingly includes one or more of the predicted three-dimensional spatial coordinates, direction of movement, and speed. Furthermore, when the predicted three-dimensional spatial coordinates indicate that the fire source is moving towards a preset sensitive area (such as important equipment, escape routes, or flammable material storage areas), or when the speed exceeds a preset speed threshold, the system generates a spread warning, indicating that the fire is rapidly approaching the sensitive area. This extension upgrades the warning information from a static location description to dynamic situational awareness, providing more comprehensive reference for emergency response decisions.

[0101] Optionally, in this embodiment, the flame detection model is a lightweight target detection model or image classification model deployed on the edge.

[0102] "Edge device" refers to the embedded or edge device located close to the image acquisition source. In this application, it refers to the microcontroller or low-power edge processor used by the first and second image acquisition nodes, such as the ESP32 chip. Since the processor frequency of edge devices is typically between tens and hundreds of MHz, and their RAM is only a few hundred KB to several MB, they cannot directly run standard object detection networks. Therefore, lightweight models are required. Lightweight processing refers to reducing the number of model parameters and computational complexity while maintaining model accuracy through techniques such as model quantization, pruning, or knowledge distillation, enabling the model to run on edge devices. Model quantization converts model weights from 32-bit floating-point numbers to 8-bit integers, thereby reducing model storage space and improving inference speed; pruning removes less contributing neuron connections from the model, sparsening the network structure; knowledge distillation uses a large model to guide the training of a smaller model, enabling the smaller model to learn the discriminative capabilities of the larger model. The above lightweight processing methods can be used individually or in combination depending on the resource constraints of the actual deployment platform.

[0103] Regarding specific model selection, the flame detection model can employ either a lightweight object detection model or a lightweight image classification model. When using an object detection model, the model takes the entire image as input, performs feature extraction and target localization via a convolutional neural network, and outputs the category labels, bounding box positions, and detection confidence scores for all flame targets in the image. Since the object detection model can directly output bounding boxes, the center point coordinates of the flame targets can be easily extracted from the bounding boxes, providing location information for subsequent binocular localization steps. Commonly used lightweight object detection models include single-stage detectors based on lightweight backbone networks such as MobileNet and EfficientNet-Lite. These models, after quantization, can be deployed on microcontroller platforms. When using an image classification model, the model only outputs the probability of whether the entire image contains a flame target, and its computational cost is typically lower than that of the object detection model. Since the classification model itself does not output target location information, in implementations using the classification model, the system needs to combine the color candidate regions extracted in step S120 to determine the location of the flame targets. Specifically, the classification model can crop the candidate regions extracted in step S120, input the cropped sub-image into the classification model for judgment, and if the probability of the flame category output by the classification model exceeds a preset confidence threshold, then the bounding rectangle of the candidate region is used as the detection box, and its center point coordinates are calculated. In another embodiment, the classification model can also use a sliding window method to judge multiple regions of the entire image one by one.

[0104] In actual deployment, a suitable model type can be selected from the two models mentioned above based on the computing power of the edge device and the specific application requirements. In this application, the implementation method using the target detection model can be more easily integrated with the binocular localization process, and is therefore a preferred implementation method.

[0105] In the implementation of the above embodiments: the flame detection model adopts a lightweight target detection model or image classification model deployed on the edge, enabling the model inference task to be completed locally at the image acquisition node without uploading images to the cloud for processing, effectively reducing the time and bandwidth consumed by data transmission. Simultaneously, because the model is lightweight, the number of parameters and computational load are reduced, allowing it to run on microcontrollers with limited memory and computing power. This avoids the problems of deployment limitations due to excessively large models or excessively high inference latency, enabling the entire detection process to be completed in real time on the edge, improving the timeliness of warnings and the practicality of the system.

[0106] Optionally, in this embodiment, the system also acquires environmental parameters collected by the environmental sensor array to assist in the judgment. Before generating the warning information, the method further includes: The system acquires environmental parameters collected by an environmental sensor array, including one or more of the following: temperature, humidity, gas concentration, and infrared flame response.

[0107] The environmental sensor group can include one or more of temperature and humidity sensors, gas sensors, and infrared flame sensors. Temperature and humidity sensors collect ambient temperature and humidity data and can be digital sensors such as the DHT11, connected to the main control node via a single-bus interface to directly output temperature and humidity values. Gas sensors detect the concentration of smoke, combustible gases, or toxic gases in the air and can be sensors such as the MQ-2 combustible gas / smoke sensor, the MQ-7 carbon monoxide sensor, or the MQ-135 air quality / hazardous gas sensor. These gas sensors typically output analog voltage signals related to gas concentration, which need to be converted to digital values ​​by an analog-to-digital converter before being read by the main control node. Infrared flame sensors detect infrared radiation of specific wavelengths emitted by flames; their output is also an analog voltage signal, which is converted to an infrared response value after analog-to-digital conversion. Since the analog signals output by the gas sensors and infrared flame sensors need to be converted to digital signals, the system can use an analog-to-digital converter such as the ADS1115. This module connects to the main control node via an I2C bus and is responsible for converting the analog outputs of each sensor into digital signals for the main control node to read.

[0108] The warning level is determined by multi-source joint judgment based on environmental parameters, flame target confirmation results, and the three-dimensional spatial coordinates of the fire source.

[0109] After acquiring environmental parameters, the system performs multi-source joint judgment based on the environmental parameters, flame target confirmation results, and the three-dimensional spatial coordinates of the fire source to determine the warning level. Multi-source joint judgment refers to integrating information from multiple sources, including visual and physical sensors, and determining the warning level of the current scene through preset logical rules, rather than relying solely on the output of a single sensor. The warning level indicates the probability and urgency of a fire and can be divided into different levels according to actual application needs. In one specific judgment method, the system first makes preliminary judgments based on the flame target confirmation results and environmental parameters, and then integrates the results of both. For example, when the flame detection model confirms the presence of a flame target, the visual part's judgment result is "flame present"; when the gas sensor detects a gas concentration exceeding a preset gas alarm threshold, or the infrared flame sensor detects an infrared radiation intensity exceeding a preset infrared alarm threshold, the environmental part's judgment result is "abnormal environment."

[0110] Based on this, if both the visual and environmental parameters indicate an anomaly, the highest warning level can be established, indicating that a fire has occurred or is highly likely to occur. If only the flame detection model confirms a flame target but the environmental parameters are normal, the next highest warning level can be established, indicating the presence of a suspected flame target, possibly a smokeless open flame, requiring manual confirmation or continuous observation. If the flame detection model does not confirm a flame target but the environmental parameters are abnormal, the lower warning level can be established, indicating a suspected environmental anomaly, but no fire has been confirmed, possibly due to interference from other non-fire factors affecting the sensor. If neither the visual nor environmental parameters show any anomalies, the situation is considered normal, and no warning is triggered. The three-dimensional spatial coordinates of the fire source are also considered when determining the warning level. For example, when the fire source is close to the monitoring equipment, the warning level can be appropriately increased under the same conditions.

[0111] Early warning information is generated based on the confirmation results of the flame target and the three-dimensional spatial coordinates of the fire source, including: generating early warning information based on the confirmation results of the flame target, the three-dimensional spatial coordinates of the fire source and the early warning level.

[0112] After determining the warning level, the system generates warning information based on the flame target confirmation result, the three-dimensional spatial coordinates of the fire source, and the warning level. The warning information is a structured message output by the system, used to notify operators or higher-level systems of the current fire monitoring status and the location of the fire source. The warning information includes at least the flame target confirmation result (e.g., whether a flame target exists and the detection confidence level), the three-dimensional spatial coordinates of the fire source (X, Y, Z values), and the warning level. In addition, the warning information may also include auxiliary information such as equipment identification, timestamps, and environmental parameters (temperature, humidity, gas concentration, infrared response), so that the recipient can fully understand the on-site situation. The generated warning information can be output through a communication module. In one implementation, the communication module connects to the network via WiFi and uploads the warning information to the cloud platform using the MQTT protocol; the data format can be encapsulated in JSON.

[0113] After receiving the warning information, the cloud platform analyzes it, displays the fire source location, warning level, and environmental parameters on the monitoring interface, and pushes alarm notifications to management personnel. In another implementation, the warning information can also be output locally, for example, by emitting sound alarms of different frequencies to indicate different warning levels, by using LED indicator colors to indicate the current status, or by directly displaying the three-dimensional coordinates of the fire source on a local screen. These output methods can be used individually or in combination depending on the actual application scenario, ensuring that warning information is delivered to relevant personnel in a timely and effective manner.

[0114] As one implementation method, the fire warning system also includes a power supply module, which provides working power to the above components. The power supply module can be powered by a regulated power supply, a mobile power supply or a battery, depending on the application scenario.

[0115] In the implementation of the above embodiments: A multi-source joint judgment is performed by integrating the flame target confirmation results output by the flame detection model, the three-dimensional spatial coordinates of the fire source calculated by binocular positioning, and environmental parameters such as temperature, humidity, gas concentration, and infrared flame response collected by the environmental sensor group. Compared to judgment methods relying solely on a single sensor or a single frame image, this approach more comprehensively reflects the actual situation on-site and effectively reduces the possibility of false alarms triggered by false alarms from a single sensor or false detections from a single frame image. Simultaneously, the warning information includes the warning level and the three-dimensional spatial coordinates of the fire source, allowing the receiver to intuitively understand the specific location and urgency of the fire source, facilitating rapid response decisions. Furthermore, the warning information is generated and uploaded only after the judgment is completed on the device side, eliminating the need for continuous transmission of original image data, reducing network bandwidth consumption, and making warning notifications more timely.

[0116] Please see Figure 3 The diagram shown is a structural schematic of a fire early warning system provided in an embodiment of this application; this application provides a fire early warning system 200, including: The first image acquisition node 210 is used to acquire the left eye image sequence; The second image acquisition node 220 is fixed relative to the first image acquisition node with a preset binocular baseline and is used to acquire the right eye image sequence. Processing module 230 is used to extract dynamic features from multiple consecutive frames of images in the left-eye image sequence and the right-eye image sequence, and to determine whether the model call triggering condition is met based on the dynamic features. The dynamic features include at least one of the following: area features of candidate regions in the image that meet the preset flame color conditions, inter-frame grayscale change features within the candidate regions, area fluctuation features of the candidate regions within a preset frame number window, and frequency domain features. The frequency domain features are obtained by performing a fast Fourier transform on the area value sequence of the candidate regions within the preset frame number window, and the frequency domain features include the main frequency component and the spectral energy ratio of the area value sequence within a preset flame flicker frequency band. The flame confirmation module 240 is used to confirm the flame target in the image through the flame detection model when the model call trigger condition is met, and output the detection result. The detection result includes whether there is a flame target, and if the flame target is confirmed to exist, the left eye center point in the left eye image and the right eye center point in the right eye image are determined. The binocular positioning module 250 is used to calculate the parallax based on the center point of the left eye and the center point of the right eye, and to solve the three-dimensional spatial coordinates of the fire source. The early warning output module 260 is used to generate early warning information based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source.

[0117] The first image acquisition node 210 is used to acquire the left-eye image sequence, and the second image acquisition node 220 is set relatively fixed to the first image acquisition node 210 with a preset binocular baseline and is used to acquire the right-eye image sequence. Both image acquisition nodes can use the same camera module, such as an OV2640 camera paired with an ESP32-CAM module, ensuring consistency in imaging quality, field of view, and resolution between the left and right images. The optical axes of the two nodes remain parallel, and the binocular baseline length is preset according to the application scenario. The two nodes need to acquire images synchronously; for example, the master node can periodically send trigger signals to both nodes simultaneously, causing them to be exposed at the same time. After installation, the camera intrinsic parameters, distortion parameters, and binocular extrinsic parameters need to be calibrated by photographing a checkerboard calibration board. The calibration results are stored in the system for subsequent positioning calculations.

[0118] The processing module 230 is used to extract dynamic features from multiple consecutive frames of images in the left and right eye image sequences, and to determine whether the model call triggering conditions are met based on the dynamic features. The dynamic features include at least one of the following: area features of candidate regions in the image that meet preset flame color conditions, inter-frame grayscale change features within the candidate regions, area fluctuation features of the candidate regions within a preset frame window, and frequency domain features. The processing module 230 can be deployed in the microcontrollers of the first image acquisition node 210 and the second image acquisition node 220, with each node processing its own acquired images, or it can be deployed in an independent processing unit for unified processing.

[0119] The flame confirmation module 240 is used to confirm the presence of a flame target in an image using a flame detection model when the model invocation trigger conditions are met, and outputs the detection results. The detection results include whether a flame target exists, and if the presence of a flame target is confirmed, the left center point of the flame target in the left eye image and the right center point in the right eye image. The flame detection model is a lightweight target detection model or image classification model deployed on the edge.

[0120] The binocular positioning module 250 is used to calculate the disparity based on the center points of the left and right eyes, and to solve the three-dimensional spatial coordinates of the fire source. The early warning output module 260 is used to generate early warning information based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source. In one embodiment, the binocular positioning module 250 is deployed in the second image acquisition node 220. The second image acquisition node 220, as the master node, receives the left eye center point sent by the first image acquisition node 210 and performs disparity calculation and three-dimensional coordinate solution locally.

[0121] In one embodiment, the system further includes an environmental sensor array, an analog-to-digital converter (ADC), and a communication module. The environmental sensor array includes one or more of a temperature and humidity sensor, a gas sensor, and an infrared flame sensor, used to collect environmental parameters. The ADC is used to convert the analog output signals from the gas sensor and the infrared flame sensor into digital signals. The communication module is used to upload environmental parameters, detection confidence levels, the three-dimensional spatial coordinates of the fire source, and the warning status to a cloud platform. In one embodiment, the communication module accesses the network via WiFi and uploads the warning information to the cloud platform in JSON format using the MQTT protocol. The cloud platform and terminal are used to parse, store, and display the data, and record fire events and remote alarm information.

[0122] In one specific implementation, both the first image acquisition node 210 and the second image acquisition node 220 are combinations of an ESP32-CAM and an OV2640 camera. The second image acquisition node 220, acting as the master node, is also connected to an environmental sensor group and a communication module. After the first image acquisition node 210 completes dynamic filtering and flame confirmation of the left-eye image, it sends the left-eye center point and confidence level to the second image acquisition node 220. After the second image acquisition node 220 completes dynamic filtering and flame confirmation of the right-eye image, it locally aggregates the left and right eye center points. The binocular positioning module 250 calculates the parallax and resolves the three-dimensional coordinates of the fire source. Finally, the early warning output module 260 generates early warning information based on the multi-source joint judgment result and uploads it to the cloud.

[0123] Optionally, in this embodiment of the application, the fire early warning system further includes an environmental sensor group for collecting environmental parameters, including one or more of temperature, humidity, gas concentration and infrared flame response; the processing module is also used to perform multi-source joint judgment based on environmental parameters, flame target confirmation results and three-dimensional spatial coordinates of the fire source to determine the early warning status or early warning level.

[0124] In one embodiment, the fire warning system 200 further includes an environmental sensor group for collecting environmental parameters. The environmental sensor group includes one or more of a temperature and humidity sensor, a gas sensor, and an infrared flame sensor. The temperature and humidity sensor collects ambient temperature and humidity, the gas sensor detects the concentration of smoke, combustible gas, or toxic gas, and the infrared flame sensor detects infrared radiation emitted by a flame. The outputs of the gas sensor and the infrared flame sensor are analog voltage signals. The system also includes an analog-to-digital converter module for converting these analog signals into digital signals for the processing module to read.

[0125] The processing module 230 is also used to perform multi-source joint judgment based on environmental parameters, flame target confirmation results, and the three-dimensional spatial coordinates of the fire source to determine the warning status or warning level. Multi-source joint judgment refers to integrating various information from visual and physical sensors and determining the warning status or warning level of the current scene through preset logical rules. For example, when the flame detection model confirms the existence of a flame target and environmental parameters are abnormal, it is determined to be the highest warning level; when only a flame target is confirmed but environmental parameters are normal, it is determined to be a suspected flame target state; when the flame detection model does not confirm a flame target but environmental parameters are abnormal, it is determined to be a suspected environmental anomaly state. After the processing module performs the above judgment, the warning information generated by the warning output module 260 includes the corresponding warning status or warning level, enabling the receiver to understand the severity of the current fire risk.

[0126] The following example illustrates a fixed, low-cost fire monitoring node. In this embodiment, the system can be installed in a fixed indoor monitoring location, such as a corner of a laboratory, the top of a small warehouse, or a wall near electrical equipment. The first and second image acquisition nodes are horizontally fixed, with a binocular baseline distance of approximately 120 mm between them. After installation, a checkerboard calibration board is used to calibrate the intrinsic parameters, distortion parameters, and binocular extrinsic parameters of the left and right cameras. The calibrated parameters are stored in the system for subsequent positioning calculations. The second image acquisition node acts as the master node, connecting to a temperature and humidity sensor, a gas sensor, an infrared flame sensor, and an analog-to-digital converter module, and is responsible for network connectivity and cloud data uploads.

[0127] After system startup, the master node first initializes the camera, sensors, network communication parameters, and binocular positioning parameters. The first and second image acquisition nodes acquire consecutive image frames at a set frame rate, each performing dynamic pre-screening on the acquired images. When multiple consecutive frames meet the trigger conditions, the node invokes a lightweight flame detection model deployed on the edge to infer the current frame. The model outputs a detection box, detection confidence score, and the flame target center point. The first image acquisition node sends the left eye center point, confidence score, and validity flag to the second image acquisition node. After confirming the flame in the right eye image and obtaining the right eye center point locally, the second image acquisition node first determines whether both left and right center points are valid. This includes checking if both center points are within the image range, if both detection confidence scores exceed thresholds, if the left eye data has not timed out, if the difference in the ordinates of the left and right center points does not exceed a preset range, and if the absolute value of the disparity is not less than a minimum threshold. After determining validity, the second image acquisition node calculates the disparity and resolves the three-dimensional spatial coordinates of the fire source. Then, combining environmental sensors and infrared flame response, it determines the warning status and finally uploads the warning information to the cloud platform.

[0128] The following embodiment illustrates the auxiliary safety perception node of the mobile inspection platform. In this embodiment, the first and second image acquisition nodes are fixed to the front end of the mobile inspection platform or robot dog. The direction of the binocular baseline is consistent with the platform's forward direction, and the field of view of the two cameras covers the area in front of the platform. The environmental sensor group is arranged in the ventilation position of the platform shell to enable the sensors to effectively detect changes in smoke and gas in the surrounding environment. As the inspection platform moves within the monitoring area according to the preset route, the system continuously performs dynamic pre-screening, end-side flame confirmation, and binocular positioning within the local field of view, and uploads the detection results, the three-dimensional spatial coordinates of the fire source, and the warning status to the remote platform via a wireless network.

[0129] Compared to fixed monitoring nodes, mobile inspection platforms may introduce camera motion and image jitter during movement, which can lead to false triggering of dynamic pre-screening. To adapt to mobile scenarios, the system can increase the value of the continuous frame threshold N, for example, from 3 frames in a fixed scenario to 5 or 8 frames, so that only dynamic changes lasting for a relatively long period can trigger model invocation. Furthermore, the system can add image stability judgment conditions, such as calculating the global motion vector of the entire image, temporarily skipping dynamic feature extraction of the current frame when the global motion exceeds a certain amplitude, or compensating for image changes between adjacent frames by combining platform odometer, inertial measurement unit, or attitude information. These adaptive adjustments do not affect the core process of the dynamic pre-screening and edge model confirmation, and are equivalent implementations of this technical solution in mobile scenarios.

[0130] The following examples illustrate alternative implementations of the model, sensor, and communication method. In this example, the lightweight flame detection model is replaced. While the aforementioned example used a FOMO-based target detection model, in this alternative implementation, the flame detection model can be replaced with other convolutional neural networks that have undergone quantization, pruning, distillation, or structural lightweighting. If replaced with an image classification model, since the classification model only outputs the probability of whether the overall image contains a flame target and not the location of the detection box, the system can crop out the candidate region obtained in the dynamic pre-screening stage and input the cropped sub-image into the classification model for judgment. When the probability of the flame category output by the classification model exceeds a preset confidence threshold, the bounding rectangle of the candidate region is used as the detection box, and the center point coordinates are extracted, thus achieving flame target confirmation and subsequent binocular localization.

[0131] In this embodiment, the environmental sensor group is also replaced. The DHT11 temperature and humidity sensor, MQ series gas sensor, and infrared flame sensor used in the previous embodiments can be replaced with other types of sensors, such as smoke sensors, carbon monoxide sensors, pyroelectric sensors, thermal imaging modules, or ultraviolet flame sensors, depending on the specific application scenario. These sensors can also provide environmental parameters for multi-source joint judgment. The communication method between nodes is not limited to the TCP protocol; UDP, serial port, I2C, SPI (Serial Peripheral Interface), Bluetooth, or WiFi direct connection can also be used to realize data transmission between the first image acquisition node and the second image acquisition node. The cloud platform is not limited to OneNet; any data access and alarm platform that supports MQTT, HTTP, or other IoT protocols can be used. The above replacement methods do not change the core concept of this application and belong to the equivalent implementation of this technical solution.

[0132] It should be understood that this system corresponds to the fire early warning method embodiment described above and is capable of performing the various steps involved in the above method embodiment. The specific functions of the system can be found in the description above, and detailed descriptions are omitted here to avoid repetition. The system includes at least one software functional module that can be stored in memory or embedded in the operating system (OS) of the device in the form of software or firmware.

[0133] Please see Figure 4 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. An electronic device 300 provided in this application includes a processor 310 and a memory 320. The memory 320 stores machine-readable instructions executable by the processor 310. When the machine-readable instructions are executed by the processor 310, the method described above is performed.

[0134] Figure 4 The components shown can be implemented using hardware, software, or a combination thereof. Electronic device 300 may be a physical device, such as a server or PC, or a virtual device, such as a virtual machine or virtualization container. Furthermore, electronic device 300 is not limited to a single device; it can be a combination of multiple devices or a cluster of numerous devices.

[0135] This application also provides a storage medium storing a computer program, which is executed by a processor to perform the above-described method.

[0136] The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0137] This application also provides a computer program product, including computer program instructions, which are executed by a processor to perform the method described above.

[0138] It should be understood that the disclosed apparatus and methods can also be implemented in other ways, given the several embodiments provided in this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0139] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0140] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. A fire early warning method, characterized in that, include: Acquire an image sequence; the image sequence includes a left-eye image sequence acquired by a first image acquisition node and a right-eye image sequence acquired by a second image acquisition node; Dynamic features are extracted from multiple consecutive frames of images in the image sequence, and the model call triggering conditions are determined based on the dynamic features. The dynamic features include at least one of the following: area features of candidate regions in the image that meet the preset flame color conditions, inter-frame grayscale change features within the candidate regions, area fluctuation features of the candidate regions within a preset frame window, and frequency domain features. The frequency domain features are obtained by performing a fast Fourier transform on the area value sequence of the candidate region within a preset frame window. The frequency domain features include the main frequency component and the proportion of spectral energy. When the model invocation triggering conditions are met, the flame detection model is used to confirm the flame target in the image and output the detection result. The detection result includes whether there is a flame target, and if the flame target is confirmed to exist, the center point of the flame target in the left eye image and the center point of the flame target in the right eye image are determined. The parallax is calculated based on the center point of the left eye and the center point of the right eye, and the three-dimensional spatial coordinates of the fire source are solved. Early warning information is generated based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source.

2. The method according to claim 1, characterized in that, The preset flame color conditions include at least one of the following: the intensity of the red channel of the pixel is higher than the intensity of the green channel and the intensity of the blue channel; the brightness of the pixel is greater than a preset brightness threshold; and the hue of the pixel is within the preset flame hue range. The model call triggering conditions include at least one of the following: the area of ​​the candidate region exceeds a first threshold, the inter-frame grayscale change feature value of the candidate region exceeds a second threshold, the area fluctuation feature value of the candidate region exceeds a third threshold, and frequency domain feature conditions; The frequency domain characteristic conditions are that the main frequency component falls within the preset flame flashing frequency band range, the spectral energy ratio exceeds the preset energy threshold, and the main frequency component does not fall within the preset interference frequency band range.

3. The method according to claim 1, characterized in that, The inter-frame grayscale change feature is determined based on the grayscale difference between the corresponding pixels in the candidate region between the current frame and the previous frame. The area fluctuation feature is determined based on the area variance or standard deviation of the candidate region within the preset frame window.

4. The method according to claim 1, characterized in that, Determining the center point of the flame target in the left-eye image and the center point of the flame target in the right-eye image includes: Obtain the coordinates of the top left corner, width, and height of the detection box output by the flame detection model; The coordinates of the center point of the detection frame are calculated based on the coordinates of the top left corner, the width, and the height. Based on the ratio between the input scale of the flame detection model and the camera calibration scale, the center point coordinates are mapped to the image coordinates under the calibration scale.

5. The method according to claim 1, characterized in that, Based on the left and right eye center points, the disparity is calculated, and the three-dimensional spatial coordinates of the fire source are solved, including: Calculate the disparity based on the x-coordinate of the left eye center point and the x-coordinate of the right eye center point; Depth is determined based on the parallax and preset camera calibration parameters; The horizontal and vertical coordinates of the fire source in three-dimensional space are calculated based on the coordinates of the left eye center point, the depth, and the camera calibration parameters.

6. The method according to claim 1, characterized in that, The flame detection model is a lightweight target detection model or image classification model deployed on the edge.

7. The method according to claim 1, characterized in that, Before generating the warning information, the method further includes: The environmental parameters collected by the environmental sensor array are acquired, including one or more of temperature, humidity, gas concentration, and infrared flame response. The warning level is determined by multi-source joint judgment based on the environmental parameters, the flame target confirmation result, and the three-dimensional spatial coordinates of the fire source; Generating early warning information based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source includes: generating early warning information based on the flame target confirmation result, the three-dimensional spatial coordinates of the fire source and the early warning level.

8. A fire early warning system, characterized in that, include: The first image acquisition node is used to acquire the left eye image sequence; The second image acquisition node is fixed relative to the first image acquisition node with a preset binocular baseline and is used to acquire the right eye image sequence. The processing module is used to extract dynamic features from multiple consecutive frames of images in the left-eye image sequence and the right-eye image sequence, and determine whether the model call triggering condition is met based on the dynamic features; the dynamic features include at least one of the following: area features of candidate regions in the image that meet preset flame color conditions, inter-frame grayscale change features within the candidate regions, and area fluctuation features of the candidate regions within a preset frame number window; The flame confirmation module is used to confirm the flame target in the image through the flame detection model when the model call triggering conditions are met, and output the detection result; the detection result includes whether there is a flame target, and if the flame target is confirmed to exist, the left center point of the flame target in the left eye image and the right center point of the flame target in the right eye image. The binocular positioning module is used to calculate the parallax based on the center point of the left eye and the center point of the right eye, and to solve the three-dimensional spatial coordinates of the fire source; The early warning output module is used to generate early warning information based on the flame target confirmation result and the three-dimensional spatial coordinates of the fire source.

9. The system according to claim 8, characterized in that, Also includes: An environmental sensor array is used to collect environmental parameters, including one or more of temperature, humidity, gas concentration, and infrared flame response. The processing module is also used to perform multi-source joint judgment based on the environmental parameters, the flame target confirmation result and the three-dimensional spatial coordinates of the fire source, and determine the warning status or warning level.

10. A computer program product, characterized in that, It includes computer program instructions that, when executed by a processor, perform the method as described in any one of claims 1 to 7.

11. An electronic device, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, perform the method as described in any one of claims 1 to 7.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, perform the method as described in any one of claims 1 to 7.

Images