Flame explosion detection method and system based on cascading of evs events and aps images

Abstract

The application provides a flame explosion detection method and system based on EVS event and APS image cascade, and belongs to the technical field of computer vision and flame detection. Through synchronous acquisition of APS gray frames and EVS asynchronous event streams by pixel-level fusion visual sensors, a unified statistical imaging noise model of APS and EVS is established, after noise parameters are calibrated by maximum likelihood estimation, a H-ESIM statistical simulator is driven to generate synthetic data, and an enhanced training data set is constructed; after recursive event denoising and double-branch feature extraction, cross-modal features are dynamically weighted and fused by an adaptive fusion module, flame discrimination output is realized by combining event rate statistical features and an SVM classifier. The application is suitable for extreme scenes such as high dynamic range and smoke shielding, has high detection precision and fast processing speed, can continuously optimize the model through a closed loop iteration, and is suitable for real-time flame explosion detection in industrial security, flammable and explosive places.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and flame detection, and in particular to a flame explosion detection method and system based on the cascading of EVS events and APS images. Background Technology

[0002] Flame and explosion detection is one of the core technologies for industrial security, warehousing and logistics, and safety management of flammable and explosive sites.

[0003] Existing multimodal fusion detection solutions generally suffer from the following drawbacks: First, there is a lack of a unified imaging noise modeling and device-level calibration process for APS and EVS, resulting in a mismatch in the noise characteristics of the two-modal data and limiting the fusion effect. Second, the training data relies on real-world data collection, and samples of flame explosion scenarios are scarce. Furthermore, there is a lack of synthetic data generation tools that closely match the characteristics of real devices, resulting in insufficient model generalization ability. Third, it has poor adaptability to extreme scenarios, and is prone to false detection and missed detection under high dynamic range and smoke-covered conditions; fourth, it lacks a closed-loop optimization mechanism after deployment, and the model accuracy continues to decline with changes in the scene environment, resulting in insufficient long-term operational reliability.

[0004] Therefore, there is a need to provide a flame explosion detection method and system based on the cascading of EVS events and APS images to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide a flame explosion detection method and system based on the cascading of EVS events and APS images. Through unified noise modeling and calibration, synthetic data augmentation, adaptive cross-modal fusion, and extreme scene-specific processing, it achieves high-speed, high-precision, and robust real-time flame explosion detection, and continuously optimizes model performance through a closed-loop iterative mechanism.

[0006] To achieve the above objectives, this invention provides a flame explosion detection method based on the cascading of EVS events and APS images, comprising the following steps: S1: Deploy a pixel-level fusion vision sensor to simultaneously acquire APS grayscale frames and EVS asynchronous event streams on the same chip. Each event includes pixel coordinates, timestamps, and event polarity. S2: Build a unified statistical imaging noise model for APS and EVS, collect standard scene data as the field calibration dataset, solve the noise parameter set through maximum likelihood estimation, obtain the joint noise statistical characteristics after calibration, synthesize RAW frames and event streams, and construct an enhanced training dataset. S3: The asynchronous event stream is denoised using a recursive event denoiser (RED) to obtain a denoised event stream. APS spatial features and EVS dynamic features are extracted and then weighted and fused to obtain fused features. S4: Calculate the event rate based on the fusion features, construct the flame discrimination feature vector according to the statistical characteristics of the event rate, complete the flame scene discrimination through the classifier, and output the structured flame detection results.

[0007] Preferably, the pixel-level fusion visual sensor in S1 adopts a shared circuit architecture, with APS pixels and EVS pixels arranged alternately on the same chip; the sensor supports three working modes: image, event, and fusion, and the hybrid visual sensing fusion mode HVS is enabled by default in the flame explosion detection scenario; the sensor's APS resolution is not less than 768×608, and the EVS time resolution is better than 1ms.

[0008] Preferably, the unified statistical imaging noise model in S2 corresponds to the pixel output formulas of APS and EVS as follows, covering four types of imaging noise: photon shot noise, dark current noise, fixed pattern noise, and quantization noise. The formula for APS pixel output is: ; In the formula, APS pixels In the passage The RAW value output at that location. This represents the expected signal strength of the corresponding pixel. For photon shot noise, This represents the shot noise gain coefficient. This is dark current noise. This is the mean coefficient for dark current. The dark current variance coefficient. Calculated based on a normal distribution. For exposure duration; This is a fixed-mode noise that includes line noise and black level correction noise; To read out noise; The EVS pixel output formula is: ; In the formula, For EVS pixels at time The detected change in voltage in the logarithmic domain. For the logarithmic change in irradiance signal term, , This is a comprehensive noise term that includes photon shot noise, dark current noise, fixed-mode noise, and readout noise. The EVS noise variance is correlated with illuminance level and time interval as follows: In the formula, Illuminance level, For EVS noise parameter set; Generate synthetic datasets using the H-ESIM statistical simulator Synthetic RAW frames and event streams are generated by combining noise statistical characteristics and compared with publicly available flame detection datasets. After merging, an enhanced training dataset is formed. , .

[0009] Preferably, the sub-steps for generating the synthetic dataset using the H-ESIM statistical simulator are as follows: S21: Perform the following steps on high frame rate input videos with a frame rate of at least 3200fps: The image is converted to RAW by inverse mapping and then corrected by inverse gamma to obtain a linear irradiance domain image. Then, the camera response domain image is obtained through inverse white balance and inverse color matrix transformation. ; ; ; In the formula, For a single frame Images, high frame rate video consists of multiple consecutive frames. The temporal sequence of images; S22: Rearrange the image into single-channel RAW format according to the CFA (Color Filter Array) rule; S23: Superimpose fixed noise terms and dark current noise terms, relying on the position dependence coefficients in the APS noise parameter set. The sampling illumination and exposure time are correlated with noise, and noisy RAW frames are output. S24: Match the camera bit depth to complete the quantization process and generate the composite RAW frame and the corresponding event stream.

[0010] Preferably, in S3, the recursive event denoising unit (RED) filters out noise based on the difference in probability distribution between the effective signal and the noise, thus obtaining the denoised event stream. The denoised event stream satisfies a signal conditional density greater than the denoising conditional probability density. In the formula, It is an asynchronous event stream. , These are the parameters for the RED network. An adaptive fusion module employing either the FlexFuser or EvPlug architecture calculates fusion weights by combining real-time scene features. : ; In the formula, For APS spatial features, The EVS dynamic features are used; the fused features obtained after weighted fusion are as follows: ; In the formula, It is the Sigmoid activation function. It is a multilayer perceptron. For feature splicing operations, The value range is [0,1].

[0011] Preferably, the event rate in S4 ; In the formula, For time window Cumulative number of events within; Flame discrimination feature vector The calculation formula is as follows: ; In the formula, The average event rate. The standard deviation of the event rate. The characteristics of flame flicker frequency. Spatial distribution density characteristics, Transpose of a vector; Using SVM classifier Complete the distinction between fire and non-fire scenes, where... For the weight vector, For bias terms; Output structured flame detection results ; In the formula, The bounding box of the flame target. To test the confidence level, To detect timestamps, For index number, This represents the total number of flame targets identified at the current detection moment.

[0012] Preferably, S3 also includes a processing flow for extreme environments, suitable for detection in high dynamic scenes and smoke-covered scenes; In high dynamic range (HDR) scenes, relying on EVS's native 120~140dB high dynamic range performance and combined with optical transmittance modulation, the absolute irradiance is decoded from the event timestamp, and the calculation formula is as follows: ; In the formula, The absolute irradiance obtained from decoding, } represents the event polarity. The EVS logarithmic field trigger threshold, The sampling interval is... For reference irradiance; In smoke-obscured scenarios, a multi-cue smoke probability discrimination model is constructed to distinguish between flame particle emission features and smoke scattering and obscuration features: ; In the formula The probability value for smoke is defined in the range [0,1]. The normalized value of APS image intensity. This represents the local density normalized value of the event. This represents the normalized value of the local brightness variance.

[0013] Preferably, it also includes a closed-loop iteration step: After deployment, data from the actual deployment scenario is collected as new calibration samples and fed back to S2 to update the noise parameter set through incremental learning. Minimize the model prediction error Complete parameter iteration and update; based on the updated parameters, drive H-ESIM to generate new synthetic data adapted to the real scene, and execute S3 after iteratively updating and enhancing the training dataset; In the formula, For loss function, For the detection model after parameter updates, This refers to the returned samples and their corresponding real labels. N This shows the total number of samples used to calculate the average loss in this study.

[0014] A flame explosion detection system based on EVS event and APS image cascade is used to implement the steps of the flame explosion detection method based on EVS event and APS image cascade, including a noise modeling and calibration module, a data preprocessing and fusion module, a flame detection and output module, and a closed-loop iterative optimization module. The data preprocessing and fusion module includes a first feature extraction network, a second feature extraction network, and an adaptive cross-modal fusion module; The closed-loop iterative optimization module collects on-site deployment data and feeds it back to the noise modeling and calibration module.

[0015] Preferably, the first feature extraction network adopts the Swin-TinyTransformer, and the second feature extraction network adopts the Event-ViT or Spike Neural Network (SNN); the flame detection and output module is equipped with the BEC-SVM algorithm based on background event counting, and the real-time processing frame rate is not less than 500 FPS.

[0016] Therefore, the present invention employs the above-mentioned flame explosion detection method and system based on the cascade of EVS events and APS images, and the beneficial technical effects are as follows: (1) A pixel-level fusion vision sensor integrated on the same chip is used to achieve precise spatiotemporal alignment of APS and EVS data through a shared circuit architecture, avoiding errors and delays in subsequent software registration, and providing a reliable data foundation for cascade fusion.

[0017] (2) This invention establishes a unified statistical model covering four types of imaging noise and completes device-level parameter calibration through maximum likelihood estimation; relying on the H-ESIM simulator to generate synthetic data that fits the characteristics of real devices, effectively expanding the scarce samples of flame explosion and improving the generalization ability of the model.

[0018] (3) The present invention calculates the fusion weight based on the dynamic characteristics of the scene and adaptively allocates the proportion of APS spatial texture information and EVS high-speed dynamic information, taking into account both the detection accuracy of static scenes and the response speed of high-speed explosion scenes.

[0019] (4) This invention is specifically designed for high dynamic range scenarios. It expands the dynamic range by decoding the absolute irradiance using EVS timestamps. For smoke-covered scenarios, it constructs a multi-clue smoke probability model to suppress artifacts and significantly improves the detection reliability under complex working conditions.

[0020] (5) This invention achieves continuous iteration of noise parameters and models through field data feedback and incremental learning, enabling the system to adapt to changes in the field environment over a long period of time and maintain stable detection performance. Attached Figure Description

[0021] Figure 1 This is a flowchart of the flame explosion detection method based on the cascading of EVS events and APS images according to the present invention; Figure 2 This is a framework diagram of the flame explosion detection system based on the cascading of EVS events and APS images according to the present invention. Detailed Implementation

[0022] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0024] Example 1 like Figure 1 As shown, this invention provides a flame explosion detection method based on the cascading of EVS events and APS images, comprising the following steps: S1: Deploy a pixel-level fusion visual sensor to simultaneously acquire APS grayscale frames on the same chip. Asynchronous event stream with EVS Each event Including pixel coordinates timestamp With event polarity The S1 pixel-level fusion visual sensor adopts a shared circuit architecture, with APS pixels and EVS pixels arranged alternately on the same chip. Hardware-level spatiotemporal alignment is achieved through shared photodiodes and readout circuits, with a time alignment error of less than 10μs and spatial alignment accuracy reaching the sub-pixel level. The sensor supports three working modes: image, event, and fusion. The Hybrid Visual Sensing Fusion Mode (HVS) is enabled by default in flame explosion detection scenarios. The sensor's APS resolution is no less than 768×608, with a frame rate of 30fps. The EVS time resolution is better than 1ms, and the dynamic range reaches 130dB.

[0025] S2: Establish a unified statistical imaging noise model for APS and EVS Standard scenario data is collected as the field calibration dataset. Through maximum likelihood Estimate and solve for the set of noise parameters The calibrated joint noise statistical characteristics are obtained, and RAW frames and event streams are synthesized to construct an enhanced training dataset. Unified statistical imaging noise model The pixel output formulas corresponding to APS and EVS are as follows, covering four types of imaging noise: photon shot noise, dark current noise, fixed pattern noise, and quantization noise. The formula for APS pixel output is: ; In the formula, APS pixels In the passage The RAW value output at that location. This represents the expected signal strength of the corresponding pixel. For photon shot noise, This represents the shot noise gain coefficient. This is dark current noise. This is the mean coefficient for dark current. The dark current variance coefficient. Calculated based on a normal distribution. For exposure duration; This is a fixed-mode noise that includes line noise and black level correction noise; To read out noise; The EVS pixel output formula is: ; In the formula, For EVS pixels at time The detected change in voltage in the logarithmic domain. For the logarithmic change in irradiance signal term, , This is a comprehensive noise term that includes photon shot noise, dark current noise, fixed-mode noise, and readout noise. The EVS noise variance is correlated with illuminance level and time interval as follows: In the formula, Illuminance level, For EVS noise parameter set; Generate synthetic datasets using the H-ESIM statistical simulator Synthetic RAW frames and event streams are generated by combining noise statistical characteristics and compared with publicly available flame detection datasets. After merging, an enhanced training dataset is formed. , .

[0026] The sub-steps for generating a synthetic dataset using the H-ESIM statistical simulator are as follows: S21: Perform the following steps on high frame rate input videos with a frame rate of at least 3200fps: The image is converted to RAW by inverse mapping and then corrected by inverse gamma to obtain a linear irradiance domain image. Then, the camera response domain image is obtained through inverse white balance and inverse color matrix transformation. ; ; ; In the formula, For a single frame Images, high frame rate video consists of multiple consecutive frames. The temporal sequence of images; S22: Rearrange the image into single-channel RAW format according to the CFA (Color Filter Array) rule; S23: Superimpose fixed noise terms and dark current noise terms, relying on the position dependence coefficients in the APS noise parameter set. The sampling illumination and exposure time are correlated with noise, and noisy RAW frames are output. S24: Match the camera bit depth to complete the quantization process and generate the composite RAW frame and the corresponding event stream.

[0027] S3: The asynchronous event stream is denoised using a recursive event denoiser (RED) to obtain a denoised event stream. APS spatial features and EVS dynamic features are extracted and then weighted and fused to obtain fused features. In S3, the recursive event denoising unit (RED) filters out noise based on the difference in probability distribution between the valid signal and the noise, thus obtaining the denoised event stream. The denoised event stream satisfies a signal conditional density greater than the denoising conditional probability density. In the formula, It is an asynchronous event stream. , These are the parameters of the RED network; in this embodiment, the motion error suppression rate (MESR) of RED can reach 0.974 under the E-MLB benchmark.

[0028] An adaptive fusion module employing either the FlexFuser or EvPlug architecture calculates fusion weights by combining real-time scene features. : ; In the formula, For APS spatial features, EVS dynamic features are generated by processing the APS image using a first feature extraction network to output APS spatial features and processing the denoised event stream using a second feature extraction network to output EVS dynamic features. The fusion features obtained after performing weighted fusion are as follows: ; In the formula, It is the Sigmoid activation function. It is a multilayer perceptron. For feature splicing operations, The value range is [0,1].

[0029] When the EvPlug architecture is selected, the feature fusion method is as follows: ;in Features of the intermediate layer of the RGB model This is a plug-and-play fusion function that outputs features and The spatial dimensions are completely consistent.

[0030] The S3 also includes a processing flow for extreme environments, suitable for detection in high dynamic scenes and smoke-covered scenes; In high dynamic range (HDR) scenes, relying on EVS's native 120~140dB high dynamic range performance and combined with optical transmittance modulation, the absolute irradiance is decoded from the event timestamp, and the calculation formula is as follows: ; In the formula, The absolute irradiance obtained from decoding, ) represents the event polarity. The EVS logarithmic field trigger threshold, The sampling interval is... For reference irradiance; In smoke-obscured scenarios, a multi-cue smoke probability discrimination model is constructed to distinguish between flame particle emission features and smoke scattering and obscuration features: ; In the formula The probability value for smoke is defined in the range [0,1]. The normalized value of APS image intensity. This represents the local density normalized value of the event. This represents the normalized value of the local brightness variance.

[0031] S4: The event rate is calculated based on fused features. A flame discrimination feature vector is constructed based on the statistical characteristics of the event rate. A classifier is then used to determine the flame scene, outputting a structured flame detection result. (Event rate in S4) ; In the formula, For time window Cumulative number of events within; Flame discrimination feature vector The calculation formula is as follows: ; In the formula, The average event rate. The standard deviation of the event rate. The characteristics of flame flicker frequency. Spatial distribution density characteristics; Using SVM classifier Complete the distinction between fire and non-fire scenes, where... For the weight vector, For bias terms; Output structured flame detection results ; In the formula, The bounding box of the flame target. To test the confidence level, For detecting timestamps.

[0032] It also includes closed-loop iteration steps: After deployment, data from the actual deployment scenario is collected as new calibration samples and fed back to S2 to update the noise parameter set through incremental learning. Minimize the model prediction error Complete parameter iteration and update; based on the updated parameters, drive H-ESIM to generate new synthetic data adapted to the real scene, and execute S3 after iteratively updating and enhancing the training dataset; In the formula, For loss function, This is the end-to-end flame detection model after parameter updates. ={ }, This refers to the returned samples and their corresponding real labels. N This shows the total number of samples used to calculate the average loss in this study.

[0033] like Figure 2 As shown, the flame explosion detection system based on EVS event and APS image cascade is used to implement the flame explosion detection method based on EVS event and APS image cascade, including a noise modeling and calibration module, a data preprocessing and fusion module, a flame detection and output module, and a closed-loop iterative optimization module. The noise modeling and calibration module is used to build a unified statistical imaging noise model for APS and EVS. The data preprocessing and fusion module includes a first feature extraction network, a second feature extraction network, and an adaptive cross-modal fusion module; The flame detection and output module solves the event rate based on fused features, builds a flame discrimination feature vector, completes the flame target discrimination through an SVM classifier, and outputs a structured detection result containing bounding boxes, confidence scores, and timestamps. The closed-loop iterative optimization module collects on-site deployment data and feeds it back to the noise modeling and calibration module.

[0034] The first feature extraction network uses the Swin-TinyTransformer, and the second feature extraction network uses either Event-ViT or Spike Neural Network (SNN). The flame detection and output module is equipped with the BEC-SVM algorithm based on background event counting, with a real-time processing frame rate of no less than 500 FPS.

[0035] Example 2 Ablation experiments, cross-sectional comparison experiments, and extreme scenario-specific tests were conducted on public and synthetic datasets. The specific experimental process and results are as follows.

[0036] Experimental environment configuration: Hardware environment: The CPU is an Intel Core i9-10900K@3.7GHz (10 cores and 20 threads), the memory is 32GB DDR4, and there is no GPU acceleration throughout the process. All inference tests are completed under single-threaded CPU to verify the real-time detection capabilities of the pure CPU platform.

[0037] Software environment: The operating system is Ubuntu 20.04LTS, the deep learning framework is PyTorch 1.12.1, the event data processing uses Metavision SDK 4.2, the SVM classifier is implemented based on scikit-learn 1.1.3, and the image processing depends on OpenCV 4.5.5.

[0038] This experiment uses the publicly available flame event detection dataset FlaDE and the synthetic dataset generated by H-ESIM to form the training set, with the specific settings as follows: The FlaDE public dataset contains 28 real-world flame event sequences, covering various scenarios including indoor open flames, outdoor combustion, low-light flames, and scenes with smoke interference, totaling over 12 million events. The sequences are randomly divided into training and testing sets in a 7:3 ratio, with 20 sequences in the training set and 8 sequences in the testing set. The testing set is further divided into three subsets: normal scenes, high dynamic range (HDR) scenes, and smoke-covered scenes, to validate detection performance under different conditions.

[0039] Synthetic dataset Based on the calibrated joint noise statistical characteristics, it was generated by the H-ESIM statistical simulator and contains 12,000 sets of flame explosion samples, covering simulation scenarios with different flame intensities, explosion velocities, light intensities, and smoke concentrations. It is used to supplement the scarce scenarios of real samples and improve the generalization ability of the model.

[0040] The final augmented training dataset is It is used to train the feature extraction network and the SVM classifier.

[0041] This experiment uses common metrics and speed metrics in the field of target detection, which are defined as follows: AP@0.5: Average detection accuracy when the Intersection over Union (IOU) threshold is 0.5, expressed as a percentage. A higher value indicates better detection accuracy. Inference Frames Per Second (FPS): The number of end-to-end time windows that can be processed on a CPU platform. It is used to measure real-time performance. The higher the value, the lower the detection latency. F1 score: The harmonic mean of precision and recall, used to comprehensively measure the accuracy and completeness of the detection, with a value range of [0,1]. To verify the technical contributions of each core module of this invention, seven sets of control experiments were set up using the controlled variable method. Core technology modules were added gradually, and performance changes were tested. Experimental parameters were uniformly set as follows: event time window length 2ms, SVM using the RBF kernel function, and the feature extraction network maintaining the same input resolution. Experimental results are shown in Table 1 below: Table 1 Ablation Experiment Results of Each Core Module

[0042] As shown in the table above, the dual-modal fusion scheme significantly improves accuracy compared to the single-modal scheme, verifying the complementarity between APS spatial texture information and EVS high-speed dynamic information. After adding unified noise modeling and synthetic data augmentation, the accuracy improved by 1.7 percentage points, indicating that noise calibration and synthetic data effectively reduced the noise difference between modes, supplemented scarce scene samples, and improved the model's generalization ability. The RED event noise reduction module filters background random noise, improving accuracy by 1.6 percentage points while only causing a slight decrease in speed. Compared with fixed-weight fusion, the adaptive fusion module improves accuracy by 0.9 percentage points, verifying the adaptability advantage of dynamic weight allocation to different scenarios; The extreme scene processing module is optimized for HDR and smoke scenes, which improves the overall accuracy by another 0.9 percentage points. The final complete solution achieves an AP@0.5 of 96.6% and an inference speed of 505.7 FPS, balancing high accuracy and high real-time performance.

[0043] To verify the overall performance advantages of this solution, four mainstream flame detection solutions were selected for a comparative analysis. All tests were conducted under the same hardware, dataset, and testing standards. The results are shown in Table 2. Table 2 Comparison of performance of different detection methods

[0044] The traditional frame difference method is the fastest but has the lowest accuracy, and can only be adapted to simple background scenes, which cannot meet the reliability requirements of detection in complex industrial environments. Single-modal deep learning solutions outperform traditional methods in accuracy, but the inference speed of CPU platforms is far below the real-time requirements of high-risk scenarios, and single-modal approaches have inherent scene defects such as easy overexposure in APS and lack of texture in EVS. While existing dual-modal deep learning fusion solutions have improved accuracy, they suffer from high model complexity and computational burden, with a CPU-based inference speed of only 41.5 FPS, which cannot meet the low-latency detection requirements of high-speed scenarios such as explosions and flash fires. This method significantly outperforms all comparable schemes in accuracy, while achieving a CPU inference speed of 505.7 FPS, which is more than 12 times faster than existing dual-modal deep learning schemes. The single-frame detection latency is approximately 1.98 ms, enabling microsecond-level detection response.

[0045] For the two most common extreme working conditions in industrial scenarios, namely high dynamic strong light and smoke obstruction, the detection performance of different schemes was tested separately to verify the effectiveness of the extreme scene processing module of the present invention. The results are shown in Table 3: Table 3 Comparison of detection performance in extreme scenarios

[0046] In HDR high-light scenes, APS images lose a lot of texture information due to overexposure, resulting in a sharp drop in accuracy. This invention supplements the dynamic range by decoding absolute irradiance using EVS timestamps, improving accuracy by 13.8 percentage points compared to ordinary fusion schemes, approaching the level of detection in conventional scenes. In smoke-covered scenarios, both APS images and event streams are affected by smoke scattering, resulting in numerous artifacts. This method distinguishes between flames and smoke and suppresses artifacts through a multi-cue smoke probability model, achieving a 13.2 percentage point improvement in accuracy compared to ordinary fusion schemes, effectively reducing false detections and false negatives in smoke environments. In both extreme scenarios, this method maintains a detection accuracy of over 92%, demonstrating significantly better scene robustness than the comparative schemes.

[0047] Input a continuous event stream and an APS frame sequence, and calculate the total time taken from data input to output of a complete detection result. The test sample size is 1000 consecutive time windows. Exclude I / O time such as data loading and file reading / writing, and take the average single frame processing time to convert it into frames per second.

[0048] Test results show that the average processing speed of this method under single-threaded CPU is 505.7 FPS, which meets the low-latency real-time flame explosion detection requirements in scenarios such as industrial plants, oil and gas stations, and warehouses.

[0049] It is worth noting that all contents not described in detail in this invention are existing technologies and are well known to those skilled in the art.

[0050] Therefore, the present invention adopts the above-mentioned flame explosion detection method and system based on the cascading of EVS events and APS images. Through unified noise modeling and calibration, synthetic data enhancement, adaptive cross-modal fusion and extreme scene-specific processing, it achieves high-speed, high-precision and robust real-time flame explosion detection. Furthermore, it continuously optimizes model performance through a closed-loop iterative mechanism, making it suitable for real-time flame explosion detection in industrial security and flammable and explosive environments.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A flame explosion detection method based on the cascading of EVS events and APS images, characterized in that, Includes the following steps: S1: Deploy a pixel-level fusion vision sensor to simultaneously acquire APS grayscale frames and EVS asynchronous event streams on the same chip. Each event includes pixel coordinates, timestamps, and event polarity. S2: Build a unified statistical imaging noise model for APS and EVS, collect standard scene data as the field calibration dataset, solve the noise parameter set through maximum likelihood estimation, obtain the joint noise statistical characteristics after calibration, synthesize RAW frames and event streams, and construct an enhanced training dataset. S3: The asynchronous event stream is denoised using a recursive event denoiser (RED) to obtain a denoised event stream. APS spatial features and EVS dynamic features are extracted and then weighted and fused to obtain fused features. S4: Calculate the event rate based on the fusion features, construct the flame discrimination feature vector according to the statistical characteristics of the event rate, complete the flame scene discrimination through the classifier, and output the structured flame detection results.

2. The flame explosion detection method based on EVS event and APS image cascading as described in claim 1, characterized in that, The pixel-level fusion vision sensor in the S1 adopts a shared circuit architecture, with APS pixels and EVS pixels arranged alternately on the same chip; the sensor supports three working modes: image, event, and fusion. The Hybrid Visual Sensing Fusion Mode (HVS) is enabled by default in the flame explosion detection scenario; the sensor's APS resolution is no less than 768×608, and the EVS time resolution is better than 1ms.

3. The flame explosion detection method based on EVS event and APS image cascading as described in claim 1, characterized in that, The unified statistical imaging noise model in S2 corresponds to the pixel output formulas for APS and EVS as follows, covering four types of imaging noise: photon shot noise, dark current noise, fixed pattern noise, and quantization noise. The formula for APS pixel output is: ； In the formula, APS pixels In the passage The RAW value output at that location. This represents the expected signal strength of the corresponding pixel. For photon shot noise, This represents the shot noise gain coefficient. This is dark current noise. This is the mean coefficient for dark current. The dark current variance coefficient. Calculated based on a normal distribution. For exposure duration; This is a fixed-mode noise that includes line noise and black level correction noise; To read out noise; The EVS pixel output formula is: ； In the formula, For EVS pixels at time The detected change in voltage in the logarithmic domain. For the logarithmic change in irradiance signal term, , This is a comprehensive noise term that includes photon shot noise, dark current noise, fixed-mode noise, and readout noise. The EVS noise variance is correlated with illuminance level and time interval as follows: ； In the formula, Illuminance level, For EVS noise parameter set; Generate synthetic datasets using the H-ESIM statistical simulator Synthetic RAW frames and event streams are generated by combining noise statistical characteristics and compared with publicly available flame detection datasets. After merging, an enhanced training dataset is formed. , .

4. The flame explosion detection method based on the cascading of EVS events and APS images according to claim 3, characterized in that, The sub-steps for generating a synthetic dataset using the H-ESIM statistical simulator are as follows: S21: Perform the following steps on high frame rate input videos with a frame rate of at least 3200fps: The image is converted to RAW by inverse mapping and then corrected by inverse gamma to obtain a linear irradiance domain image. Then, the camera response domain image is obtained through inverse white balance and inverse color matrix transformation. ; ； ； In the formula, For a single frame Images, high frame rate video consists of multiple consecutive frames. The temporal sequence of images; S22: Rearrange the image into single-channel RAW format according to the CFA (Color Filter Array) rule; S23: Superimpose fixed noise terms and dark current noise terms, relying on the position dependence coefficients in the APS noise parameter set. The sampling illumination and exposure time are correlated with noise, and noisy RAW frames are output. S24: Match the camera bit depth to complete the quantization process and generate the composite RAW frame and the corresponding event stream.

5. The flame explosion detection method based on EVS event and APS image cascading as described in claim 1, characterized in that, In S3, the recursive event denoising unit (RED) filters out noise based on the difference in probability distribution between the valid signal and the noise, thus obtaining the denoised event stream. The denoised event stream satisfies a signal conditional density greater than the denoising conditional probability density. In the formula, It is an asynchronous event stream. , These are the parameters for the RED network. An adaptive fusion module employing either the FlexFuser or EvPlug architecture calculates fusion weights by combining real-time scene features. : ； In the formula, For APS spatial features, The EVS dynamic features are used; the fused features obtained after weighted fusion are as follows: ； In the formula, It is the Sigmoid activation function. It is a multilayer perceptron. For feature splicing operations, The value range is [0,1].

6. The flame explosion detection method based on EVS event and APS image cascading as described in claim 1, characterized in that, Event rate in S4 ; In the formula, For time window Cumulative number of events within; Flame discrimination feature vector The calculation formula is as follows: ； In the formula, The average event rate. The standard deviation of the event rate. The characteristics of flame flicker frequency. Spatial distribution density characteristics, Transpose of a vector; Using SVM classifier Complete the distinction between fire and non-fire scenes, where... For the weight vector, For bias terms; Output structured flame detection results ; In the formula, The bounding box of the flame target. To test the confidence level, To detect timestamps, For index number, This represents the total number of flame targets identified at the current detection moment.

7. The flame explosion detection method based on EVS event and APS image cascading as described in claim 5, characterized in that, The S3 also includes a processing flow for extreme environments, suitable for detection in high dynamic scenes and smoke-covered scenes; In high dynamic range (HDR) scenes, relying on EVS's native 120~140dB high dynamic range performance and combined with optical transmittance modulation, the absolute irradiance is decoded from the event timestamp, and the calculation formula is as follows: ； In the formula, The absolute irradiance obtained from decoding, For the polarity of the event, The EVS logarithmic field trigger threshold, The sampling interval is... For reference irradiance; In smoke-obscured scenarios, a multi-cue smoke probability discrimination model is constructed to distinguish between flame particle emission features and smoke scattering and obscuration features: ； In the formula The probability value for smoke is defined in the range [0,1]. The normalized value of APS image intensity. This represents the local density normalized value of the event. This represents the normalized value of the local brightness variance.

8. The flame explosion detection method based on the cascading of EVS events and APS images according to claim 1, characterized in that, It also includes closed-loop iteration steps: After deployment, data from the actual deployment scenario is collected as new calibration samples and fed back to S2 to update the noise parameter set through incremental learning. Minimize the model prediction error Complete parameter iteration and update; based on the updated parameters, drive H-ESIM to generate new synthetic data adapted to the real scene, and execute S3 after iteratively updating and enhancing the training dataset; In the formula, For loss function, For the detection model after parameter updates, For the returned samples and their corresponding true labels, N represents the total number of samples used to calculate the average loss.

9. A flame explosion detection system based on EVS event and APS image cascading, used to implement the steps of the flame explosion detection method based on EVS event and APS image cascading as described in any one of claims 1-8, characterized in that: It includes a noise modeling and calibration module, a data preprocessing and fusion module, a flame detection and output module, and a closed-loop iterative optimization module; The data preprocessing and fusion module includes a first feature extraction network, a second feature extraction network, and an adaptive cross-modal fusion module; The closed-loop iterative optimization module collects on-site deployment data and feeds it back to the noise modeling and calibration module.

10. The flame explosion detection system based on EVS event and APS image cascading as described in claim 9, characterized in that: The first feature extraction network uses the Swin-TinyTransformer, and the second feature extraction network uses either Event-ViT or Spike Neural Network (SNN). The flame detection and output module is equipped with the BEC-SVM algorithm based on background event counting, with a real-time processing frame rate of no less than 500 FPS.

Images