Infrared visible light flame detection system and method

Abstract

The invention discloses an infrared visible light flame detection system and method, and belongs to the technical field of computer vision detection. In order to solve the problems that sensor data is very redundant and has time delay, and the current flame detection technology is difficult to effectively integrate information from different modals and accurately position the flame position, the flame position is detected by analyzing infrared and visible light image data. The method is suitable for fire detection in large factories, warehouses and outdoor open large spaces such as forest parks, the target model can detect flames at high speed and high precision in a complex environment, the flames can be detected more accurately by combining thermal imaging data captured by an infrared camera and detail images provided by a visible light camera, and the target model can be applied to fire detection. Even under the influence of negative factors such as light, temperature and shielding, the stability and accuracy of the system can be kept, the accuracy and efficiency of flame detection are improved, and the method is especially suitable for severe environment conditions and occasions where a traditional method is difficult to apply.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision detection, and in particular to an infrared visible light flame detection system and method thereof. Background Art

[0002] Fire is one of the major disasters facing mankind today. Fire refers to a disaster caused by uncontrolled combustion in time or space. There are many causes of fire, including lightning strikes, spontaneous combustion, careless use of open flames, and improper use of gas or electrical appliances. Fire not only threatens people's lives and safety, but also causes huge property losses. According to the severity of the fire, it can be divided into general fire, major fire and extremely large fire. If it is not detected in time and corresponding measures are not taken in its early stage, it may lead to large-scale safety accidents, resulting in immeasurable consequences.

[0003] However, there are still the following problems in actual operation: Traditional fire detection systems are mostly based on sensor recognition. However, the data obtained in this way is often very redundant and has a certain delay, which often leads to inaccurate results. At the same time, current flame detection technology is difficult to effectively integrate information from different modes, and accurately identify the temperature anomalies caused by the flame and locate the flame position. The early selection of image features consumes a lot of manpower and material resources, and it is difficult to make accurate judgments without affecting real-time performance. Summary of the invention

[0004] The object of the present invention is to provide an infrared visible light flame detection system and method thereof to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: an infrared visible light flame detection system, comprising: Image acquisition module for: Collect infrared images and visible light images synchronously, and process the infrared images and visible light images synchronously to make the two images consistent in time and space; Feature extraction module, used to: Select the VGG16 deep convolutional network model as the feature extraction model, use the feature extraction model to extract temperature anomaly features from infrared images, and use the feature extraction model to extract the color and shape features of the fire source from visible light images; Feature Registration Module for: A feature registration network is introduced, and the affine transformation parameters are calculated through the registration network to register the infrared image to the visible light image. The correlation between the infrared feature map and the visible light feature map is solved to obtain a correlation map with corresponding correlation. The correlation map is input into the regression network model to predict the affine transformation parameters of the infrared image to be registered to the visible light image. Based on the affine transformation parameters, all the infrared images to be registered are registered to the visible light image. Feature fusion module for: Based on the Vision Transformer algorithm, the registered infrared image and visible light image are fused to generate a fused feature map containing both infrared image features and visible light image features; Flame detection modules for: Introduce the flame detection network to perform target detection on the fused feature map, detect the flame position on the fused feature map based on the YOLOv8 algorithm and output the flame position coordinates; Result output module, used for: The flame detection results are generated based on the flame position coordinates and output in the form of visualization and data. At the same time, the data and alarm information are remotely transmitted to the monitoring center and mobile devices through network connection.

[0006] Furthermore, the image acquisition module includes: Infrared cameras for: Capture infrared images with thermal radiation information; Visible light cameras for: Capture visible light images with texture information; Wherein, the infrared camera and the visible light camera both have high-resolution sensors.

[0007] Furthermore, the feature extraction module includes: Infrared feature extraction unit, used for: Extract temperature anomaly features from infrared images, input the infrared images into the VGG16 model, extract the feature map of the infrared image through the forward propagation of VGG16, perform activation function operations on the feature map, highlight the temperature anomaly features, and output the extracted infrared feature map; Visible light feature extraction unit, used for: The color and shape features of the fire source are extracted from the visible light image, and the visible light image is input into the VGG16 model. Through the forward propagation of VGG16, the feature map of the visible light image is extracted, and the feature map is processed by applying the hole-spatial convolution pooling pyramid structure to increase the receptive field and capture more feature map details, and the extracted visible light feature map is output.

[0008] Furthermore, the feature registration module includes: Parameter calculation unit, used for: Input the feature maps of the infrared image and visible light image to be registered, and use the feature registration network to process the two sets of feature maps. The network estimates the affine transformation parameters by comparing the similarity of the feature maps. The affine transformation parameters include rotation, scaling, and translation. The affine transformation parameters are used to describe the geometric transformation of the infrared image relative to the visible light image, and the calculated affine transformation parameters are output.

[0009] Furthermore, the feature registration module further includes: Feature map registration unit, used to: Input the feature map of the infrared image, the feature map of the visible light image, and the affine transformation parameters, perform geometric transformation on the feature map of the infrared image according to the affine transformation parameters to align it with the feature map of the visible light image, and fill the pixel values ​​of the transformed feature map through bilinear interpolation to make the size and resolution of the feature map consistent with the feature map of the visible light image. Figure 1 The registered infrared image feature map is output.

[0010] Furthermore, the feature fusion module further includes: Feature preprocessing unit, used for: Preprocess the features of the registered infrared image and visible light image, input the feature maps of the registered infrared image and visible light image, normalize the feature maps to make the scale and range of the feature maps consistent, remove noise in the feature maps, enhance the feature maps to highlight the feature information in the feature maps, and output the preprocessed feature maps; Feature fusion unit, used for: The features of the preprocessed infrared light image and the visible light image are fused, the feature maps of the preprocessed infrared light image and the visible light image are input, the feature maps are fused pixel by pixel and feature by feature to generate a fused feature map, the fused feature map is smoothed and sharpened to improve the fusion effect, and the fused feature map is output.

[0011] Furthermore, the flame detection module comprises: Detection network building blocks for: Construct a deep learning model for flame detection, select a deep learning framework to construct a flame detection network, wherein constructing the flame detection network includes defining a network structure, selecting an activation function, setting a loss function, initializing the flame detection network, and outputting the constructed flame detection network model.

[0012] Furthermore, the flame detection module further includes: Flame position detection unit for: The fused feature map is input into the flame detection network for forward propagation processing to obtain the flame position coordinates output by the flame detection network. The output coordinates are decoded and processed to obtain the actual flame position, and the detected flame position coordinates are output to the result output module.

[0013] Furthermore, the result output module includes: Display devices for: Real-time display of flame detection results; Results visualization unit for: Input the flame position coordinates output by the flame detection module, mark the flame position on the original image and the fusion feature map according to the coordinate information, generate visualization results, format the flame position coordinates, detection time, and image source information into structured data, and output visualization results and structured data; Telecommunications unit for: Input visualization results and structured data, package the data into a format for network transmission, and send the data to the monitoring center and the designated receiving end of the mobile device through the network connection. The receiving end parses the data packet and restores the visualization results and structured data; An alarm generating unit for: Generate an alarm when a flame is detected, and notify relevant personnel in a variety of ways. Enter the flame detection results. When a flame is detected, generate an alarm message according to the preset alarm rules and send the alarm message to the relevant personnel's devices and applications.

[0014] Another technical problem to be solved by the present invention is to provide an infrared visible light flame detection method, comprising the following steps: Step 1: Image acquisition and synchronous processing: use an infrared camera to capture infrared images with thermal radiation information, use a visible light camera to capture visible light images with texture information, and synchronously process the infrared image and visible light image to ensure that the two images are synchronized in time and space; Step 2: Feature extraction: extract temperature anomaly features from infrared images and output infrared feature maps, extract color and shape features of fire sources from visible light images and output visible light feature maps; Step 3: Feature registration: introduce the feature registration network, calculate the affine transformation parameters, register the infrared image to the visible light image, solve the correlation between the infrared feature map and the visible light feature map, obtain a correlation map with corresponding correlation, predict the affine transformation parameters of the infrared image to the visible light image, and based on the affine transformation parameters, register all the infrared images to be registered to the visible light image; Step 4: Feature fusion: preprocess the features of the registered infrared image and visible light image, fuse the preprocessed features of the infrared image and visible light image, and generate a fused feature map; Step 5: Flame detection: Build a deep learning model for flame detection and initialize it. Input the fused feature map into the flame detection network for forward propagation processing to obtain the flame position coordinates. Decode and process the output coordinates to obtain the actual flame position. Step 6: Result output and alarm, generate visual results and structured data, display the flame detection results in real time through the display device, send the data to the monitoring center and mobile devices, generate an alarm when a flame is detected, and notify relevant personnel in various ways.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention detects the flame position by analyzing infrared and visible light image data, and is suitable for fire detection in large factories, warehouses and large outdoor open spaces such as forest parks. The target model can detect the existence of flames at high speed and high precision in complex environments. By combining the thermal imaging data captured by the infrared camera and the detailed image provided by the visible light camera, the flame can be detected more accurately. Even under the influence of negative factors such as light, temperature, and occlusion, the stability and accuracy of the system can be maintained, thereby improving the accuracy and efficiency of flame detection, especially in harsh environmental conditions and occasions where traditional methods are difficult to apply.

[0016] 2. The present invention extracts temperature anomaly features from infrared images, highlights temperature anomaly areas associated with flames, helps the system identify flames more accurately, extracts color and shape features of fire sources from visible light images, and increases the receptive field to capture more detail information, thereby improving the extraction effect of visible light features of flames. By extracting flame features from infrared and visible light images respectively and fusing these features, the system can more comprehensively analyze the characteristics of flames, thereby improving the accuracy of flame detection. The design of the feature extraction module enables the system to accurately detect flames in complex environments, such as light changes, occlusions, etc.

[0017] 3. The present invention compares the similarity of the feature maps of the infrared light image and the visible light image through a feature registration network, estimates the precise affine transformation parameters, and can accurately describe the geometric transformation of the infrared light image relative to the visible light image, so as to align it with the feature map of the visible light image, so that the infrared light image and the visible light image are aligned at the feature level, and the features of the two images are fused to generate a fused feature map containing richer information, thereby improving the accuracy of flame detection. Due to the difference in imaging principles between infrared images and visible light images, their adaptability to different environmental conditions is also different. Through feature fusion, the system can comprehensively utilize the advantages of the two images, thereby maintaining good detection performance under different environmental conditions.

[0018] 4. The present invention obtains the flame position coordinates by inputting the fused feature map into the flame detection network for forward propagation processing. By decoding and processing the output coordinates, the system can more accurately locate the actual position of the flame, which helps to respond to fire incidents in a timely and accurate manner and reduce false alarms and missed alarms. Through the training and learning of the deep learning model, the flame detection module can adapt to the flame detection needs in different scenarios. Due to the fusion of the features of infrared and visible light images, the system can maintain stable detection performance in complex environments, such as light changes, occlusions, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a module schematic diagram of the infrared visible light flame detection system of the present invention; Figure 2 This is a schematic diagram of the steps of infrared visible light flame detection of the present invention; Figure 3 It is a schematic diagram of the detection process of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In order to solve the problem that most traditional fire detection systems are based on sensor recognition, the data obtained in this way is often very redundant and has a certain delay, which often leads to inaccurate results. At the same time, the current flame detection technology is difficult to effectively integrate information from different modes, accurately identify the temperature anomaly generated by the flame and locate the flame position. The early stage of image feature selection consumes a lot of manpower and material resources, and it is difficult to make accurate judgments without affecting real-time performance. Please refer to Figure 1-3 , the present invention provides the following technical solutions: An infrared visible light flame detection system, comprising: Image acquisition module for: Collect infrared images and visible light images synchronously, and process the infrared images and visible light images synchronously to make the two images consistent in time and space; Feature extraction module, used to: Select the VGG16 deep convolutional network model as the feature extraction model, use the feature extraction model to extract temperature anomaly features from infrared images, and use the feature extraction model to extract the color and shape features of the fire source from visible light images; Feature Registration Module for: A feature registration network is introduced, and the affine transformation parameters are calculated through the registration network to register the infrared image to the visible light image. The correlation between the infrared feature map and the visible light feature map is solved to obtain a correlation map with corresponding correlation. The correlation map is input into the regression network model to predict the affine transformation parameters of the infrared image to be registered to the visible light image. Based on the affine transformation parameters, all the infrared images to be registered are registered to the visible light image. Feature fusion module for: Based on the Vision Transformer algorithm, the registered infrared image and visible light image are fused to generate a fused feature map containing both infrared image features and visible light image features; Flame detection modules for: Introduce the flame detection network to perform target detection on the fused feature map, detect the flame position on the fused feature map based on the YOLOv8 algorithm and output the flame position coordinates; Result output module, used for: The flame detection results are generated based on the flame position coordinates and output in the form of visualization and data. At the same time, the data and alarm information are remotely transmitted to the monitoring center and mobile devices through network connection.

[0022] Specifically, the flame position is detected by analyzing infrared and visible light image data. It is suitable for fire detection in large factories, warehouses and large outdoor open spaces such as forest parks. The target model can detect the presence of flames at high speed and high precision in complex environments. By combining the thermal imaging data captured by the infrared camera and the detailed images provided by the visible light camera, the flame can be detected more accurately. Even under the influence of negative factors such as light, temperature, and occlusion, the system stability and accuracy can be maintained, improving the accuracy and efficiency of flame detection, especially in harsh environmental conditions and occasions where traditional methods are difficult to apply.

[0023] In the above embodiment, by combining the data of infrared images and visible light images, the system can more comprehensively analyze the characteristics of the flame, including temperature anomalies, color, shape, etc., thereby improving the accuracy of detection. At the same time, the use of advanced deep learning models and algorithms, such as VGG16 and YOLOv8, enables the system to detect the presence of flames at high speed and high precision in complex environments. Even under the influence of negative factors such as light, temperature, and occlusion, the system can maintain stability and accuracy. This is due to the fusion of infrared and visible light images, as well as the effective operation of the feature registration module, which enables the system to better adapt to various harsh environmental conditions. In the above embodiment, this technical solution is not only applicable to indoor environments such as large factories and warehouses, but also to fire detection in large outdoor open spaces such as forest parks, and has broad application prospects. The result output module can output the flame detection results in a visual and data form, and remotely transmit the data and alarm information to the monitoring center and mobile devices through a network connection, so that relevant personnel can obtain fire information in a timely manner and take corresponding countermeasures.

[0024] Image acquisition module, including: Infrared cameras for: Capture infrared images with thermal radiation information; Visible light cameras for: Capture visible light images with texture information; Wherein, the infrared camera and the visible light camera both have high-resolution sensors.

[0025] In the above embodiment, by combining the data of infrared images and visible light images, the system can more comprehensively analyze the characteristics of the flame, including temperature anomalies, color, shape, etc., thereby improving the accuracy of detection. At the same time, the use of high-resolution sensors enables the system to detect the presence of flames with high precision in complex environments. Even under the influence of negative factors such as light, temperature, and occlusion, the system can maintain stability and accuracy. This is due to the fusion of infrared and visible light images, which enables the system to better adapt to various harsh environmental conditions.

[0026] Feature extraction module, including: Infrared feature extraction unit, used for: Extract temperature anomaly features from infrared images, input the infrared images into the VGG16 model, extract the feature map of the infrared image through the forward propagation of VGG16, perform activation function operations on the feature map, highlight the temperature anomaly features, and output the extracted infrared feature map.

[0027] Visible light feature extraction unit, used for: The color and shape features of the fire source are extracted from the visible light image, and the visible light image is input into the VGG16 model. Through the forward propagation of VGG16, the feature map of the visible light image is extracted, and the feature map is processed by applying the hole-spatial convolution pooling pyramid structure to increase the receptive field and capture more feature map details, and the extracted visible light feature map is output.

[0028] In the above embodiment, the infrared feature extraction unit uses the VGG16 model to extract temperature anomaly features from the infrared image. Through activation function operation processing, the temperature anomaly area related to the flame is highlighted, which helps the system to identify the flame more accurately. The visible light feature extraction unit also uses the VGG16 model to extract the color and shape features of the fire source from the visible light image, and applies the hole-space convolution pooling pyramid structure to increase the receptive field, capture more detailed information, and improve the extraction effect of the visible light features of the flame. In the above embodiment, by extracting flame features from infrared and visible light images respectively and fusing these features, the system can analyze the characteristics of flames more comprehensively, thereby improving the accuracy of flame detection. The design of the feature extraction module enables the system to accurately detect flames in complex environments, such as light changes, occlusions, etc. In the above embodiment, the feature extraction module adopts the deep learning model VGG16, which has powerful feature extraction capabilities and can adapt to the flame detection needs in different scenarios. By combining infrared and visible light image data, the system can better adapt to various harsh environmental conditions, such as night, foggy days, etc., and maintain stable detection performance. The feature extraction module uses the forward propagation of the VGG16 model to quickly extract image features, combined with the efficient processing of the void-spatial convolution pooling pyramid structure, to improve the processing efficiency of the system, which enables the system to perform well in real-time flame detection applications and respond to fire incidents in a timely manner.

[0029] Feature registration module, including: Parameter calculation unit, used for: Input the feature maps of the infrared image and visible light image to be registered, and use the feature registration network to process the two sets of feature maps. The network estimates the affine transformation parameters by comparing the similarity of the feature maps. The affine transformation parameters include rotation, scaling, and translation. The affine transformation parameters are used to describe the geometric transformation of the infrared image relative to the visible light image, and the calculated affine transformation parameters are output.

[0030] Feature map registration unit, used to: Input the feature map of the infrared image, the feature map of the visible light image, and the affine transformation parameters, perform geometric transformation on the feature map of the infrared image according to the affine transformation parameters to align it with the feature map of the visible light image, and fill the pixel values ​​of the transformed feature map through bilinear interpolation to make the size and resolution of the feature map consistent with the feature map of the visible light image. Figure 1 The registered infrared image feature map is output.

[0031] In the above embodiment, the parameter calculation unit compares the similarity of the feature maps of the infrared image and the visible light image through the feature registration network, and estimates the precise affine transformation parameters, which can accurately describe the geometric transformation of the infrared image relative to the visible light image. This feature-based registration method is more robust and accurate than the traditional pixel-based method, especially when processing infrared and visible light images with large differences. In the above embodiment, the feature map registration unit geometrically transforms the feature map of the infrared light image according to the affine transformation parameters to align it with the feature map of the visible light image. This process fills the pixel values ​​through bilinear interpolation to ensure that the transformed feature map is consistent with the feature map of the visible light image in size and resolution, effectively reducing false detections caused by image differences and improving the stability of the system in various environments. In the above embodiment, the design of the feature registration module enables the infrared light image and the visible light image to be aligned at the feature level, providing a more accurate and consistent data basis for subsequent feature fusion and flame detection, helping to improve the efficiency of flame detection, and enabling the system to respond to fire incidents more quickly.

[0032] Feature fusion module, including: Feature preprocessing unit, used for: The features of the registered infrared light image and visible light image are preprocessed, the feature maps of the registered infrared light image and visible light image are input, the feature maps are normalized to make the scale and range of the feature maps consistent, the noise in the feature maps is removed, the feature maps are enhanced to highlight the feature information in the feature maps, and the preprocessed feature maps are output.

[0033] Feature fusion unit, used for: The features of the preprocessed infrared light image and the visible light image are fused, the feature maps of the preprocessed infrared light image and the visible light image are input, the feature maps are fused pixel by pixel and feature by feature to generate a fused feature map, the fused feature map is smoothed and sharpened to improve the fusion effect, and the fused feature map is output.

[0034] In the above embodiment, the features of the infrared light image and the visible light image are preprocessed by the feature preprocessing unit to remove noise and highlight the feature information, thereby providing a high-quality data basis for subsequent feature fusion. The feature fusion unit further fuses the features of the two images to generate a fused feature map containing richer information, thereby improving the accuracy of flame detection. Due to the difference in imaging principles between infrared images and visible light images, their adaptability to different environmental conditions is also different. Through feature fusion, the system can comprehensively utilize the advantages of the two images, thereby maintaining good detection performance under different environmental conditions and enhancing the robustness of the system.

[0035] Flame detection module, including: Detection network building blocks for: Construct a deep learning model for flame detection, select a deep learning framework to construct a flame detection network, wherein constructing the flame detection network includes defining a network structure, selecting an activation function, setting a loss function, initializing the flame detection network, and outputting the constructed flame detection network model.

[0036] Flame position detection unit for: The fused feature map is input into the flame detection network for forward propagation processing to obtain the flame position coordinates output by the flame detection network. The output coordinates are decoded and processed to obtain the actual flame position, and the detected flame position coordinates are output to the result output module.

[0037] In the above embodiment, the detection network construction unit provides a powerful model foundation for flame detection by constructing a deep learning model and carefully selecting the network structure, activation function and loss function, so that the system can more accurately identify flame features from the fused feature map, thereby improving the accuracy of flame detection. The flame position detection unit inputs the fused feature map into the flame detection network for forward propagation processing to obtain the flame position coordinates. By decoding and processing the output coordinates, the system can more accurately locate the actual position of the flame, which helps to respond to fire incidents in a timely and accurate manner and reduce false alarms and missed alarms. In the above embodiment, the deep learning model can quickly process input data during the forward propagation process and output the flame position coordinates, so that the system can perform well in real-time flame detection applications, provide fire alarms and location information in a timely manner, and win precious time for fire fighting. Through the training and learning of the deep learning model, the flame detection module can adapt to the flame detection needs in different scenarios. At the same time, due to the fusion of the features of infrared light and visible light images, the system can still maintain stable detection performance in complex environments, such as light changes, occlusions, etc.

[0038] The result output module includes: Display devices for: Real-time display of flame detection results; Results visualization unit for: Input the flame position coordinates output by the flame detection module, mark the flame position on the original image and the fusion feature map according to the coordinate information, generate visualization results, format the flame position coordinates, detection time, and image source information into structured data, and output visualization results and structured data; Telecommunications unit for: Input visualization results and structured data, package the data into a format for network transmission, and send the data to the monitoring center and the designated receiving end of the mobile device through the network connection. The receiving end parses the data packet and restores the visualization results and structured data; An alarm generating unit for: Generate an alarm when a flame is detected, and notify relevant personnel in a variety of ways. Enter the flame detection results. When a flame is detected, generate an alarm message according to the preset alarm rules and send the alarm message to the relevant personnel's devices and applications.

[0039] In the above embodiment, by fusing the features of infrared and visible light images, the system can identify flames more accurately and reduce false alarms and missed alarms. The display device displays the flame detection results in real time. The result visualization unit formats information such as the flame position into structured data for easy monitoring and analysis. The remote communication unit sends data to the monitoring center and mobile devices in real time, so that relevant personnel can quickly obtain fire information and respond. The alarm generation unit generates an alarm when a flame is detected and notifies relevant personnel in various ways to improve the efficiency of fire emergency response.

[0040] This embodiment now proposes an infrared visible light flame detection method, comprising the following steps: Step 1: Image acquisition and synchronous processing: use an infrared camera to capture infrared images with thermal radiation information, use a visible light camera to capture visible light images with texture information, and synchronously process the infrared image and visible light image to ensure that the two images are synchronized in time and space; Step 2: Feature extraction: extract temperature anomaly features from infrared images and output infrared feature maps, extract color and shape features of fire sources from visible light images and output visible light feature maps; Step 3: Feature registration: introduce the feature registration network, calculate the affine transformation parameters, register the infrared image to the visible light image, solve the correlation between the infrared feature map and the visible light feature map, obtain a correlation map with corresponding correlation, predict the affine transformation parameters of the infrared image to the visible light image, and based on the affine transformation parameters, register all the infrared images to be registered to the visible light image; Step 4: Feature fusion: preprocess the features of the registered infrared image and visible light image, fuse the preprocessed features of the infrared image and visible light image, and generate a fused feature map; Step 5: Flame detection: Build a deep learning model for flame detection and initialize it. Input the fused feature map into the flame detection network for forward propagation processing to obtain the flame position coordinates. Decode and process the output coordinates to obtain the actual flame position. Step 6: Result output and alarm, generate visual results and structured data, display the flame detection results in real time through the display device, send the data to the monitoring center and mobile devices, generate an alarm when a flame is detected, and notify relevant personnel in various ways.

[0041] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An infrared visible light flame detection system, characterized in that: include: Image acquisition module for: Collect infrared images and visible light images synchronously, and process the infrared images and visible light images synchronously to make the two images consistent in time and space; Feature extraction module, used to: Select the VGG16 deep convolutional network model as the feature extraction model, use the feature extraction model to extract temperature anomaly features from infrared images, and use the feature extraction model to extract the color and shape features of the fire source from visible light images; Feature Registration Module for: A feature registration network is introduced, and the affine transformation parameters are calculated through the registration network to register the infrared image to the visible light image. The correlation between the infrared feature map and the visible light feature map is solved to obtain a correlation map with corresponding correlation. The correlation map is input into the regression network model to predict the affine transformation parameters of the infrared image to be registered to the visible light image. Based on the affine transformation parameters, all the infrared images to be registered are registered to the visible light image. Feature fusion module for: Based on the Vision Transformer algorithm, the registered infrared image and visible light image are fused to generate a fused feature map containing both infrared image features and visible light image features; Flame detection modules for: Introduce the flame detection network, use the fused features to perform target detection, predict the flame position on the fused feature map based on the YOLOv8 algorithm and output the flame position coordinates; Result output module, used for: The flame detection results are generated based on the flame position coordinates and output in the form of visualization and data. At the same time, the data and alarm information are remotely transmitted to the monitoring center and mobile devices through network connection.

2. An infrared visible light flame detection system as claimed in claim 1, characterized in that: The image acquisition module comprises: Infrared cameras for: Capture infrared images with thermal radiation information; Visible light cameras for: Capture visible light images with texture information; Wherein, the infrared camera and the visible light camera both have high-resolution sensors.

3. An infrared visible light flame detection system as claimed in claim 1, characterized in that: The feature extraction module comprises: Infrared feature extraction unit, used for: Extract temperature anomaly features from infrared images, input the infrared images into the VGG16 model, extract the feature map of the infrared image through the forward propagation of VGG16, perform activation function operations on the feature map, highlight the temperature anomaly features, and output the extracted infrared feature map; Visible light feature extraction unit, used for: The color and shape features of the fire source are extracted from the visible light image, and the visible light image is input into the VGG16 model. Through the forward propagation of VGG16, the feature map of the visible light image is extracted, and the feature map is processed by applying the hole-spatial convolution pooling pyramid structure to increase the receptive field and capture more feature map details, and the extracted visible light feature map is output.

4. An infrared visible light flame detection system as claimed in claim 1, characterized in that: The feature registration module comprises: Parameter calculation unit, used for: Input the feature maps of the infrared image and visible light image to be registered, and use the feature registration network to process the two sets of feature maps. The network estimates the affine transformation parameters by comparing the similarity of the feature maps. The affine transformation parameters include rotation, scaling, and translation. The affine transformation parameters are used to describe the geometric transformation of the infrared image relative to the visible light image, and the calculated affine transformation parameters are output.

5. An infrared visible light flame detection system as claimed in claim 4, characterized in that: The feature registration module further includes: Feature map registration unit, used to: Input the feature map of the infrared image, the feature map of the visible light image, and the affine transformation parameters, perform geometric transformation on the feature map of the infrared image according to the affine transformation parameters to align it with the feature map of the visible light image, fill the pixel values ​​of the transformed feature map through bilinear interpolation to make the size and resolution of the feature map consistent with those of the visible light image, and output the aligned feature map of the infrared image.

6. An infrared visible light flame detection system as claimed in claim 1, characterized in that: The feature fusion module further includes: Feature preprocessing unit, used for: Preprocess the features of the registered infrared image and visible light image, input the feature maps of the registered infrared image and visible light image, normalize the feature maps to make the scale and range of the feature maps consistent, remove noise in the feature maps, enhance the feature maps to highlight the feature information in the feature maps, and output the preprocessed feature maps; Feature fusion unit, used for: The features of the preprocessed infrared light image and the visible light image are fused, the feature maps of the preprocessed infrared light image and the visible light image are input, the feature maps are fused pixel by pixel and feature by feature to generate a fused feature map, the fused feature map is smoothed and sharpened to improve the fusion effect, and the fused feature map is output.

7. An infrared visible light flame detection system as claimed in claim 1, characterized in that: The flame detection module comprises: Detection network building blocks for: Construct a deep learning model for flame detection, select a deep learning framework to construct a flame detection network, wherein constructing the flame detection network includes defining a network structure, selecting an activation function, setting a loss function, initializing the flame detection network, and outputting the constructed flame detection network model.

8. An infrared visible light flame detection system as claimed in claim 7, characterized in that: The flame detection module further includes: Flame position detection unit for: The fused feature map is input into the flame detection network for forward propagation processing to obtain the flame position coordinates output by the flame detection network. The output coordinates are decoded and processed to obtain the actual flame position, and the detected flame position coordinates are output to the result output module.

9. An infrared visible light flame detection system as claimed in claim 1, characterized in that: The result output module comprises: Display devices for: Real-time display of flame detection results; Results visualization unit for: Input the flame position coordinates output by the flame detection module, mark the flame position on the original image and the fusion feature map according to the coordinate information, generate visualization results, format the flame position coordinates, detection time, and image source information into structured data, and output visualization results and structured data; Telecommunications unit for: Input visualization results and structured data, package the data into a format for network transmission, and send the data to the monitoring center and the designated receiving end of the mobile device through the network connection. The receiving end parses the data packet and restores the visualization results and structured data; An alarm generating unit for: Generate an alarm when a flame is detected, and notify relevant personnel in a variety of ways. Enter the flame detection results. When a flame is detected, generate an alarm message according to the preset alarm rules and send the alarm message to the relevant personnel's devices and applications.

10. An infrared visible light flame detection system, implemented based on the infrared visible light flame detection system according to any one of claims 1 to 9, characterized in that: The steps include: Step 1: Image acquisition and synchronous processing: use an infrared camera to capture infrared images with thermal radiation information, use a visible light camera to capture visible light images with texture information, and synchronously process the infrared image and visible light image to ensure that the two images are synchronized in time and space; Step 2: Feature extraction: extract temperature anomaly features from infrared images and output infrared feature maps, extract color and shape features of fire sources from visible light images and output visible light feature maps; Step 3: Feature registration: introduce the feature registration network, calculate the affine transformation parameters, register the infrared image to the visible light image, solve the correlation between the infrared feature map and the visible light feature map, obtain a correlation map with corresponding correlation, predict the affine transformation parameters of the infrared image to the visible light image, and based on the affine transformation parameters, register all the infrared images to be registered to the visible light image; Step 4: Feature fusion: preprocess the features of the registered infrared image and visible light image, fuse the preprocessed features of the infrared image and visible light image, and generate a fused feature map; Step 5: Flame detection: Build a deep learning model for flame detection and initialize it. Input the fused feature map into the flame detection network for forward propagation processing to obtain the flame position coordinates. Decode and process the output coordinates to obtain the actual flame position. Step 6: Result output and alarm, generate visual results and structured data, display the flame detection results in real time through the display device, send the data to the monitoring center and mobile devices, generate an alarm when a flame is detected, and notify relevant personnel in various ways.

Images