Gas detection method, program product and device

Abstract

This invention discloses a gas detection method, program product, and device. The method includes: acquiring consecutive frame images; forming input data for a pre-trained gas detection model based on the consecutive frame images; outputting gas detection data through the gas detection model based on the input data, wherein the gas detection data includes gas position data in the current frame and a current gas feature map in the current frame; determining a gas mask map based on the gas detection data; and color-displaying the target gas region indicated by the gas mask map in the current frame.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a gas detection method, computer program product, and gas detection equipment. Background Technology

[0002] In recent years, the demand for gas detection and monitoring in industrial, environmental, and safety monitoring fields has been increasing. Traditional gas detection methods typically rely on sensors installed at fixed locations, but their performance is poor in complex backgrounds and various scenarios. For example, in dynamic scenarios where the environment is constantly changing, the accuracy of gas detection using traditional methods is low. Current gas detection algorithms mainly focus on the detection of specific fixed scenarios or specific gases, and the joint detection of multiple gases in complex backgrounds remains a challenge. For instance, multiple harmful gases, such as formaldehyde, hydrogen sulfide, and carbon monoxide, are often present simultaneously in industrial production environments. In urban environments, gas sources are diverse and complex, including vehicle exhaust, industrial waste gas, and combustion emissions. In these situations, traditional gas detection methods struggle to provide real-time detection and do not offer a clear visual representation of the detected gas area. Summary of the Invention

[0003] To address the existing technical problems, this invention provides a gas detection method, a computer program product, and a gas detection device, which can accurately detect gas areas and display the gas areas in color, making it easy for users to intuitively observe changes in the gas areas.

[0004] In a first aspect, a gas detection method is provided, comprising: acquiring consecutive frame images; forming input data for a pre-trained gas detection model based on the consecutive frame images; outputting gas detection data through the gas detection model based on the input data, wherein the gas detection data includes gas position data in the current frame of the consecutive frame images and a current gas feature map in the current frame; determining a gas mask map based on the gas detection data; and color-displaying the target gas region indicated by the gas mask map in the current frame.

[0005] In a second aspect, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the gas detection method as described in any of the first aspects of this application.

[0006] Thirdly, a gas detection device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the gas detection method as described in any of the first aspects of this application.

[0007] Fourthly, a storage medium is provided, including a computer program, storing the computer program, which, when executed by a processor, causes the processor to perform the gas detection method described in any of the first aspects of this application.

[0008] This application embodiment acquires continuous frame images and forms input data for a pre-trained gas detection model based on these images. Since gas is a changing target, forming input data based on continuous frame images facilitates better capture of gas features. Because the gas detection model is a pre-trained model with fixed parameters, it can directly process continuous frame images, meeting real-time requirements. Furthermore, based on gas position data, it further extracts precise gas mask regions from the current gas feature map, thereby improving the accuracy of gas detection. In the current frame, the target gas region indicated by the gas mask is colored for easy observation of changes in the gas region by the user. Attached Figure Description

[0009] Figure 1 This is a diagram illustrating the application environment of a gas detection method in one embodiment;

[0010] Figure 2 This is a flowchart of a gas detection method in one embodiment;

[0011] Figure 3 This is a schematic diagram showing the gas distribution in one embodiment;

[0012] Figure 4 This is a flowchart showing the gas distribution in a target gas region in a gas detection method according to one embodiment;

[0013] Figure 5 This is a schematic diagram illustrating the generation of a connection feature map in one embodiment;

[0014] Figure 6 This is a schematic diagram illustrating the generation of a connection feature map in another embodiment;

[0015] Figure 7 This is a schematic diagram of training a gas detection model in one embodiment;

[0016] Figure 8 This is a schematic diagram of a gas detection device in one embodiment;

[0017] Figure 9 This is a schematic diagram of a gas detection device in one embodiment. Detailed Implementation

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0020] In the following description, the expression “some embodiments” refers to a subset of all possible embodiments. However, it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] See Figure 1 This diagram illustrates the application environment of a gas detection method in one embodiment. The gas detection method is applied in a gas detection device 10, which includes an image acquisition device 12, a processor 13, and a memory 14. The image acquisition device 12 acquires continuous frame images in front of the gas detection device 10, and the processor 13 detects gas regions in the continuous frame images based on the images acquired by the image acquisition device 12. The memory 14 stores the program and data corresponding to the implementation of the gas detection method.

[0022] The gas detection device 10 includes, but is not limited to, handheld detection devices, non-handheld autonomously movable detection devices, and non-handheld, non-autonomous detection devices. Handheld detection devices include, but are not limited to, handheld imaging devices with infrared thermal imaging capabilities and handheld imaging devices with visible light imaging capabilities. Non-handheld, non-autonomous detection devices include, but are not limited to, autonomously movable detection devices with infrared thermal imaging capabilities and autonomously movable detection devices with visible light imaging capabilities. Non-handheld, non-autonomous detection devices include, but are not handheld and not autonomously movable devices with infrared thermal imaging capabilities and non-handheld and not autonomously movable devices with visible light imaging capabilities. Using handheld and non-handheld, autonomously movable detection devices, users can perform gas detection on the current scene during movement or on a fixed scene. Non-handheld, non-autonomous detection devices can perform gas detection on a fixed scene. Therefore, the gas detection method provided in this application can be applied to various complex moving and changing scenes as well as fixed scenes.

[0023] The image acquisition device 12 can be a combination of one or more sensors. The image acquisition device 12 can be a monocular vision sensor or a multi-view vision sensor. For example, it can be a combination of one or more sensors, such as a thermal imaging sensor, a visible light image sensor, a millimeter-wave sensor, a lidar sensor, an infrared thermal imaging sensor, and a depth sensor.

[0024] The processor 13 can be one or more. When there are multiple processors 13, they can be integrated on a single chip or independently located on each chip. The gas detection device 10 can be installed on any type of mobile body, such as vehicles, electric vehicles, hybrid electric vehicles, motorcycles, bicycles, personal mobile devices, airplanes, drones, ships, or robots, etc. The gas detection device 10 can also be fixedly installed on a fixed device or at a fixed location in a fixed scene.

[0025] The gas detection device 10 may also include other sensor modules, including but not limited to environmental sensing sensors and motion attitude sensors. Environmental sensing sensors include, but are not limited to, one or more combinations of the following: brightness sensors, temperature sensors, haze sensors, etc. Motion attitude sensors include, but are not limited to, one or more combinations of the following: inertial measurement units (IMUs), velocity sensors, acceleration sensors, gyroscope sensors, geomagnetic sensors, rotation vector sensors, steering wheel angle sensors, level sensors, tilt sensors, vibration sensors, displacement sensors, and gravity sensors, etc.

[0026] The gas detection device 10 may also include a display terminal for displaying images.

[0027] Please see Figure 2 This is a flowchart of a gas detection method provided in an embodiment of this application. The gas detection method is applied in a gas detection device and includes the following steps:

[0028] S11. Obtain consecutive frame images.

[0029] In this embodiment, continuous frame images include, but are not limited to, continuous frame infrared images and continuous frame fused images obtained by fusing continuous frame infrared images and continuous frame visible light images. Continuous frame infrared images and / or continuous frame visible light images are acquired by the image acquisition device 12. When acquiring continuous frame infrared images and continuous frame visible light images synchronously in time, mapping the continuous frame infrared images and continuous frame visible light images to the same image coordinate system can achieve spatial synchronization of the continuous frame infrared images and continuous frame visible light images. Then, using an image fusion method, the continuous frame infrared images and continuous frame visible light images are fused to obtain a continuous frame fused image. Since the gas is changing, using continuous frame images makes it easier to extract gas features subsequently.

[0030] S12. Based on the continuous frame images, input data for a pre-trained gas detection model is formed. Based on the input data, gas detection data is output through the gas detection model. The gas detection data includes gas position data in the current frame of the continuous frame images and the current gas feature map in the current frame.

[0031] In this embodiment, the gas detection model is trained using a training dataset. The gas detection model includes a feature extraction model and a gas location detection network. After training, the parameters in the feature extraction model and the gas location detection network are fixed. Input data is obtained by preprocessing consecutive frame images. Preprocessing includes, but is not limited to, denoising, cropping, and background processing. In an optional implementation, consecutive frame images can also be directly used as input data.

[0032] In this embodiment, since the gas detection model is a model pre-trained using a training dataset, the feature extraction model can extract gas feature maps based on the input data. Based on the gas feature maps, the gas location detection network can detect more accurate gas location data.

[0033] S13. Based on the gas detection data, determine the gas mask pattern.

[0034] In this embodiment, the gas mask image is used to further extract gas regions from the current gas feature map based on gas location data. The location boxes represented by the gas mask image are more precise than those corresponding to the gas location data. In the gas mask image, non-gas regions are represented as 0, while the pixel values ​​of the target gas regions are retained to facilitate further processing of the target gas regions.

[0035] S14. In the current frame, the target gas region indicated by the gas mask is colored and displayed.

[0036] In the above embodiments, continuous frame images are acquired, and input data for a pre-trained gas detection model is formed based on these continuous frame images. Since gas is a changing target, forming input data based on continuous frame images facilitates better capture of gas features. Because the gas detection model is a pre-trained model with fixed parameters, it can directly process continuous frame images, meeting real-time requirements. Furthermore, based on gas position data, a precise gas mask region is further extracted from the current gas feature map, thereby improving the accuracy of gas detection. In the current frame, the target gas region indicated by the gas mask is colored and displayed, allowing users to intuitively observe changes in the gas region.

[0037] In some embodiments, the consecutive frame images are consecutive frame infrared images, and the input data for forming a pre-trained gas detection model based on the consecutive frame images includes:

[0038] The consecutive frames of infrared images are merged to obtain a multi-channel merged image, and the input data is obtained based on the merged image.

[0039] In this embodiment, a multi-channel merged image is obtained by merging consecutive infrared images. This merged image can represent the motion state of the gas. Moreover, merging consecutive infrared images beforehand and inputting it into the gas detection model can improve the detection speed of the gas detection model. For example, eight consecutive single-channel images can be merged into one eight-channel image, where each channel represents the motion state. An eight-channel image can be an eight-frame video disguised as an image, which retains the motion characteristics of the gas and is subsequently used to detect the position of the gas in the eighth frame.

[0040] With the support of image processing and deep learning technologies, consecutive frame images are fused into a single multi-channel image to satisfy motion features and serve as input data for a gas detection model. The motion features of the gas are extracted from the input data, and gas detection and localization are performed using adaptive learning and the fusion of multiple feature extraction models to suit different scenarios and complex backgrounds.

[0041] In the above embodiments, consecutive frame infrared images are merged to obtain a multi-channel merged image. A single multi-channel image can represent the motion state of the gas. Moreover, merging consecutive frame infrared images beforehand and then inputting them into the gas detection model can improve the detection speed of the gas detection model.

[0042] In some embodiments, determining the gas mask pattern based on the gas detection data includes:

[0043] Based on the gas location data, the feature region to be processed is obtained in the current gas feature map;

[0044] Image processing operations are performed on the feature region to be processed to extract gas region shape data, and the gas mask is determined based on the gas region shape data.

[0045] In this embodiment, the location data corresponding to the feature region to be processed in the current gas feature map is the same as the gas location data. Image processing operations include, but are not limited to, thresholding, filtering, erosion, and dilation. The gas region shape data is obtained through further image processing of the feature region to be processed; therefore, the gas region indicated by the gas mask map is more accurate than the feature region to be processed.

[0046] In the above embodiments, further image processing operations are performed on the current gas feature map to extract a precise gas mask map region, thereby improving the accuracy of gas detection.

[0047] In some embodiments, coloring the target gas region indicated by the gas mask in the current frame includes:

[0048] In the current frame, the gas distribution in the target gas region is displayed in different colors, with different colors corresponding to different relative gas concentrations.

[0049] In this embodiment, the gas distribution in the target gas region is overlaid and displayed on the acquired current frame image. When the current frame image includes an infrared image and / or a visible light image, the gas distribution in the target gas region is overlaid and displayed on the infrared image and / or visible light image. The relative gas concentration score represents the probability of gas being present at a pixel; the higher the relative gas concentration score, the higher the probability of gas being present. In application scenarios, the higher the gas concentration at a location, the greater the probability of detecting the gas using a gas detection method. Therefore, by using different colors to represent the distribution of different relative gas concentrations based on the relative gas concentration scores, users can intuitively observe changes in gas concentration, such as... Figure 3 As shown, Figure 3 This is a schematic diagram showing the gas distribution in one embodiment. Figure 3 The diffusion range and direction of the gas can be directly observed. Simultaneously, the gas distribution in the target area is displayed in the current frame, relative to the background, making it easier for the user to observe the gas diffusion direction. The blue areas indicate a low probability of gas distribution, which can be understood as edges; the red areas indicate a high probability of gas distribution, which can be understood as gas outlets or the center of the gas. The relative gas concentration score is a concentration estimate probability. The colored areas represent the diffusion range, and the shape reflects the diffusion direction, for example... Figure 3The gas in the image diffuses towards the upper left. Therefore, the intuitive color-coding display allows users to visually understand the gas distribution and concentration, directly obtaining gas concentration estimates and the gas diffusion range and direction. Furthermore, because the gas is continuously monitored in real-time, and the video is continuous, a similar colored image can be seen in each frame.

[0050] In one optional implementation, the gas distribution in the target gas region is displayed using a gradient of colors based on the relative gas concentration, where darker colors correspond to a higher relative gas concentration than lighter colors. This gradient color display allows users to more intuitively observe the changing trends and diffusion directions of the gas concentration.

[0051] In the above embodiments, different colors are used to represent the distribution of different relative gas concentrations, which allows users to intuitively observe the changes in gas concentration. At the same time, the gas distribution of the target area is displayed in the current frame, which, relative to the background in the current frame, makes it easier for users to observe the direction of gas diffusion, thereby facilitating the observation of the dynamic changes in the gas area.

[0052] In some embodiments, such as Figure 4 As shown, Figure 4 This is a flowchart illustrating the gas distribution in a target gas region during a gas detection method in one embodiment; S14 further includes:

[0053] S141. Based on the confidence data corresponding to the current gas feature map, obtain the confidence level of each pixel in the target region.

[0054] In this embodiment, the confidence level corresponding to each pixel represents the probability value that gas exists at the location of each pixel.

[0055] S142. Based on the confidence level corresponding to each pixel, calculate the relative gas concentration score corresponding to each pixel.

[0056] Optionally, calculating the relative gas concentration score for each pixel based on the confidence level includes:

[0057] Based on the confidence level corresponding to each pixel, calculate the first gas relative concentration score corresponding to each pixel;

[0058] Based on the position data corresponding to the gas mask image, a target gas image region is obtained in the current frame. From the target gas image region, a gas cumulative histogram is obtained. Based on the cumulative histogram, a second gas relative concentration score corresponding to each pixel is obtained; or,

[0059] Obtain the current frame and the previous preset frame, perform differential processing on the current frame and the previous preset frame using a frame difference algorithm to obtain a gas differential image, and obtain the third gas relative concentration score corresponding to each pixel based on the gas differential image; or,

[0060] The current frame and the previous preset frame are obtained. Based on the position data corresponding to the gas mask, the gas region corresponding to the current frame and the gas region corresponding to the previous preset frame are obtained. The gas region corresponding to the current frame and the gas region corresponding to the previous preset frame are differentially processed using the frame difference algorithm to obtain a differential image. Based on the differential image, the fourth gas relative concentration score corresponding to each pixel is obtained.

[0061] The relative gas concentration score of each pixel is determined based on the first relative gas concentration score corresponding to each pixel, or based on the first relative gas concentration score corresponding to each pixel and at least one of the following relative gas concentration scores: the second relative gas concentration score corresponding to each pixel, the third relative gas concentration score corresponding to each pixel, and the fourth relative gas concentration score corresponding to each pixel.

[0062] In this embodiment, the relative gas concentration score represents the probability that gas is present at the location of each pixel. The first relative gas concentration score is obtained from the confidence level output by a pre-trained gas detection model. The fourth relative gas concentration score is obtained by frame difference processing based on the current gas feature map output by the pre-trained gas detection model. Since the detection data output by the pre-trained gas detection model is more accurate, coloring based on the first and fourth relative gas concentration scores results in a more rounded coloring range. The second and third relative gas concentration scores are obtained by processing the original continuous frame images, which preserves the gas plume. The histogram processing method and frame difference algorithm can be implemented using existing techniques, and will not be elaborated further here.

[0063] In this embodiment, when calculating the relative gas concentration score of each pixel, the relative gas concentration score of the first gas is weighted with the relative gas concentration score of at least one gas to obtain a weighted score, and the weighted score of each pixel is used as the relative gas concentration score of each pixel.

[0064] S143. Based on the gas relative concentration score corresponding to each pixel and the mapping relationship between the gas relative concentration score and the color value, color each pixel in the current frame is colored.

[0065] In this embodiment, the relative gas concentration corresponds to the color value. For example, the relative gas concentration between [0,5] corresponds to yellow, the relative gas concentration between [5,10] corresponds to red, and the relative gas concentration between [10,15] corresponds to green.

[0066] Based on the outputs of the gas detection model and the feature extraction model, each pixel can be colored, and the coloring effect can be appended to the current frame image to visually display the gas distribution and concentration. The relative gas concentration score for each pixel is obtained by weighting the scores according to the confidence level of the detection results. Color mapping technology can be used to map the relative gas concentration score of each pixel to different color values, and weighted according to the confidence level of the detection results to accurately represent the gas location and concentration.

[0067] Based on the confidence data corresponding to the current gas feature map, gas features such as shape, orientation, size, and the probability of gas presence can be obtained. The relative concentration score of the third gas corresponding to each pixel is obtained through the frame difference algorithm. That is, the score map composed of grayscale features and motion features can fine-tune the shape of the feature map output by the gas detection model to make it closer to the gas outline. In other words, by combining the information inside the trained gas detection model with the traditional model, the effect of machine learning segmentation can be improved.

[0068] In the above embodiments, the relative concentration scores of various gases are calculated based on multiple methods, and the relative concentration score of each pixel is calculated based on the relative concentration scores of various gases. This can accurately represent the probability that gas is present at the location of each pixel, which facilitates more accurate coloring based on the relative concentration score of each pixel.

[0069] In some embodiments, the method further includes:

[0070] In the current frame, the position box corresponding to the gas mask is displayed in a first display mode, and the position box corresponding to the gas position data is displayed in a second display mode, wherein the position box corresponding to the gas mask is located within the position box corresponding to the gas position data.

[0071] In this embodiment, the location bounding box corresponding to the gas mask image is more precise than the location bounding box corresponding to the gas location data. By displaying these two types of location bounding boxes in different ways, users can more intuitively observe the dynamic changes of the gas region. For example, the first display method is a solid line bounding box, and the second display method is a dashed line bounding box; the first display method uses a first color, and the second display method uses a second color, distinguishing the two display methods by color.

[0072] In the above embodiments, the location boxes corresponding to the gas mask image and the gas location data are displayed in different ways, which makes it easier for users to observe the dynamic changes of the gas region more intuitively.

[0073] In some embodiments, the gas detection model includes multiple feature extraction models and a gas location detection network. The multiple feature extraction models are used to obtain gas feature maps corresponding to each feature extraction model based on the input data. The gas location detection network is used to output the gas detection data based on the gas feature maps corresponding to each feature extraction model.

[0074] In this embodiment, since the gas detection model is pre-trained using a training dataset, each feature extraction model can extract different gas feature maps based on the input data. Using multiple feature extraction models allows for the extraction of gas feature maps under scene changes from continuously changing images, thus better adapting to complex and changing scenes, such as moving scenes. Because the gas feature maps extracted by multiple feature extraction models better represent the gas characteristics under scene changes, the gas position detection network can detect more accurate gas position data based on the gas feature maps corresponding to each feature extraction model.

[0075] After the gas detection model is trained, the parameters in the feature extraction model are fixed. Therefore, based on the input data, the model can output gas feature maps corresponding to each feature extraction model, or gas feature maps corresponding to each feature extraction model and their confidence scores. The confidence score of a gas feature map represents the probability that a gas region exists in the feature map. A feature extraction model can correspond to one or more gas feature maps; that is, for a single feature extraction model, one or more gas feature maps can be output. The network structure of the feature extraction model includes, but is not limited to, convolutional layers, downsampling layers, pooling layers, and activation layers.

[0076] In this embodiment, consecutive frame images are acquired and input into the input data of a pre-trained gas detection model. Since gas is a changing target, the input data is formed based on consecutive frame images, which facilitates better capture of gas features. Since the gas detection model is a pre-trained model, multiple feature extraction models obtain gas feature maps corresponding to each feature extraction model based on the input data. In this way, each feature extraction model can extract different gas feature maps based on the input data. Using multiple feature extraction models can obtain different gas feature maps under scene changes from consecutive frame images, thus better adapting to complex changing scenes. Based on the gas feature maps corresponding to each feature extraction model, the gas position detection network can detect more accurate gas position data. Therefore, since the gas detection model is pre-trained and the parameters in the model are fixed, it can directly process consecutive frame images, which can meet real-time requirements. Moreover, extracting different gas feature maps based on multiple feature extraction models can improve the accuracy of gas detection in complex backgrounds and scenes.

[0077] In one optional approach, gas position data is output through a gas position detection network based on the gas feature maps corresponding to each feature extraction model. In another optional approach, gas position data is output through a gas position detection network based on the gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps. Employing multiple feature extraction models can obtain different gas feature maps under scene changes from continuously changing images, thus better adapting to complex and changing scenes. Based on the gas feature maps corresponding to each feature extraction model, the gas position detection network can detect more accurate gas position data.

[0078] In the above embodiments, multiple feature extraction models can be used to obtain different gas feature maps under scene changes from images with continuous frame changes, thus making it more adaptable to complex and changing scenes. The confidence level of the gas feature maps can also be obtained. Based on the gas feature maps and confidence levels corresponding to each feature extraction model, the gas location detection network can detect more accurate gas location data and improve the accuracy of gas detection in complex backgrounds and scenes. Using machine learning methods, gas can be detected and located based on different gas features extracted by multiple feature models. In addition, a strategy of fusing multiple feature models can be used to combine the output results of multiple feature models to improve the accuracy and robustness of gas detection.

[0079] In some embodiments, the step of outputting gas detection data through the gas detection model based on the input data includes:

[0080] Based on the input data, gas feature maps corresponding to each feature extraction model are obtained through multiple feature extraction models, or gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps are obtained.

[0081] Based on the gas feature maps corresponding to each feature extraction model, or based on the gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps, the gas location detection network outputs the gas detection data.

[0082] In this embodiment, after the gas detection model is trained, the parameters in the feature extraction model are fixed. Therefore, based on the input data, the model can output gas feature maps corresponding to each feature extraction model, or gas feature maps corresponding to each feature extraction model and their confidence scores. The confidence score of a gas feature map represents the probability that a gas region exists in the feature map. A feature extraction model can correspond to one or more gas feature maps; that is, for a single feature extraction model, one or more gas feature maps can be output. The network structure of the feature extraction model includes, but is not limited to, convolutional layers, downsampling layers, pooling layers, and activation layers.

[0083] In one optional approach, gas position data is output through a gas position detection network based on the gas feature maps corresponding to each feature extraction model. In another optional approach, gas position data is output through a gas position detection network based on the gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps. Employing multiple feature extraction models can obtain different gas feature maps under scene changes from continuously changing images, thus better adapting to complex and changing scenes. Based on the gas feature maps corresponding to each feature extraction model, the gas position detection network can detect more accurate gas position data.

[0084] In the above embodiments, multiple feature extraction models can be used to obtain different gas feature maps under scene changes from images with continuous frame changes, thus making it more adaptable to complex changing scenes. The confidence level of the gas feature maps can also be obtained. Based on the gas feature maps and the confidence level of the gas feature maps corresponding to each feature extraction model, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0085] Optionally, the step of outputting the gas detection data through the gas location detection network based on the gas feature maps corresponding to each feature extraction model, or based on the gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps, includes:

[0086] By connecting the gas feature maps corresponding to each feature extraction model through the connection network in the gas detection model, a connection feature map is obtained;

[0087] Based on the connection feature map, the gas detection data is output through the gas position detection network.

[0088] In this embodiment, a connection network is used to connect the gas feature maps corresponding to various feature extraction models together, generating a connection feature map with more feature information. This can be used to increase the number of channels or feature dimensions in the deep learning model, so as to better capture the relationships between different gas features in subsequent layers. The connection network can be a convolutional network. Thus, the connection feature map includes more gas feature data and can represent the relationships between different gas features. Therefore, based on the connection feature map, the gas location detection network can detect more accurate gas detection data.

[0089] Optionally, each feature extraction model is trained based on a specific scenario. One model is trained for each scenario with overfitting. During testing, the output of this overfitted model has higher confidence and more obvious features, while the feature maps of other models are less obvious. Then, the head parts of these feature extraction models are connected by a convolutional layer to output a result, namely a gas connection feature map. For example, if there are five feature extraction models, the gas connection feature map is output by a large model composed of five smaller models. That is, each scenario activates a feature extraction model pathway, resulting in a gas feature map corresponding to a feature extraction model. Then, the selection weights in the convolutional layer are used to connect and fuse the gas feature maps corresponding to the various feature extraction models.

[0090] like Figure 5 This is a schematic diagram of generating a connection feature map in one embodiment. The gas detection model includes two feature extraction models, namely feature extraction models A1 and A2. Feature extraction model A1 outputs three feature maps A12, A13, and A14, and feature extraction model A2 outputs three feature maps A22, A23, and A24. Then, A12, A13, A14, A22, A23, and A24 are used as inputs to the connection network to output the connection feature map.

[0091] In the above embodiments, the gas feature maps corresponding to each feature extraction model are connected through the connection network in the gas detection model to obtain a connection feature map. This connection feature map includes more gas feature data and can represent the relationship between different gas features. Based on the connection feature map, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and scenes.

[0092] Optionally, the step of outputting the gas detection data through the gas location detection network based on the gas feature maps corresponding to each feature extraction model, or based on the gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps, includes:

[0093] Based on the confidence level of the gas feature map, target gas feature maps that meet the confidence level conditions are selected from the gas feature maps corresponding to each feature extraction model.

[0094] Based on the target gas feature map, the gas detection data is output through the gas location detection network.

[0095] In this embodiment, the target gas feature map represents a gas feature map with a confidence level higher than a preset confidence level. That is, gas feature maps with high confidence are used for subsequent gas detection, while gas feature maps with low confidence are discarded. This improves the feature representation of gas feature maps with high confidence and increases the accuracy of gas region detection.

[0096] Optionally, the step of outputting the gas detection data through the gas position detection network based on the target gas feature map includes at least one of the following:

[0097] When there is only one target gas feature map, the target gas feature map is used as the input of the gas position detection network, and the gas detection data is output through the gas position detection network.

[0098] When there are multiple target gas feature maps, the multiple target gas feature maps are connected through the connection network in the gas detection model to obtain a target connection feature map. Based on the target connection feature map, the gas detection data is output through the gas position detection network.

[0099] like Figure 6 This is a schematic diagram of generating connection feature maps in another embodiment; feature extraction model A1 outputs three feature maps A12 with a confidence level of 0.81; A13 with a confidence level of 0.5; and A14 with a confidence level of 0.85; feature extraction model A2 outputs three feature maps A22 with a confidence level of 0.45; A23 with a confidence level of 0.91; and A24 with a confidence level of 0.95; the preset confidence level is 0.8, so according to the preset confidence level, the target gas feature maps A12, A14, A23, and A24 are obtained. After connecting A12, A14, A23, and A24, the target connection feature map is obtained.

[0100] In the above embodiments, gas feature maps are filtered by the confidence level corresponding to the gas feature maps. Gas feature maps with high confidence levels are used for subsequent gas location detection, thereby activating the feature expression of gas feature maps with high confidence levels and suppressing the feature expression of gas feature maps with low confidence levels. This better highlights the features of gas regions in the current scene, enabling the gas location detection network to detect more accurate gas location data and improve the accuracy of gas detection in complex backgrounds and scenes.

[0101] In some embodiments,

[0102] The feature extraction model includes a multi-scale feature network. The process of obtaining gas feature maps corresponding to each feature extraction model based on the input data includes:

[0103] Based on the input data, the multi-scale feature networks in each feature extraction model output gas feature maps of multiple different scales corresponding to each feature extraction model.

[0104] In this embodiment, a multi-scale feature network is used to extract multiple gas feature maps of different sizes based on the input data. The multi-scale feature network includes, but is not limited to, FPN (Feature Pyramid Network) and PAN (Pyramid Attention Network) structures. FPN conveys strong semantic features from top to bottom, while PAN conveys strong localization features from bottom to top. Working together, they aggregate parameters from different backbone layers to different detection layers, maximizing the preservation of both the location and category information of the gas target. The multi-scale network obtains multiple input feature maps of different scales through multiple downsampling operations. These multiple input feature maps of different scales are then fed into the network structure formed by FPN and PAN for processing, resulting in multiple gas feature maps of different scales. For example, as... Figure 5 or Figure 6 As shown, feature extraction model A1 outputs three gas feature maps respectively. A12, A13, and A14 correspond to different scales. For example, the scale of A12 is 20*20, the scale of A13 is 100*100, and the scale of A14 is 400*400. Then, A12 is used to detect gas in smaller gas regions, A13 is used to detect gas in medium-sized gas regions, and A14 is used to detect gas in large gas regions.

[0105] In the above embodiments, the multi-scale feature network outputs multiple gas feature maps of different scales corresponding to each feature extraction model, which is more adaptable to the detection of gas regions of different sizes in complex scenes and backgrounds. Based on gas feature maps of different scales, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and scenes.

[0106] In some embodiments, the method further includes: each of the feature extraction models corresponds to its own sample dataset, wherein different sample datasets are collected from different gas detection scenarios.

[0107] In this embodiment, common gas detection scenarios can be classified into multiple scenarios. Multiple sample images are then collected in each scenario, and a feature extraction model for each scenario is pre-trained. For example, gas detection scenarios can be divided into indoor and outdoor scenarios. Indoor scenarios can be further subdivided into office scenarios, factory scenarios, oil field indoor scenarios, etc. Outdoor scenarios can be further subdivided into oil field outdoor scenarios, factory outdoor scenarios, etc., such as feature extraction model A for oil field outdoor scenarios and feature extraction model B for factory outdoor scenarios. A large number of sample images are collected for each scenario to train the corresponding feature extraction model. The collected sample images can include rich backgrounds, including but not limited to the sky, black objects, grass, pipes, glass, blue sky, white clouds, and the sea. In this way, during training, the corresponding feature extraction model can learn the main features of the scenario and rich background features. After the feature extraction models are trained and applied to the gas detection model, the gas detection model can be applied to various complex scenarios and backgrounds. This application embodiment utilizes artificial intelligence technology to acquire sample data under various complex backgrounds and multiple scenarios, thereby achieving gas detection in complex backgrounds and multiple scenarios. It collects a large amount of video data and classifies the data according to the scenario. Then, multiple feature extraction models are trained using different sample datasets, and these feature extraction models are fused to obtain a gas detection model.

[0108] In the above embodiments, each feature extraction model corresponds to its own sample dataset. Different sample datasets are collected from different gas detection scenarios. In this way, the feature extraction model can learn the main features and rich background features of the corresponding scenario from the corresponding sample dataset during the training process. This enables the gas detection model to detect more accurate gas location data and improve the accuracy of gas detection in complex backgrounds and scenarios.

[0109] In some embodiments, such as Figure 7 As shown, Figure 7This is a schematic diagram of training a gas detection model in one embodiment. The method further includes:

[0110] S51. Obtain the training dataset, wherein each training image in the training dataset includes consecutive training images and gas label data corresponding to the consecutive training images.

[0111] In this embodiment, the training dataset may include multiple sample datasets, and the training image is a consecutive frame training image randomly selected from the multiple sample datasets. Since the multiple sample datasets are obtained from multiple gas detection scenarios, the training dataset is also based on multiple gas detection scenarios. The gas region in the current frame of the consecutive training image is used as the corresponding gas label data. For example, if the consecutive training images are the first frame, the second frame, and the third frame is the current frame, then the gas region in the third frame is used as the corresponding gas label data.

[0112] The training dataset can be derived from a large amount of video data, including various scenarios from industrial, environmental, and safety monitoring fields. This video data should contain the movement and distribution of various gases. The collected data should be preprocessed, including frame sampling, resolution adjustment, and annotation, in order to facilitate subsequent model training and evaluation.

[0113] S52. Construct an initial gas detection model, which includes multiple trained feature extraction models and an initial gas location detection network.

[0114] In this embodiment, to accelerate the training of the gas detection model, each feature extraction model can be pre-trained, and then trained separately based on its corresponding sample dataset. During training, appropriate hyperparameters, learning rates, and batch sizes need to be set, and data augmentation and regularization operations need to be performed to improve the model's generalization ability and robustness.

[0115] Optionally, before obtaining the pre-trained gas detection model, the method further includes:

[0116] Based on each of the aforementioned sample datasets, each trained feature extraction model is obtained.

[0117] The training of each feature extraction model based on each of the aforementioned sample datasets includes:

[0118] Obtain the sample dataset, which includes each sample image comprising consecutive frame sample images and gas feature map labels corresponding to the consecutive frame sample images;

[0119] Construct the initial feature extraction model;

[0120] Based on the sample dataset, the initial feature extraction model corresponding to the sample dataset is iteratively trained until the training termination condition is met, thereby obtaining the trained feature extraction model corresponding to the sample dataset.

[0121] In this embodiment, firstly, a large amount of video data is collected as a sample dataset and classified according to different scenes. Then, for each scene, one or more feature extraction models are trained using the corresponding sample dataset. These trained feature extraction models are fused to combine the advantages of different models and improve the accuracy and robustness of gas detection. The continuous frame sample images include, but are not limited to, continuous frame sample infrared images, continuous frame sample visible light images, and continuous frame sample fused images obtained by fusing continuous frame sample infrared images and continuous frame sample visible light images. Each feature extraction model is trained according to the above steps. During the training of a feature extraction model, sample images are acquired from the sample dataset, sample image input data is formed based on the sample images, and the gas feature map corresponding to the sample image input data is output by the feature extraction model in the current iteration. Based on the feature loss function, the gas feature map corresponding to the sample image input data, and the label of the gas feature map corresponding to the sample image input data, the loss value in the current iteration is output. Based on the loss value in the current iteration, it is determined whether the current iteration meets the training termination condition. If the current iteration meets the training termination condition, the feature extraction model after the iteration termination is used as the trained feature extraction model; if the current iteration does not meet the training termination condition, sample images are continuously acquired from the sample dataset for training. Training termination conditions include, but are not limited to, the number of iterations, and the loss value being less than a preset loss value. Feature loss functions include, but are not limited to, the cross-entropy function and the mean squared error function.

[0122] S53. The initial gas detection model is iteratively trained using the training dataset until the loss function converges, thus obtaining the pre-trained gas detection model.

[0123] Optionally, the step of iteratively training the initial gas detection model using the training dataset until the loss function converges to obtain the pre-trained gas detection model includes:

[0124] Training images are obtained from the training dataset. Training image input data for the gas detection model of the current iteration is formed based on the training images. Based on the training image input data, gas sample feature maps corresponding to each feature extraction model are output through multiple feature extraction models in the current iteration. Based on the gas sample feature maps corresponding to each feature extraction model, gas sample location data corresponding to the training images are output through the gas location detection network in the current iteration.

[0125] Based on the loss function, the loss value between the gas sample location data corresponding to the training image and the gas label data corresponding to the training image is calculated to obtain the loss value of the current iteration;

[0126] Based on the loss value of the current iteration, determine whether the current iteration meets the iteration termination condition. If the current iteration does not meet the iteration termination condition, continue to acquire training images from the training dataset and iteratively train the gas detection model. If the current iteration meets the iteration termination condition, use the gas detection model after stopping the iteration as the pre-trained gas detection model.

[0127] In this embodiment, consecutive training frames are merged to obtain a multi-channel training image input data. For example, eight consecutive training frames are arranged by channel and merged into a 512*640*8 image. The label of this training image is the gas location label of the last frame. Training termination conditions include, but are not limited to, the number of iterations, the loss value being less than a preset loss value, etc. The loss function includes, but is not limited to, the cross-entropy function, the mean squared error function, etc. After the feature extraction model is trained based on the sample dataset, the initial parameters of the feature extraction model can be obtained. When the gas detection model is trained as a whole based on the training dataset, the initial parameters of the feature extraction model will be fine-tuned and optimized so that the optimized parameters enable the gas location detection network to detect more accurate gas regions. Moreover, since the training dataset is formed by multiple sample datasets from multiple gas detection scenarios, the gas detection model can learn gas features and rich background features in multiple scenarios and backgrounds during training. After training, it can better adapt to gas detection in complex scenarios and backgrounds. Therefore, when the trained gas detection model is applied to motion scenarios, such as handheld detection devices or non-handheld, autonomously movable detection devices, the scene changes constantly during the motion process. However, since the gas detection model is trained based on training datasets collected in various gas detection scenarios, it can better adapt to such motion scenarios.

[0128] Optionally, after training, the gas detection model can be optimized using techniques such as model compression, pruning, and quantization to reduce computational and storage resources. Furthermore, multiple feature extraction models can be fused together using strategies such as voting, weighted averaging, or model ensemble to combine the outputs of multiple models and improve the accuracy and robustness of gas detection.

[0129] Optionally, the training of the gas detection model can be performed on a computing device equipped with high-performance computing power, such as a GPU or cloud computing platform, to perform complex image processing and deep learning computations. Furthermore, to improve the efficiency and accuracy of the algorithm, parameter tuning and optimization are necessary. For example, cross-validation can be used to determine the hyperparameters of the feature extraction model, such as kernel size and number of layers. Simultaneously, data augmentation techniques, such as translation, rotation, and scaling, can be applied to expand the training dataset and improve the model's robustness and generalization ability.

[0130] In the above embodiments, the corresponding feature extraction models are first trained based on each sample dataset to obtain the initial parameters of the feature extraction models. Then, the gas detection model is trained based on the training dataset, which can accelerate the convergence speed of the gas detection model. Since the training dataset is formed by multiple sample datasets from multiple gas detection scenarios, the gas detection model can learn gas features and rich background features in multiple scenarios and backgrounds, improving the accuracy of gas detection in complex backgrounds and scenarios. This approach comprehensively considers the influence of different scenarios and complex backgrounds, classifying data during the data collection phase and training corresponding models for each scenario. This adapts to the gas detection needs of different scenarios and improves adaptability to complex backgrounds. Multi-model fusion strategy: This method trains multiple models using different datasets and then fuses these models, combining the output results of multiple models to improve the accuracy and robustness of gas detection. By adopting a multi-model fusion strategy, the advantages of each model can be fully utilized to improve the overall detection performance.

[0131] In some embodiments, the continuous frame image includes any one of the following: continuous frame infrared image, continuous frame visible light image, and continuous frame fused image obtained by fusing continuous frame infrared image and continuous frame visible light image.

[0132] In this embodiment, the consecutive frame images are consecutive frame infrared images. Acquiring infrared images can reduce the influence of environmental factors, such as adverse weather conditions like rain and fog. Moreover, infrared images can capture both colorless and colored gases, improving the accuracy of subsequent gas detection. Furthermore, the consecutive frame images are consecutive frame fused images, which can simultaneously capture both infrared and visible light features, thus providing more features for subsequent gas detection.

[0133] When one or more embodiments of this application are combined, this application has at least the following advantages:

[0134] At least one embodiment of this application combines image processing and deep learning techniques to automatically extract gas features from multiple frames of infrared images and distinguish and locate different gases. It can also extract the probability of gas positions in the current frame image and assign pseudo-color, which is then overlaid on the current frame image. Furthermore, this algorithm achieves adaptability to complex backgrounds and various scenes through adaptive learning and multi-model fusion.

[0135] At least one embodiment of this application obtains a gas connectivity feature map from a feature extraction model, and then applies thresholding, filtering, and morphological processing to extract the gas's position and shape information. Next, the gas features are further enhanced by combining grayscale data from infrared frames and frame differencing. Grayscale provides brightness information of the gas, while frame differencing is used to detect gas motion. This comprehensive processing helps improve the accuracy and robustness of gas detection and provides a more accurate basis for subsequent analysis and visualization.

[0136] At least one embodiment of this application, through steps such as preprocessing, feature extraction, gas detection and localization, and color display, combines image processing and deep learning technologies to implement a multi-gas detection and color display algorithm applicable to complex backgrounds and various scenarios. This algorithm enables the detection and localization of multiple gases and displays the detection results in an intuitive color-coded manner, allowing users to visually understand the gas distribution and concentration. This method has broad application prospects and can play an important role in fields such as industrial, environmental, and safety monitoring.

[0137] At least one embodiment of this application employs a multi-model fusion strategy: multiple feature extraction models are trained using different sample datasets, and these models are then fused together to combine their outputs, thereby improving the accuracy and robustness of gas detection. By adopting this multi-model fusion strategy, the advantages of each model can be fully utilized, improving overall detection performance. Considering the impact of different scenarios and complex backgrounds, data was classified during the data collection phase, and a corresponding feature extraction model was trained for each scenario. This adapts to the gas detection needs of different scenarios and improves adaptability to complex backgrounds.

[0138] At least one embodiment of this application employs gas feature extraction and coloring. Through feature extraction technology, the motion features of gas are extracted from consecutive input frame images, and a gas detection model trained using machine learning methods is used for gas detection and localization. Simultaneously, by weighted coloring of each pixel and appending the coloring effect to the current frame image, the distribution and concentration of gas are visually displayed. This gas feature extraction and coloring method improves the accuracy and visualization effect of gas detection.

[0139] In another aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the gas detection method described in any embodiment of this application.

[0140] In the computer program product, the optional implementation form of the program module architecture of the computer program that implements each step of the gas detection method can be a gas detection device.

[0141] Please see Figure 8 One embodiment of this application provides a gas detection device, including: an acquisition module 81 for acquiring continuous frame images; a detection module 82 for forming input data for a pre-trained gas detection model based on the continuous frame images, and outputting gas detection data through the gas detection model based on the input data, wherein the gas detection data includes gas position data in the current frame of the continuous frame images and a current gas feature map in the current frame; the detection module 82 is further configured to determine a gas mask map based on the gas detection data; and a display module 83 for coloring and displaying the target gas region indicated by the gas mask map in the current frame.

[0142] Optionally, the detection module 82 is also used for:

[0143] The consecutive frames of infrared images are merged to obtain a multi-channel merged image, and the input data is obtained based on the merged image.

[0144] Optionally, the detection module 82 is also used for:

[0145] Based on the gas location data, the feature region to be processed is obtained in the current gas feature map;

[0146] Image processing operations are performed on the feature region to be processed to extract gas region shape data, and the gas mask is determined based on the gas region shape data.

[0147] Optionally, the display module 83 is also used for:

[0148] In the current frame, the gas distribution in the target gas region is displayed in different colors, with different colors corresponding to different relative gas concentrations.

[0149] Optionally, the display module 83 is also used for:

[0150] Based on the confidence data corresponding to the current gas feature map, obtain the confidence level of each pixel in the target region;

[0151] Calculate the relative gas concentration score for each pixel based on the confidence level corresponding to each pixel.

[0152] Based on the relative gas concentration score corresponding to each pixel and the mapping relationship between the relative gas concentration score and the color value, each pixel is colored in the current frame.

[0153] Optionally, the display module 83 is also used for:

[0154] Based on the confidence level corresponding to each pixel, calculate the first gas relative concentration score corresponding to each pixel;

[0155] Based on the position data corresponding to the gas mask image, a target gas image region is obtained in the current frame. From the target gas image region, a gas cumulative histogram is obtained. Based on the cumulative histogram, a second gas relative concentration score corresponding to each pixel is obtained; or,

[0156] Obtain the current frame and the previous preset frame, perform differential processing on the current frame and the previous preset frame using a frame difference algorithm to obtain a gas differential image, and obtain the third gas relative concentration score corresponding to each pixel based on the gas differential image; or,

[0157] The current frame and the previous preset frame are obtained. Based on the position data corresponding to the gas mask, the gas region corresponding to the current frame and the gas region corresponding to the previous preset frame are obtained. The gas region corresponding to the current frame and the gas region corresponding to the previous preset frame are differentially processed using the frame difference algorithm to obtain a differential image. Based on the differential image, the fourth gas relative concentration score corresponding to each pixel is obtained.

[0158] The relative gas concentration score of each pixel is determined based on the first relative gas concentration score corresponding to each pixel, or based on the first relative gas concentration score corresponding to each pixel and at least one of the following: the second relative gas concentration score corresponding to each pixel, the third relative gas concentration score corresponding to each pixel, and the fourth relative gas concentration score corresponding to each pixel.

[0159] Optionally, the display module 83 is also used for:

[0160] In the current frame, the position box corresponding to the gas mask is displayed in a first display mode, and the position box corresponding to the gas position data is displayed in a second display mode, wherein the position box corresponding to the gas mask is located within the position box corresponding to the gas position data.

[0161] Optionally, the gas detection model includes multiple feature extraction models and a gas location detection network. The multiple feature extraction models are used to obtain gas feature maps corresponding to each feature extraction model based on the input data. The gas location detection network is used to output the gas detection data based on the gas feature maps corresponding to each feature extraction model.

[0162] Optionally, the detection module 82 is also used for:

[0163] Based on the input data, gas feature maps corresponding to each feature extraction model are obtained through multiple feature extraction models, or gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps are obtained.

[0164] Based on the gas feature maps corresponding to each feature extraction model, or based on the gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps, the gas location detection network outputs the gas detection data.

[0165] Optionally, the detection module 82 is also used for:

[0166] By connecting the gas feature maps corresponding to each feature extraction model through the connection network in the gas detection model, a connection feature map is obtained;

[0167] Based on the connection feature map, the gas detection data is output through the gas position detection network.

[0168] Optionally, the detection module 82 is also used for:

[0169] Based on the confidence level of the gas feature map, target gas feature maps that meet the confidence level conditions are selected from the gas feature maps corresponding to each feature extraction model.

[0170] Based on the target gas feature map, the gas detection data is output through the gas location detection network.

[0171] Optionally, the detection module 82 is also used for:

[0172] When there is only one target gas feature map, the target gas feature map is used as the input of the gas position detection network, and the gas detection data is output through the gas position detection network.

[0173] When there are multiple target gas feature maps, the multiple target gas feature maps are connected through the connection network in the gas detection model to obtain a target connection feature map. Based on the target connection feature map, the gas detection data is output through the gas position detection network.

[0174] Optionally, the feature extraction model includes a multi-scale feature network; optionally, the detection module 82 is further configured to:

[0175] Based on the input data, the multi-scale feature networks in each feature extraction model output gas feature maps of multiple different scales corresponding to each feature extraction model.

[0176] Optionally, each of the feature extraction models corresponds to its own sample dataset, wherein different sample datasets are collected from different gas detection scenarios.

[0177] Optionally, the continuous frame image includes any one of the following: continuous frame infrared image, continuous frame visible light image, and continuous frame fused image obtained by fusing continuous frame infrared image and continuous frame visible light image.

[0178] It will be understood by those skilled in the art that Figure 8 The structure of the gas detection device does not constitute a limitation on the gas detection device. Each module can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the gas detection device, or stored in software in the memory of the gas detection device, so that the processor can call and execute the operations corresponding to each module. In other embodiments, the gas detection device may include more or fewer modules than shown in the figures.

[0179] Please see Figure 9 In another aspect of the embodiments of this application, a gas detection device 10 is also provided, including a processor 13 and a memory 14. The memory 14 stores a computer program. When the computer program is executed by the processor, the processor 13 performs the steps of the gas detection method provided in any of the above embodiments of this application.

[0180] The processor 13 serves as the control center, connecting various parts of the gas detection device via various interfaces and lines. It executes software programs and / or modules stored in the memory 14, and calls data stored in the memory 14 to perform various functions and process data within the gas detection device. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user page, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 13.

[0181] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the gas detection device, etc. In addition, the memory 14 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 14 may also include a memory processor to provide the processor 13 with access to the memory 14.

[0182] In another aspect, this application also provides a storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the gas detection method provided in any of the above embodiments of this application.

[0183] Those skilled in the art will understand that all or part of the processes in the methods provided in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0184] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A gas detection method, characterized in that, include: Acquire consecutive frame images, wherein the consecutive frame images include any one of the following: consecutive frame infrared images, and consecutive frame fused images obtained by fusing consecutive frame infrared images and consecutive frame visible light images; Based on the continuous frame images, input data for a pre-trained gas detection model is formed. Based on the input data, gas detection data is output through the gas detection model. The gas detection data includes gas position data in the current frame of the continuous frame images and the current gas feature map in the current frame. Based on the gas detection data, a gas mask pattern is determined; In the current frame, the target gas region indicated by the gas mask is colored and displayed. The gas detection data further includes: confidence data corresponding to the current gas feature map. The coloring and displaying of the target gas region indicated by the gas mask includes: obtaining the confidence level of each pixel in the target gas region based on the confidence data corresponding to the current gas feature map. Calculate the relative gas concentration score for each pixel based on the confidence level corresponding to each pixel. Based on the relative gas concentration score corresponding to each pixel and the mapping relationship between the relative gas concentration score and the color value, each pixel is colored in the current frame.

2. The gas detection method as described in claim 1, characterized in that, The consecutive frame images are consecutive frame infrared images, and the input data for forming the pre-trained gas detection model based on the consecutive frame images includes: The consecutive frames of infrared images are merged to obtain a multi-channel merged image, and the input data is obtained based on the merged image.

3. The gas detection method as described in claim 1, characterized in that, The process of determining the gas mask image based on the gas detection data includes: Based on the gas location data, the feature region to be processed is obtained in the current gas feature map; Image processing operations are performed on the feature region to be processed to extract gas region shape data, and the gas mask is determined based on the gas region shape data.

4. The gas detection method as described in claim 1, characterized in that, The step of coloring and displaying the target gas region indicated by the gas mask in the current frame includes: In the current frame, the gas distribution in the target gas region is displayed in different colors, with different colors corresponding to different relative gas concentrations.

5. The gas detection method as described in claim 1, characterized in that, The calculation of the relative gas concentration score for each pixel based on the confidence level includes: Based on the confidence level corresponding to each pixel, calculate the first gas relative concentration score corresponding to each pixel; Based on the position data corresponding to the gas mask image, a target gas image region is obtained in the current frame. From the target gas image region, a gas cumulative histogram is obtained. Based on the cumulative histogram, a second gas relative concentration score corresponding to each pixel is obtained; or, Obtain the current frame and the previous preset frame, perform differential processing on the current frame and the previous preset frame using a frame difference algorithm to obtain a gas differential image, and obtain the third gas relative concentration score corresponding to each pixel based on the gas differential image; or, The current frame and the previous preset frame are obtained. Based on the position data corresponding to the gas mask, the gas region corresponding to the current frame and the gas region corresponding to the previous preset frame are obtained. The gas region corresponding to the current frame and the gas region corresponding to the previous preset frame are differentially processed using the frame difference algorithm to obtain a differential image. Based on the differential image, the fourth gas relative concentration score corresponding to each pixel is obtained. The relative gas concentration score for each pixel is determined based on the first relative gas concentration score for each pixel and at least one of the following: the second relative gas concentration score for each pixel, the third relative gas concentration score for each pixel, and the fourth relative gas concentration score for each pixel.

6. The gas detection method as described in claim 1, characterized in that, The method further includes: In the current frame, the position box corresponding to the gas mask is displayed in a first display mode, and the position box corresponding to the gas position data is displayed in a second display mode, wherein the position box corresponding to the gas mask is located within the position box corresponding to the gas position data.

7. The gas detection method as described in claim 1, characterized in that, The gas detection model includes multiple feature extraction models and a gas location detection network. The multiple feature extraction models are used to obtain gas feature maps corresponding to each feature extraction model based on the input data. The gas location detection network is used to output the gas detection data based on the gas feature maps corresponding to each feature extraction model.

8. The gas detection method as described in claim 7, characterized in that, The step of outputting gas detection data based on the input data through the gas detection model includes: Based on the input data, gas feature maps corresponding to each feature extraction model are obtained through multiple feature extraction models, or gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps are obtained. Based on the gas feature maps corresponding to each feature extraction model, or based on the gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps, the gas location detection network outputs the gas detection data.

9. The gas detection method as described in claim 8, characterized in that, The process of outputting gas detection data through the gas location detection network based on gas feature maps corresponding to each feature extraction model, or based on gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps, includes: By connecting the gas feature maps corresponding to each feature extraction model through the connection network in the gas detection model, a connection feature map is obtained; Based on the connection feature map, the gas detection data is output through the gas position detection network.

10. The gas detection method as described in claim 8, characterized in that, The process of outputting gas detection data through the gas location detection network based on gas feature maps corresponding to each feature extraction model, or based on gas feature maps corresponding to each feature extraction model and the confidence level of the gas feature maps, includes: Based on the confidence level of the gas feature map, target gas feature maps that meet the confidence level conditions are selected from the gas feature maps corresponding to each feature extraction model. Based on the target gas feature map, the gas detection data is output through the gas location detection network.

11. The gas detection method as described in claim 10, characterized in that, The step of outputting gas detection data through the gas location detection network based on the target gas feature map includes at least one of the following: When there is only one target gas feature map, the target gas feature map is used as the input of the gas position detection network, and the gas detection data is output through the gas position detection network. When there are multiple target gas feature maps, the multiple target gas feature maps are connected through the connection network in the gas detection model to obtain a target connection feature map. Based on the target connection feature map, the gas detection data is output through the gas position detection network.

12. The gas detection method as described in claim 8, characterized in that, The feature extraction model includes a multi-scale feature network. The process of obtaining gas feature maps corresponding to each feature extraction model based on the input data includes: Based on the input data, the multi-scale feature networks in each feature extraction model output gas feature maps of multiple different scales corresponding to each feature extraction model.

13. The gas detection method as described in claim 7, characterized in that, Each of the aforementioned feature extraction models corresponds to its own sample dataset, wherein different sample datasets are collected from different gas detection scenarios.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the gas detection method as described in any one of claims 1 to 13.

15. A gas detection device, characterized in that, It includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the gas detection method as described in any one of claims 1 to 13.

16. A storage medium comprising a computer program, characterized in that, The device contains a computer program that, when executed by a processor, causes the processor to perform the gas detection method as described in any one of claims 1 to 13.

Images