Gas detection method, program product and device

Abstract

The invention discloses a gas detection method, a program product and equipment. The method comprises the following steps: acquiring continuous frame images; based on the continuous frame images, input data of a pre-trained gas detection model is formed, based on the input data, gas detection data are output through the gas detection model, and the gas detection data comprise gas position data in a current frame in the continuous frame images and a current gas feature map in the current frame; determining a gas mask pattern based on the gas detection data; and in the current frame, performing coloring display on a target gas area indicated by the gas mask pattern.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a gas detection method, a computer program product and a gas detection device. Background Art

[0002] In recent years, the demand for gas detection and monitoring in the fields of industry, environment and safety monitoring has continued to increase. Traditional gas detection methods usually rely on sensors installed at a fixed location for detection, but the application effect is poor in complex backgrounds and multiple scenarios. For example, in sports scenes, the scene is constantly changing. Using traditional gas detection methods, the accuracy of the detected gas area is low. At present, the existing gas detection algorithms mainly focus on the detection of specific fixed scenes or specific gases, and the joint detection of complex backgrounds and multiple gases is still challenging. For example, in industrial production environments, multiple harmful gases such as formaldehyde, hydrogen sulfide and carbon monoxide often exist at the same time. In urban environments, the sources of gas are diverse and complex, such as automobile exhaust, industrial waste gas and combustion emissions. In view of these situations, traditional gas detection methods are difficult to meet real-time requirements and cannot intuitively see the detected gas area. Summary of the invention

[0003] In order to solve the existing technical problems, the present invention provides a gas detection method, a computer program product and a gas detection device, which can accurately detect the gas area and display the gas area in color, so that the user can intuitively observe the changes in the gas area.

[0004] In a first aspect, a gas detection method is provided, comprising: acquiring continuous frame images; forming input data of a pre-trained gas detection model based on the continuous frame images; and outputting gas detection data through the gas detection model based on the input data, wherein the gas detection data comprises gas position data in a current frame in the continuous frame images and a current gas characteristic map in the current frame; determining a gas mask map based on the gas detection data; and displaying a target gas area indicated by the gas mask map in the current frame by coloring.

[0005] In a second aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the gas detection method as described in any one of the first aspects of the present application is implemented.

[0006] In a third aspect, a gas detection device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the gas detection method as described in any one of the first aspects of the present application.

[0007] In a fourth aspect, a storage medium is provided, comprising a computer program, storing the computer program, and when the computer program is executed by a processor, the processor executes the gas detection method described in any one of the first aspects of the present application.

[0008] The embodiment of the present application obtains continuous frame images, and forms input data of a pre-trained gas detection model based on the continuous frame images. Since gas is a changing target, forming input data based on continuous frame images facilitates better capture of gas characteristics. Since the gas detection model is a pre-trained model, the parameters in the model have been solidified, and the continuous frame images can be directly processed to meet real-time requirements. Moreover, based on the gas position data, a precise gas mask image area is further extracted in the current gas characteristic image, thereby improving the accuracy of gas detection. In the current frame, the target gas area indicated by the gas mask image is colored and displayed, so that the user can intuitively observe the changes in the gas area. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A diagram showing an application environment of a gas detection method in an embodiment;

[0010] Figure 2 is a flow chart of a gas detection method in one embodiment;

[0011] Figure 3 is a schematic diagram showing gas distribution in one embodiment;

[0012] Figure 4 A flow chart showing gas distribution in a target gas area in a gas detection method in one embodiment;

[0013] Figure 5 A schematic diagram of generating a connection feature graph in one embodiment;

[0014] Figure 6 A schematic diagram of generating a connection feature graph in another embodiment;

[0015] Figure 7 A schematic diagram of training a gas detection model in one embodiment;

[0016] Figure 8 is a schematic diagram of a gas detection device in one embodiment;

[0017] Fig. 9 Schematic diagram of a gas detection device in one embodiment. DETAILED DESCRIPTION

[0018] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0020] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] See also Figure 1 , is an application environment diagram of a gas detection method in an embodiment. The gas detection method is applied to a gas detection device 10, and the gas detection device 10 includes an image acquisition device 12, a processor 13 and a memory 14. The image acquisition device 12 is used to acquire continuous frame images in front of the gas detection device 10, and the processor 13 detects the gas area in the continuous frame images according to the continuous frame images acquired by the image acquisition device 12. The memory 14 is used to store programs and data corresponding to the implementation of the gas detection method.

[0022] The gas detection device 10 includes but is not limited to a handheld detection device, a non-handheld autonomously movable detection device, and a non-handheld non-autonomously movable detection device. Among them, the handheld detection device includes but is not limited to a handheld imaging device with an infrared thermal imaging function and a handheld imaging device with a visible light imaging function. Among them, the non-handheld autonomously movable detection device includes but is not limited to an autonomously movable detection device with an infrared thermal imaging function and an autonomously movable detection device with a visible light imaging function. The non-handheld non-autonomously movable detection device includes but is not limited to a non-handheld and non-autonomously movable device with an infrared thermal imaging function and a non-handheld and non-autonomously movable device with a visible light imaging function. Among them, through the handheld detection device and the non-handheld autonomously movable detection device, the user can perform gas detection on the current scene in motion during the movement, and can also perform gas detection on a fixed scene. Gas detection can be performed in a fixed scene through a non-handheld non-autonomously movable detection device. Therefore, the gas detection method provided in the present application can be applied to various complex motion change scenes and fixed scenes.

[0023] The image acquisition device 12 may be a combination of one or more sensors. The image acquisition device 12 may be a monocular vision sensor or a multi-vision sensor. For example, it may be a combination of one or more sensors selected from the group consisting of a thermal imaging sensor, a visible light image sensor, a millimeter wave sensor, a laser radar sensor, an infrared thermal imaging sensor, and a depth sensor.

[0024] The processor 13 may be one or more. When there are multiple processors 13, the multiple processors may be integrated on one chip or independently arranged on each chip. The gas detection device 10 is a device installed on any type of mobile body, such as a vehicle, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship or a robot, etc. The gas detection device 10 may also be fixedly installed on a fixed device or at a fixed position in a fixed scene.

[0025] The gas detection device 10 may also include other sensor modules, including but not limited to environmental perception sensors and motion posture sensors. Environmental perception sensors include but are not limited to one or more combinations of the following sensors: brightness sensors, temperature sensors, haze sensors and other environmental sensors. Motion posture sensors include but are not limited to one or more combinations of the following: inertial sensors (Inertial Measurement Unit, IMU), speed sensors, acceleration sensors, gyroscope sensors, geomagnetic sensors, rotation vector sensors, steering wheel angle sensors, level sensors, tilt sensors, vibration sensors, displacement sensors and gravity sensors, etc.

[0026] The gas detection device 10 may further include a display terminal for displaying images.

[0027] See also Figure 2 , is a flow chart of a gas detection method provided in an embodiment of the present application. The gas detection method is applied in a gas detection device, and the gas detection method includes the following steps:

[0028] S11, acquiring continuous frame images.

[0029] In this embodiment, the continuous frame images include but are not limited to continuous frame infrared images and continuous frame fused images obtained by fusing continuous frame infrared images with continuous frame visible light images. The continuous frame infrared images and / or continuous frame visible light images are collected by the image acquisition device 12. When the continuous frame infrared images and the continuous frame visible light images are synchronously collected in time, the continuous frame infrared images and the continuous frame visible light images are mapped to the same image coordinate system, so that the spatial synchronization of the continuous frame infrared images and the continuous frame visible light images can be achieved, and then the continuous frame infrared images and the continuous frame visible light images are fused by the image fusion method to obtain the continuous frame fused image. Since the gas is changing, it is more convenient to extract the characteristics of the gas later based on the continuous frame images.

[0030] S12. Based on the continuous frame images, input data of a pre-trained gas detection model is formed. Based on the input data, gas detection data is output through the gas detection model, and the gas detection data includes gas position data in a current frame in the continuous frame images and a current gas characteristic map in the current frame.

[0031] In this embodiment, the gas detection model is obtained by training with a training data set. The gas detection model includes a feature extraction model and a gas position detection network. After the training is completed, the parameters in the feature extraction model and the gas position detection network have been solidified. After preprocessing the continuous frame images, the input data is obtained, and the preprocessing includes but is not limited to denoising, cropping, background processing, etc. In an optional implementation, the continuous frame images can also be directly used as input data.

[0032] In this embodiment, since the gas detection model is a model pre-trained by a training data set, the feature extraction model can extract a gas characteristic map based on the input data. Based on the gas characteristic map, the gas position detection network can detect more accurate gas position data.

[0033] S13. Determine a gas mask image based on the gas detection data.

[0034] In this embodiment, the gas mask is based on the gas position data, and the gas area is further extracted in the current gas feature map. Compared with the position box corresponding to the gas position data, the position box represented by the gas mask is more accurate. In the gas mask, the non-gas area is represented as 0, and the pixel value of the pixel point in the target gas area is retained to facilitate the subsequent further processing of the target gas area.

[0035] S14. In the current frame, the target gas region indicated by the gas mask image is displayed in color.

[0036] In the above embodiment, continuous frame images are acquired, and input data of a pre-trained gas detection model is formed based on the continuous frame images. Since gas is a changing target, forming input data based on continuous frame images facilitates better capture of gas characteristics. Since the gas detection model is a pre-trained model, the parameters in the model have been solidified, and the continuous frame images can be directly processed to meet real-time requirements. Moreover, based on the gas position data, a precise gas mask image area is further extracted in the current gas characteristic image, thereby improving the accuracy of gas detection. In the current frame, the target gas area indicated by the gas mask image is colored and displayed, so that the user can intuitively observe the changes in the gas area.

[0037] In some embodiments, the continuous frame images are continuous frame infrared images, and the input data for forming a pre-trained gas detection model based on the continuous frame images includes:

[0038] The continuous frame infrared images are merged to obtain a multi-channel merged image, and the input data is obtained based on the merged image.

[0039] In this embodiment, a merged image of multiple channels is obtained by merging continuous infrared images. The merged image can represent the motion state of the gas. Moreover, after the continuous infrared images are merged in advance and input into the gas detection model, the detection speed of the gas detection model can be improved. For example, eight consecutive single-channel images are fused into an eight-channel image, so that the channels in the eight-channel image represent the motion state. An eight-channel image can be an eight-frame video disguised as an image, which retains the motion characteristics of the gas and is subsequently used to detect the position of the eighth frame of gas.

[0040] With the support of image processing and deep learning technology, continuous frame images are fused into a multi-channel image to meet the motion characteristics and used as the input data of the gas detection model. The motion characteristics of the gas are extracted from the input data, and the adaptive learning and fusion of multiple feature extraction models are used to perform gas detection and positioning to suit different scenarios and complex backgrounds.

[0041] In the above embodiment, continuous frame infrared images are merged to obtain a multi-channel merged image. A multi-channel image can represent the movement state of the gas. Moreover, the continuous frame infrared images are merged in advance and then input into the gas detection model, which can improve the detection speed of the gas detection model.

[0042] In some embodiments, based on the gas detection data, determining a gas mask includes:

[0043] Based on the gas position data, acquiring a feature area to be processed in the current gas feature map;

[0044] An image processing operation is performed on the feature area to be processed to extract gas area shape data, and the gas mask image is determined based on the gas area shape data.

[0045] In this embodiment, in the current gas feature map, the position data corresponding to the feature area to be processed is the same as the gas position data. The image processing operation includes but is not limited to thresholding operation, filtering operation, corrosion operation, expansion operation, etc. The gas area shape data is obtained by further image processing of the feature area to be processed. Therefore, the gas area indicated by the gas mask image is more accurate than the feature area to be processed.

[0046] In the above embodiment, further image processing operations are performed on the current gas characteristic image to extract an accurate gas mask image area, thereby improving the accuracy of gas detection.

[0047] In some embodiments, in the current frame, coloring and displaying the target gas region indicated by the gas mask image includes:

[0048] In the current frame, the gas distribution in the target gas region is displayed in different colors, wherein different colors correspond to different relative concentrations of the gas.

[0049] In this embodiment, the gas distribution in the target gas area is superimposed and displayed in the current frame image collected. When the current frame image includes an infrared image and / or a visible light image, the gas distribution in the target gas area is superimposed and displayed in the infrared image and / or the visible light image. The gas relative concentration score represents the probability value of the existence of gas at the pixel point. The larger the gas relative concentration score, the higher the probability of existence here. In the application scenario, the greater the gas concentration, the greater the probability of gas detected by the gas detection method. Therefore, according to the gas relative concentration score, different colors are used to represent the distribution of different gas relative concentrations, so that users can intuitively observe the changes in gas concentration, such as Figure 3 As shown, Figure 3 is a schematic diagram showing gas distribution in an embodiment, through Figure 3 The diffusion range and diffusion direction of the gas can be directly observed. At the same time, the gas distribution of the target area is displayed in the current frame. Relative to the background in the current frame, it is convenient for users to observe the diffusion direction of the gas. The probability of gas distribution in the blue area is small, which can be understood as the edge. The probability of gas distribution in the red area is large, which can be understood as the gas outlet position or the gas center. The relative concentration score of the gas is a concentration estimation probability. The colored part is the diffusion range, and the shape can reflect the diffusion direction, such as Figure 3The gas in the figure diffuses to the upper left. Therefore, through the intuitive coloring display method, users can intuitively understand the distribution and concentration of the gas, and directly obtain the gas concentration estimation data and the gas diffusion range and direction. And because the gas is detected continuously in real time, the video is continuous, so a similar colored image can be seen in each frame.

[0050] In an optional implementation, the gas distribution in the target gas area is displayed in gradient colors according to the relative concentration of the gas, wherein the relative concentration of the gas corresponding to the dark color is greater than the relative concentration of the gas corresponding to the light color. The display method of gradient colors makes it easier for users to observe the changing trend and diffusion direction of the gas concentration more intuitively.

[0051] In the above embodiment, different colors are used to represent the distribution of relative concentrations of different gases, so that the user can intuitively observe the changes in gas concentration. At the same time, the gas distribution of the target area is displayed in the current frame, relative to the background in the current frame, which can facilitate the user to observe the diffusion direction of the gas, thereby facilitating the user to observe the dynamic changes of the gas area.

[0052] In some embodiments, Figure 4 As shown, Figure 4 The flow chart of the gas detection method in one embodiment showing the gas distribution in the target gas area; S14 also includes:

[0053] S141. Obtain the confidence corresponding to each pixel in the target area according to the confidence data corresponding to the current gas characteristic map.

[0054] In this embodiment, the confidence level corresponding to each pixel point represents the probability value of the existence of gas at the position of each pixel point.

[0055] S142. Calculate the relative gas concentration score corresponding to each pixel based on the confidence level corresponding to each pixel.

[0056] Optionally, the calculating the gas relative concentration score corresponding to each pixel point based on the confidence level corresponding to each pixel point includes:

[0057] Based on the confidence level corresponding to each pixel point, calculate the first gas relative concentration score corresponding to each pixel point;

[0058] According to the position data corresponding to the gas mask image, a target gas image area is acquired in the current frame, a gas cumulative histogram is obtained from the target gas image area, and a second gas relative concentration score corresponding to each pixel is obtained based on the cumulative histogram; or,

[0059] Acquire the current frame and a previous preset frame of the current frame, perform differential processing on the current frame and the previous preset frame using a frame difference algorithm to obtain a gas differential image, and obtain a third gas relative concentration score corresponding to each pixel point based on the gas differential image; or,

[0060] Acquire the current frame and a previous preset frame of the current frame, acquire a gas region corresponding to the current frame and a gas region corresponding to the previous preset frame according to position data corresponding to the gas mask image, perform differential processing on the gas region corresponding to the current frame and the gas region corresponding to the previous preset frame using a frame difference algorithm to obtain a differential image, and obtain a fourth gas relative concentration score corresponding to each pixel point based on the differential image;

[0061] Determine the gas relative concentration score corresponding to each pixel based on the first gas relative concentration score corresponding to each pixel, or based on the first gas relative concentration score corresponding to each pixel and at least one of the following gas relative concentration scores: the second gas relative concentration score corresponding to each pixel, the third gas relative concentration score corresponding to each pixel, and the fourth gas relative concentration score corresponding to each pixel.

[0062] In this embodiment, the gas relative concentration score represents the probability value of the existence of gas at the position of each pixel. The first gas relative concentration score is obtained based on the confidence of the pre-trained gas detection model output, and the fourth gas relative concentration score is obtained by frame difference processing based on the current gas characteristic map output by the pre-trained gas detection model. Since the detection data output by the pre-trained gas detection model is more accurate, coloring based on the first gas relative concentration score and the fourth gas relative concentration score can make the coloring range more rounded. The second gas relative concentration score and the third gas relative concentration score are obtained by processing based on the original continuous frame image, and the gas plume can be retained. The histogram processing method and the frame difference algorithm can be processed using the methods in the prior art, which will not be repeated here.

[0063] In this embodiment, when calculating the gas relative concentration score of each pixel point, the first gas relative concentration score is weighted with at least one gas relative concentration score to obtain a weighted score, and the weighted score of each pixel point is used as the gas relative concentration score of each pixel point.

[0064] S143. Color each pixel in the current frame based on the gas relative concentration score corresponding to each pixel and the mapping relationship between the gas relative concentration score and the color value.

[0065] In this embodiment, the relative gas concentration corresponds to the color value. For example, when the relative gas concentration is between [0,5], the corresponding color is yellow, when the relative gas concentration is between [5,10], the corresponding color is red, and when the relative gas concentration is between [10,15], the corresponding color is green.

[0066] According to the output results of the gas detection model and feature extraction model, each pixel can be colored and the coloring effect can be added to the current frame image to intuitively display the distribution and concentration of the gas. Among them, the relative gas concentration score corresponding to each pixel is weighted according to the confidence of the detection result. The relative gas concentration score corresponding to each pixel can be mapped to different color values ​​using color mapping technology, and weighted according to the confidence of the detection result to accurately represent the location and concentration of the gas.

[0067] According to the confidence data corresponding to the current gas characteristic map, the gas characteristics can be obtained, such as shape, direction, size, and the probability of gas existence. The relative concentration score of the third gas corresponding to each pixel is obtained through the frame difference algorithm. That is, the score map composed of grayscale features and motion features can be used to fine-tune the shape of the feature map output by the gas detection model to make it closer to the gas contour. That is, the effect of machine learning segmentation is improved by combining the information inside the trained gas detection model with the traditional model.

[0068] In the above embodiment, multiple gas relative concentration scores are calculated based on multiple methods, and the gas relative concentration score of each pixel is calculated based on the multiple gas relative concentration scores. This can accurately represent the probability of the existence of gas at the position of each pixel, so that subsequent coloring can be more accurate based on the gas relative concentration score corresponding to each pixel.

[0069] In some embodiments, the method further comprises:

[0070] In the current frame, a position frame corresponding to the gas mask image is displayed in a first display mode, and a position frame corresponding to the gas position data is displayed in a second display mode, wherein the position frame corresponding to the gas mask image is located within the position frame corresponding to the gas position data.

[0071] In this embodiment, the position frame corresponding to the gas mask image is more accurate than the position frame corresponding to the gas position data. The two position frames are displayed in different display modes, so that the user can more intuitively observe the dynamic changes of the gas area. For example, the first display mode is a solid line frame, and the second display mode is a dotted line frame; the first display mode is a first color, and the second display mode is a second color, and the two display modes are distinguished by color.

[0072] In the above embodiments, the position frame corresponding to the gas mask image and the position frame corresponding to the gas position data are displayed in different ways, so that the user can observe the dynamic changes of the gas area more intuitively.

[0073] In some embodiments, the gas detection model includes multiple feature extraction models and a gas position detection network. The multiple feature extraction models are used to obtain gas characteristic maps corresponding to each feature extraction model based on the input data, and the gas position detection network is used to output the gas detection data based on the gas characteristic maps corresponding to each feature extraction model.

[0074] In this embodiment, since the gas detection model is a model pre-trained with a training data set, each feature extraction model can extract different gas feature maps based on the input data, and multiple feature extraction models can be used to obtain gas feature maps under scene changes from images of continuous frame changes, so as to be more adaptable to scenes under complex changes, such as sports scenes. Since the gas feature maps extracted by multiple feature extraction models can better represent the gas characteristics under scene changes, the gas position detection network can detect more accurate gas position data based on the gas feature maps corresponding to each feature extraction model.

[0075] After the gas detection model training is completed, the parameters in the feature extraction model have been solidified, so based on the input data, the gas feature graphs corresponding to each feature extraction model, or the gas feature graphs corresponding to each feature extraction model and the confidence of the gas feature graphs can be output, where the confidence of the gas feature graph represents the probability that a gas region exists in the feature graph. The gas feature graphs corresponding to one feature extraction model can be one or more, that is, for one feature extraction model, one or more gas feature graphs can be output. The network structure of the feature extraction model includes but is not limited to convolutional layers, downsampling layers, pooling layers, and activation layers.

[0076] In this embodiment, continuous frame images are obtained and input into the input data of a pre-trained gas detection model. Since the gas is a changing target, the input data is formed based on the continuous frame images to better capture the characteristics of the gas. Since the gas detection model is a pre-trained model, the multiple feature extraction models respectively obtain the gas feature maps corresponding to each feature extraction model based on the input data. In this way, each feature extraction model can extract different gas feature maps based on the input data. By using multiple feature extraction models, different gas feature maps under scene changes can be obtained from the continuous frame changing images, so that it can better adapt to scenes under complex changes. Based on the gas feature maps corresponding to each feature extraction model, the gas position detection network can detect more accurate gas position data. Therefore, since the gas detection model is pre-trained, the parameters in the model have been solidified, and the continuous frame images can be directly processed, which can meet the real-time requirements. Moreover, different gas feature maps are extracted based on multiple feature extraction models, which can improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0077] In an optional manner, based on the gas characteristic graphs corresponding to each feature extraction model, the gas position data is output through the gas position detection network. In an optional manner, based on the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs, the gas position data is output through the gas position detection network. Using multiple feature extraction models can obtain different gas characteristic graphs under scene changes from images of continuous frame changes, so as to be more adaptable to scenes under complex changes. Based on the gas characteristic graphs corresponding to each feature extraction model, the gas position detection network can detect more accurate gas position data.

[0078] In the above embodiment, a variety of feature extraction models are used to obtain different gas characteristic maps under scene changes from images of continuous frame changes, so that it can better adapt to scenes under complex changes, and the confidence of the gas characteristic map can be obtained. Based on the gas characteristic map corresponding to each feature extraction model and the confidence of the gas characteristic map, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and complex scenes; using machine learning methods, the gas is detected and located according to different gas features extracted by multiple feature models. In addition, a strategy of fusing multiple feature models can be used to integrate the output results of multiple feature models to improve the accuracy and robustness of gas detection.

[0079] In some embodiments, outputting the gas detection data through the gas detection model based on the input data includes:

[0080] Based on the input data, a gas characteristic graph corresponding to each feature extraction model is obtained through the plurality of feature extraction models, or a gas characteristic graph corresponding to each feature extraction model and a confidence level of the gas characteristic graph are obtained;

[0081] Based on the gas characteristic graphs corresponding to each feature extraction model, or based on the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs, the gas detection data is output through the gas position detection network.

[0082] In this embodiment, after the gas detection model training is completed, the parameters in the feature extraction model have been solidified, so the gas characteristic graphs corresponding to each feature extraction model, or the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs can be output based on the input data, wherein the confidence of the gas characteristic graph represents the probability that a gas region exists in the characteristic graph. The gas characteristic graphs corresponding to one feature extraction model can be one or more, that is, for one feature extraction model, one or more gas characteristic graphs can be output. The network structure of the feature extraction model includes but is not limited to convolutional layers, downsampling layers, pooling layers, and activation layers.

[0083] In an optional manner, based on the gas characteristic graphs corresponding to each feature extraction model, the gas position data is output through the gas position detection network. In an optional manner, based on the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs, the gas position data is output through the gas position detection network. Using multiple feature extraction models can obtain different gas characteristic graphs under scene changes from images of continuous frame changes, so as to be more adaptable to scenes under complex changes. Based on the gas characteristic graphs corresponding to each feature extraction model, the gas position detection network can detect more accurate gas position data.

[0084] In the above embodiment, a variety of feature extraction models are used to obtain different gas characteristic maps under scene changes from images of continuous frame changes, so that it can better adapt to scenes under complex changes, and the confidence of the gas characteristic map can be obtained. Based on the gas characteristic map corresponding to each feature extraction model and the confidence of the gas characteristic map, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0085] Optionally, the outputting the gas detection data through the gas position detection network based on the gas characteristic graphs corresponding to the respective feature extraction models, or based on the gas characteristic graphs corresponding to the respective feature extraction models and the confidence of the gas characteristic graphs, includes:

[0086] The gas characteristic graphs corresponding to the respective feature extraction models are connected through the connection network in the gas detection model to obtain a connection characteristic graph;

[0087] Based on the connection characteristic map, the gas detection data is output through the gas position detection network.

[0088] In this embodiment, the connection network is used to connect the gas feature graphs corresponding to the various feature extraction models to generate a connection feature graph with more feature information, which can be used to increase the number of channels or feature dimensions in the deep learning model so as to better capture the relationship between different gas features in subsequent layers. The connection network can be a convolutional network. In this way, the connection feature graph includes more gas feature data and can represent the relationship between different gas features. Therefore, based on the connection feature graph, the gas position detection network can detect more accurate gas detection data.

[0089] Optionally, each feature extraction model is trained based on each scenario, and a model is overfitted for each scenario. In this way, during testing, the confidence level of the result output by the overfitted model is higher and the features are more obvious. On the contrary, the feature maps of other models are less obvious. Then the head parts of each feature extraction model are connected through a layer of convolution to output a result, namely, the gas connection feature map. For example, there are five feature extraction models in total, and the gas connection feature map is output by a large model composed of five small models, namely, each scenario activates a feature extraction model pathway to obtain a gas feature map corresponding to the feature extraction model, and then the selection weights in the convolution layer are used to connect and fuse the gas feature maps corresponding to each feature extraction model.

[0090] like Figure 5 It is a schematic diagram of generating a connection feature graph in an embodiment. The gas detection model includes two feature extraction models, namely feature extraction models A1 and A2, wherein feature extraction model A1 outputs three feature graphs A12, A13, and A14, and feature extraction model A2 outputs three feature graphs A22, A23, and A24. Then A12, A13, A14, A22, A23, and A24 are used as inputs of a connection network to output a connection feature graph.

[0091] In the above embodiment, the gas characteristic graphs corresponding to the various feature extraction models are connected through the connection network in the gas detection model to obtain a connection characteristic graph. Such a connection characteristic graph includes more gas characteristic data and can represent the relationship between different gas characteristics. Based on the connection characteristic graph, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0092] Optionally, the outputting the gas detection data through the gas position detection network based on the gas characteristic graphs corresponding to the respective feature extraction models, or based on the gas characteristic graphs corresponding to the respective feature extraction models and the confidence of the gas characteristic graphs, includes:

[0093] Based on the confidence of the gas characteristic graph, a target gas characteristic graph that meets the confidence condition is screened from the gas characteristic graphs corresponding to each feature extraction model;

[0094] Based on the target gas characteristic map, the gas detection data is output through the gas position detection network.

[0095] In this embodiment, the target gas characteristic graph represents a gas characteristic graph with a confidence level higher than a preset confidence level, that is, the gas characteristic graph with a high confidence level is used for subsequent gas detection, and the gas characteristic graph with a low confidence level is discarded, thereby improving the feature expression of the gas characteristic graph with a high confidence level and improving the accuracy of gas area detection.

[0096] Optionally, the outputting the gas detection data through the gas position detection network based on the target gas characteristic graph includes at least one of the following:

[0097] When there is only one target gas characteristic graph, the target gas characteristic graph is used as an input of the gas position detection network, and the gas detection data is output through the gas position detection network;

[0098] When there are multiple target gas characteristic graphs, the multiple target gas characteristic graphs are connected through the connection network in the gas detection model to obtain a target connection characteristic graph, and based on the target connection characteristic graph, the gas detection data is output through the gas position detection network.

[0099] like Figure 6 It is a schematic diagram of generating a connection feature map in another embodiment; the feature extraction model A1 outputs three feature maps A12, corresponding to a confidence level of 0.81; A13, corresponding to a confidence level of 0.5; A14, corresponding to a confidence level of 0.85; the feature extraction model A2 outputs three feature maps A22, corresponding to a confidence level of 0.45; A23, corresponding to a confidence level of 0.91; A24, corresponding to a confidence level of 0.95; the preset confidence level is 0.8, then according to the preset confidence level, the target gas feature maps are screened as A12, A14, A23 and A24, and after connecting A12, A14, A23 and A24, the target connection feature map is obtained.

[0100] In the above embodiment, the gas characteristic maps are screened by the confidence levels corresponding to the gas characteristic maps, and the gas characteristic maps with high confidence levels are used for subsequent gas position detection, thereby activating the feature expression of the gas characteristic maps with high confidence levels and suppressing the feature expression of the gas characteristic maps with low confidence levels, thereby highlighting the features of the gas area in the current scene. In this way, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0101] In some embodiments,

[0102] The feature extraction model includes a multi-scale feature network. Based on the input data, the gas feature graphs corresponding to the respective feature extraction models are obtained through the plurality of feature extraction models, including:

[0103] Based on the input data, through the multi-scale feature network in each of the feature extraction models, gas feature maps of multiple different scales corresponding to each feature extraction model are output.

[0104] In this embodiment, a multi-scale feature network is used to extract multiple gas feature maps of different sizes based on input data. The multi-scale feature network includes but is not limited to the FPN (Feature Pyramid Network) network structure and the PAN (Pyramid Attention Network) network structure. FPN conveys strong semantic features from top to bottom, while PAN conveys strong positioning features from bottom to top. They work together to aggregate parameters of different detection layers from different backbone layers, so that the location information and category information of the gas target are retained to the greatest extent. The multi-scale network obtains multiple input feature maps of different scales through multiple different downsampling operations, and then the multiple input feature maps of different scales are respectively input into the network structure formed by FPN and PAN for processing to obtain multiple gas feature maps of different scales. For example, Figure 5 or Figure 6 As shown, the feature extraction model A1 outputs three gas feature maps respectively, and the corresponding scales of A12, A13, and A14 are different. For example, the scale of A12 is 20*20, the scale of A13 is 100*100, and the scale of A14 is 400*400. Then, A12 is used for gas detection in smaller gas areas, A13 is used for gas detection in medium-sized gas areas, and A14 is used for gas detection in large gas areas.

[0105] In the above embodiment, each feature extraction model corresponds to a plurality of gas feature maps of different scales through the multi-scale feature network output, which can better adapt to the detection of gas areas of different sizes in complex scenes and complex backgrounds. Based on gas feature maps of different scales, the gas position detection network can detect more accurate gas position data, which can improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0106] In some embodiments, the method further includes: each of the feature extraction models corresponds to a respective sample data set, wherein different sample data sets are collected from different gas detection scenarios.

[0107] In this embodiment, common gas detection scenes can be classified to obtain multiple scenes, and then multiple sample images are collected in each scene, and the feature extraction model of each scene is pre-trained. For example, the gas detection scene can be divided into indoor scenes and outdoor scenes. The indoor scene can be further subdivided into office scenes, factory scenes, oil field indoor scenes, and the like. The outdoor scene can be further subdivided into oil field outdoor scenes, factory outdoor scenes, and the like, such as the oil field outdoor scene corresponding to feature extraction model A, and the factory outdoor scene corresponding to feature extraction model B. A large number of sample images are collected in each scene to train the feature extraction model corresponding to the scene, and the collected sample images may include rich backgrounds, including but not limited to the sky, black objects, grass, pipes, glass, blue sky, white clouds, sea, and the like. In this way, during the training process, the corresponding feature extraction model can learn the main features of the scene and the rich background features during the training process. After the training of each feature extraction model is completed and each feature extraction model is used in the gas detection model, the gas detection model can be applied to a variety of complex scenes and a variety of complex backgrounds. In the embodiment of the present application, based on artificial intelligence technology, sample data in various complex backgrounds and multiple scenes are obtained, so as to realize gas detection in complex backgrounds and multiple scenes, by collecting a large amount of video data and classifying the data according to the scene. Then, multiple feature extraction models are trained using different sample data sets, and these feature extraction models are fused to obtain a gas detection model.

[0108] In the above embodiments, each feature extraction model corresponds to its own sample data set, and different sample data sets are collected from different gas detection scenarios. In this way, the feature extraction model can learn the main features and rich background features of the corresponding scene from the corresponding sample data set during the training process. In this way, the gas detection model can detect more accurate gas location data and improve the accuracy of gas detection in complex backgrounds and complex scenarios.

[0109] In some embodiments, Figure 7 As shown, Figure 7is a schematic diagram of training a gas detection model in an embodiment, wherein the method further comprises:

[0110] S51, obtaining a training data set, wherein each training image in the training data set includes a continuous frame training image and gas label data corresponding to the continuous frame training image.

[0111] In this embodiment, the training data set may include multiple sample data sets, and the training image is a continuous frame training image randomly extracted from the multiple sample data sets. Since the multiple sample data sets are obtained from multiple gas detection scenes, the training data set is also obtained based on multiple gas detection scenes. The gas area in the current frame of the continuous frame training image is used as the corresponding gas label data. For example, the continuous frame training image is the first frame, the second frame, and the third frame, and the third frame is the current frame, then the gas area in the third frame is used as the corresponding gas label data.

[0112] The training data set can come from a large amount of video data. A large amount of video data includes various scenes from the fields of industry, environment and safety monitoring. These video data should contain the movement and distribution of various gases. The collected data should be preprocessed, including frame sampling, resolution adjustment and labeling, etc., to facilitate subsequent model training and evaluation.

[0113] S52: construct an initial gas detection model, wherein the initial gas detection model includes a plurality of trained feature extraction models and an initial gas position detection network.

[0114] In this embodiment, in order to speed up the training of the gas detection model, each feature extraction model can be pre-trained, and then each feature extraction model can be trained separately based on its corresponding sample data set. During the training process, it is necessary to set appropriate hyperparameters, learning rate and batch size, and perform data enhancement and regularization operations to improve the generalization ability and robustness of the model.

[0115] Optionally, before obtaining the pre-trained gas detection model, the method further includes:

[0116] Based on each of the sample data sets, training is performed to obtain each trained feature extraction model;

[0117] Wherein, based on each of the sample data sets, training each trained feature extraction model comprises:

[0118] Acquire the sample data set, wherein the sample data set includes each sample image including a continuous frame sample image, and a gas characteristic map label corresponding to the continuous frame sample image;

[0119] Build an initial feature extraction model;

[0120] Based on the sample data set, the initial feature extraction model corresponding to the sample data set is iteratively trained until a training termination condition is met, thereby obtaining a trained feature extraction model corresponding to the sample data set.

[0121] In this embodiment, first, a large amount of video data is collected as a sample data set, and classified according to different scenes. Then, for each scene, one or more feature extraction models are trained using the corresponding sample data set. These trained feature extraction models are fused to combine the advantages of different models and improve the accuracy and robustness of gas detection. The continuous frame sample images include but are not limited to continuous frame sample infrared images, continuous frame sample visible light images, and continuous frame sample fusion images obtained by fusing continuous frame sample infrared images with continuous frame sample visible light images. Train each feature extraction model according to the above steps. In the process of training a feature extraction model, a sample image is obtained from a sample data set, sample image input data is formed based on the sample image, and a gas feature map corresponding to the sample image input data is output through the feature extraction model in the current iteration process. Based on the feature loss function, the gas feature map corresponding to the sample image input data, and the gas feature map label corresponding to the sample image input data, the loss value in the current iteration is output. Based on the loss value in the current iteration, it is judged whether the current iteration meets the training termination condition. If the current iteration meets the training termination condition, the feature extraction model after the iteration termination is used as the trained feature extraction model; if the current iteration does not meet the training termination condition, the sample image is continued to be obtained from the sample data set for training. The training termination conditions include but are not limited to the number of iterations, the loss value is less than the preset loss value, etc. The feature loss function includes but is not limited to the cross entropy function, the mean square error function, etc.

[0122] S53. Iteratively train the initial gas detection model using the training data set until the loss function converges, thereby obtaining the pre-trained gas detection model.

[0123] Optionally, iteratively training the initial gas detection model using the training data set until the loss function converges to obtain the pre-trained gas detection model includes:

[0124] Acquire a training image from the training data set, form training image input data of the gas detection model of the current iteration based on the training image, output a gas sample feature map corresponding to each feature extraction model through multiple feature extraction models in the current iteration based on the training image input data, and output gas sample position data corresponding to the training image through the gas position detection network in the current iteration based on the gas sample feature map corresponding to each feature extraction model;

[0125] Based on the loss function, the loss value between the gas sample position data corresponding to the training image and the gas label data corresponding to the training image is calculated to obtain the loss value of the current iteration;

[0126] Based on the loss value of the current iteration, determine whether the current iteration meets the iteration termination condition. If the current iteration does not meet the iteration termination condition, continue to obtain training images from the training data set and iteratively train the gas detection model; if the current iteration meets the iteration termination condition, the gas detection model after stopping the iteration will be used as the pre-trained gas detection model.

[0127] In this embodiment, continuous frame training images are merged to obtain a multi-channel training image input data. For example, 8 consecutive frames of training images are arranged by channel and merged into a 512*640*8 image. The label of this training image is the gas position label of the last frame. The training termination conditions include but are not limited to the number of iterations, the loss value is less than the preset loss value, etc. The loss function includes but is not limited to the cross entropy function, the mean square error function, etc. After the feature extraction model is trained based on the sample data set, the initial parameters in the feature extraction model can be obtained. When the gas detection model is trained as a whole based on the training data set, the initial parameters in the feature extraction model will be fine-tuned and optimized so that the optimized parameters can better enable the gas position detection network to detect more accurate gas areas. Moreover, since the training data set is formed by multiple sample data sets, and the multiple sample data sets are from multiple gas detection scenes, during the training process, the gas detection model can learn the gas features and rich background features in multiple scenes and backgrounds. After the training is completed, it can better adapt to gas detection in complex scenes and complex backgrounds. Therefore, when the trained gas detection model is applied to motion scenes, such as handheld detection equipment and non-handheld autonomously movable detection equipment, the scene is constantly changing during the motion, but the gas detection model is trained based on the training data sets collected in various gas detection scenarios during the training process. Therefore, the gas detection model can better adapt to this motion scene.

[0128] Optionally, after training, the gas detection model can be optimized, including model compression, pruning, and quantization techniques to reduce the model's computational and storage resources. In addition, it is possible to consider fusing multiple feature extraction models, and through strategies such as voting, weighted averaging, or model integration, the output results of multiple models can be integrated to improve the accuracy and robustness of gas detection.

[0129] Optionally, the training of the gas detection model can be performed on a computing device that has a high-performance computing device, such as a GPU or a cloud computing platform, to perform complex image processing and deep learning calculations. In addition, in order to improve the efficiency and accuracy of the algorithm, parameter tuning and optimization are required. For example, cross-validation is used to determine the hyperparameters of the feature extraction model, such as the convolution kernel size and the number of layers. At the same time, data enhancement techniques such as translation, rotation, and scaling are applied to expand the training data set and improve the robustness and generalization ability of the model.

[0130] In the above embodiment, the feature extraction models corresponding to each other are first trained based on each sample data set to obtain the initial parameters of the feature extraction model, and then the gas detection model is trained based on the training data set, so as to accelerate the convergence speed of the gas detection model. Since the training data set is formed by multiple sample data sets, and the multiple sample data sets are from multiple gas detection scenes, the gas detection model can learn the gas features and rich background features under multiple scenes and backgrounds, and can improve the accuracy of gas detection under complex backgrounds and complex scenes. It is comprehensively applied to different scenes and complex backgrounds, taking into account the influence of different scenes and complex backgrounds, and is classified in the data collection stage, and the corresponding model is trained for each scene. In this way, it can adapt to the gas detection needs of different scenes and improve the adaptability to complex backgrounds. Multi-model fusion strategy: This method trains multiple models using different data sets, fuses these models, and integrates the output results of multiple models to improve the accuracy and robustness of gas detection. By adopting the strategy of multi-model fusion, the advantages of each model can be fully utilized to improve the overall detection performance.

[0131] In some embodiments, the continuous frame images include any one of the following: continuous frame infrared images, continuous frame visible light images, and continuous frame fused images obtained by fusing continuous frame infrared images with continuous frame visible light images.

[0132] In this embodiment, when the continuous frame images are continuous frame infrared images, collecting infrared images can reduce the impact of environmental factors, such as the impact of bad weather such as rainy days and foggy days, and the infrared images can also capture colorless gases and colored gases, which can improve the accuracy of subsequent gas detection. When the continuous frame images are continuous frame fusion images, infrared features and visible light features can be captured at the same time, which can provide more features for subsequent gas detection.

[0133] When one or more of the above embodiments of the present application are combined, the present application has at least the following effective effects:

[0134] At least one embodiment of the present application combines image processing and deep learning technology, which can automatically extract the characteristics of gases in multiple frames of infrared images, and distinguish and locate different gases. At the same time, it can extract the probability of the gas position in the current frame image and assign pseudo-color, which is superimposed on the current frame image. In addition, the algorithm achieves adaptability to complex backgrounds and multiple scenes through adaptive learning and multi-model fusion.

[0135] At least one embodiment of the present application obtains a gas connection feature map from a feature extraction model, and then applies thresholding, filtering, and morphological processing to extract the location and shape information of the gas. Then, the grayscale and frame difference method of the infrared frame are combined to further enhance the gas characteristics. The grayscale provides the brightness information of the gas, and the frame difference method is used to detect the movement of the gas. This comprehensive processing process can help improve the accuracy and robustness of gas detection and provide a more accurate basis for subsequent analysis and visualization.

[0136] At least one embodiment of the present application realizes a multi-gas detection and coloring algorithm suitable for complex backgrounds and multiple scenarios through preprocessing, feature extraction, gas detection and positioning, and coloring display, combined with image processing and deep learning technology, and realizes the detection and positioning of multiple gases, and displays the detection results in an intuitive coloring way, so that users can intuitively understand the distribution and concentration of the gas. This method has broad application prospects and can play an important role in the fields of industry, environment and safety monitoring.

[0137] At least one embodiment of the present application adopts a multi-model fusion strategy: by using different sample data sets to train multiple feature extraction models, and fusing these feature extraction models, the output results of multiple feature extraction models are integrated to improve the accuracy and robustness of gas detection. By adopting a multi-model fusion strategy, the advantages of each model can be fully utilized to improve the overall detection performance. Taking into account the impact of different scenes and complex backgrounds, classification is performed during the data collection stage, and a corresponding feature extraction model is trained for each scene. This can adapt to the gas detection needs of different scenes and improve adaptability to complex backgrounds.

[0138] At least one embodiment of the present application uses gas feature extraction and coloring. Through feature extraction technology, the motion features of the gas are extracted from the input continuous frame images, and the gas detection model trained by machine learning methods is used to detect and locate the gas. At the same time, by weighted coloring of each pixel and adding the coloring effect to the current frame image, the distribution and concentration of the gas are intuitively displayed. This method of gas feature extraction and coloring improves the accuracy and visualization of gas detection.

[0139] On the other hand, the present application provides a computer program product, including a computer program, which implements the gas detection method described in any embodiment of the present application when executed by a processor.

[0140] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the gas detection method can be a gas detection device.

[0141] See also Figure 8 An embodiment of the present application provides a gas detection device, including: an acquisition module 81, used to acquire continuous frame images; a detection module 82, used to form input data of a pre-trained gas detection model based on the continuous frame images, and output gas detection data through the gas detection model based on the input data, wherein the gas detection data includes gas position data in the current frame of the continuous frame images and a current gas characteristic map in the current frame; the detection module 82 is also used to determine a gas mask map based on the gas detection data; a display module 83 is used to color and display the target gas area indicated by the gas mask map in the current frame.

[0142] Optionally, the detection module 82 is further used for:

[0143] The continuous frame infrared images are merged to obtain a multi-channel merged image, and the input data is obtained based on the merged image.

[0144] Optionally, the detection module 82 is further used for:

[0145] Based on the gas position data, acquiring a feature area to be processed in the current gas feature map;

[0146] An image processing operation is performed on the feature area to be processed to extract gas area shape data, and the gas mask image is determined based on the gas area shape data.

[0147] Optionally, the display module 83 is also used for:

[0148] In the current frame, the gas distribution in the target gas region is displayed in different colors, wherein different colors correspond to different relative concentrations of the gas.

[0149] Optionally, the display module 83 is further used for:

[0150] According to the confidence data corresponding to the current gas characteristic map, obtaining the confidence corresponding to each pixel in the target area;

[0151] Based on the confidence level corresponding to each pixel, the relative gas concentration score corresponding to each pixel is calculated;

[0152] Based on the gas relative concentration score corresponding to each pixel and the mapping relationship between the gas relative concentration score and the color value, each pixel in the current frame is colored.

[0153] Optionally, the display module 83 is also used for:

[0154] Based on the confidence level corresponding to each pixel point, calculate the first gas relative concentration score corresponding to each pixel point;

[0155] According to the position data corresponding to the gas mask image, a target gas image area is acquired in the current frame, a gas cumulative histogram is obtained from the target gas image area, and a second gas relative concentration score corresponding to each pixel is obtained based on the cumulative histogram; or,

[0156] Acquire the current frame and a previous preset frame of the current frame, perform differential processing on the current frame and the previous preset frame using a frame difference algorithm to obtain a gas differential image, and obtain a third gas relative concentration score corresponding to each pixel point based on the gas differential image; or,

[0157] Acquire the current frame and a previous preset frame of the current frame, acquire a gas region corresponding to the current frame and a gas region corresponding to the previous preset frame according to position data corresponding to the gas mask image, perform differential processing on the gas region corresponding to the current frame and the gas region corresponding to the previous preset frame using a frame difference algorithm to obtain a differential image, and obtain a fourth gas relative concentration score corresponding to each pixel point based on the differential image;

[0158] Determine the gas relative concentration score corresponding to each pixel based on the first gas relative concentration score corresponding to each pixel, or based on the first gas relative concentration score corresponding to each pixel and at least one of the following: the second gas relative concentration score corresponding to each pixel, the third gas relative concentration score corresponding to each pixel, and the fourth gas relative concentration score corresponding to each pixel.

[0159] Optionally, the display module 83 is also used for:

[0160] In the current frame, a position frame corresponding to the gas mask image is displayed in a first display mode, and a position frame corresponding to the gas position data is displayed in a second display mode, wherein the position frame corresponding to the gas mask image is located within the position frame corresponding to the gas position data.

[0161] Optionally, the gas detection model includes multiple feature extraction models and a gas position detection network, the multiple feature extraction models are used to obtain the gas characteristic graphs corresponding to each feature extraction model based on the input data, and the gas position detection network is used to output the gas detection data based on the gas characteristic graphs corresponding to each feature extraction model.

[0162] Optionally, the detection module 82 is further used for:

[0163] Based on the input data, a gas characteristic graph corresponding to each feature extraction model is obtained through the plurality of feature extraction models, or a gas characteristic graph corresponding to each feature extraction model and a confidence level of the gas characteristic graph are obtained;

[0164] Based on the gas characteristic graphs corresponding to each feature extraction model, or based on the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs, the gas detection data is output through the gas position detection network.

[0165] Optionally, the detection module 82 is further used for:

[0166] The gas characteristic graphs corresponding to the respective feature extraction models are connected through the connection network in the gas detection model to obtain a connection characteristic graph;

[0167] Based on the connection characteristic map, the gas detection data is output through the gas position detection network.

[0168] Optionally, the detection module 82 is further used for:

[0169] Based on the confidence of the gas characteristic graph, a target gas characteristic graph that meets the confidence condition is screened from the gas characteristic graphs corresponding to each feature extraction model;

[0170] Based on the target gas characteristic map, the gas detection data is output through the gas position detection network.

[0171] Optionally, the detection module 82 is further used for:

[0172] When there is only one target gas characteristic graph, the target gas characteristic graph is used as an input of the gas position detection network, and the gas detection data is output through the gas position detection network;

[0173] When there are multiple target gas characteristic graphs, the multiple target gas characteristic graphs are connected through the connection network in the gas detection model to obtain a target connection characteristic graph, and based on the target connection characteristic graph, the gas detection data is output through the gas position detection network.

[0174] Optionally, the feature extraction model includes a multi-scale feature network. Optionally, the detection module 82 is further used for:

[0175] Based on the input data, through the multi-scale feature network in each of the feature extraction models, gas feature maps of multiple different scales corresponding to each feature extraction model are output.

[0176] Optionally, each of the feature extraction models corresponds to a respective sample data set, wherein different sample data sets are collected from different gas detection scenarios.

[0177] Optionally, the continuous frame images include any one of the following: continuous frame infrared images, continuous frame visible light images, and continuous frame fused images obtained by fusing continuous frame infrared images with continuous frame visible light images.

[0178] It can be understood by those skilled in the art that Figure 8 The structure of the gas detection device does not constitute a limitation on the gas detection device, and each module can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of the processor in the gas detection device in the form of hardware, or can be stored in the memory in the gas detection device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. In other embodiments, the gas detection device may include more or fewer modules than those shown in the figure.

[0179] See also Fig. 9 On the other hand, a gas detection device 10 is provided, including a processor 13 and a memory 14. The memory 14 stores a computer program. When the computer program is executed by the processor, the processor 13 executes the steps of the gas detection method provided in any of the above embodiments of the present application.

[0180] The processor 13 is the control center, which uses various interfaces and lines to connect various parts of the entire gas detection device, and executes various functions of the gas detection device and processes data by running or executing software programs and / or modules stored in the memory 14, and calling data stored in the memory 14. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 13.

[0181] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the gas detection device, etc. In addition, the memory 14 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 14 may also include a memory processor to provide the processor 13 with access to the memory 14.

[0182] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the gas detection method provided in any of the above embodiments of the present application.

[0183] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0184] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A gas detection method, characterized in that: include: Acquire continuous frame images; Based on the continuous frame images, forming input data of a pre-trained gas detection model, and based on the input data, outputting gas detection data through the gas detection model, the gas detection data including gas position data in a current frame in the continuous frame images and a current gas feature map in the current frame; Determining a gas mask based on the gas detection data; In the current frame, the target gas region indicated by the gas mask image is displayed in color.

2. The gas detection method according to claim 1, characterized in that: The continuous frame images are continuous frame infrared images, and the input data for forming a pre-trained gas detection model based on the continuous frame images includes: The continuous frame infrared images are merged to obtain a multi-channel merged image, and the input data is obtained based on the merged image.

3. The gas detection method according to claim 1, characterized in that: Determining a gas mask image based on the gas detection data includes: Based on the gas position data, acquiring a feature area to be processed in the current gas feature map; An image processing operation is performed on the feature area to be processed to extract gas area shape data, and the gas mask image is determined based on the gas area shape data.

4. The gas detection method according to claim 1, characterized in that: In the current frame, coloring and displaying the target gas area indicated by the gas mask image includes: In the current frame, the gas distribution in the target gas region is displayed in different colors, wherein different colors correspond to different relative concentrations of the gas.

5. The gas detection method according to claim 4, characterized in that: The gas detection data further includes: confidence data corresponding to the current gas characteristic graph, and the gas distribution in the target gas area is displayed in different colors in the current frame, including: According to the confidence data corresponding to the current gas characteristic map, obtaining the confidence corresponding to each pixel in the target area; Based on the confidence level corresponding to each pixel, the relative gas concentration score corresponding to each pixel is calculated; Based on the gas relative concentration score corresponding to each pixel and the mapping relationship between the gas relative concentration score and the color value, each pixel in the current frame is colored.

6. The gas detection method according to claim 5, characterized in that: The step of calculating the relative gas concentration score corresponding to each pixel point based on the confidence level corresponding to each pixel point includes: Based on the confidence level corresponding to each pixel point, calculate the first gas relative concentration score corresponding to each pixel point; According to the position data corresponding to the gas mask image, a target gas image area is acquired in the current frame, a gas cumulative histogram is obtained from the target gas image area, and a second gas relative concentration score corresponding to each pixel is obtained based on the cumulative histogram; or, Acquire the current frame and a previous preset frame of the current frame, perform differential processing on the current frame and the previous preset frame using a frame difference algorithm to obtain a gas differential image, and obtain a third gas relative concentration score corresponding to each pixel point based on the gas differential image; or, Acquire the current frame and a previous preset frame of the current frame, acquire a gas region corresponding to the current frame and a gas region corresponding to the previous preset frame according to position data corresponding to the gas mask image, perform differential processing on the gas region corresponding to the current frame and the gas region corresponding to the previous preset frame using a frame difference algorithm to obtain a differential image, and obtain a fourth gas relative concentration score corresponding to each pixel point based on the differential image; Determine the gas relative concentration score corresponding to each pixel based on the first gas relative concentration score corresponding to each pixel, or based on the first gas relative concentration score corresponding to each pixel and at least one of the following: the second gas relative concentration score corresponding to each pixel, the third gas relative concentration score corresponding to each pixel, and the fourth gas relative concentration score corresponding to each pixel.

7. The gas detection method according to claim 1, characterized in that: The method further comprises: In the current frame, a position frame corresponding to the gas mask image is displayed in a first display mode, and a position frame corresponding to the gas position data is displayed in a second display mode, wherein the position frame corresponding to the gas mask image is located within the position frame corresponding to the gas position data.

8. The gas detection method according to claim 1, characterized in that: The gas detection model includes multiple feature extraction models and a gas position detection network. The multiple feature extraction models are used to obtain gas characteristic maps corresponding to each feature extraction model based on the input data. The gas position detection network is used to output the gas detection data based on the gas characteristic maps corresponding to each feature extraction model.

9. The gas detection method according to claim 8, characterized in that: The step of outputting gas detection data based on the input data through the gas detection model comprises: Based on the input data, a gas characteristic graph corresponding to each feature extraction model is obtained through the plurality of feature extraction models, or a gas characteristic graph corresponding to each feature extraction model and a confidence level of the gas characteristic graph are obtained; Based on the gas characteristic graphs corresponding to each feature extraction model, or based on the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs, the gas detection data is output through the gas position detection network.

10. The gas detection method according to claim 9, characterized in that: The method outputting the gas detection data through the gas position detection network based on the gas characteristic graphs corresponding to the respective feature extraction models, or based on the gas characteristic graphs corresponding to the respective feature extraction models and the confidence of the gas characteristic graphs, comprises: The gas characteristic graphs corresponding to the respective feature extraction models are connected through the connection network in the gas detection model to obtain a connection characteristic graph; Based on the connection characteristic map, the gas detection data is output through the gas position detection network.

11. The gas detection method according to claim 9, characterized in that: The method outputting the gas detection data through the gas position detection network based on the gas characteristic graphs corresponding to the respective feature extraction models, or based on the gas characteristic graphs corresponding to the respective feature extraction models and the confidence of the gas characteristic graphs, comprises: Based on the confidence of the gas characteristic graph, a target gas characteristic graph that meets the confidence condition is screened from the gas characteristic graphs corresponding to each feature extraction model; Based on the target gas characteristic map, the gas detection data is output through the gas position detection network.

12. The gas detection method according to claim 11, characterized in that: The outputting of the gas detection data through the gas position detection network based on the target gas characteristic diagram includes at least one of the following: When there is only one target gas characteristic graph, the target gas characteristic graph is used as an input of the gas position detection network, and the gas detection data is output through the gas position detection network; When there are multiple target gas characteristic graphs, the multiple target gas characteristic graphs are connected through the connection network in the gas detection model to obtain a target connection characteristic graph, and based on the target connection characteristic graph, the gas detection data is output through the gas position detection network.

13. The gas detection method according to claim 9, characterized in that: The feature extraction model includes a multi-scale feature network. Based on the input data, the gas feature graphs corresponding to the respective feature extraction models are obtained through the plurality of feature extraction models, including: Based on the input data, through the multi-scale feature network in each of the feature extraction models, gas feature maps of multiple different scales corresponding to each feature extraction model are output.

14. The gas detection method according to claim 8, characterized in that: Each of the feature extraction models corresponds to a respective sample data set, wherein different sample data sets are collected from different gas detection scenarios.

15. The gas detection method according to claim 1, characterized in that: The continuous frame images include any one of the following: continuous frame infrared images, continuous frame visible light images, and continuous frame fused images obtained by fusing continuous frame infrared images with continuous frame visible light images.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the gas detection method according to any one of claims 1 to 15 is implemented.

17. A gas detection device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the gas detection method according to any one of claims 1 to 15.

18. A storage medium comprising a computer program, characterized in that A computer program is stored, and when the computer program is executed by a processor, the processor executes the gas detection method according to any one of claims 1 to 15.

Images