Gas detection method, program product and device

Abstract

The invention discloses a gas detection method, a program product and equipment. The method comprises the following steps: acquiring continuous frame images; based on the continuous frame images, forming input data of a pre-trained gas detection model; based on the input data, outputting gas position data in the current frame in the continuous frame images through the gas detection model; wherein the gas detection model comprises a plurality of feature extraction models and a gas position detection network, the plurality of feature extraction models are used for respectively obtaining gas feature maps corresponding to the feature extraction models based on the input data, and the gas position detection network is used for detecting the gas position of the gas detection model based on the gas feature maps corresponding to the feature extraction models. And outputting the gas position data.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a gas detection method, a computer program product and a gas detection device. Background Art

[0002] In recent years, the demand for gas detection and monitoring in the fields of industry, environment and safety monitoring has been increasing. Traditional gas detection methods usually rely on sensors installed at a fixed location for detection, but the application effect is poor in complex backgrounds and multiple scenarios. For example, in sports scenes, the scene is constantly changing. Using traditional gas detection methods, the accuracy of the detected gas area is low. At present, the existing gas detection algorithms mainly focus on the detection of specific fixed scenes or specific gases, and the joint detection of complex backgrounds and multiple gases is still challenging. For example, in industrial production environments, there are often multiple harmful gases at the same time, such as formaldehyde, hydrogen sulfide and carbon monoxide. In urban environments, the sources of gas are diverse and complex, such as automobile exhaust, industrial waste gas and combustion emissions. In view of these situations, traditional gas detection methods are difficult to meet the real-time and complex background and multiple scenarios. Application requirements. Summary of the invention

[0003] In order to solve the existing technical problems, the present invention provides a gas detection method, a computer program product and a gas detection device, which can improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0004] In a first aspect, a gas detection method is provided, comprising: acquiring continuous frame images; forming input data of a pre-trained gas detection model based on the continuous frame images; and outputting gas position data in a current frame in the continuous frame images through the gas detection model based on the input data; wherein the gas detection model comprises a plurality of feature extraction models and a gas position detection network, the plurality of feature extraction models being used to obtain gas characteristic maps corresponding to each feature extraction model based on the input data, and the gas position detection network being used to output the gas position data based on the gas characteristic maps corresponding to each feature extraction model.

[0005] In a second aspect, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the gas detection method as described in any one of the first aspects of the present application is implemented.

[0006] In a third aspect, a gas detection device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the gas detection method as described in any one of the first aspects of the present application.

[0007] In a fourth aspect, a storage medium is provided, comprising a computer program, storing the computer program, and when the computer program is executed by a processor, the processor executes the gas detection method described in any one of the first aspects of the present application.

[0008] The embodiment of the present application obtains continuous frame images and inputs the continuous frame images into the input data of a pre-trained gas detection model. Since the gas is a changing target, the input data is formed based on the continuous frame images to facilitate better capture of the characteristics of the gas. Since the gas detection model is a pre-trained model, the multiple feature extraction models respectively obtain the gas feature maps corresponding to each feature extraction model based on the input data. In this way, each feature extraction model can extract different gas feature maps based on the input data. By using multiple feature extraction models, different gas feature maps under scene changes can be obtained from the images of continuous frame changes, so that it can better adapt to scenes under complex changes. Based on the gas feature maps corresponding to each feature extraction model, the gas position detection network can detect more accurate gas position data. Therefore, since the gas detection model is pre-trained, the parameters in the model have been solidified, and the continuous frame images can be directly processed, which can meet the real-time requirements. Moreover, different gas feature maps are extracted based on multiple feature extraction models, which can improve the accuracy of gas detection in complex backgrounds and complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A diagram showing an application environment of a gas detection method in an embodiment;

[0010] Figure 2 is a flow chart of a gas detection method in one embodiment;

[0011] Figure 3 A schematic diagram of generating a connection feature graph in one embodiment;

[0012] Figure 4 A schematic diagram of generating a connection feature graph in another embodiment;

[0013] Figure 5 A schematic diagram of training a gas detection model in one embodiment;

[0014] Figure 6 is a schematic diagram of a gas detection device in one embodiment;

[0015] Figure 7 Schematic diagram of a gas detection device in one embodiment. DETAILED DESCRIPTION

[0016] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments of the specification.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the technical field of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0018] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0019] See also Figure 1 , is an application environment diagram of a gas detection method in an embodiment. The gas detection method is applied to a gas detection device 10, and the gas detection device 10 includes an image acquisition device 12, a processor 13 and a memory 14. The image acquisition device 12 is used to acquire continuous frame images in front of the gas detection device 10, and the processor 13 detects the gas area in the continuous frame images according to the continuous frame images acquired by the image acquisition device 12. The memory 14 is used to store programs and data corresponding to the implementation of the gas detection method.

[0020] The gas detection device 10 includes but is not limited to a handheld detection device, a non-handheld autonomously movable detection device, and a non-handheld non-autonomously movable detection device. Among them, the handheld detection device includes but is not limited to a handheld imaging device with an infrared thermal imaging function and a handheld imaging device with a visible light imaging function. Among them, the non-handheld autonomously movable detection device includes but is not limited to an autonomously movable detection device with an infrared thermal imaging function and an autonomously movable detection device with a visible light imaging function. The non-handheld non-autonomously movable detection device includes but is not limited to a non-handheld and non-autonomously movable device with an infrared thermal imaging function and a non-handheld and non-autonomously movable device with a visible light imaging function. Among them, through the handheld detection device and the non-handheld autonomously movable detection device, the user can perform gas detection on the current scene in motion during the movement, and can also perform gas detection on a fixed scene. Gas detection can be performed in a fixed scene through a non-handheld non-autonomously movable detection device. Therefore, the gas detection method provided in the present application can be applied to various complex motion change scenes and fixed scenes.

[0021] The image acquisition device 12 may be a combination of one or more sensors. The image acquisition device 12 may be a monocular vision sensor or a multi-vision sensor. For example, it may be a combination of one or more sensors selected from the group consisting of a thermal imaging sensor, a visible light image sensor, a millimeter wave sensor, a laser radar sensor, an infrared thermal imaging sensor, and a depth sensor.

[0022] The processor 13 may be one or more. When there are multiple processors 13, the multiple processors may be integrated on one chip or independently arranged on each chip. The gas detection device 10 is a device installed on any type of mobile body, such as a vehicle, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship or a robot, etc. The gas detection device 10 may also be fixedly installed on a fixed device or at a fixed position in a fixed scene.

[0023] The gas detection device 10 may also include other sensor modules, including but not limited to environmental perception sensors and motion posture sensors. Environmental perception sensors include but are not limited to one or more combinations of the following sensors: brightness sensors, temperature sensors, haze sensors and other environmental sensors. Motion posture sensors include but are not limited to one or more combinations of the following: inertial sensors (Inertial Measurement Unit, IMU), speed sensors, acceleration sensors, gyroscope sensors, geomagnetic sensors, rotation vector sensors, steering wheel angle sensors, level sensors, tilt sensors, vibration sensors, displacement sensors and gravity sensors, etc.

[0024] The gas detection device 10 may further include a display terminal for displaying images.

[0025] See also Figure 2 , is a flow chart of a gas detection method provided in an embodiment of the present application. The gas detection method is applied in a gas detection device, and the gas detection method includes the following steps:

[0026] S11, acquiring continuous frame images.

[0027] In this embodiment, the continuous frame images include but are not limited to continuous frame infrared images and continuous frame fused images obtained by fusing continuous frame infrared images with continuous frame visible light images. The continuous frame infrared images and / or continuous frame visible light images are collected by the image acquisition device 12. When the continuous frame infrared images and the continuous frame visible light images are synchronously collected in time, the continuous frame infrared images and the continuous frame visible light images are mapped to the same image coordinate system, so that the spatial synchronization of the continuous frame infrared images and the continuous frame visible light images can be achieved, and then the continuous frame infrared images and the continuous frame visible light images are fused by the image fusion method to obtain the continuous frame fused image. Since the gas is changing, it is more convenient to extract the characteristics of the gas later based on the continuous frame images.

[0028] S12. Based on the continuous frame images, input data of a pre-trained gas detection model is formed.

[0029] In this embodiment, the gas detection model is obtained by training with a training data set. The gas detection model includes multiple feature extraction models and a gas position detection network. After the training is completed, the parameters in the feature extraction model and the gas position detection network have been solidified. After preprocessing the continuous frame images, the input data is obtained, and the preprocessing includes but is not limited to denoising, cropping, background processing, etc. In an optional implementation, the continuous frame images can also be directly used as input data.

[0030] S13. Based on the input data, output the gas position data in the current frame in the continuous frame images through the gas detection model.

[0031] In this embodiment, since the gas detection model is a model pre-trained with a training data set, each feature extraction model can extract different gas feature maps based on the input data, and multiple feature extraction models can be used to obtain gas feature maps under scene changes from images of continuous frame changes, so as to be more adaptable to scenes under complex changes, such as sports scenes. Since the gas feature maps extracted by multiple feature extraction models can better represent the gas characteristics under scene changes, the gas position detection network can detect more accurate gas position data based on the gas feature maps corresponding to each feature extraction model.

[0032] In the above embodiment, continuous frame images are obtained and input into the input data of a pre-trained gas detection model. Since the gas is a changing target, the input data is formed based on the continuous frame images to better capture the characteristics of the gas. Since the gas detection model is a pre-trained model, the plurality of feature extraction models respectively obtain the gas feature maps corresponding to each feature extraction model based on the input data. In this way, each feature extraction model can extract a different gas feature map based on the input data. By using a plurality of feature extraction models, different gas feature maps under scene changes can be obtained from the images of continuous frame changes, thereby being more adaptable to scenes under complex changes. The gas position detection network can detect more accurate gas position data based on the gas characteristic map corresponding to each feature extraction model. Therefore, since the gas detection model is pre-trained and the parameters in the model have been solidified, continuous frame images can be directly processed, which can meet the real-time requirements. Different gas characteristic maps can be extracted based on multiple feature extraction models, which can improve the accuracy of gas detection in complex backgrounds and complex scenes. Using machine learning methods, the gas can be detected and located according to the different gas features extracted by multiple feature models. In addition, the strategy of fusing multiple feature models can be used to integrate the output results of multiple feature models to improve the accuracy and robustness of gas detection.

[0033] In some embodiments, the continuous frame images are continuous frame infrared images, and the input data for forming a pre-trained gas detection model based on the continuous frame images includes:

[0034] The continuous frame infrared images are merged to obtain a multi-channel merged image, and the input data is obtained based on the merged image.

[0035] In this embodiment, a merged image of multiple channels is obtained by merging continuous infrared images. The merged image can represent the motion state of the gas. Moreover, after the continuous infrared images are merged in advance and input into the gas detection model, the detection speed of the gas detection model can be improved. For example, eight consecutive single-channel images are fused into an eight-channel image, so that the channels in the eight-channel image represent the motion state. An eight-channel image can be an eight-frame video disguised as an image, which retains the motion characteristics of the gas and is subsequently used to detect the position of the eighth frame of gas.

[0036] With the support of image processing and deep learning technology, continuous frame images are fused into a multi-channel image to meet the motion characteristics and used as the input data of the gas detection model. The motion characteristics of the gas are extracted from the input data, and the adaptive learning and fusion of multiple feature extraction models are used to perform gas detection and positioning to suit different scenarios and complex backgrounds.

[0037] In the above embodiment, continuous frame infrared images are merged to obtain a multi-channel merged image. A multi-channel image can represent the movement state of the gas. Moreover, the continuous frame infrared images are merged in advance and then input into the gas detection model, which can improve the detection speed of the gas detection model.

[0038] In some embodiments, outputting the gas position data in the current frame of the continuous frame images through the gas detection model based on the input data includes:

[0039] Based on the input data, a gas characteristic graph corresponding to each feature extraction model is obtained through the plurality of feature extraction models, or a gas characteristic graph corresponding to each feature extraction model and a confidence level of the gas characteristic graph are obtained;

[0040] Based on the gas characteristic graphs corresponding to each feature extraction model, or based on the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs, the gas position data is output through the gas position detection network.

[0041] In this embodiment, after the gas detection model training is completed, the parameters in the feature extraction model have been solidified, so the gas characteristic graphs corresponding to each feature extraction model, or the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs can be output based on the input data, wherein the confidence of the gas characteristic graph represents the probability that a gas region exists in the characteristic graph. The gas characteristic graphs corresponding to one feature extraction model can be one or more, that is, for one feature extraction model, one or more gas characteristic graphs can be output. The network structure of the feature extraction model includes but is not limited to convolutional layers, downsampling layers, pooling layers, and activation layers.

[0042] In an optional manner, based on the gas characteristic graphs corresponding to each feature extraction model, the gas position data is output through the gas position detection network. In an optional manner, based on the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs, the gas position data is output through the gas position detection network. Using multiple feature extraction models can obtain different gas characteristic graphs under scene changes from images of continuous frame changes, so as to be more adaptable to scenes under complex changes. Based on the gas characteristic graphs corresponding to each feature extraction model, the gas position detection network can detect more accurate gas position data.

[0043] In the above embodiment, a variety of feature extraction models are used to obtain different gas characteristic maps under scene changes from images of continuous frame changes, so that it can better adapt to scenes under complex changes, and the confidence of the gas characteristic map can be obtained. Based on the gas characteristic map corresponding to each feature extraction model and the confidence of the gas characteristic map, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0044] Optionally, the outputting the gas position data through the gas position detection network based on the gas characteristic graphs corresponding to the respective feature extraction models, or based on the gas characteristic graphs corresponding to the respective feature extraction models and the confidence of the gas characteristic graphs, includes:

[0045] The gas characteristic graphs corresponding to the respective feature extraction models are connected through the connection network in the gas detection model to obtain a connection characteristic graph;

[0046] Based on the connection characteristic graph, the gas position data is output through the gas position detection network.

[0047] In this embodiment, the connection network is used to connect the gas feature graphs corresponding to the various feature extraction models to generate a connection feature graph with more feature information, which can be used to increase the number of channels or feature dimensions in the deep learning model so as to better capture the relationship between different gas features in subsequent layers. The connection network can be a convolutional network. In this way, the connection feature graph includes more gas feature data and can represent the relationship between different gas features. Therefore, based on the connection feature graph, the gas position detection network can detect more accurate gas position data.

[0048] Optionally, each feature extraction model is trained based on each scenario, and a model is overfitted for each scenario. In this way, during testing, the confidence level of the result output by the overfitted model is higher and the features are more obvious. On the contrary, the feature maps of other models are less obvious. Then the head parts of each feature extraction model are connected through a layer of convolution to output a result, namely, the gas connection feature map. For example, there are five feature extraction models in total, and the gas connection feature map is output by a large model composed of five small models, namely, each scenario activates a feature extraction model pathway to obtain a gas feature map corresponding to the feature extraction model, and then the selection weights in the convolution layer are used to connect and fuse the gas feature maps corresponding to each feature extraction model.

[0049] like Figure 3It is a schematic diagram of generating a connection feature graph in an embodiment. The gas detection model includes two feature extraction models, namely feature extraction models A1 and A2, wherein feature extraction model A1 outputs three feature graphs A12, A13, and A14, and feature extraction model A2 outputs three feature graphs A22, A23, and A24. Then A12, A13, A14, A22, A23, and A24 are used as inputs of a connection network to output a connection feature graph.

[0050] In the above embodiment, the gas characteristic graphs corresponding to the various feature extraction models are connected through the connection network in the gas detection model to obtain a connection characteristic graph. Such a connection characteristic graph includes more gas characteristic data and can represent the relationship between different gas characteristics. Based on the connection characteristic graph, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0051] Optionally, the outputting the gas position data through the gas position detection network based on the gas characteristic graphs corresponding to the respective feature extraction models, or based on the gas characteristic graphs corresponding to the respective feature extraction models and the confidence of the gas characteristic graphs, includes:

[0052] Based on the confidence of the gas characteristic graph, a target gas characteristic graph that meets the confidence condition is screened from the gas characteristic graphs corresponding to each feature extraction model;

[0053] Based on the target gas characteristic map, the gas position data is output through the gas position detection network.

[0054] In this embodiment, the target gas characteristic graph represents a gas characteristic graph with a confidence level higher than a preset confidence level, that is, the gas characteristic graph with a high confidence level is used for subsequent gas detection, and the gas characteristic graph with a low confidence level is discarded, thereby improving the feature expression of the gas characteristic graph with a high confidence level and improving the accuracy of gas area detection.

[0055] Optionally, the outputting the gas position data through the gas position detection network based on the target gas characteristic graph includes at least one of the following:

[0056] When there is only one target gas characteristic graph, the target gas characteristic graph is used as an input of the gas position detection network, and the gas position data is output through the gas position detection network;

[0057] When there are multiple target gas characteristic graphs, the multiple target gas characteristic graphs are connected through the connection network in the gas detection model to obtain a target connection characteristic graph, and based on the target connection characteristic graph, the gas position data is output through the gas position detection network.

[0058] like Figure 4 It is a schematic diagram of generating a connection feature map in another embodiment; the feature extraction model A1 outputs three feature maps A12, corresponding to a confidence level of 0.81; A13, corresponding to a confidence level of 0.5; A14, corresponding to a confidence level of 0.85; the feature extraction model A2 outputs three feature maps A22, corresponding to a confidence level of 0.45; A23, corresponding to a confidence level of 0.91; A24, corresponding to a confidence level of 0.95; the preset confidence level is 0.8, then according to the preset confidence level, the target gas feature maps are screened as A12, A14, A23 and A24, and after connecting A12, A14, A23 and A24, the target connection feature map is obtained.

[0059] In the above embodiment, the gas characteristic maps are screened by the confidence levels corresponding to the gas characteristic maps, and the gas characteristic maps with high confidence levels are used for subsequent gas position detection, thereby activating the feature expression of the gas characteristic maps with high confidence levels and suppressing the feature expression of the gas characteristic maps with low confidence levels, thereby highlighting the features of the gas area in the current scene. In this way, the gas position detection network can detect more accurate gas position data and improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0060] In some embodiments, the method further comprises:

[0061] Output the connection feature graph or the target connection feature graph.

[0062] In the above embodiment, by outputting a connection feature map or a target connection feature map, it is convenient for the user to continue further accurate detection in the gas location area, which can improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0063] In some embodiments, the feature extraction model includes a multi-scale feature network, and the gas feature graphs corresponding to each feature extraction model are obtained based on the input data through multiple feature extraction models, including:

[0064] Based on the input data, through the multi-scale feature network in each of the feature extraction models, gas feature maps of multiple different scales corresponding to each feature extraction model are output.

[0065] In this embodiment, a multi-scale feature network is used to extract multiple gas feature maps of different sizes based on input data. The multi-scale feature network includes but is not limited to the FPN (Feature Pyramid Network) network structure and the PAN (Pyramid Attention Network) network structure. FPN conveys strong semantic features from top to bottom, while PAN conveys strong positioning features from bottom to top. They work together to aggregate parameters of different detection layers from different backbone layers, so that the location information and category information of the gas target are retained to the greatest extent. The multi-scale network obtains multiple input feature maps of different scales through multiple different downsampling operations, and then the multiple input feature maps of different scales are respectively input into the network structure formed by FPN and PAN for processing to obtain multiple gas feature maps of different scales. For example, Figure 3 or Figure 4 As shown, the feature extraction model A1 outputs three gas feature maps respectively, and the corresponding scales of A12, A13, and A14 are different. For example, the scale of A12 is 20*20, the scale of A13 is 100*100, and the scale of A14 is 400*400. Then, A12 is used for gas detection in smaller gas areas, A13 is used for gas detection in medium-sized gas areas, and A14 is used for gas detection in large gas areas.

[0066] In the above embodiment, each feature extraction model corresponds to a plurality of gas feature maps of different scales through the multi-scale feature network output, which can better adapt to the detection of gas areas of different sizes in complex scenes and complex backgrounds. Based on gas feature maps of different scales, the gas position detection network can detect more accurate gas position data, which can improve the accuracy of gas detection in complex backgrounds and complex scenes.

[0067] In some embodiments, the method further includes: each of the feature extraction models corresponds to a respective sample data set, wherein different sample data sets are collected from different gas detection scenarios.

[0068] In this embodiment, common gas detection scenes can be classified to obtain multiple scenes, and then multiple sample images are collected in each scene, and the feature extraction model of each scene is pre-trained. For example, the gas detection scene can be divided into indoor scenes and outdoor scenes. The indoor scene can be further subdivided into office scenes, factory scenes, oil field indoor scenes, and the like. The outdoor scene can be further subdivided into oil field outdoor scenes, factory outdoor scenes, and the like, such as the oil field outdoor scene corresponding to feature extraction model A, and the factory outdoor scene corresponding to feature extraction model B. A large number of sample images are collected in each scene to train the feature extraction model corresponding to the scene, and the collected sample images may include rich backgrounds, including but not limited to the sky, black objects, grass, pipes, glass, blue sky, white clouds, sea, and the like. In this way, during the training process, the corresponding feature extraction model can learn the main features of the scene and the rich background features during the training process. After the training of each feature extraction model is completed and each feature extraction model is used in the gas detection model, the gas detection model can be applied to a variety of complex scenes and a variety of complex backgrounds. In the embodiment of the present application, based on artificial intelligence technology, sample data in various complex backgrounds and multiple scenes are obtained, so as to realize gas detection in complex backgrounds and multiple scenes, by collecting a large amount of video data and classifying the data according to the scene. Then, multiple feature extraction models are trained using different sample data sets, and these feature extraction models are fused to obtain a gas detection model.

[0069] In the above embodiments, each feature extraction model corresponds to its own sample data set, and different sample data sets are collected from different gas detection scenarios. In this way, the feature extraction model can learn the main features and rich background features of the corresponding scene from the corresponding sample data set during the training process. In this way, the gas detection model can detect more accurate gas location data and improve the accuracy of gas detection in complex backgrounds and complex scenarios.

[0070] In some embodiments, Figure 5 As shown, Figure 5 is a schematic diagram of training a gas detection model in an embodiment, wherein the method further comprises:

[0071] S51, obtaining a training data set, wherein each training image in the training data set includes a continuous frame training image and gas label data corresponding to the continuous frame training image.

[0072] In this embodiment, the training data set may include multiple sample data sets, and the training image is a continuous frame training image randomly extracted from the multiple sample data sets. Since the multiple sample data sets are obtained from multiple gas detection scenes, the training data set is also obtained based on multiple gas detection scenes. The gas area in the current frame of the continuous frame training image is used as the corresponding gas label data. For example, the continuous frame training image is the first frame, the second frame, and the third frame, and the third frame is the current frame, then the gas area in the third frame is used as the corresponding gas label data.

[0073] The training data set can come from a large amount of video data. A large amount of video data includes various scenes from the fields of industry, environment and safety monitoring. These video data should contain the movement and distribution of various gases. The collected data should be preprocessed, including frame sampling, resolution adjustment and labeling, etc., to facilitate subsequent model training and evaluation.

[0074] S52: construct an initial gas detection model, wherein the initial gas detection model includes a plurality of trained feature extraction models and an initial gas position detection network.

[0075] In this embodiment, in order to speed up the training of the gas detection model, each feature extraction model can be pre-trained, and then each feature extraction model can be trained separately based on its corresponding sample data set. During the training process, it is necessary to set appropriate hyperparameters, learning rate and batch size, and perform data enhancement and regularization operations to improve the generalization ability and robustness of the model.

[0076] Optionally, before obtaining the pre-trained gas detection model, the method further includes:

[0077] Based on each of the sample data sets, training is performed to obtain each trained feature extraction model;

[0078] Wherein, based on each of the sample data sets, training each trained feature extraction model comprises:

[0079] Acquire the sample data set, wherein the sample data set includes each sample image including a continuous frame sample image, and a gas characteristic map label corresponding to the continuous frame sample image;

[0080] Build an initial feature extraction model;

[0081] Based on the sample data set, the initial feature extraction model corresponding to the sample data set is iteratively trained until a training termination condition is met, thereby obtaining a trained feature extraction model corresponding to the sample data set.

[0082] In this embodiment, first, a large amount of video data is collected as a sample data set, and classified according to different scenes. Then, for each scene, one or more feature extraction models are trained using the corresponding sample data set. These trained feature extraction models are fused to combine the advantages of different models and improve the accuracy and robustness of gas detection. The continuous frame sample images include but are not limited to continuous frame sample infrared images, continuous frame sample visible light images, and continuous frame sample fusion images obtained by fusing continuous frame sample infrared images with continuous frame sample visible light images. Train each feature extraction model according to the above steps. In the process of training a feature extraction model, a sample image is obtained from a sample data set, sample image input data is formed based on the sample image, and a gas feature map corresponding to the sample image input data is output through the feature extraction model in the current iteration process. Based on the feature loss function, the gas feature map corresponding to the sample image input data, and the gas feature map label corresponding to the sample image input data, the loss value in the current iteration is output. Based on the loss value in the current iteration, it is judged whether the current iteration meets the training termination condition. If the current iteration meets the training termination condition, the feature extraction model after the iteration termination is used as the trained feature extraction model; if the current iteration does not meet the training termination condition, the sample image is continued to be obtained from the sample data set for training. The training termination conditions include but are not limited to the number of iterations, the loss value is less than the preset loss value, etc. The feature loss function includes but is not limited to the cross entropy function, the mean square error function, etc.

[0083] S53. Iteratively train the initial gas detection model using the training data set until the loss function converges, thereby obtaining the pre-trained gas detection model.

[0084] Optionally, iteratively training the initial gas detection model using the training data set until the loss function converges to obtain the pre-trained gas detection model includes:

[0085] Acquire a training image from the training data set, form training image input data of the gas detection model of the current iteration based on the training image, output a gas sample feature map corresponding to each feature extraction model through multiple feature extraction models in the current iteration based on the training image input data, and output gas sample position data corresponding to the training image through the gas position detection network in the current iteration based on the gas sample feature map corresponding to each feature extraction model;

[0086] Based on the loss function, the loss value between the gas sample position data corresponding to the training image and the gas label data corresponding to the training image is calculated to obtain the loss value of the current iteration;

[0087] Based on the loss value of the current iteration, determine whether the current iteration meets the iteration termination condition. If the current iteration does not meet the iteration termination condition, continue to obtain training images from the training data set and iteratively train the gas detection model; if the current iteration meets the iteration termination condition, the gas detection model after stopping the iteration will be used as the pre-trained gas detection model.

[0088] In this embodiment, continuous frame training images are merged to obtain a multi-channel training image input data. For example, 8 consecutive frames of training images are arranged by channel and merged into a 512*640*8 image. The label of this training image is the gas position label of the last frame. The training termination conditions include but are not limited to the number of iterations, the loss value is less than the preset loss value, etc. The loss function includes but is not limited to the cross entropy function, the mean square error function, etc. After the feature extraction model is trained based on the sample data set, the initial parameters in the feature extraction model can be obtained. When the gas detection model is trained as a whole based on the training data set, the initial parameters in the feature extraction model will be fine-tuned and optimized so that the optimized parameters can better enable the gas position detection network to detect more accurate gas areas. Moreover, since the training data set is formed by multiple sample data sets, and the multiple sample data sets are from multiple gas detection scenes, during the training process, the gas detection model can learn the gas features and rich background features in multiple scenes and backgrounds. After the training is completed, it can better adapt to gas detection in complex scenes and complex backgrounds. Therefore, when the trained gas detection model is applied to motion scenes, such as handheld detection equipment and non-handheld autonomously movable detection equipment, the scene is constantly changing during the motion, but the gas detection model is trained based on the training data sets collected in various gas detection scenarios during the training process. Therefore, the gas detection model can better adapt to this motion scene.

[0089] Optionally, after training, the gas detection model can be optimized, including model compression, pruning, and quantization techniques to reduce the model's computational and storage resources. In addition, it is possible to consider fusing multiple feature extraction models, and through strategies such as voting, weighted averaging, or model integration, the output results of multiple models can be integrated to improve the accuracy and robustness of gas detection.

[0090] Optionally, the training of the gas detection model can be performed on a computing device that has a high-performance computing device, such as a GPU or a cloud computing platform, to perform complex image processing and deep learning calculations. In addition, in order to improve the efficiency and accuracy of the algorithm, parameter tuning and optimization are required. For example, cross-validation is used to determine the hyperparameters of the feature extraction model, such as the convolution kernel size and the number of layers. At the same time, data enhancement techniques such as translation, rotation, and scaling are applied to expand the training data set and improve the robustness and generalization ability of the model.

[0091] In the above embodiment, the feature extraction models corresponding to each other are first trained based on each sample data set to obtain the initial parameters of the feature extraction model, and then the gas detection model is trained based on the training data set, so as to accelerate the convergence speed of the gas detection model. Since the training data set is formed by multiple sample data sets, and the multiple sample data sets are from multiple gas detection scenes, the gas detection model can learn the gas features and rich background features under multiple scenes and backgrounds, and can improve the accuracy of gas detection under complex backgrounds and complex scenes. It is comprehensively applied to different scenes and complex backgrounds, taking into account the influence of different scenes and complex backgrounds, and is classified in the data collection stage, and the corresponding model is trained for each scene. In this way, it can adapt to the gas detection needs of different scenes and improve the adaptability to complex backgrounds. Multi-model fusion strategy: This method trains multiple models using different data sets, fuses these models, and integrates the output results of multiple models to improve the accuracy and robustness of gas detection. By adopting the strategy of multi-model fusion, the advantages of each model can be fully utilized to improve the overall detection performance.

[0092] In some embodiments, the continuous frame images include any one of the following: continuous frame infrared images, continuous frame visible light images, and continuous frame fused images obtained by fusing continuous frame infrared images with continuous frame visible light images.

[0093] In this embodiment, when the continuous frame images are continuous frame infrared images, collecting infrared images can reduce the impact of environmental factors, such as the impact of bad weather such as rainy days and foggy days, and the infrared images can also capture colorless gases and colored gases, which can improve the accuracy of subsequent gas detection. When the continuous frame images are continuous frame fusion images, infrared features and visible light features can be captured at the same time, which can provide more features for subsequent gas detection.

[0094] On the other hand, the present application provides a computer program product, including a computer program, which implements the gas detection method described in any embodiment of the present application when executed by a processor.

[0095] Among them, in the computer program product, an optional implementation form of the program module architecture of the computer program that implements each step of the gas detection method can be a gas detection device.

[0096] See also Figure 6An embodiment of the present application provides a gas detection device, comprising: an acquisition module 61, used to acquire continuous frame images; a determination module 62, used to form input data of a pre-trained gas detection model based on the continuous frame images; a detection module 63, used to output gas position data in the current frame of the continuous frame images through the gas detection model based on the input data; wherein the gas detection model includes multiple feature extraction models and a gas position detection network, the multiple feature extraction models are used to obtain gas feature maps corresponding to each feature extraction model based on the input data, and the gas position detection network is used to output the gas position data based on the gas feature maps corresponding to each feature extraction model.

[0097] Optionally, the detection module 63 is further used for:

[0098] The continuous frame infrared images are merged to obtain a multi-channel merged image, and the input data is obtained based on the merged image.

[0099] Optionally, the detection module 63 is further used for:

[0100] Based on the input data, a gas characteristic graph corresponding to each feature extraction model is obtained through the plurality of feature extraction models, or a gas characteristic graph corresponding to each feature extraction model and a confidence level of the gas characteristic graph are obtained;

[0101] Based on the gas characteristic graphs corresponding to each feature extraction model, or based on the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs, the gas position data is output through the gas position detection network.

[0102] Optionally, the detection module 63 is further used for:

[0103] The gas characteristic graphs corresponding to the respective feature extraction models are connected through the connection network in the gas detection model to obtain a connection characteristic graph;

[0104] Based on the connection characteristic graph, the gas position data is output through the gas position detection network.

[0105] Optionally, the detection module 63 is further used for:

[0106] Based on the confidence of the gas characteristic graph, a target gas characteristic graph that meets the confidence condition is screened from the gas characteristic graphs corresponding to each feature extraction model;

[0107] Based on the target gas characteristic map, the gas position data is output through the gas position detection network.

[0108] Optionally, the detection module 63 is further used for:

[0109] When there is only one target gas characteristic graph, the target gas characteristic graph is used as an input of the gas position detection network, and the gas position data is output through the gas position detection network;

[0110] When there are multiple target gas characteristic graphs, the multiple target gas characteristic graphs are connected through the connection network in the gas detection model to obtain a target connection characteristic graph, and based on the target connection characteristic graph, the gas position data is output through the gas position detection network.

[0111] Optionally, the detection module 63 is further used for:

[0112] Output the connection feature graph or the target connection feature graph.

[0113] Optionally, the detection module 63 is further used for:

[0114] Based on the input data, through the multi-scale feature network in each of the feature extraction models, gas feature maps of multiple different scales corresponding to each feature extraction model are output.

[0115] Optionally, each of the feature extraction models corresponds to a respective sample data set, wherein different sample data sets are collected from different gas detection scenarios.

[0116] Optionally, the gas training detection device further includes a training module 64, which is used to:

[0117] Acquire a training data set, wherein each training image in the training data set includes a continuous frame training image and gas label data corresponding to the continuous frame training image;

[0118] Constructing an initial gas detection model, wherein the initial gas detection model includes a plurality of trained feature extraction models and an initial gas position detection network;

[0119] The initial gas detection model is iteratively trained using the training data set until the loss function converges, thereby obtaining the pre-trained gas detection model.

[0120] Optionally, the training module 64 is further used to:

[0121] Acquire a training image from the training data set, form training image input data of the gas detection model of the current iteration based on the training image, output a gas sample feature map corresponding to each feature extraction model through multiple feature extraction models in the current iteration based on the training image input data, and output gas sample position data corresponding to the training image through the gas position detection network in the current iteration based on the gas sample feature map corresponding to each feature extraction model;

[0122] Based on the loss function, the loss value between the gas sample position data corresponding to the training image and the gas label data corresponding to the training image is calculated to obtain the loss value of the current iteration;

[0123] Based on the loss value of the current iteration, determine whether the current iteration meets the iteration termination condition. If the current iteration does not meet the iteration termination condition, continue to obtain training images from the training data set and iteratively train the gas detection model; if the current iteration meets the iteration termination condition, the gas detection model after stopping the iteration will be used as the pre-trained gas detection model.

[0124] Optionally, the training module 64 is further used to:

[0125] Based on each of the sample data sets, training is performed to obtain each trained feature extraction model;

[0126] Wherein, based on each of the sample data sets, training each trained feature extraction model comprises:

[0127] Acquire the sample data set, wherein the sample data set includes each sample image including a continuous frame sample image, and a gas characteristic map label corresponding to the continuous frame sample image;

[0128] Build an initial feature extraction model;

[0129] Based on the sample data set, the initial feature extraction model corresponding to the sample data set is iteratively trained until a training termination condition is met, thereby obtaining a trained feature extraction model corresponding to the sample data set.

[0130] Optionally, the continuous frame images include any one of the following: continuous frame infrared images, continuous frame visible light images, and continuous frame fused images obtained by fusing continuous frame infrared images with continuous frame visible light images.

[0131] It can be understood by those skilled in the art that Figure 6 The structure of the gas detection device does not constitute a limitation on the gas detection device, and each module can be implemented in whole or in part by software, hardware, or a combination thereof. The above modules can be embedded in or independent of the processor in the gas detection device in the form of hardware, or can be stored in the memory in the gas detection device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. In other embodiments, the gas detection device may include more or fewer modules than those shown in the figure.

[0132] See also Figure 7On the other hand, a gas detection device 10 is provided, including a processor 13 and a memory 14. The memory 14 stores a computer program. When the computer program is executed by the processor, the processor 13 executes the steps of the gas detection method provided in any of the above embodiments of the present application.

[0133] The processor 13 is the control center, which uses various interfaces and lines to connect various parts of the entire gas detection device, and executes various functions of the gas detection device and processes data by running or executing software programs and / or modules stored in the memory 14, and calling data stored in the memory 14. Optionally, the processor 13 may include one or more processing cores; preferably, the processor 13 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 13.

[0134] The memory 14 can be used to store software programs and modules. The processor 13 executes various functional applications and data processing by running the software programs and modules stored in the memory 14. The memory 14 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the gas detection device, etc. In addition, the memory 14 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 14 may also include a memory processor to provide the processor 13 with access to the memory 14.

[0135] On the other hand, an embodiment of the present application further provides a storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the gas detection method provided in any of the above embodiments of the present application.

[0136] Those skilled in the art can understand that all or part of the processes in the methods provided in the above embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0137] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A gas detection method, characterized in that: include: Acquire continuous frame images; Based on the continuous frame images, forming input data of a pre-trained gas detection model; Based on the input data, the gas position data in the current frame of the continuous frame image is output through the gas detection model; wherein the gas detection model includes multiple feature extraction models and a gas position detection network, the multiple feature extraction models are used to obtain the gas characteristic graphs corresponding to each feature extraction model based on the input data, and the gas position detection network is used to output the gas position data based on the gas characteristic graphs corresponding to each feature extraction model.

2. The gas detection method according to claim 1, characterized in that: The continuous frame images are continuous frame infrared images, and the input data for forming a pre-trained gas detection model based on the continuous frame images includes: The continuous frame infrared images are merged to obtain a multi-channel merged image, and the input data is obtained based on the merged image.

3. The gas detection method according to claim 1, characterized in that: The step of outputting the gas position data in the current frame of the continuous frame images through the gas detection model based on the input data comprises: Based on the input data, a gas characteristic graph corresponding to each feature extraction model is obtained through the plurality of feature extraction models, or a gas characteristic graph corresponding to each feature extraction model and a confidence level of the gas characteristic graph are obtained; Based on the gas characteristic graphs corresponding to each feature extraction model, or based on the gas characteristic graphs corresponding to each feature extraction model and the confidence of the gas characteristic graphs, the gas position data is output through the gas position detection network.

4. The gas detection method according to claim 3, characterized in that: The method outputting the gas position data through the gas position detection network based on the gas characteristic graphs corresponding to the respective feature extraction models, or based on the gas characteristic graphs corresponding to the respective feature extraction models and the confidence of the gas characteristic graphs, comprises: The gas characteristic graphs corresponding to the respective feature extraction models are connected through the connection network in the gas detection model to obtain a connection characteristic graph; Based on the connection characteristic graph, the gas position data is output through the gas position detection network.

5. The gas detection method according to claim 3, characterized in that: The method outputting the gas position data through the gas position detection network based on the gas characteristic graphs corresponding to the respective feature extraction models, or based on the gas characteristic graphs corresponding to the respective feature extraction models and the confidence of the gas characteristic graphs, comprises: Based on the confidence of the gas characteristic graph, a target gas characteristic graph that meets the confidence condition is screened from the gas characteristic graphs corresponding to each feature extraction model; Based on the target gas characteristic map, the gas position data is output through the gas position detection network.

6. The gas detection method according to claim 5, characterized in that: The outputting of the gas position data through the gas position detection network based on the target gas characteristic diagram includes at least one of the following: When there is only one target gas characteristic graph, the target gas characteristic graph is used as an input of the gas position detection network, and the gas position data is output through the gas position detection network; When there are multiple target gas characteristic graphs, the multiple target gas characteristic graphs are connected through the connection network in the gas detection model to obtain a target connection characteristic graph, and based on the target connection characteristic graph, the gas position data is output through the gas position detection network.

7. The gas detection method according to claim 4 or 6, characterized in that: The method further comprises: Output the connection feature graph or the target connection feature graph.

8. The gas detection method according to claim 3, characterized in that: The feature extraction model includes a multi-scale feature network. Based on the input data, the gas feature graphs corresponding to the respective feature extraction models are obtained through the plurality of feature extraction models, including: Based on the input data, through the multi-scale feature network in each of the feature extraction models, gas feature maps of multiple different scales corresponding to each feature extraction model are output.

9. The gas detection method according to claim 1, characterized in that: Each of the feature extraction models corresponds to a respective sample data set, wherein different sample data sets are collected from different gas detection scenarios.

10. The gas detection method according to any one of claims 1 to 9, characterized in that: The method further comprises: Acquire a training data set, wherein each training image in the training data set includes a continuous frame training image and gas label data corresponding to the continuous frame training image; Constructing an initial gas detection model, wherein the initial gas detection model includes a plurality of trained feature extraction models and an initial gas position detection network; The initial gas detection model is iteratively trained using the training data set until the loss function converges, thereby obtaining the pre-trained gas detection model.

11. The gas detection method according to claim 10, characterized in that: The iterative training of the initial gas detection model by the training data set until the loss function converges to obtain the pre-trained gas detection model includes: Acquire a training image from the training data set, form training image input data of the gas detection model of the current iteration based on the training image, output a gas sample feature map corresponding to each feature extraction model through multiple feature extraction models in the current iteration based on the training image input data, and output gas sample position data corresponding to the training image through the gas position detection network in the current iteration based on the gas sample feature map corresponding to each feature extraction model; Based on the loss function, the loss value between the gas sample position data corresponding to the training image and the gas label data corresponding to the training image is calculated to obtain the loss value of the current iteration; Based on the loss value of the current iteration, determine whether the current iteration meets the iteration termination condition. If the current iteration does not meet the iteration termination condition, continue to obtain training images from the training data set and iteratively train the gas detection model; if the current iteration meets the iteration termination condition, the gas detection model after stopping the iteration will be used as the pre-trained gas detection model.

12. The gas detection method according to claim 10, characterized in that: Before obtaining the pre-trained gas detection model, the method further includes: Based on each of the sample data sets, training is performed to obtain each trained feature extraction model; Wherein, based on each of the sample data sets, training each trained feature extraction model comprises: Acquire the sample data set, wherein the sample data set includes each sample image including a continuous frame sample image, and a gas characteristic map label corresponding to the continuous frame sample image; Build an initial feature extraction model; Based on the sample data set, the initial feature extraction model corresponding to the sample data set is iteratively trained until a training termination condition is met, thereby obtaining a trained feature extraction model corresponding to the sample data set.

13. The gas detection method according to claim 1, characterized in that: The continuous frame images include any one of the following: continuous frame infrared images, continuous frame visible light images, and continuous frame fused images obtained by fusing continuous frame infrared images with continuous frame visible light images.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the gas detection method according to any one of claims 1 to 13 is implemented.

15. A gas detection device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the gas detection method according to any one of claims 1 to 14.

16. A storage medium comprising a computer program, characterized in that A computer program is stored, and when the computer program is executed by a processor, the processor executes the gas detection method according to any one of claims 1 to 14.

Images