Convolutional neural network-based gas leakage detection method and system

Abstract

The invention relates to a gas leakage detection method and system based on a convolutional neural network, and relates to the technical field of the Internet of Things. Environmental change data is collected through an environmental sensor, infrared image data is collected through an infrared imaging sensor, and based on the environmental change data, a region segmentation strategy used for influencing the infrared image data is determined; and through a pre-trained convolutional neural network, based on reference information corresponding to the infrared image data, carrying out region segmentation and identification to obtain processed infrared image data, the processed infrared image data being used for determining whether there is leakage of the target gas in the to-be-detected region. The data of the environment sensor and the infrared imaging sensor are combined for application, and the infrared image data are processed based on the environment data, so that the processing mode is matched with an actual environment scene, the processing precision of the infrared image data is improved, the precision of a gas leakage detection result is also improved, and the potential safety hazard of gas leakage is further reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of Internet of Things, and in particular, to a gas leakage detection method and system based on convolutional neural network. Background Art

[0002] With the development of Internet of Things technology, it has been applied to various detection scenarios. In these detection scenarios, data acquisition devices such as sensors can be configured to collect relevant data, which are then analyzed to obtain detection results.

[0003] In the related technologies, the detection of gas leakage is usually based on a single sensing technology and integrated with the Internet of Things technology to achieve detection. This detection technology, due to the single data, may not be able to achieve accurate detection of gas leakage, and thus cannot effectively reduce the safety hazards of gas leakage. Summary of the invention

[0004] The purpose of the present disclosure is to provide a gas leakage detection method and system based on convolutional neural network, which can improve the accuracy of gas leakage detection and thus reduce the safety hazards of gas leakage.

[0005] In order to achieve the above-mentioned objectives, in a first aspect, the present disclosure provides a gas leakage detection method based on a convolutional neural network, comprising: obtaining environmental change data collected by an environmental sensor of an area to be detected and infrared image data collected by an infrared imaging sensor; determining reference information corresponding to the infrared image data based on the environmental change data, wherein the reference information is used to influence a regional segmentation strategy of the infrared image data; performing regional segmentation and identification on the infrared image data based on the infrared image data and the reference information corresponding to the infrared image data through a pre-trained convolutional neural network to obtain processed infrared image data, wherein the processed infrared image data includes a target image with an image area corresponding to a target gas identified; determining a gas leakage detection result based on the processed infrared image data, wherein the gas leakage detection result is used to characterize the leakage of the target gas in the area to be detected.

[0006] Optionally, the environmental change data includes environmental parameters corresponding to multiple moments, and determining the reference information corresponding to the infrared image data based on the environmental change data includes: determining original environmental parameter characteristics based on the environmental parameters corresponding to the multiple moments; adjusting the original environmental parameter characteristics based on the type and amount of the environmental change data to obtain target environmental parameter characteristics, wherein the target environmental parameter characteristics are related to the gas circulation index of the area to be detected; determining the reference information corresponding to the infrared image data based on the target environmental parameter characteristics through a pre-trained evaluation model, wherein the reference information includes the gas circulation index of the area to be detected.

[0007] Optionally, the original environmental parameter characteristics are adjusted according to the type and data volume of the environmental change data to obtain target environmental parameter characteristics, including: determining the degree of influence of the environmental change data on the gas circulation index of the area to be detected according to the type of the environmental change data; determining the confidence level of the environmental change data according to the data volume of the environmental change data; if the degree of influence is higher than a preset degree of influence and / or the confidence level is lower than a preset confidence level, then, the original environmental parameter characteristics are expanded, and the expanded original environmental parameter characteristics are determined as the target environmental parameter characteristics.

[0008] Optionally, the pre-trained convolutional neural network includes a pre-trained region segmentation network and a pre-trained region identification network, the infrared image data includes multiple infrared images, and the pre-trained convolutional neural network is used to perform region segmentation and identification on the infrared image data according to the infrared image data and reference information corresponding to the infrared image data to obtain processed infrared image data, including: performing region segmentation on the multiple infrared images according to the reference information corresponding to the infrared image data through the pre-trained region segmentation network to obtain multiple segmented infrared images; and identifying the image areas corresponding to the target gas in the multiple segmented infrared images through the pre-trained region identification network to obtain multiple identified infrared images, and determining the multiple identified infrared images as the processed infrared image data.

[0009] Optionally, the gas leakage detection method also includes: acquiring a training data set, the training data set includes multiple training samples, each training sample includes: a sample image, sample reference information, a first label image and a second label image, the first label image is an image obtained by performing region segmentation on the sample image based on the sample reference information, and the second label image is an image obtained by marking the region where the target gas is located on the sample image; training a region segmentation network to be trained according to the sample images, the sample reference information and the first label image in the multiple training samples to obtain the pre-trained region segmentation network; training a region identification network to be trained according to at least the sample images and the second label image in the multiple training samples to obtain the pre-trained region identification network; determining the pre-trained convolutional neural network according to the pre-trained region segmentation network and the pre-trained region identification network.

[0010] Optionally, the training of the region identification network to be trained is performed at least according to the sample images and the second label images in the multiple training samples to obtain the pre-trained region identification network, including: performing initial training on the region identification network to be trained according to at least part of the sample images and the second label images in the multiple training samples to obtain the initially trained region identification network; testing the initially trained region identification network according to at least part of the first label images in the multiple training samples to obtain a network test result, wherein the number of samples used for the test is less than the number of samples used for the initial training; and optimizing the initially trained region identification network according to the network test result to obtain the pre-trained region identification network.

[0011] Optionally, the processed infrared image data includes multiple identified infrared images, and determining the gas leakage detection result based on the processed infrared image data includes: determining the total number of the multiple identified infrared images; determining the number of target images; determining area information of the image area corresponding to the target gas identified in the target image, the area information being used to characterize the distribution of the image area corresponding to the target gas in the target image; determining the leakage probability of the target gas in the area to be detected based on the total number, the number of target images and the area information; and determining the gas leakage detection result based on the leakage probability of the target gas.

[0012] Optionally, the gas leakage detection result includes the leakage probability of the target gas in the area to be detected, the reference information includes the gas circulation index of the area to be detected, and the gas leakage method further includes: determining whether it is necessary to perform gas leakage protection on the area to be detected based on the leakage probability of the target gas; when it is determined that the area to be detected needs to be protected from gas leakage, generating a gas leakage protection strategy based on the gas circulation index, and executing the gas leakage protection strategy; when it is determined that it is not necessary to perform gas leakage protection on the area to be detected, adding a preset safety mark to the leakage probability of the target gas and the gas circulation index, and storing them.

[0013] Optionally, the gas leakage method also includes: when the gas leakage protection strategy includes outputting warning information indicating gas leakage in the area to be detected, after a preset time period, obtaining new environmental change data collected by the environmental sensor and new infrared image data collected by the infrared imaging sensor, and the preset time period is determined according to the area information of the area to be detected; generating an updated training data set according to the new environmental change data and the new infrared image data; and updating the pre-trained convolutional neural network according to the updated training data set to obtain an updated trained convolutional neural network.

[0014] In a second aspect, the present disclosure provides a gas leakage detection system based on a convolutional neural network, comprising: an environmental sensor arranged in an area to be detected, for collecting environmental change data of the area to be detected; an infrared imaging sensor arranged in the area to be detected, for collecting infrared image data of the area to be detected; and a detection device, which is respectively connected to the environmental sensor and the infrared imaging sensor, and is used to execute the gas leakage detection method based on a convolutional neural network described in the first aspect of the present disclosure.

[0015] Through the above technical solution, environmental change data is collected through environmental sensors, infrared image data is collected through infrared imaging sensors, and based on the environmental change data, a regional segmentation strategy for influencing infrared image data is determined, and then, through a pre-trained convolutional neural network, based on the reference information corresponding to the infrared image data, regional segmentation and identification are performed to obtain processed infrared image data, and the processed infrared image data can be used to determine whether there is a target gas leak in the area to be detected. This technical solution combines the data of environmental sensors and infrared imaging sensors, processes the infrared image data based on environmental data, so that the processing method matches the actual environmental scene, improves the processing accuracy of infrared image data, and then, the accuracy of the final gas leak detection result is also improved. Therefore, this technical solution can improve the accuracy of gas leak detection, thereby reducing the safety hazards of gas leaks.

[0016] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0018] Figure 1 It is a structural block diagram of a gas leakage detection system based on a convolutional neural network according to an exemplary embodiment.

[0019] Figure 2 It is a flow chart of a gas leakage detection method based on a convolutional neural network according to an exemplary embodiment.

[0020] Figure 3 It is a schematic diagram of the structure of a pre-trained convolutional neural network according to an exemplary embodiment.

[0021] Figure 4 The figure is a schematic diagram of infrared image processing according to an exemplary embodiment.

[0022] Figure 5 It is a structural block diagram of a gas leakage detection device based on a convolutional neural network according to an exemplary embodiment.

[0023] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0024] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0025] With the development of Internet of Things technology, it has been applied to various detection scenarios. In these detection scenarios, data acquisition devices such as sensors can be configured to collect relevant data, which are then analyzed to obtain detection results.

[0026] In the related technologies, the detection of gas leakage is usually based on a single sensing technology and integrated with the Internet of Things technology to achieve detection. This detection technology, due to the single data, may not be able to achieve accurate detection of gas leakage, and thus cannot effectively reduce the safety hazards of gas leakage.

[0027] Based on this, the embodiment of the present disclosure provides a technical solution, which collects environmental change data through environmental sensors, collects infrared image data through infrared imaging sensors, and determines the regional segmentation strategy that affects the infrared image data based on the environmental change data. Then, through a pre-trained convolutional neural network, based on the reference information corresponding to the infrared image data, regional segmentation and identification are performed to obtain processed infrared image data. The processed infrared image data can be used to determine whether there is a leakage of target gas in the area to be detected.

[0028] This technical solution combines the data from environmental sensors and infrared imaging sensors, processes the infrared image data based on the environmental data, matches the processing method with the actual environmental scene, improves the processing accuracy of the infrared image data, and thus, improves the accuracy of the final gas leak detection results.

[0029] Therefore, this technical solution can improve the accuracy of gas leakage detection and thus reduce the safety hazards of gas leakage.

[0030] Regarding gas leak detection, for example: natural gas leak detection, polluted gas leak detection and harmful gas leak detection, etc.

[0031] In these scenarios, detection systems based on IoT architecture are usually deployed to achieve effective detection of gas leaks.

[0032] Figure 1 is a structural block diagram of a gas leakage detection system based on a convolutional neural network according to an exemplary embodiment. Figure 1 As shown, the system includes: an environmental sensor, an infrared imaging sensor and a detection device.

[0033] The detection device is connected to an environmental sensor and an infrared imaging sensor respectively. The environmental sensor and the infrared imaging sensor are arranged in the area to be detected.

[0034] The area to be inspected may be, for example, a kitchen area, a workshop area of ​​a factory, etc.

[0035] In some embodiments, the environmental sensor may involve various types of sensors for detecting environmental parameters, such as temperature, humidity, and wind level, etc. Accordingly, the environmental sensor may include but is not limited to a temperature sensor, a humidity sensor, and a wind sensor.

[0036] In some embodiments, according to the cost requirements of different scenarios, one or more types of environmental sensors may be configured in a detection area, which is not limited here.

[0037] In some embodiments, the infrared imaging sensor may be: an infrared imaging device such as an infrared imager, which can capture infrared images. Regarding the principle of infrared imaging, infrared imaging is a technology that uses infrared radiation for imaging. Infrared radiation is the part of the electromagnetic spectrum with a wavelength longer than visible light, usually referring to radiation with a wavelength between 0.75 and 1000 microns. Infrared imaging technology can detect thermal radiation emitted by objects and convert these radiations into images. Infrared imaging technology is a mature imaging technology. For details, please refer to the mature technology in this field, which will not be introduced in detail here.

[0038] In some embodiments, according to the cost requirements of different scenarios, one or more infrared imaging sensors may be configured in a region to be detected, and infrared imaging sensors with different precisions may be configured.

[0039] In some embodiments, the detection device may be any electronic device having a data processing function, a visual display function, etc. For example, a host computer, etc.

[0040] In some embodiments, the environmental sensor can continuously collect environmental parameters to obtain environmental change data. The infrared imaging sensor can continuously collect infrared images to obtain infrared image data.

[0041] Therefore, the environmental change data may include environmental parameters corresponding to multiple moments, and the infrared image data may include multiple infrared images. These multiple infrared images may correspond to different moments or different acquisition viewing angles, which are not limited here.

[0042] Figure 2 is a flow chart of a gas leakage detection method based on a convolutional neural network according to an exemplary embodiment. The method can be applied to Figure 1 Testing equipment such as Figure 2 As shown, the method comprises the following steps:

[0043] Step S21, acquiring environmental change data collected by the environmental sensor of the area to be detected and infrared image data collected by the infrared imaging sensor.

[0044] Step S22: determining reference information corresponding to the infrared image data according to the environmental change data, wherein the reference information is used to influence a regional segmentation strategy of the infrared image data.

[0045] Step S23, using a pre-trained convolutional neural network, the infrared image data is segmented and marked according to the infrared image data and reference information corresponding to the infrared image data to obtain processed infrared image data, wherein the processed infrared image data includes a target image with an image area corresponding to the target gas marked therein.

[0046] Step S24, determining a gas leakage detection result according to the processed infrared image data, where the gas leakage detection result is used to characterize the leakage of the target gas in the area to be detected.

[0047] In step S21, the environmental change data and the infrared image data may be relevant data collected in the current detection cycle. Both the environmental sensor and the infrared imaging sensor may collect corresponding data according to the collection cycle.

[0048] Furthermore, the environmental sensor and the infrared imaging sensor can upload the collected data to the detection device so that the detection device can perform detection.

[0049] In step S22, reference information for a regional segmentation strategy affecting infrared image data can be determined based on the environmental change data. It is understandable that the gas conditions reflected by infrared images are different in different environments. In the related art, image processing is usually performed only based on infrared images to determine whether gas is leaking. This method cannot adapt to different environments, resulting in poor accuracy of the final detection result.

[0050] Therefore, the regional segmentation of infrared images can be assisted based on the environmental change data.

[0051] In some embodiments, the environmental change data includes environmental parameters corresponding to multiple moments, and step S22 may include: determining original environmental parameter characteristics based on the environmental parameters corresponding to multiple moments; adjusting the original environmental parameter characteristics based on the type and amount of environmental change data to obtain target environmental parameter characteristics, and the target environmental parameter characteristics are related to the gas circulation index of the area to be detected; determining reference information corresponding to the infrared image data based on the target environmental parameter characteristics through a pre-trained evaluation model, and the reference information includes the gas circulation index of the area to be detected.

[0052] In this implementation, the original environmental parameter features can be extracted based on the environmental parameters corresponding to multiple moments. The original environmental parameter features can include: environmental parameter variation rules, environmental parameter mean values, environmental parameter standard deviations, and other data features determined based on environmental parameter statistical analysis.

[0053] In some embodiments, if different types of environmental parameters are involved, the different types of environmental parameters can be processed in corresponding ways respectively, and finally input into the evaluation model for evaluation.

[0054] In some embodiments, the target environment parameter feature may be a feature obtained by adjusting the original environment parameter feature.

[0055] As an optional implementation, the original environmental parameter characteristics are adjusted according to the type and amount of environmental change data to obtain target environmental parameter characteristics, including: determining the degree of influence of the environmental change data on the gas circulation index of the detection area according to the type of environmental change data; determining the confidence level of the environmental change data according to the amount of environmental change data; if the degree of influence is higher than the preset degree of influence and / or the confidence level is lower than the preset confidence level, then, the original environmental parameter characteristics are expanded, and the expanded original environmental parameter characteristics are determined as the target environmental parameter characteristics.

[0056] In some embodiments, the influence levels corresponding to different environmental parameter types may be predefined, and the influence level of the environmental change data on the gas circulation index of the area to be detected may be determined according to the environmental parameter types involved in the current environmental change data.

[0057] For example, temperature has a smaller impact on the gas circulation index, humidity has a larger impact on the gas circulation index, and wind force level has a larger impact on the gas circulation index.

[0058] The gas circulation index can characterize the flow rate of gas in the area to be tested. In the case of good gas circulation, even if there is a gas leak, the test results will be affected.

[0059] In some embodiments, the data volume of the environment change data may be the number of frames of the environment parameter involved in the environment change data. The more frames there are, the higher the confidence of the environment change data.

[0060] Therefore, the confidence levels corresponding to different frame numbers may be preconfigured, and the current confidence level may be determined according to the current frame number.

[0061] In some embodiments, the preset influence degree may be 50%, the preset confidence level may be 50%, or other values ​​may be used.

[0062] Furthermore, in the case where the impact level is too high and / or the confidence level is too low, the original environmental parameter features may be expanded to ensure the comprehensiveness of the features, thereby improving the accuracy of the reference information.

[0063] In some embodiments, in other cases, the original environmental parameter characteristics may be directly determined as the target environmental parameter characteristics.

[0064] In some embodiments, expanding the original environmental parameter features may include: increasing the complexity of the original environmental parameter features. For example, if there are only three original environmental parameter features, two more may be added.

[0065] Furthermore, the obtained target environmental parameter characteristics can be used to determine the gas circulation index of the area to be detected.

[0066] In some embodiments, the target environment parameter characteristics are input into a pre-trained evaluation model to obtain the gas circulation index of the area to be detected.

[0067] In some embodiments, the evaluation model may be a random forest model, a large model, etc., which is not limited here.

[0068] In some embodiments, the training data corresponding to the evaluation model may include: sample environmental parameter characteristics and gas circulation index labels. The evaluation model to be trained is trained by using the training data so that the evaluation model can learn the relationship between the environmental parameter characteristics and the gas circulation index, and then can predict the gas circulation index based on the environmental parameter characteristics.

[0069] In step S23, region segmentation and identification are performed based on the infrared image data and reference information through a pre-trained convolutional neural network to obtain a target image including an image region corresponding to the target gas.

[0070] Figure 3 is a schematic diagram of a pre-trained convolutional neural network structure according to an exemplary embodiment. Figure 3 The network includes: a pre-trained region segmentation network and a pre-trained region identification network.

[0071] Among them, the region segmentation network is used to achieve region segmentation, and the region identification network is used to achieve region identification through object recognition.

[0072] In some embodiments, the specific network structures of the region segmentation network and the region identification network can refer to the mature convolutional neural network technology in the art. For example, the network may involve convolutional layers, pooling layers, etc.

[0073] In some embodiments, the infrared image data may include multiple infrared images, and step S23 may include: performing regional segmentation on the multiple infrared images according to reference information corresponding to the infrared image data through a pre-trained region segmentation network to obtain multiple segmented infrared images; using a pre-trained region identification network to identify the image regions corresponding to the target gas in the multiple segmented infrared images to obtain multiple identified infrared images, and determining the multiple identified infrared images as processed infrared image data.

[0074] In this embodiment, the pre-trained region segmentation network first performs region segmentation on each infrared image, and then the region identification network is used to detect the image region where the target gas is located for identification.

[0075] Figure 4 is a schematic diagram of infrared image processing according to an exemplary embodiment. Figure 4 As shown, the infrared image can be first segmented into multiple regions through the region segmentation network. The specific segmentation results of these multiple regions depend on the reference information. Then, the image features of each image region are detected through the region identification network to see if they match the target gas. If they match, the image region identification can be added. If they do not match, no corresponding processing is required.

[0076] In some embodiments, the target gas may be a gas to be detected, or a gas related to the gas to be detected, which may be fuel gas, polluted gas, harmful gas, etc.

[0077] In some embodiments, the training process of the convolutional neural network may include: obtaining a training data set, the training data set includes multiple training samples, each training sample includes: a sample image, sample reference information, a first label image and a second label image, the first label image is an image obtained by performing region segmentation on the sample image based on the sample reference information, and the second label image is an image obtained by marking the region where the target gas is located in the sample image; according to the sample images, sample reference information and first label images in the multiple training samples, a region segmentation network to be trained is trained to obtain a pre-trained region segmentation network; according to at least the sample images and the second label images in the multiple training samples, a region identification network to be trained is trained to obtain a pre-trained region identification network; according to the pre-trained region segmentation network and the pre-trained region identification network, a pre-trained convolutional neural network is determined.

[0078] In some embodiments, training samples can be obtained through actual measurement or simulation, and labels can be configured manually or through artificial intelligence.

[0079] The training sample includes a sample image, a first label image obtained by segmenting the sample image based on sample reference information, and a second label image obtained by marking the target gas region of the sample image.

[0080] The sample image, the sample reference information and the first label image can be used to train a region segmentation network.

[0081] Furthermore, the sample image and the second label image can be used to train the region identification network to be trained.

[0082] Furthermore, the two networks trained separately are integrated according to the input-output relationship to obtain a pre-trained convolutional neural network.

[0083] In some embodiments, in addition to the sample image and the second label image, the first label image may also be combined to train the region identification network.

[0084] Therefore, as an optional implementation, the region identification network to be trained is trained at least according to the sample images and the second label images in multiple training samples to obtain a pre-trained region identification network, including: according to at least part of the sample images and the second label images in the multiple training samples, the region identification network to be trained is initially trained to obtain the initially trained region identification network; according to at least part of the first label images in the multiple training samples, the initially trained region identification network is tested to obtain a network test result, wherein the number of samples used for the test is less than the number of samples used for the initial training; according to the network test result, the initially trained region identification network is optimized to obtain a pre-trained region identification network.

[0085] In this implementation, a portion of the sample images and the second label images may be randomly selected to perform initial training on the region identification network, and then a portion of the first label images may be randomly selected and input into the initially trained region identification network to obtain a region identification result output by the initially trained region identification network. The region identification result is compared with the second label image corresponding to the first label image, and the initially trained region identification network is optimized according to the comparison result.

[0086] For example, if the region identification result obtained through testing is significantly different from the second label image corresponding to the first label image, more sample images and second label images are selected for training, or the second label image corresponding to the sample image is updated for training.

[0087] Through this implementation, the region identification network can better identify the region output by the region segmentation network, thereby improving the model processing accuracy.

[0088] Furthermore, the processed infrared image data may include a plurality of marked infrared images, and in step S24, the gas leakage detection result may be determined based on the plurality of infrared images.

[0089] As an optional implementation, step S24 includes: determining the total number of multiple infrared images after identification; determining the number of target images; determining the area information of the image area corresponding to the target gas identified in the target image, the area information is used to characterize the distribution of the image area corresponding to the target gas in the target image; determining the leakage probability of the target gas in the area to be detected based on the total number, the number of target images and the area information; determining the gas leakage detection result based on the leakage probability of the target gas.

[0090] In some embodiments, the region information may include: region size, region density, and region quantity, etc.

[0091] In some embodiments, the leakage probability may be expressed as: weight 1×number of target images / total number+weight 2×region information.

[0092] Among them, weight 1 represents the influence of the proportion of the number of target images on the leakage probability, and weight 2 represents the influence of the regional information on the leakage probability. The greater the influence, the greater the corresponding weight. The sum of weight 1 and weight 2 can be 1, and weight 1 can be less than weight 2.

[0093] In some embodiments, the larger the region size, the region density, and the number of regions, the greater the probability of leakage; the larger the proportion of target images, the greater the probability of leakage.

[0094] Therefore, the leakage probability of the target gas can be used as the final gas leakage detection result.

[0095] In some embodiments, after step S24, corresponding processing may be performed based on the detection result.

[0096] Therefore, the gas leakage method may also include: determining whether it is necessary to perform gas leakage protection on the area to be detected based on the leakage probability of the target gas; when it is determined that the area to be detected needs to be protected from gas leakage, generating a gas leakage protection strategy based on the gas circulation index, and executing the gas leakage protection strategy; when it is determined that it is not necessary to perform gas leakage protection on the area to be detected, adding a preset safety mark for the leakage probability and gas circulation index of the target gas, and storing it.

[0097] In some embodiments, if the leakage probability of the target gas is higher than the preset leakage probability, the area to be detected needs to be protected from gas leakage. Different target gases may correspond to different preset leakage probabilities, which are configured specifically in combination with application scenarios.

[0098] In some embodiments, for fuel gas, when the gas circulation index is high, the protection strength of the gas leakage protection strategy can be low, and when the gas circulation index is low, the protection strength of the gas leakage protection strategy can be high. For harmful gas or polluted gas, when the gas circulation index is low, the protection strength of the gas leakage protection strategy can be low, and when the gas circulation index is high, the protection strength of the gas leakage protection strategy can be high.

[0099] When it is determined that the protection intensity is low, the gas leakage protection strategy may include outputting warning information indicating gas leakage in the area to be detected. When it is determined that the protection intensity is high, the gas leakage protection strategy may include arranging protection personnel to go to the area to be detected.

[0100] In some embodiments, if it is determined that the area to be detected does not need to be protected from gas leakage, the leakage probability and gas circulation index of the target gas can be added with a preset safety mark and stored for easy query or other applications.

[0101] In some embodiments, the method may further include: when the gas leakage protection strategy includes outputting warning information indicating gas leakage in the area to be detected, after a preset time period, obtaining new environmental change data collected by the environmental sensor and new infrared image data collected by the infrared imaging sensor, and the preset time period is determined according to the area information of the area to be detected; generating an updated training data set according to the new environmental change data and the new infrared image data; and updating the pre-trained convolutional neural network according to the updated training data set to obtain an updated trained convolutional neural network.

[0102] In some embodiments, the larger the area size, area density and area number, the longer the preset duration. The preset durations corresponding to different area information can be predefined by simulation testing, actual measurement, etc., and then the matching preset duration is determined according to the current area information.

[0103] In some embodiments, the new environmental change data and the new infrared image data may be related data corresponding to multiple moments within a preset time period.

[0104] It is understandable that, since the user may perform corresponding processing after the early warning, the data of the area to be detected may also change. At this time, this part of the data can be collected to generate a new training data set.

[0105] In some embodiments, the tag corresponding to the new environmental change data may be reference information uploaded by the processing personnel. The tag corresponding to the new infrared image data may be a processed image uploaded by the processing personnel.

[0106] Furthermore, using the updated training data set, the network is trained again according to the training method of the aforementioned embodiment to achieve network update training.

[0107] In some embodiments, when executing the gas leakage protection strategy, warning information can be output through the corresponding warning device, or by calling the protection personnel to inform the protection personnel to handle the situation.

[0108] It is understandable that the gas leakage detection results can be used in more applications, such as repairing the equipment in the detection area, etc. The equipment is, for example, a gas pipeline, a gas energy storage device, etc.

[0109] Figure 5 is a structural block diagram of a gas leakage detection device based on a convolutional neural network according to an exemplary embodiment. Figure 5 As shown, the device comprises:

[0110] The acquisition module 501 is used to acquire the environmental change data collected by the environmental sensor of the area to be detected and the infrared image data collected by the infrared imaging sensor.

[0111] The detection module 502 is used to determine the reference information corresponding to the infrared image data according to the environmental change data, and the reference information is used to affect the regional segmentation strategy of the infrared image data; through a pre-trained convolutional neural network, the infrared image data is regionally segmented and marked according to the infrared image data and the reference information corresponding to the infrared image data to obtain processed infrared image data, and the processed infrared image data includes a target image with an image area corresponding to the target gas marked; based on the processed infrared image data, a gas leakage detection result is determined, and the gas leakage detection result is used to characterize the leakage of the target gas in the area to be detected.

[0112] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0113] Figure 6 FIG. 6 is a block diagram of an electronic device 600 according to an exemplary embodiment. Figure 6 As shown, the electronic device 600 may include: a processor 601 and a memory 602. The electronic device 600 may also include one or more of a multimedia component 603, an input / output (I / O) interface 604, and a communication component 605.

[0114] The processor 601 is used to control the overall operation of the electronic device 600 to complete all or part of the steps in the gas leakage detection method based on the convolutional neural network. The memory 602 is used to store various types of data to support the operation of the electronic device 600, and these data may include, for example, instructions for any application or method used to operate on the electronic device 600, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 603 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 602 or sent through the communication component 605. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 604 provides an interface between the processor 601 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 605 may include: Wi-Fi module, Bluetooth module, NFC module.

[0115] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned XXXX method.

[0116] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned gas leakage detection method based on a convolutional neural network are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 602 including program instructions, and the above-mentioned program instructions can be executed by the processor 601 of the electronic device 600 to complete the above-mentioned gas leakage detection method based on a convolutional neural network.

[0117] In another exemplary embodiment, a computer program product is also provided, which includes a computer program that can be executed by a processor, and when the computer program is executed by the processor, the steps of the above-mentioned gas leakage detection method based on convolutional neural network are implemented.

[0118] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0119] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0120] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A gas leakage detection method based on convolutional neural network, characterized in that: include: Acquire environmental change data collected by environmental sensors in the area to be detected and infrared image data collected by infrared imaging sensors; Determining reference information corresponding to the infrared image data according to the environmental change data, wherein the reference information is used to influence a regional segmentation strategy of the infrared image data; By using a pre-trained convolutional neural network, the infrared image data is segmented and marked according to the infrared image data and reference information corresponding to the infrared image data, so as to obtain processed infrared image data, wherein the processed infrared image data includes a target image with an image area corresponding to the target gas marked therein; A gas leakage detection result is determined according to the processed infrared image data, and the gas leakage detection result is used to characterize the leakage situation of the target gas in the area to be detected.

2. The gas leakage detection method according to claim 1, characterized in that: The environmental change data includes environmental parameters corresponding to a plurality of moments respectively, and determining the reference information corresponding to the infrared image data according to the environmental change data includes: Determining original environmental parameter characteristics according to the environmental parameters corresponding to the multiple moments respectively; According to the type and amount of the environmental change data, the original environmental parameter characteristics are adjusted to obtain target environmental parameter characteristics, where the target environmental parameter characteristics are related to the gas circulation index of the area to be detected; The reference information corresponding to the infrared image data is determined by a pre-trained evaluation model according to the target environment parameter characteristics, and the reference information includes a gas circulation index of the area to be detected.

3. The gas leakage detection method according to claim 2, characterized in that: The adjusting the original environmental parameter characteristics according to the type and amount of the environmental change data to obtain the target environmental parameter characteristics includes: Determining, according to the type of the environmental change data, the degree of influence of the environmental change data on the gas circulation index of the area to be detected; Determining the confidence level of the environmental change data according to the data volume of the environmental change data; If the impact degree is higher than a preset impact degree and / or the confidence level is lower than a preset confidence level, the original environmental parameter feature is expanded, and the expanded original environmental parameter feature is determined as the target environmental parameter feature.

4. The gas leakage detection method according to claim 1, characterized in that: The pre-trained convolutional neural network includes a pre-trained region segmentation network and a pre-trained region identification network, the infrared image data includes a plurality of infrared images, and the pre-trained convolutional neural network performs region segmentation and identification on the infrared image data according to the infrared image data and reference information corresponding to the infrared image data to obtain processed infrared image data, including: Performing regional segmentation on the plurality of infrared images respectively according to the reference information corresponding to the infrared image data through the pre-trained regional segmentation network to obtain a plurality of segmented infrared images; The image regions corresponding to the target gas are respectively identified in the multiple segmented infrared images through the pre-trained region identification network to obtain the multiple identified infrared images, and the multiple identified infrared images are determined as the processed infrared image data.

5. The gas leakage method according to claim 4, characterized in that: The gas leakage detection method further comprises: Acquire a training data set, the training data set comprising a plurality of training samples, each training sample comprising: a sample image, sample reference information, a first label image and a second label image, the first label image being an image obtained by performing region segmentation on the sample image based on the sample reference information, and the second label image being an image obtained by marking a region where a target gas is located on the sample image; Training the region segmentation network to be trained according to the sample images, the sample reference information and the first label image in the multiple training samples to obtain the pre-trained region segmentation network; Training the region identification network to be trained at least according to the sample image and the second label image in the plurality of training samples to obtain the pre-trained region identification network; The pre-trained convolutional neural network is determined according to the pre-trained region segmentation network and the pre-trained region identification network.

6. The gas leakage method according to claim 5, characterized in that: The step of training the region identification network to be trained based on at least the sample image and the second label image in the plurality of training samples to obtain the pre-trained region identification network includes: Performing initial training on the region identification network to be trained according to at least part of the sample images and the second label image in the plurality of training samples to obtain an initially trained region identification network; Testing the initially trained region identification network according to at least part of the first label images in the plurality of training samples to obtain a network test result, wherein the number of samples used for the test is less than the number of samples used for the initial training; According to the network test result, the initially trained region identification network is optimized to obtain the pre-trained region identification network.

7. The gas leakage method according to claim 1, characterized in that: The processed infrared image data includes a plurality of marked infrared images, and determining the gas leakage detection result according to the processed infrared image data includes: Determining the total number of the plurality of infrared images after the marking; determining the number of the target images; Determine region information of an image region corresponding to a target gas identified in the target image, wherein the region information is used to characterize distribution of the image region corresponding to the target gas in the target image; Determining the leakage probability of the target gas in the area to be detected according to the total number, the number of the target images and the area information; The gas leakage detection result is determined according to the leakage probability of the target gas.

8. The gas leakage method according to claim 1, characterized in that: The gas leakage detection result includes the leakage probability of the target gas in the area to be detected, the reference information includes the gas circulation index of the area to be detected, and the gas leakage method further includes: Determining whether it is necessary to perform gas leakage protection on the area to be detected according to the leakage probability of the target gas; In the case where it is determined that the area to be detected needs to be protected from gas leakage, a gas leakage protection strategy is generated according to the gas circulation index, and the gas leakage protection strategy is executed; In the case where it is determined that the area to be detected does not need to be protected from gas leakage, a preset safety mark is added to the leakage probability of the target gas and the gas circulation index, and the marks are stored.

9. The gas leakage method according to claim 8, characterized in that: The gas leakage method further comprises: In the case where the gas leakage protection strategy includes outputting warning information indicating gas leakage in the area to be detected, after a preset time period, new environmental change data collected by the environmental sensor and new infrared image data collected by the infrared imaging sensor are obtained, and the preset time period is determined according to the area information of the area to be detected; Generate an updated training data set according to the new environmental change data and the new infrared image data; The pre-trained convolutional neural network is updated and trained according to the updated training data set to obtain an updated and trained convolutional neural network.

10. A gas leak detection system based on convolutional neural network, characterized in that: include: An environmental sensor is arranged in the area to be detected, and is used to collect environmental change data of the area to be detected; An infrared imaging sensor is arranged in the area to be detected, and is used to collect infrared image data of the area to be detected; A detection device, connected to the environmental sensor and the infrared imaging sensor respectively, is used to execute the gas leakage detection method based on convolutional neural network as described in any one of claims 1 to 9.

Images