Gas leakage detection method and system based on convolutional neural network

Abstract

The present disclosure relates to a gas leakage detection method and system based on a convolutional neural network, and relates to the technical field of Internet of Things. The environmental sensor is used to collect environmental change data, and the infrared imaging sensor is used to collect infrared image data. Based on the environmental change data, a region segmentation strategy for affecting the infrared image data is determined. Through a pre-trained convolutional neural network, region segmentation and identification are performed based on the reference information corresponding to the infrared image data, and processed infrared image data is obtained. The processed infrared image data is used to determine whether there is a leakage of a target gas in a to-be-detected region. The data of the environmental sensor and the infrared imaging sensor are combined and applied, the infrared image data is processed based on the environmental data, the processing method is matched with the actual environmental scene, the processing accuracy of the infrared image data is improved, the accuracy of the gas leakage detection result is also improved, and the safety hazard of gas leakage is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of Internet of Things (IoT) technology, and more specifically, to a gas leak detection method and system based on convolutional neural networks. Background Technology

[0002] With the development of IoT technology, IoT technology is being applied to various detection scenarios. In these scenarios, data acquisition devices such as sensors can be configured to collect relevant data, which is then analyzed to obtain detection results.

[0003] In related technologies, the detection of gas leaks is usually based on a single sensing technology and integrated with Internet of Things (IoT) technology. However, due to the limited data, this detection technology may not be able to accurately detect gas leaks and thus cannot effectively reduce the safety hazards of gas leaks. Summary of the Invention

[0004] The purpose of this disclosure is to provide a gas leak detection method and system based on convolutional neural networks, which can improve the accuracy of gas leak detection and thus reduce the safety hazards of gas leaks.

[0005] To achieve the above objectives, in a first aspect, this disclosure provides a gas leak detection method based on a convolutional neural network, comprising: acquiring environmental change data collected by an environmental sensor and infrared image data collected by an infrared imaging sensor in an area to be detected; determining reference information corresponding to the infrared image data based on the environmental change data, wherein the reference information is used to influence the region segmentation strategy of the infrared image data; performing region segmentation and labeling on the infrared image data using a pre-trained convolutional neural network based on the infrared image data and the reference information corresponding to the infrared image data, thereby obtaining processed infrared image data, wherein the processed infrared image data includes a target image that identifies an image region corresponding to a target gas; and determining a gas leak detection result based on the processed infrared image data, wherein the gas leak detection result is used to characterize the leakage status of the target gas in the area to be detected.

[0006] Optionally, the environmental change data includes environmental parameters corresponding to multiple time points. Determining the reference information corresponding to the infrared image data based on the environmental change data includes: determining original environmental parameter features based on the environmental parameters corresponding to the multiple time points; adjusting the original environmental parameter features according to the type and amount of the environmental change data to obtain target environmental parameter features, wherein the target environmental parameter features are related to the gas flow index of the area to be detected; and determining the reference information corresponding to the infrared image data based on the target environmental parameter features using a pre-trained evaluation model, wherein the reference information includes the gas flow index of the area to be detected.

[0007] Optionally, adjusting the original environmental parameter features according to the type and amount of the environmental change data to obtain the target environmental parameter features includes: determining the degree of influence of the environmental change data on the gas flow index of the area to be detected according to the type of the environmental change data; determining the confidence level of the environmental change data according to the amount of the environmental change data; if the degree of influence is higher than a preset degree of influence and / or the confidence level is lower than a preset confidence level, then the original environmental parameter features are expanded, and the expanded original environmental parameter features are determined as the target environmental parameter features.

[0008] Optionally, the pre-trained convolutional neural network includes a pre-trained region segmentation network and a pre-trained region labeling network. The infrared image data includes multiple infrared images. The step of using the pre-trained convolutional neural network to perform region segmentation and labeling on the infrared image data according to the infrared image data and the reference information corresponding to the infrared image data to obtain processed infrared image data includes: using the pre-trained region segmentation network to perform region segmentation on the multiple infrared images according to the reference information corresponding to the infrared image data, to obtain multiple segmented infrared images; using the pre-trained region labeling network to label the image regions corresponding to the target gas in the multiple segmented infrared images, to obtain multiple labeled infrared images, and identifying the multiple labeled infrared images as the processed infrared image data.

[0009] Optionally, the gas leak detection method further includes: acquiring a training dataset, the training dataset including multiple training samples, each training sample including: a sample image, sample reference information, a first label image, and a second label image, wherein the first label image is an image obtained by performing region segmentation on the sample image based on the sample reference information, and the second label image is an image obtained by identifying the region where the target gas is located on the sample image; training a region segmentation network to be trained based on the sample image, the sample reference information, and the first label image from the multiple training samples to obtain the pre-trained region segmentation network; training a region labeling network to be trained based at least on the sample image and the second label image from the multiple training samples to obtain the pre-trained region labeling network; and determining the pre-trained convolutional neural network based on the pre-trained region segmentation network and the pre-trained region labeling network.

[0010] Optionally, training the region labeling network to be trained based at least on the sample images and the second label images from the plurality of training samples to obtain the pre-trained region labeling network includes: performing initial training on the region labeling network to be trained based on at least a portion of the sample images and the second label images from the plurality of training samples to obtain an initially trained region labeling network; testing the initially trained region labeling network based on at least a portion of the first label images from the plurality of training samples to obtain network test results, wherein the number of samples used for testing is less than the number of samples used for initial training; and optimizing the initially trained region labeling network based on the network test results to obtain the pre-trained region labeling network.

[0011] Optionally, the processed infrared image data includes multiple identified infrared images. Determining the gas leak detection result based on the processed infrared image data includes: determining the total number of identified infrared images; determining the number of target images; determining the region information of the image region corresponding to the identified target gas in the target image, the region information being used to characterize the distribution of the image region corresponding to the target gas in the target image; determining the leakage probability of the target gas in the detection area based on the total number, the number of target images, and the region information; and determining the gas leak detection result based on the leakage probability of the target gas.

[0012] Optionally, the gas leak detection result includes the leakage probability of the target gas in the area to be detected, and the reference information includes the gas flow index of the area to be detected. The gas leak detection method further includes: determining whether gas leak protection is needed for the area to be detected based on the leakage probability of the target gas; if it is determined that gas leak protection is needed for the area to be detected, generating a gas leak protection strategy based on the gas flow index and executing the gas leak protection strategy; if it is determined that gas leak protection is not needed for the area to be detected, adding a preset safety flag to the leakage probability of the target gas and the gas flow index, and storing it.

[0013] Optionally, the gas leak detection method further includes: when the gas leak protection strategy includes outputting early warning information indicating a gas leak in the area to be detected, after a preset time period, acquiring new environmental change data collected by the environmental sensor and new infrared image data collected by the infrared imaging sensor, wherein the preset time period is determined based on the area information of the area to be detected; generating an updated training dataset based on the new environmental change data and the new infrared image data; and updating and training the pre-trained convolutional neural network based on the updated training dataset to obtain an updated trained convolutional neural network.

[0014] Secondly, this disclosure provides a gas leak detection system based on a convolutional neural network, comprising: an environmental sensor disposed in a detection area for collecting environmental change data of the detection area; an infrared imaging sensor disposed in the detection area for collecting infrared image data of the detection area; and a detection device connected to the environmental sensor and the infrared imaging sensor respectively, for executing the gas leak detection method based on a convolutional neural network described in the first aspect of this disclosure.

[0015] The above technical solution involves collecting environmental change data using environmental sensors and infrared image data using infrared imaging sensors. Based on the environmental change data, a region segmentation strategy affecting the infrared image data is determined. Then, a pre-trained convolutional neural network, based on reference information corresponding to the infrared image data, performs region segmentation and labeling to obtain processed infrared image data. This processed infrared image data can be used to determine whether a target gas leak exists in the area to be detected. This technical solution combines data from environmental sensors and infrared imaging sensors, processing infrared image data based on environmental data. This ensures the processing method matches the actual environmental scene, improving the processing accuracy of the infrared image data. Consequently, the accuracy of the final gas leak detection result is also improved. Therefore, this technical solution can improve the accuracy of gas leak detection, thereby reducing the safety hazards of gas leaks.

[0016] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0018] Figure 1 This is a structural block diagram of a gas leak detection system based on a convolutional neural network, according to an exemplary embodiment.

[0019] Figure 2 This is a flowchart illustrating a gas leak detection method based on a convolutional neural network according to an exemplary embodiment.

[0020] Figure 3 This is a schematic diagram illustrating the structure of a pre-trained convolutional neural network according to an exemplary embodiment.

[0021] Figure 4 This is a schematic diagram illustrating infrared image processing according to an exemplary embodiment.

[0022] Figure 5 This is a block diagram illustrating the structure of a gas leak detection device based on a convolutional neural network, according to an exemplary embodiment.

[0023] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0024] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0025] With the development of IoT technology, IoT technology is being applied to various detection scenarios. In these scenarios, data acquisition devices such as sensors can be configured to collect relevant data, which is then analyzed to obtain detection results.

[0026] In related technologies, the detection of gas leaks is usually based on a single sensing technology and integrated with Internet of Things (IoT) technology. However, due to the limited data, this detection technology may not be able to accurately detect gas leaks and thus cannot effectively reduce the safety hazards of gas leaks.

[0027] Based on this, the present disclosure provides a technical solution that collects environmental change data through an environmental sensor and infrared image data through an infrared imaging sensor. Based on the environmental change data, a region segmentation strategy for affecting the infrared image data is determined. Then, through a pre-trained convolutional neural network, region segmentation and labeling are performed based on the reference information corresponding to the infrared image data to obtain processed infrared image data. The processed infrared image data can be used to determine whether there is a leak of target gas in the area to be detected.

[0028] This technical solution combines data from environmental sensors and infrared imaging sensors, processes infrared image data based on environmental data, and matches the processing method with the actual environmental scene, thereby improving the processing accuracy of infrared image data and ultimately improving the accuracy of the gas leak detection results.

[0029] Therefore, this technical solution can improve the accuracy of gas leak detection, thereby reducing the safety hazards of gas leaks.

[0030] Regarding the detection of gas leaks, such as the detection of natural gas leaks, the detection of pollutant gas leaks, and the detection of harmful gas leaks.

[0031] In these scenarios, detection systems based on IoT architecture are typically deployed to effectively detect gas leaks.

[0032] Figure 1 This is a structural block diagram of a gas leak detection system based on a convolutional neural network, according to an exemplary embodiment. Figure 1 As shown, the system includes: an environmental sensor, an infrared imaging sensor, and a detection device.

[0033] The detection equipment is connected to an environmental sensor and an infrared imaging sensor, respectively. The environmental sensor and the infrared imaging sensor are positioned in the area to be detected.

[0034] The area to be tested could be, for example, a kitchen area or a factory workshop area.

[0035] In some embodiments, the environmental sensor may involve various types of sensors for detecting environmental parameters. Environmental parameters include, for example, temperature, humidity, and wind speed; accordingly, the environmental sensor may include, but is not limited to, temperature sensors, humidity sensors, and wind sensors.

[0036] In some embodiments, depending on the cost requirements of different scenarios, one or more types of environmental sensors may be configured in a detection area, without limitation.

[0037] In some embodiments, the infrared imaging sensor can be an infrared imaging device such as an infrared imager, which can acquire infrared images. Regarding the principle of infrared imaging, it is a technique that uses infrared radiation to create images. Infrared radiation is the portion of the electromagnetic spectrum with wavelengths longer than visible light, typically referring to radiation with wavelengths between 0.75 and 1000 micrometers. Infrared imaging technology can detect the thermal radiation emitted by objects and convert this radiation into images. Infrared imaging technology is a mature imaging technology; for details, please refer to other mature technologies in the field, which will not be described in detail here.

[0038] In some embodiments, depending on the cost requirements of different scenarios, one or more infrared imaging sensors can be configured in a detection area, and infrared imaging sensors with different accuracies can be configured.

[0039] In some embodiments, the detection device can be various electronic devices that have data processing capabilities, visualization capabilities, etc. For example, a host computer.

[0040] In some embodiments, environmental sensors can continuously acquire environmental parameters to obtain environmental change data. Infrared imaging sensors can continuously acquire infrared images to obtain infrared image data.

[0041] Therefore, environmental change data can include environmental parameters corresponding to multiple times, and infrared image data can include multiple infrared images, which can correspond to different times or different acquisition angles, without limitation here.

[0042] Figure 2 This is a flowchart illustrating a gas leak detection method based on a convolutional neural network according to an exemplary embodiment. This method can be applied to... Figure 1 The testing equipment in, such as Figure 2 As shown, the method includes the following steps:

[0043] Step S21: Obtain environmental change data collected by the environmental sensor and infrared image data collected by the infrared imaging sensor in the area to be detected.

[0044] Step S22: Based on the environmental change data, determine the reference information corresponding to the infrared image data. The reference information is used to influence the region segmentation strategy of the infrared image data.

[0045] Step S23: Using a pre-trained convolutional neural network, the infrared image data is segmented and labeled according to the infrared image data and the reference information corresponding to the infrared image data to obtain processed infrared image data. The processed infrared image data includes the target image with the image region corresponding to the target gas.

[0046] Step S24: Based on the processed infrared image data, determine the gas leak detection result. The gas leak detection result is used to characterize the leakage of the target gas in the area to be detected.

[0047] In step S21, the environmental change data and infrared image data can be relevant data collected within the current detection cycle. Both the environmental sensor and the infrared imaging sensor can collect corresponding data according to the acquisition cycle.

[0048] Furthermore, environmental sensors and infrared imaging sensors can upload the collected data to the detection equipment so that the detection equipment can perform detection.

[0049] In step S22, reference information for the region segmentation strategy affecting infrared image data can be determined based on environmental change data. It is understood that the gas conditions reflected in infrared images differ under different environments. In related technologies, image processing is typically performed solely based on infrared images to determine whether a gas leak has occurred. This approach cannot adapt to different environments, resulting in poor accuracy of the final detection results.

[0050] Therefore, region segmentation of infrared images can be assisted based on environmental change data.

[0051] In some embodiments, the environmental change data includes environmental parameters corresponding to multiple times. Step S22 may include: determining original environmental parameter features based on the environmental parameters corresponding to multiple times; adjusting the original environmental parameter features according to the type and amount of environmental change data to obtain target environmental parameter features, wherein the target environmental parameter features are related to the gas flow index of the area to be detected; and determining reference information corresponding to the infrared image data based on the target environmental parameter features using a pre-trained evaluation model, wherein the reference information includes the gas flow index of the area to be detected.

[0052] In this implementation, the original environmental parameter features can be extracted based on the environmental parameters corresponding to multiple time points. These original environmental parameter features may include: the variation patterns of environmental parameters, the mean of environmental parameters, the standard deviation of environmental parameters, and other data features determined through statistical analysis of environmental parameters.

[0053] In some embodiments, if different types of environmental parameters are involved, the different types of environmental parameters can be processed in the corresponding manner, and finally all of them can be input into the evaluation model for evaluation.

[0054] In some embodiments, the target environmental parameter features may be features obtained by adjusting the original environmental parameter features.

[0055] As an optional implementation method, the original environmental parameter features are adjusted according to the type and amount of environmental change data to obtain target environmental parameter features. This includes: determining the degree of influence of environmental change data on the gas flow index of the area to be detected based on the type of environmental change data; determining the confidence level of environmental change data based on the amount of environmental change data; if the degree of influence is higher than the preset degree of influence and / or the confidence level is lower than the preset confidence level, then the original environmental parameter features are expanded, and the expanded original environmental parameter features are determined as the target environmental parameter features.

[0056] In some embodiments, the degree of influence corresponding to different types of environmental parameters can be predefined. Based on the types of environmental parameters involved in the current environmental change data, the degree of influence of the environmental change data on the gas flow index of the area to be detected can be determined.

[0057] For example, temperature has a relatively small impact on the gas flow index, humidity has a relatively large impact, and wind speed has a relatively large impact.

[0058] The gas flow index characterizes the rate at which gas flows through the area being tested. Even with a gas leak, the test results will be affected if gas flow is good.

[0059] In some embodiments, the amount of environmental change data can be the number of frames related to the environmental parameters involved in the environmental change data. The more frames, the higher the confidence level of the environmental change data.

[0060] Therefore, by pre-configuring the confidence levels corresponding to different frame numbers, the current confidence level can be determined based on the current frame number.

[0061] In some embodiments, the preset influence level can be 50%, the preset confidence level can be 50%, or it can be other values.

[0062] Furthermore, in cases where the impact is too high and / or the confidence level is too low, the original environmental parameter features can be expanded to ensure the comprehensiveness of the features, thereby improving the accuracy of the reference information.

[0063] In some embodiments, in other cases, the original environmental parameter characteristics can be directly determined as the target environmental parameter characteristics.

[0064] In some embodiments, extending the original environmental parameter features may include increasing the complexity of the original environmental parameter features. For example, if there were only three original environmental parameter features, two more may be added.

[0065] Furthermore, the obtained target environmental parameter characteristics can be used to determine the gas flow index of the area to be detected.

[0066] In some embodiments, by inputting the target environmental parameter features into a pre-trained evaluation model, the gas flow index of the area to be detected can be obtained.

[0067] In some embodiments, the evaluation model may be a random forest model, a large model, etc., and there is no limitation herein.

[0068] In some embodiments, the training data corresponding to the evaluation model may include: sample environmental parameter features and gas flow index labels. The evaluation model to be trained is trained using this training data, enabling the model to learn the relationship between environmental parameter features and the gas flow index, and thus predict the gas flow index based on the environmental parameter features.

[0069] In step S23, a pre-trained convolutional neural network is used to perform region segmentation and labeling based on infrared image data and reference information to obtain a target image including the image region corresponding to the target gas.

[0070] Figure 3 This is a schematic diagram illustrating the structure of a pre-trained convolutional neural network according to an exemplary embodiment, such as... Figure 3 The network includes a pre-trained region segmentation network and a pre-trained region labeling network.

[0071] Among them, the region segmentation network is used to achieve region segmentation, and the region identification network is used to achieve region identification through object recognition.

[0072] In some embodiments, the specific network structures of the region segmentation network and the region identification network can refer to mature convolutional neural network techniques in the art. For example, the network may involve convolutional layers, pooling layers, etc.

[0073] In some embodiments, the infrared image data may include multiple infrared images, and step S23 may include: using a pre-trained region segmentation network to segment the multiple infrared images according to the reference information corresponding to the infrared image data, to obtain multiple segmented infrared images; using a pre-trained region labeling network to label the image regions corresponding to the target gas in the segmented multiple infrared images, to obtain multiple labeled infrared images, and determining the labeled multiple infrared images as the processed infrared image data.

[0074] In this implementation, the pre-trained region segmentation network first segments each infrared image into regions, and then uses a region labeling network to detect and label the image region where the target gas is located.

[0075] Figure 4 This is a schematic diagram illustrating infrared image processing according to an exemplary embodiment, such as... Figure 4 As shown, the infrared image can first be segmented into multiple regions using a region segmentation network. The specific segmentation result depends on the reference information. Next, a region labeling network is used to detect whether the image features of each region match the target gas. If they match, image region labels can be added. If they do not match, no further processing is required.

[0076] In some embodiments, the target gas may be the gas to be detected or a gas related to the gas to be detected, such as fuel gas, polluting gas, and harmful gas.

[0077] In some embodiments, the training process of a convolutional neural network may include: acquiring a training dataset, the training dataset including multiple training samples, each training sample including: a sample image, sample reference information, a first label image and a second label image, the first label image being an image obtained after region segmentation of the sample image based on the sample reference information, and the second label image being an image obtained after identifying the region where the target gas is located in the sample image; training a region segmentation network to be trained based on the sample image, sample reference information and first label image from the multiple training samples to obtain a pre-trained region segmentation network; training a region labeling network to be trained based on at least the sample image and second label image from the multiple training samples to obtain a pre-trained region labeling network; and determining a pre-trained convolutional neural network based on the pre-trained region segmentation network and the pre-trained region labeling network.

[0078] In some embodiments, training samples can be obtained through actual testing or simulation, and labels can be configured manually or through artificial intelligence.

[0079] The training samples include sample images, a first label image obtained by segmenting the sample images into regions based on sample reference information, and a second label image obtained by identifying the regions where the target gas is located in the sample images.

[0080] The sample images, sample reference information, and first label images can be used to train the region segmentation network.

[0081] Additionally, the sample images and the second label images can be used to train the region labeling network to be trained.

[0082] Then, by integrating the two separately trained networks according to their input-output relationship, a pre-trained convolutional neural network can be obtained.

[0083] In some embodiments, in addition to the sample image and the second label image, the first label image can also be combined to train the region identification network.

[0084] Therefore, as an optional implementation, training the region labeling network to be trained using at least sample images and second label images from multiple training samples to obtain a pre-trained region labeling network includes: performing initial training on the region labeling network to be trained using at least a portion of sample images and second label images from multiple training samples to obtain an initially trained region labeling network; testing the initially trained region labeling network using at least a portion of first label images from multiple training samples to obtain network test results, wherein the number of samples used for testing is less than the number of samples used for initial training; and optimizing the initially trained region labeling network based on the network test results to obtain a pre-trained region labeling network.

[0085] In this implementation, a subset of sample images and second label images can be randomly selected first to train the region labeling network. Then, a subset of first label images can be randomly selected and input into the initially trained region labeling network to obtain the region labeling result output by the initially trained network. This region labeling result is compared with the second label image corresponding to the first label image, and the initially trained region labeling network is optimized based on the comparison result.

[0086] For example, if the region labeling result obtained through testing differs significantly from the second label image corresponding to the first label image, then more sample images and second label images can be selected for training, or the second label image corresponding to the sample image can be updated for training, etc.

[0087] This implementation method enables the region labeling network to better label the regions output by the region segmentation network, thereby improving the model processing accuracy.

[0088] Furthermore, the processed infrared image data may include multiple identified infrared images. In step S24, the gas leak detection result can be determined based on these multiple infrared images.

[0089] As an optional implementation, step S24 includes: determining the total number of identified infrared images; determining the number of target images; determining the region information of the image region corresponding to the identified target gas in the target image, wherein the region information is used to characterize the distribution of the image region corresponding to the target gas in the target image; determining the leakage probability of the target gas in the area to be detected based on the total number, the number of target images, and the region information; and determining the gas leakage detection result based on the leakage probability of the target gas.

[0090] In some embodiments, the area information may include: area size, area density, and number of areas.

[0091] In some embodiments, the leakage probability can be expressed as: weight 1 × number of target images / total number + weight 2 × region information.

[0092] Among them, weight 1 represents the influence of the proportion of target images on the leakage probability, and weight 2 represents the influence of regional information on the leakage probability. The greater the influence, the greater the corresponding weight. The sum of weight 1 and weight 2 can be 1, and weight 1 can be less than weight 2.

[0093] In some embodiments, the larger the region size, region density, and number of regions, the greater the probability of leakage; the larger the proportion of target images, the greater the probability of leakage.

[0094] Therefore, the probability of leakage of the target gas can be used as the final gas leak detection result.

[0095] In some embodiments, after step S24, corresponding processing can be performed based on the detection results.

[0096] Therefore, the gas leak detection method may further include: determining whether gas leak protection is required for the area to be detected based on the leak probability of the target gas; if it is determined that gas leak protection is required for the area to be detected, generating a gas leak protection strategy based on the gas flow index and executing the gas leak protection strategy; if it is determined that gas leak protection is not required for the area to be detected, adding preset safety indicators to the leak probability and gas flow index of the target gas and storing them.

[0097] In some embodiments, if the leakage probability of the target gas is higher than a preset leakage probability, gas leakage protection is required for the area to be detected. Different target gases may correspond to different preset leakage probabilities, which are configured according to the specific application scenario.

[0098] In some embodiments, for flammable gases, the protection strength of the gas leak protection strategy can be lower when the gas flow index is high, and higher when the gas flow index is low. Conversely, for harmful or polluting gases, the protection strength of the gas leak protection strategy can be lower when the gas flow index is low, and higher when the gas flow index is high.

[0099] When the protection level is determined to be low, a gas leak protection strategy may include outputting a warning message indicating a gas leak in the area to be detected. When the protection level is determined to be high, a gas leak protection strategy may include dispatching protection personnel to the area to be detected.

[0100] In some embodiments, if it is determined that gas leakage protection for the area to be tested is not required, the leakage probability and gas flow index of the target gas can be stored with a preset safety flag for easy querying or other applications.

[0101] In some embodiments, the method may further include: when the gas leak protection strategy includes outputting early warning information indicating gas leaks in the area to be detected, after a preset time period, acquiring new environmental change data collected by an environmental sensor and new infrared image data collected by an infrared imaging sensor, the preset time period being determined based on the area information of the area to be detected; generating an updated training dataset based on the new environmental change data and the new infrared image data; and updating and training a pre-trained convolutional neural network based on the updated training dataset to obtain an updated trained convolutional neural network.

[0102] In some embodiments, the larger the area size, area density, and number of areas, the longer the preset duration. Preset durations corresponding to different area information can be predefined through simulation testing, actual measurements, etc., and then the matching preset duration can be determined based on the current area information.

[0103] In some embodiments, new environmental change data and new infrared image data may be related data corresponding to multiple moments within a preset time period.

[0104] It's understandable that users might take action after an alert is issued, causing changes to the data in the area to be detected. In this case, this data can be collected to generate a new training dataset.

[0105] In some embodiments, the tags corresponding to new environmental change data may be reference information uploaded by the processor. Similarly, the tags corresponding to new infrared image data may be processed images uploaded by the processor.

[0106] Then, by using the updated training dataset, the network is trained again according to the training method described in the aforementioned embodiment, thereby achieving network update training.

[0107] In some embodiments, when implementing a gas leak protection strategy, warning information can be output through corresponding early warning devices, or the protection personnel can be notified to take action by making a phone call.

[0108] It's understandable that gas leak detection results can have many other applications, such as inspecting and repairing equipment in the area being tested. Examples of such equipment include gas pipelines and gas storage devices.

[0109] Figure 5 This is a block diagram illustrating the structure of a gas leak detection device based on a convolutional neural network, according to an exemplary embodiment. Figure 5 As shown, the device includes:

[0110] The acquisition module 501 is used to acquire environmental change data collected by the environmental sensor and infrared image data collected by the infrared imaging sensor in the area to be detected.

[0111] The detection module 502 is used to determine reference information corresponding to the infrared image data based on the environmental change data, wherein the reference information is used to influence the region segmentation strategy of the infrared image data; through a pre-trained convolutional neural network, the infrared image data is segmented and labeled according to the infrared image data and the reference information corresponding to the infrared image data to obtain processed infrared image data, wherein the processed infrared image data includes a target image of an image region labeled with the target gas; and based on the processed infrared image data, a gas leak detection result is determined, wherein the gas leak detection result is used to characterize the leakage of the target gas in the area to be detected.

[0112] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0113] Figure 6 This is a block diagram illustrating an electronic device 600 according to an exemplary embodiment. For example... Figure 6 As shown, the electronic device 600 may include a processor 601 and a memory 602. The electronic device 600 may also include one or more of a multimedia component 603, an input / output (I / O) interface 604, and a communication component 605.

[0114] The processor 601 controls the overall operation of the electronic device 600 to complete all or part of the steps in the gas leak detection method based on convolutional neural networks described above. The memory 602 stores various types of data to support the operation of the electronic device 600. This data may include, for example, instructions for any application or method operating on the electronic device 600, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 603 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 602 or transmitted via communication component 605. The audio component also includes at least one speaker for outputting audio signals. I / O interface 604 provides an interface between processor 601 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 605 is used for wired or wireless communication between the electronic device 600 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof; therefore, the corresponding communication component 605 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0115] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the XXXX method described above.

[0116] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the gas leak detection method based on a convolutional neural network described above. For example, the computer-readable storage medium may be the memory 602 including the program instructions described above, which may be executed by the processor 601 of the electronic device 600 to complete the gas leak detection method based on a convolutional neural network described above.

[0117] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the gas leak detection method based on a convolutional neural network described above.

[0118] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0119] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0120] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A gas leak detection method based on convolutional neural networks, characterized in that, include: Acquire environmental change data collected by environmental sensors and infrared image data collected by infrared imaging sensors in the area to be detected; Based on the environmental change data, reference information corresponding to the infrared image data is determined, and the reference information is used to influence the region segmentation strategy of the infrared image data; By using a pre-trained convolutional neural network, the infrared image data is segmented and labeled according to the infrared image data and the reference information corresponding to the infrared image data, and processed infrared image data is obtained. The processed infrared image data includes a target image that is labeled with the image region corresponding to the target gas. Based on the processed infrared image data, a gas leak detection result is determined, which is used to characterize the leakage of the target gas in the area to be detected. The environmental change data includes environmental parameters corresponding to multiple times. Determining the reference information corresponding to the infrared image data based on the environmental change data includes: Based on the environmental parameters corresponding to the multiple time points, the original environmental parameter characteristics are determined; Based on the type and amount of the environmental change data, the original environmental parameter features are adjusted to obtain target environmental parameter features, which are related to the gas flow index of the area to be detected. Using a pre-trained evaluation model, reference information corresponding to the infrared image data is determined based on the target environmental parameter characteristics. The reference information includes the gas flow index of the area to be detected.

2. The gas leak detection method according to claim 1, characterized in that, The step of adjusting the original environmental parameter features based on the type and volume of the environmental change data to obtain the target environmental parameter features includes: Based on the type of environmental change data, determine the degree of influence of the environmental change data on the gas flow index of the area to be detected; The confidence level of the environmental change data is determined based on the amount of data. If the degree of influence is higher than the preset degree of influence and / or the confidence level is lower than the preset confidence level, then the original environmental parameter features are expanded, and the expanded original environmental parameter features are determined as the target environmental parameter features.

3. The gas leak detection method according to claim 1, characterized in that, The pre-trained convolutional neural network includes a pre-trained region segmentation network and a pre-trained region labeling network. The infrared image data includes multiple infrared images. The pre-trained convolutional neural network performs region segmentation and labeling on the infrared image data based on the infrared image data and corresponding reference information to obtain processed infrared image data, including: The pre-trained region segmentation network performs region segmentation on the multiple infrared images based on the reference information corresponding to the infrared image data, thereby obtaining multiple segmented infrared images. The pre-trained region labeling network is used to label the image regions corresponding to the target gas in the segmented infrared images, resulting in labeled infrared images. These labeled infrared images are then identified as the processed infrared image data.

4. The gas leak detection method according to claim 3, characterized in that, The gas leak detection method further includes: Obtain a training dataset, which includes multiple training samples. Each training sample includes: a sample image, sample reference information, a first label image, and a second label image. The first label image is an image obtained by performing region segmentation on the sample image based on the sample reference information, and the second label image is an image obtained by identifying the region where the target gas is located in the sample image. Based on the sample images, the sample reference information, and the first label image from the plurality of training samples, the region segmentation network to be trained is trained to obtain the pre-trained region segmentation network. The region labeling network to be trained is trained based on at least the sample images from the plurality of training samples and the second label image to obtain the pre-trained region labeling network; The pre-trained convolutional neural network is determined based on the pre-trained region segmentation network and the pre-trained region identifier network.

5. The gas leak detection method according to claim 4, characterized in that, The step of training the region labeling network to be trained based at least on the sample images and the second label images from the plurality of training samples to obtain the pre-trained region labeling network includes: Based on at least a portion of the sample images from the plurality of training samples and the second label image, the region labeling network to be trained is initially trained to obtain the initially trained region labeling network. The region labeling network trained initially is tested based on at least a portion of the first label images from the plurality of training samples to obtain network test results, wherein the number of samples used for testing is less than the number of samples used for initial training. Based on the network test results, the initially trained region labeling network is optimized to obtain the pre-trained region labeling network.

6. The gas leak detection method according to claim 1, characterized in that, The processed infrared image data includes multiple labeled infrared images. Determining the gas leak detection result based on the processed infrared image data includes: Determine the total number of the multiple infrared images after the identification; Determine the number of target images; Determine the region information of the image region corresponding to the target gas identified in the target image, wherein the region information is used to characterize the distribution of the image region corresponding to the target gas in the target image; Based on the total number, the number of target images, and the region information, the leakage probability of the target gas in the region to be detected is determined; The gas leak detection result is determined based on the leakage probability of the target gas.

7. The gas leak detection method according to claim 1, characterized in that, The gas leak detection result includes the leak probability of the target gas in the area to be detected, the reference information includes the gas flow index of the area to be detected, and the gas leak detection method further includes: Based on the leakage probability of the target gas, determine whether gas leakage protection is needed for the area to be detected; If it is determined that gas leakage protection is required for the area to be detected, a gas leakage protection strategy is generated based on the gas flow index, and the gas leakage protection strategy is executed. If it is determined that gas leakage protection for the area to be detected is not required, a preset safety indicator is added to the leakage probability of the target gas and the gas flow index, and stored.

8. The gas leak detection method according to claim 7, characterized in that, The gas leakage method further includes: In the case where the gas leak protection strategy includes outputting early warning information indicating gas leak in the area to be detected, after a preset time, new environmental change data collected by the environmental sensor and new infrared image data collected by the infrared imaging sensor are acquired, and the preset time is determined according to the area information of the area to be detected. Based on the new environmental change data and the new infrared image data, an updated training dataset is generated; Based on the updated training dataset, the pre-trained convolutional neural network is updated and trained to obtain an updated and trained convolutional neural network.

9. A gas leak detection system based on a convolutional neural network, characterized in that, include: An environmental sensor is installed in the area to be detected to collect environmental change data of the area to be detected. An infrared imaging sensor is installed in the area to be detected to collect infrared image data of the area to be detected. The detection device is connected to the environmental sensor and the infrared imaging sensor respectively, and is used to perform the gas leak detection method based on convolutional neural network as described in any one of claims 1 to 8.

Images