Leakage detection method and related apparatus, electronic device and storage medium

Abstract

The application discloses a leakage detection method and related device, electronic equipment and storage medium, wherein the leakage detection method comprises: based on the motion region in the to-be-detected image, a first region is extracted; and the first region is an image region suspected to have a leakage target, and the leakage target comprises at least one of a leakage gas and a spouting liquid; based on the first image feature of the first region, a detection result is detected; and the detection result comprises whether the leakage target exists in the first region; and based on the first region and the detection result, a second region is extracted from the motion region; and the second region is an image region suspected to have a seepage liquid; and whether the seepage liquid exists in the second region is identified. The above scheme can improve the accuracy of leakage detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a leakage detection method and related device, electronic equipment and storage medium. BACKGROUND

[0002] In real scenarios, due to natural aging, human damage and many other factors of containing bodies such as pipelines, tank cars and the like, gas, liquid and the like may be leaked. Therefore, accurate and reliable leakage detection is particularly important.

[0003] At present, leakage detection generally relies on detection equipment, and common detection equipment includes ultrasonic detectors, photoelectric water immersion sensors, water leakage controllers and the like. However, when the detection equipment performs leakage detection, it can only determine the position interval of the fault in the monitoring range of the equipment, and cannot provide the accurate position of the fault point in real time. Therefore, how to improve the accuracy of leakage detection has become a problem to be solved. SUMMARY

[0004] The technical problem solved by the present application is to provide a leakage detection method and related device, electronic equipment and storage medium, which can improve the accuracy of leakage detection.

[0005] In order to solve the above technical problem, the first aspect of the present application provides a leakage detection method, comprising: based on a motion region in a to-be-detected image, a first region is extracted; and the first region is an image region suspected to contain a leakage target, and the leakage target includes at least one of a leakage gas and a spouting liquid; based on a first image feature of the first region, a detection result is detected; and the detection result includes whether the leakage target exists in the first region; based on the first region and the detection result, a second region is extracted from the motion region; and the second region is an image region suspected to contain a spouting liquid; and whether the spouting liquid exists in the second region is identified.

[0006] In order to solve the above technical problem, the second aspect of the present application provides a leakage detection device, comprising a first extraction module, a feature detection module, a second extraction module and an image recognition module, the first extraction module is used for extracting a first region based on a motion region in a to-be-detected image; and the first region is an image region suspected to contain a leakage target, and the leakage target includes at least one of a leakage gas and a spouting liquid; the feature detection module is used for detecting a detection result based on a first image feature of the first region; and the detection result includes whether the leakage target exists in the first region; the second extraction module is used for extracting a second region from the motion region based on the first region and the detection result; and the second region is an image region suspected to contain a spouting liquid; and the image recognition module is used for identifying whether the spouting liquid exists in the second region.

[0007] To solve the above technical problems, the third aspect of the present application provides an electronic device, comprising a memory and a processor coupled with each other, the memory stores program instructions, and the processor is configured to execute the program instructions to implement the leakage detection method in the first aspect.

[0008] To solve the above technical problems, the fourth aspect of the present application provides a computer readable storage medium, which stores program instructions capable of being executed by a processor, and the program instructions are used to implement the leakage detection method in the first aspect.

[0009] The above scheme extracts a first region based on a motion region in a to-be-detected image; the first region is an image region suspected to contain a leakage target, and the leakage target includes at least one of a leakage gas and a spouting liquid; a detection result is detected based on a first image feature of the first region, and the detection result includes whether the leakage target exists in the first region. On this basis, a second region is extracted from the motion region based on the first region and the detection result, and the second region is an image region suspected to contain a leakage liquid, so as to identify whether the leakage liquid exists in the second region. On one hand, by using the first image feature of the first region, it is detected whether the leakage gas or the spouting liquid exists in the first region, which can simultaneously detect the leakage of the gas and the spouting liquid, and helps to improve the applicability of the leakage detection method. On the other hand, after the relatively easy-to-find leakage gas and spouting liquid are detected to obtain the detection result, the second region suspected to contain the leakage is extracted from the motion region based on the detection result, and it is identified whether the second region contains the relatively difficult-to-find leakage, so as to comprehensively detect various leakages from easy to difficult. Therefore, the accuracy of the leakage detection can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a flowchart of an embodiment of the leakage detection method of the present application;

[0011] Figure 2 is a process diagram of an embodiment of the leakage detection method of the present application;

[0012] Figure 3 is a framework diagram of an embodiment of the leakage detection device of the present application;

[0013] Figure 4 is a framework diagram of an embodiment of the electronic device of the present application;

[0014] Figure 5 is a framework diagram of an embodiment of the computer readable storage medium of the present application. DETAILED DESCRIPTION

[0015] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0016] In the following description, for the purposes of explanation, not limitation, specific details are set forth such as particular architectures, interfaces, techniques, etc. in order to provide a thorough understanding of the application.

[0017] The terms "system" and "network" are often used interchangeably herein. The term "and / or", as used herein, merely means one or all of the associated listed items. For example, A and / or B can mean A alone, A and B together, or B alone. In addition, the character " / " as used herein generally means that the associated objects before and after the " / " are in an "or" relationship. In addition, "multiple" herein means two or more than two.

[0018] Please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the leak detection method of the present application. Specifically, it can include the following steps:

[0019] Step S11: based on the motion region in the to-be-detected image, a first region is extracted.

[0020] In one implementation scenario, the to-be-detected image can be a captured image of a to-be-detected scene by a camera device. The to-be-detected scene can be set according to specific applications. For example, for a transportation application scenario, the to-be-detected scene can be a transportation pipeline, and the to-be-detected image can be an image captured by the camera device for the transportation pipeline. For a production application scenario, the to-be-detected scene can be a production workshop pipeline, and the to-be-detected image can be an image captured by the camera device for the production workshop pipeline. The to-be-detected scene can be selected according to actual conditions, which will not be described here. In addition, the camera device can also be set according to specific application scenarios. For example, for an outdoor application scenario, the camera device can be a waterproof camera. For an indoor application scenario, the camera device can be a normal network camera. The camera device can be selected according to actual conditions, which will not be specifically limited.

[0021] In one implementation scenario, the motion region can be obtained by background difference method. Of course, it can also be obtained by frame difference method, and it can also be obtained by optical flow method. The method for obtaining the motion region in the to-be-detected image can be selected according to actual conditions, which will not be specifically limited. In addition, it should be noted that in the embodiment of the present disclosure, the first region is an image region suspected to have a leakage target. More accurately, the first region is an image region suspected to have a leakage target in the to-be-detected image. It should be noted that the leakage target can include at least one of a leakage gas and a spouting liquid, that is, the leakage target is a leakage object with a large leakage amplitude.

[0022] In one implementation scenario, since the leakage gas and the spouting liquid usually have the characteristic of large leakage amplitude, the motion region can be directly taken as the first region.

[0023] In another implementation scenario, different from the foregoing first region extraction manner, in order to further improve the accuracy of the first region, a leakage target can be detected from the to-be-detected image first to obtain a third region. In this process, the leakage target can be detected from the to-be-detected image by using a deep learning network model. Exemplarily, the deep learning network model can include, but is not limited to, an SSD (Single Shot Multi-Box Detector), an RCNN (Region based Convolutional Neural Network), and the like. The deep learning network model can be selected according to actual conditions, and is not limited herein. On this basis, the intersection of the motion region and the third region is taken as the first region. In addition, in order to improve the accuracy of the deep learning network model, a plurality of sample images can be collected in advance, and the sample images are labeled with sample regions of leakage gas and spouting liquid. On this basis, the sample images can be detected for leakage gas and spouting liquid by using the deep learning network model to obtain predicted regions. Then, the network parameters of the deep learning network model are adjusted based on the difference between the sample regions and the predicted regions. Specifically, the specific manner of the difference measurement can refer to a loss function such as an intersection over union. The adjustment manner of the network parameters can refer to an optimization manner such as gradient descent, and is not described herein again. The foregoing manner determines the intersection of the motion region and the third region as the first region, which can avoid the detection error caused by the single manner of determining the first region as much as possible, and further improves the accuracy of determining the first region and the accuracy of the leakage detection.

[0024] In yet another implementation scenario, different from the foregoing two manners of extracting the first region from the reference motion region, the to-be-detected image can also be detected for leakage gas and spouting liquid to obtain a third region. Then, the third region is directly determined as the first region. It should be noted that the foregoing several manners of extracting the first region are only several possible implementation manners in actual application scenarios, and do not limit the specific extraction manner of the first region.

[0025] Step S12: detecting a detection result based on the first image feature of the first region.

[0026] In the embodiments of the present disclosure, the detection result includes whether there is a leakage target in the first region, and the leakage target includes at least one of leakage gas and spouting liquid. It should be noted that the first region can only have leakage gas, can only have spouting liquid, can have both leakage gas and spouting liquid, or can have neither leakage gas nor spouting liquid.

[0027] In an implementation scenario, the first image feature of the first region can be extracted by a feature extraction model. The feature extraction model can be a CNN (convolution neural network) or an FCN (Fully Convolutional Networks), which can be selected according to actual conditions and is not specifically limited here.

[0028] In an implementation scenario, the detection result can be analyzed based on the feature distance between the first image feature and the reference image features of various leakage categories. For example, the feature distance between the first image feature and the reference image features of various leakage categories can be calculated by the Euclidean distance or the Manhattan distance, which is not specifically limited here. On this basis, the detection result can be obtained based on the leakage category to which the reference image feature corresponding to the minimum feature distance belongs. The above method determines the leakage category with the minimum feature distance between the first image feature and the reference image features of the leakage category as the detection result, which helps to improve the accuracy of leakage detection.

[0029] In a specific implementation scenario, the various leakage categories can include, but are not limited to, presence of leakage gas, presence of spouting liquid, presence of leakage gas and spouting liquid, absence of leakage gas and spouting liquid, and the like, which is not limited here.

[0030] In a specific implementation scenario, in order to obtain the reference image features of the plurality of leakage categories, sample images of the plurality of leakage categories can be obtained first. The sample images can include images of different leakage categories under different illuminations and different scenes. After obtaining the sample images, sample image features of the sample images of the plurality of leakage categories can be extracted based on a feature extraction model. The feature extraction model can be a CNN (convolution neural network) or an FCN (Fully Convolutional Networks), which can be selected according to actual conditions and is not specifically limited here. On this basis, for each leakage category, the reference image feature of the leakage category can be obtained based on the feature statistics of the sample image features of the sample images belonging to the same leakage category. The reference image feature can be an average value of the sample image features of the sample images of the same leakage category, or a result of weighted processing of the sample image features of the sample images of the same leakage category, which can be selected according to actual conditions and is not specifically limited here. The above method obtains the reference image feature of the leakage category by performing feature statistics on the sample image features, which can make the reference image feature representative and help improve the accuracy of leakage detection.

[0031] In a specific implementation scenario, in order to improve the accuracy of the feature extraction model, sample images of different leakage categories can be collected in advance, and the feature extraction model can be used to extract features of the sample images of different leakage categories respectively to obtain sample image features of each sample image. On this basis, for each sample image, other sample images belonging to the same leakage category as the sample image are obtained as positive example images, and sample images belonging to different leakage categories from the sample image are obtained as negative example images, and a first distance between the sample image features of the sample image and the sample image features of the positive example images thereof is obtained, a second distance between the sample image features of the sample image and the sample image features of the negative example images thereof is obtained, and then based on the first distance and the second distance, a network loss of the feature extraction model is obtained. It should be noted that the network loss is positively correlated with the first distance and negatively correlated with the second distance. On this basis, the network parameters of the feature extraction model can be adjusted based on the network loss. The specific adjustment process of the network parameters can refer to optimization methods such as gradient descent, which will not be described here.

[0032] In a specific implementation scenario, after the reference image features of various leakage categories are extracted based on the feature extraction model, and the feature distances between the first image features and the reference image features of various leakage categories are calculated, if the leakage category to which the reference image feature corresponding to the minimum feature distance belongs is the presence of leakage gas, it can be determined that the detection result includes the presence of leakage gas in the first region, if the leakage category to which the reference image feature corresponding to the minimum feature distance belongs is the presence of spouting liquid, it can be determined that the detection result includes the presence of spouting liquid in the first region, and if the leakage category to which the reference image feature corresponding to the minimum feature distance belongs is the absence of leakage gas and spouting liquid, it is determined that the detection result includes the absence of leakage gas and spouting liquid in the first region. Other cases can be similarly deduced, which will not be exemplified one by one here.

[0033] In an implementation scenario, if it is determined that the detection result includes the presence of a leakage target, the color of the leakage target can also be identified. For example, the color classification model can be used to identify the first region to obtain the color corresponding to the leakage target. In addition, the color classification model can refer to the technical details of the multi-classification neural network, which will not be described here.

[0034] In an implementation scenario, if it is determined that the detection result includes the presence of a leakage target, the bounding rectangle of the first region can also be obtained, and based on the bounding rectangle, the leakage position can be obtained. In this case, the detection result can also include the leakage position. For example, the top point position, center position, etc. of the aforementioned bounding rectangle can be used as the leakage position, which is not limited here.

[0035] Step S13: Extracting a second region from the motion region based on the first region and the detection result.

[0036] In the embodiments of the present disclosure, the second region is an image region in which the leaked liquid is suspected to exist, and more specifically, the second region is an image region in which the leaked liquid is suspected to exist in the to-be-detected image.

[0037] In one implementation scenario, in response to the detection result including the existence of the leakage target in the first region, the first region can be excluded from the motion region to obtain the second region. In the above manner, in the case where it is determined according to the detection result that the leakage target exists in the first region, the first region is excluded from the motion region to obtain the second region, so that the second region can exist only in the suspected leaked liquid as much as possible, which helps to improve the accuracy of subsequent detection.

[0038] In another implementation scenario, in response to the detection result including the non-existence of the leakage target in the first region, the motion region can be directly taken as the second region. In the above manner, in the case where it is determined according to the detection result that the leakage target does not exist in the first region, the motion region is directly taken as the second region, so that the case of missing leakage detection due to the existence of multiple leakage categories can be avoided as much as possible, which helps to improve the accuracy of subsequent detection.

[0039] In one implementation scenario, in a real scene, the captured image of the to-be-detected scene by the camera device can be sequentially taken as the to-be-detected image for leakage detection. After the second region is obtained, in order to verify whether the second region in the current to-be-detected image is reliable, the second region extracted from the current to-be-detected image can be taken as a target region, and the second region extracted from the captured image before the current to-be-detected image can be taken as a reference region. It should be noted that the captured image before the current to-be-detected image can be the previous image of the to-be-detected image, or can be an image spaced several frames from the to-be-detected image. The captured image before the current to-be-detected image can be selected according to actual conditions, which is not limited here. In addition, the extraction manner of the second region in the captured image before the current to-be-detected image can refer to the extraction manner of the second region in the current to-be-detected image in the foregoing disclosed embodiments, which will not be described here. On this basis, the target region and the reference region are compared and analyzed to obtain an analysis result, and the analysis result includes whether the second region in the current to-be-detected image is reliable. In the above manner, by obtaining the captured image before the current to-be-detected image and extracting the second region, and taking the second region as the reference region, the case of detection error caused by a single image can be avoided as much as possible, thereby improving the accuracy of leakage detection.

[0040] In a specific implementation scenario, comparative analysis based on a target region and a reference region can be performed. This comparison can be conducted across several dimensions to obtain comparison values ​​for each dimension. These dimensions include at least one of the following: edge direction histogram, area, and length. It should be noted that these comparison values ​​across several dimensions are used to measure the similarity between the target region and the reference region. Therefore, the comparison values ​​for each dimension can be determined by calculating the similarity or rate of change between the target region and the reference region along several identical dimensions. For example, the edge direction histogram can be determined by calculating the similarity of the edge direction histograms, using the following formula:

[0041]

[0042] in, This is the histogram of the edge directions of the motion region at time t. for Histogram of the edge orientation of the motion region at any given moment; N represents the number of histograms. A correlation coefficient closer to 1 indicates a higher similarity in the edge direction histograms of the two images. Area and length can be represented by the rate of change of area and the rate of change of length, respectively. The formulas for calculating the rate of change of area S and the rate of change of length L are as follows:

[0043]

[0044]

[0045] in, Let be the area of ​​the region of motion at time t. for The area of ​​the region of constant motion Let be the boundary length of the region of motion at time t. for The boundary length of the moving area at any given time. Based on the comparison values ​​obtained in each dimension, further analysis is performed to obtain the analysis results. When the comparison values ​​in each dimension all meet preset conditions, the second region in the current image under test is considered reliable. The preset conditions can be set to the comparison values ​​in each dimension being within a certain threshold range. When the comparison values ​​in any dimension do not meet the preset conditions, the second region in the current image under test is considered unreliable. It should be noted that when the second region in the current image under test is reliable, it indicates that there is suspected leakage liquid in the second region. This method, by comparing the target area and the reference area in several dimensions to determine their similarity in several dimensions, and thus obtaining the analysis results, helps to improve the comprehensiveness and accuracy of leak detection.

[0046] Step S14: Identify whether there is any leakage liquid in the second area.

[0047] In an implementation scenario, as described above, in order to improve the efficiency of leakage detection, the second region can be analyzed for trustworthiness first, and then, in response to the analysis result including that the second region in the current image to be tested is trustworthy, the step of identifying whether there is a leakage liquid in the second region is performed; otherwise, in response to the analysis result including that the second region in the current image to be tested is untrustworthy, it can be directly determined that there is no leakage liquid in the current image to be tested. Through the above-mentioned manner, the trustworthiness of the second region is verified, and the use efficiency of the leakage detection is further improved, and thus the use experience is improved.

[0048] In an implementation scenario, the leakage categories of the reference images in the reference image library can include presence of leakage gas, presence of gushing liquid, presence of leakage liquid, and absence of all leakages. Of course, the leakage categories of the reference images in the reference image library can also include only presence of leakage liquid and absence of leakage liquid. The leakage categories of the reference images in the reference image library can be selected according to actual conditions, which are not limited herein.

[0049] In an implementation scenario, in order to improve the accuracy of determining whether there is a leakage liquid in the second region, the trained leakage liquid identification model can be used to identify the second region to determine whether there is a leakage liquid in the second region. Exemplarily, the leakage liquid identification model can include, but is not limited to, SSD (Single Shot Multi-Box Detector), RCNN (Region based Convolutional Neural Network, region-based convolutional neural network), etc. The leakage liquid identification model can be selected according to actual conditions, which are not limited herein.

[0050] In a specific implementation scenario, in order to improve the accuracy of the leakage liquid identification model, sample images of different image categories can be collected in advance, the image categories representing whether there is a leakage liquid in the corresponding target image, and the sample images of different image categories are identified based on the leakage liquid identification model to obtain the identification results of the sample images. Based on the identification results of the sample images, the network parameters of the leakage liquid identification model are adjusted. The specific adjustment process of the network parameters can refer to optimization methods such as gradient descent, which are not described herein again.

[0051] In one implementation scenario, whether the second region has the leaking liquid can be determined based on the image category of the second region and the at least one target image, and the image category represents whether the leaking liquid exists in the corresponding target image. It should be noted that the target image is part of the reference image library. After the at least one target image is selected from the reference image library, the image category of the at least one selected target image can be further used to determine whether the second region has the leaking liquid. For example, if the image category of the at least one target image indicates that the target image with the leaking liquid exists most frequently, it can be determined that the second region has the leaking liquid. Otherwise, it can be directly determined that the second region does not have the leaking liquid.

[0052] In another implementation scenario, in order to improve the accuracy of determining whether the second region has the leaking liquid, the similarity of the second region to each reference image in the reference image library can be obtained first. Specifically, the similarity of the second region to each reference image can be determined based on the image features of the second region and the image features of each reference image. For example, the image features of the second region and each reference image in the reference image library can be extracted based on a feature extraction model, and then the image features of the second region and each reference image are used to calculate the similarity. The similarity of the second region to each reference image in the reference image library can be determined. The calculation of the similarity can be selected according to the actual situation, which is not limited here. After obtaining the similarity, at least one target image can be determined from the reference image library based on the similarity. For example, Y (such as 1, 10, 15, etc.) images with a similarity greater than X (such as 0.8, 0.9, etc.) can be selected as target images. Based on the image category of each target image in the at least one target image, whether the second region has the leaking liquid can be determined. The above method can help to improve the influence of the to-be-detected image on the leakage detection result due to the external environment, and thus improve the accuracy of the leakage detection.

[0053] In one specific implementation scenario, the image category can include, but is not limited to, the presence of the leaking liquid, the absence of the leaking liquid, and the like, which is not limited here.

[0054] In a specific implementation scenario, after obtaining the image categories of the target images, the first quantity of the target images of which the image categories represent the presence of the leaked liquid can be counted, and the quantities of the target images of which the image categories represent different images can also be counted respectively. It should be noted that when all the image categories are the absence of the leaked liquid, i.e., the first quantity is 0, it can be directly determined that the second region is free of the leaked liquid. After determining the first quantity, the recognition result (i.e., whether the second region is free of the leaked liquid) can be obtained based on the proportion of the quantity of the first quantity in the at least one selected target image. Specifically, the proportion of the quantity of the first quantity in the at least one selected target image and the proportions of the quantities of other various image categories in the at least one selected target image can be calculated. When the proportion of the quantity of the image category representing the presence of the leaked liquid is the highest among all the proportions, it can be determined that the second region is free of the leaked liquid. Otherwise, it is determined that the second region is free of the leaked liquid. In the above manner, the quantity of the target image of the leaked liquid is counted, and whether the second region is free of the leaked liquid is determined based on the proportion of the quantity, so as to reduce the misjudgment caused by the external environment as much as possible, and further improve the accuracy of the leakage detection.

[0055] In an implementation scenario, similar to the foregoing determination of the detection result including at least one of the presence of the leaked gas and the ejection of the liquid, if it is determined that the second region is free of the leaked liquid, the color of the leaked liquid in the second region can also be identified. For example, the color of the leaked liquid can be obtained by identifying the second region through a color classification model. In addition, the color classification model can refer to the technical details of the multi-classification neural network, which will not be described here.

[0056] In an implementation scenario, if it is determined that the second region is free of the leaked liquid, the bounding rectangle of the second region can also be obtained, and the leakage position can be obtained based on the bounding rectangle. In this case, the recognition result can also include the leakage position. For example, the top point position, the center position, and the like of the foregoing bounding rectangle can be used as the leakage position, which is not specifically limited here.

[0057] Please refer to Figure 2 , Figure 2 is a process schematic diagram of an embodiment of the leakage detection method of the present application, as shown in Figure 2As shown, the moving area in the to-be-detected image can be obtained by background difference method, and the target detection network is used for detecting the to-be-detected image to obtain the third area. It should be noted that the target detection network can be trained by using the leakage gas training data, the spouting liquid training data and the background image training data before use, and the parameters of the target detection network can be adjusted. On this basis, the intersection of the moving area and the third area is taken as the first area. After obtaining the first area, the first image feature of the first area is extracted, and the feature distance between the first image feature and the reference image feature of each leakage category is obtained. The leakage category corresponding to the smallest feature distance is determined as the detection result including the leakage category existing in the first area. If at least one of the leakage gas and the spouting liquid exists in the first area, the bounding rectangle of the first area can also be output, and the specific position of the at least one of the leakage gas and the spouting liquid existing in the first area can be obtained. The color recognition model can also be used to identify the color in the first area, and the color of the leakage target existing in the first area can be determined. In order to obtain more accurate detection result, the to-be-detected image can also be detected. According to the detection result including whether the leakage target exists in the first area, the acquisition mode of the second area can be determined. That is, if the detection result includes at least one of the leakage gas and the spouting liquid existing in the first area, the first area is removed from the moving area to obtain the second area; if the detection result includes that the leakage gas and the spouting liquid do not exist in the first area, the moving area is taken as the second area. After determining the second area, the N (N is a positive integer) reference images with the highest similarity between the second area and each reference image in the reference image library can be obtained. The leakage category with the highest occurrence frequency in the N reference images is selected as the image category of the target image. If the image category of the target image is the seepage liquid, the seepage liquid exists in the second area. If the seepage liquid exists in the second area, the bounding rectangle of the second area can be output, and the specific position of the seepage liquid existing in the second area can be obtained. The color recognition model can also be used to identify the color in the second area, and the color of the seepage liquid existing in the second area can be determined. It should be noted that the detection result of the to-be-detected image can include one leakage category, for example, the leakage gas. The detection result can also include two leakage categories, for example, the spouting liquid and the seepage liquid. The detection result of the to-be-detected image can be determined according to actual conditions, which is not limited herein.

[0058] The scheme extracts a first region based on a motion region in a to-be-detected image, and the first region is an image region in which a leakage target is suspected to exist, and the leakage target includes at least one of a leakage gas and a spouting liquid. Then, a detection result is detected based on a first image feature of the first region, and the detection result includes whether the leakage target exists in the first region. Then, a second region is extracted from the motion region based on the first region and the detection result, and the second region is an image region in which a leakage liquid is suspected to exist. Then, whether the leakage liquid exists in the second region is identified. In this way, whether the leakage gas or the spouting liquid exists in the first region is detected based on the first image feature of the first region, so that the gas and the spouting liquid can be detected simultaneously, and the applicability of the leakage detection method is improved. In addition, after the relatively easy-to-find leakage gas and the spouting liquid are detected to obtain the detection result, the second region in which the leakage is suspected to exist is extracted from the motion region based on the detection result, and whether the relatively difficult-to-find leakage exists in the second region is determined by identifying the second region. Therefore, various leakages can be comprehensively detected from easy to difficult. Therefore, the accuracy of the leakage detection can be improved.

[0059] Please refer to Figure 3 , Figure 3 is a schematic diagram of an embodiment of a leakage detection device. The leakage detection device 30 includes a first extraction module 31, a feature detection module 32, a second extraction module 33, and an image recognition module 34. The first extraction module 31 is configured to extract a first region based on a motion region in a to-be-detected image, and the first region is an image region in which a leakage target is suspected to exist, and the leakage target includes at least one of a leakage gas and a spouting liquid. The feature detection module 32 is configured to detect a detection result based on a first image feature of the first region, and the detection result includes whether the leakage target exists in the first region. The second extraction module 33 is configured to extract a second region from the motion region based on the first region and the detection result, and the second region is an image region in which a leakage liquid is suspected to exist. The image recognition module 34 is configured to identify whether the leakage liquid exists in the second region.

[0060] The scheme detects whether the leakage gas or the spouting liquid exists in the first region based on the first image feature of the first region, so that the gas and the spouting liquid can be detected simultaneously, and the applicability of the leakage detection method is improved. In addition, after the relatively easy-to-find leakage gas and the spouting liquid are detected to obtain the detection result, the second region in which the leakage is suspected to exist is extracted from the motion region based on the detection result, and whether the relatively difficult-to-find leakage exists in the second region is determined by identifying the second region. Therefore, various leakages can be comprehensively detected from easy to difficult. Therefore, the accuracy of the leakage detection can be improved.

[0061] In some disclosed embodiments, the image recognition module 34 comprises a recognition sub-module configured to recognize the second region by using the trained liquid leakage recognition model to determine whether the second region contains liquid leakage; the image recognition module 34 further comprises a determination sub-module configured to determine whether the second region contains liquid leakage based on the second region and the image category of the at least one target image, wherein the image category represents whether the corresponding target image contains liquid leakage.

[0062] Therefore, by determining whether the second region contains liquid leakage through the liquid leakage model or through the image category of the target image, the accuracy of the recognition result of whether the second region contains liquid leakage under different circumstances is improved, and thus the accuracy of the leakage detection is improved.

[0063] In some disclosed embodiments, the determination sub-module comprises a first determination unit configured to determine the at least one target image from the reference image library based on the similarity of the second region to each reference image in the reference image library; and the determination sub-module comprises a second determination unit configured to determine whether the second region contains liquid leakage based on the image category of each target image in the at least one target image.

[0064] Therefore, by obtaining the similarity of the second region to each reference image in the reference image library, the target image is determined, and then whether the second region contains liquid leakage is determined, which helps to reduce the influence of the external environment on the leakage detection result of the to-be-detected image, and thus the accuracy of the leakage detection is improved.

[0065] In some disclosed embodiments, the camera device sequentially performs leakage detection on the captured images of the to-be-detected scene as to-be-detected images, the leakage detection device 30 comprises a region extraction module configured to extract the second region in the current to-be-detected image as a target region, and extract the second region in the captured image before the current to-be-detected image as a reference region; the leakage detection device 30 further comprises a comparison and analysis module configured to perform comparison and analysis based on the target region and the reference region to obtain an analysis result, wherein the analysis result comprises whether the second region in the current to-be-detected image is reliable; and the image recognition module 34 further comprises an execution sub-unit configured to execute the step of recognizing whether the second region contains liquid leakage in response to the analysis result comprising that the second region in the current to-be-detected image is reliable.

[0066] Therefore, by obtaining the captured image before the current to-be-detected image and extracting the second region, and taking the second region as the reference region, the situation of detection error caused by a single image is avoided as much as possible, and thus the accuracy of the leakage detection is improved.

[0067] In some disclosed embodiments, the comparison analysis module comprises a comparison sub-module, which is configured to compare the target region and the reference region in a plurality of dimensions respectively to obtain comparison values of each dimension, and the plurality of dimensions comprise at least one of the following: an edge direction histogram, an area, and a length. The comparison analysis module further comprises an analysis sub-module, which is configured to analyze based on the comparison values of each dimension to obtain an analysis result.

[0068] Therefore, by comparing the target region and the reference region in a plurality of dimensions respectively, determining the similarity of the two in the plurality of dimensions, and then obtaining an analysis result, the comprehensiveness and accuracy of the leakage detection can be improved.

[0069] In some disclosed embodiments, the leakage detection device 30 comprises an image acquisition module, which is configured to acquire reference image features of a plurality of leakage categories. The plurality of leakage categories comprise: presence of a leakage gas, presence of a spouting liquid, absence of a leakage gas and spouting liquid. The feature detection module 32 comprises an analysis sub-module, which is configured to analyze based on feature distances between the first image feature and the reference image features of various leakage categories to obtain a detection result.

[0070] Therefore, by determining the leakage category with the smallest feature distance between the first image feature and the reference image features of the leakage category as the detection result, the accuracy of the leakage detection can be improved.

[0071] In some disclosed embodiments, the first image feature is extracted by a feature extraction model from the first region. The image acquisition module comprises an extraction sub-module, which is configured to extract sample image features of sample images of a plurality of leakage categories based on the feature extraction model respectively. The image acquisition module further comprises a statistical sub-module, which is configured to, for each leakage category, perform feature statistics based on the sample image features of the sample images belonging to the same leakage category to obtain the reference image features of the leakage category.

[0072] Therefore, by performing feature statistics on the sample image features to obtain the reference image features of the leakage category, the reference image features can be representative, and the accuracy of the leakage detection can be improved.

[0073] In some disclosed embodiments, the first extraction module comprises a detection sub-module, which is configured to detect a leakage target in the to-be-detected image to obtain a third region. The first extraction module further comprises a determination sub-module, which is configured to determine an intersection of the motion region and the third region as the first region.

[0074] Therefore, by determining the intersection of the motion region and the third region as the first region, the detection error caused by determining the first region in a single way is avoided as much as possible, the accuracy of determining the first region is improved, and the accuracy of the leakage detection is further improved.

[0075] In some disclosed embodiments, the second extraction module 33 comprises a rejection sub-module configured to, in response to the detection result comprising that there is a leakage target in the first region, reject the first region from the motion region to obtain the second region; and the second extraction module 33 further comprises a determination sub-module configured to, in response to the detection result comprising that there is no leakage target in the first region, take the motion region as the second region.

[0076] Therefore, by verifying whether there is a leakage target in the first region and then determining the second region, the situation that the leakage detection is wrong due to the existence of multiple leakage categories is avoided as much as possible, and the accuracy of the leakage detection is further improved.

[0077] Please refer to Figure 4 , Figure 4 is a schematic diagram of a framework of an embodiment of the electronic device. The electronic device 40 comprises a memory 41 and a processor 42 coupled with each other. The memory 41 stores program instructions. The processor 42 is configured to execute the program instructions to implement the steps in any of the above leakage detection method embodiments. Specifically, the electronic device 40 can include but is not limited to a desktop computer, a notebook computer, a server, a mobile phone, a tablet computer, and the like, which are not limited herein.

[0078] Specifically, the processor 42 is configured to control itself and the memory 41 to implement the steps in any of the above leakage detection method embodiments. The processor 42 can also be referred to as a CPU (Central Processing Unit). The processor 42 can be an integrated circuit chip with processing capability. The processor 42 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 42 can be implemented by an integrated circuit chip jointly.

[0079] The above scheme, on the one hand, detects whether there is a leakage gas or a spouting liquid in the first region through the first image feature of the first region, can simultaneously detect the leakage of the gas and the spouting liquid, and helps to improve the applicability of the leakage detection method. On the other hand, after the detection of the relatively easy-to-find leakage gas and spouting liquid obtains a detection result, the second region suspected to have a leakage is extracted from the motion region based on the detection result, and whether the second region has a relatively difficult-to-find leakage is determined by recognizing the second region, so that various leakages can be comprehensively detected from easy to difficult. Therefore, the accuracy of the leakage detection can be improved.

[0080] Please refer to Figure 5 , Figure 5 is a schematic diagram of the framework of an embodiment of the computer readable storage medium of the present application. The computer readable storage medium 50 stores program instructions 51 capable of being run by the processor, and the program instructions 51 are used to implement the steps in any of the above leakage detection method embodiments.

[0081] The above scheme, on the one hand, detects whether there is a leakage gas or a spouting liquid in the first region through the first image feature of the first region, can simultaneously detect the leakage of the gas and the spouting liquid, and helps to improve the applicability of the leakage detection method. On the other hand, after the detection of the relatively easy-to-find leakage gas and spouting liquid obtains a detection result, the second region suspected to have a leakage is extracted from the motion region based on the detection result, and whether the second region has a relatively difficult-to-find leakage is determined by recognizing the second region, so that various leakages can be comprehensively detected from easy to difficult. Therefore, the accuracy of the leakage detection can be improved.

[0082] In some embodiments, the device provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0083] The above description of various embodiments tends to emphasize the differences between various embodiments, and the same or similar parts can be mutually referred to. For the sake of brevity, it will not be repeated here.

[0084] In several embodiments provided in the present application, it should be understood that the disclosed methods and apparatuses can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely an example, and the division of the modules or units can be different, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, and can be electrical, mechanical or other forms.

[0085] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0086] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present separately, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0087] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0088] If the technical solution of the present application involves personal information, the product applying the technical solution of the present application has clearly informed the personal information processing rules before processing the personal information and obtained the personal independent consent. If the technical solution of the present application involves sensitive personal information, the product applying the technical solution of the present application has obtained the personal independent consent before processing the sensitive personal information and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent sign is set to inform that it has entered the personal information collection range and will collect personal information. If the individual voluntarily enters the collection range, it is considered to agree to collect personal information. Or on the device for processing personal information, through the pop-up information or by asking the individual to upload his / her personal information, the individual's authorization is obtained under the condition of using obvious signs / information to inform the personal information processing rules. The personal information processing rules can include personal information processor, personal information processing purpose, processing method and type of processed personal information, etc.

Claims

1. A leak detection method, characterized in that, include: Based on the motion region in the image to be tested, a first region is extracted; wherein, the first region is an image region suspected of containing a leak target, and the leak target includes at least one of leaking gas and ejected liquid; Based on the first image features of the first region, a detection result is obtained; wherein, the detection result includes whether the leakage target exists in the first region; Based on the first region and the detection result, a second region is extracted from the motion region; wherein, the second region is an image region suspected of having leaking liquid, in response to the detection result including the presence of a leaking target in the first region, the first region is removed from the motion region to obtain the second region, in response to the detection result including the absence of a leaking target in the first region, the motion region is used as the second region; Identify whether there is any leaking liquid in the second area.

2. The method according to claim 1, characterized in that, The step of identifying whether there is leaking liquid in the second area includes: Using the trained leakage liquid identification model, the second region is identified to determine whether leakage liquid exists in the second region; or Based on the image category of the second region and at least one target image, it is determined whether there is leaking liquid in the second region; wherein, the image category indicates whether there is leaking liquid in the corresponding target image.

3. The method according to claim 2, characterized in that, The determination of whether there is leaking liquid in the second region based on the image category of the second region and at least one target image includes: Based on the similarity between the second region and each reference image in the reference image library, at least one target image is determined from the reference image library; Based on the image category of each of the at least one target image, it is determined whether there is leaking liquid in the second region.

4. The method according to claim 1, characterized in that, The image captured by the camera device of the scene under test is sequentially used as the image under test for leak detection. After extracting the second region from the motion region based on the first region and the detection result, and before identifying whether there is leaking liquid in the second region, the method further includes: The second region extracted from the current image to be tested is taken as the target region, and the second region extracted from the image taken before the current image to be tested is taken as the reference region. The analysis results are obtained by comparing the target region with the reference region; wherein, the analysis results include whether the second region in the current image to be tested is reliable; The step of identifying whether there is leaking liquid in the second area includes: In response to the analysis result including the belief that the second region in the current image to be tested is reliable, the step of identifying whether there is leakage liquid in the second region is performed.

5. The method according to claim 4, characterized in that, The comparison analysis based on the target region and the reference region to obtain the analysis results includes: The target region and the reference region are compared in several dimensions to obtain the comparison value of each dimension; wherein, the several dimensions include at least one of the following: edge direction histogram, area, and length; The analysis results are obtained by analyzing the comparison values ​​of each of the aforementioned dimensions.

6. The method according to claim 1, characterized in that, Before obtaining the detection result based on the first image features of the first region, the method further includes: Acquire reference image features for several leakage categories; wherein, the several leakage categories include: presence of leaking gas, presence of erupting liquid, absence of leaking gas and erupting liquid; The detection result obtained based on the first image features of the first region includes: The detection results are obtained by analyzing the feature distances between the first image features and reference image features for each of the various leakage categories.

7. The method according to claim 6, characterized in that, The first image feature is extracted from the first region by a feature extraction model. The step of obtaining reference image features for several leakage categories includes: Based on the feature extraction model, sample image features of sample images of the several leakage categories are extracted respectively; For each type of leakage, feature statistics are performed based on the sample image features of sample images belonging to the same leakage category to obtain the reference image features of the leakage category.

8. The method according to claim 1, characterized in that, The first region is extracted based on the motion region in the image to be tested, including: The leak target is detected in the image to be tested, and a third region is obtained; The intersection of the motion region and the third region is taken as the first region.

9. A leak detection device, characterized in that, include: The first extraction module is used to extract a first region based on the motion region in the image to be tested; wherein the first region is an image region suspected of containing a leak target, and the leak target includes at least one of leaking gas and ejected liquid; The feature detection module is used to detect and obtain a detection result based on the first image features of the first region; wherein, the detection result includes whether the leakage target exists in the first region; The second extraction module is used to extract a second region from the motion region based on the first region and the detection result; wherein the second region is an image region suspected of having leaking liquid, and in response to the detection result including the presence of a leaking target in the first region, the first region is removed from the motion region to obtain the second region, and in response to the detection result including the absence of a leaking target in the first region, the motion region is used as the second region; An image recognition module is used to identify whether there is leaking liquid in the second area.

10. An electronic device, characterized in that, The method includes a memory and a processor coupled to each other, the memory storing program instructions, and the processor executing the program instructions to implement the leakage detection method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The system stores program instructions that can be executed by a processor, the program instructions being used to implement the leakage detection method according to any one of claims 1 to 8.

Images