Leak Detection Behavior Compliance Audit Method, Device, Electronic Device and Storage Medium

Abstract

The present invention provides a method, device, electronic device and computer storage medium for leak detection compliance auditing. A camera is provided on the probe handle of an FID detector, and multiple leak detection images of a target sealing point during leak detection are obtained through the camera. The probe front detection frame and the sealing point component detection frame in the image are obtained based on computer vision technology, and the overlap is calculated. The compliance of the image is judged based on the overlap. Finally, the compliance of the leak detection process is determined based on the compliance status of each image. This solves the problems of lack of supervision measures for the leak detection process and difficulty in tracing fraudulent behavior in the prior art.

Description

Technical Field

[0001] The present invention belongs to the field of LDAR technology, and in particular relates to a leakage detection behavior compliance audit method, device, electronic device and storage medium. Background Art

[0002] Leak Detection and Repair (LDAR), as an equipment integrity management technology, can achieve source control and reduction of VOCs fugitive emissions from industrial enterprises. LDAR efforts in my country have been ongoing since 2013 and have become a mandatory regulatory requirement for petrochemical and chemical companies to control VOC emissions from equipment and components. LDAR is currently a key tool for industrial enterprises to reduce fugitive leaks and control odor pollution.

[0003] Due to a lack of supporting regulatory and enforcement technologies, the current quality of LDAR work fails to support decision-making at industrial enterprises. Years of LDAR work by enterprises have been plagued by issues such as incomplete inspection point records, non-standard inspection processes, and poor data integrity. Statistics show leakage rates ranging from 0.02% to 0.2%, while the actual leakage rate is 5-10 times higher. Traditional LDAR audits struggle to detect non-compliance issues hidden within the inspection process, deviating from the original purpose of LDAR work and rendering it ineffective. Summary of the Invention

[0004] Based on this, in order to address the above technical issues, a leakage detection behavior compliance audit method, device, electronic device and storage medium are provided.

[0005] The technical solution adopted in the present invention is as follows:

[0006] As a first aspect of the present invention, a method for compliance auditing of leak detection behavior is provided, comprising:

[0007] S101, acquiring multiple leak detection images during a leak detection process of a target sealing point through a camera, wherein the camera is provided on a probe handle of an FID detector;

[0008] S102, inputting each leak detection image into a pre-trained deep learning neural network model to generate a probe front detection frame and a sealing point component detection frame in each leak detection image;

[0009] S103, determining the overlapping area between the probe front detection frame and the sealing point assembly detection frame in each leak detection image, and dividing the overlapping area by the area of the probe front detection frame to obtain the overlap degree corresponding to each leak detection image;

[0010] S104. Compare the overlap of each leak detection image with the overlap threshold. If the overlap is greater than the overlap threshold, the detection result of the corresponding leak detection image is compliant; otherwise, it is questionable. If the detection result of each leak detection image meets the preset conditions, the leak detection process is compliant; otherwise, it is questionable.

[0011] As a second aspect of the present invention, a leakage detection behavior compliance audit device is provided, comprising:

[0012] A leak detection image acquisition module is used to acquire multiple leak detection images during the leak detection process of the target sealing point through a camera, wherein the camera is provided on the probe handle of the FID detector;

[0013] A detection frame determination module is used to input each leak detection image into a pre-trained deep learning neural network model to generate a probe front detection frame and a sealing point component detection frame in each leak detection image;

[0014] An overlap calculation module is used to determine the overlapping area between the probe front detection frame and the sealing point assembly detection frame in each leak detection image, and divide the overlapping area by the area of the probe front detection frame to obtain the overlap corresponding to each leak detection image;

[0015] The audit module is used to compare the overlap of each leakage detection image with the overlap threshold. If the overlap is greater than the overlap threshold, the detection result of the corresponding leakage detection image is compliant; otherwise, it is questionable. If the detection result of each leakage detection image meets the preset conditions, the leakage detection process is compliant; otherwise, it is questionable.

[0016] As a third aspect of the present invention, an electronic device is provided, comprising a storage module, wherein the storage module comprises instructions loaded and executed by a processor, wherein when the instructions are executed, the processor executes a leakage detection behavior compliance audit method according to the first aspect.

[0017] As a fourth aspect of the present invention, a computer-readable storage medium is provided, which stores one or more programs. When the one or more programs are executed by a processor, they implement a leakage detection behavior compliance audit method according to the first aspect.

[0018] The present invention sets a camera on the probe handle of the FID detector, and obtains multiple leak detection pictures of the target sealing point during leak detection through the camera. Based on computer vision technology, the probe front detection frame and the sealing point component detection frame in the picture are obtained, and the overlap is calculated. The compliance of the picture is judged according to the overlap. Finally, according to the compliance of each picture, it is determined whether the leakage detection process is compliant, which solves the problems of lack of supervision measures for the leakage detection process and difficulty in tracing fraudulent behavior in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments:

[0020] Figure 1 A flowchart of a leak detection behavior compliance audit method provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of a leak detection behavior compliance audit device provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of a detection angle according to an embodiment of the present invention;

[0024] Figure 5 Schematic diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will illustrate the implementation of the present invention in conjunction with the drawings in the specification. It should be noted that the implementation methods involved in this specification are not exhaustive and do not represent the only implementation methods of the present invention. The following corresponding embodiments are only for the purpose of clearly illustrating the invention content of the patent of this invention and are not intended to limit its implementation methods. For ordinary technicians in this field, different forms of changes and modifications can be made based on the description of this embodiment. Any obvious changes or modifications that belong to the technical concept and invention content of the present invention are also within the scope of protection of the present invention.

[0026] like Figure 1 As shown, the embodiment of the present application provides a method for compliance review of leak detection behavior, the specific process of which is as follows:

[0027] S101. Acquire multiple leakage detection images during a leakage detection process of a target sealing point through a camera.

[0028] The camera is located on the probe handle of the FID detector. In actual applications, LDAR inspectors use a handheld FID detector to detect leaks at a certain sealing point component (where a leak may occur). To obtain a high-quality inspection video, the FID detector's moving speed can be controlled below 7 cm / s. At the same time, the probe's inspection angle must be within the range of a 60° angle between the normal N and the cross-section of the sealing point inspection surface. Inspection angles exceeding 90° should be avoided as much as possible (there is a high probability that the sealing point component will be seriously missing). See [Note: The following sentences appear to be unrelated and should be omitted.] Figure 4 .

[0029] During leak detection at sealing points, video is captured by a camera and uploaded to a cloud platform in real time or offline via a wireless network (4G / 5G / WiFi). To ensure the stability of video transmission, videos are uploaded in 10-minute increments. The cloud platform determines the time period for leak detection at the target sealing point and extracts multiple leak detection images within that time period from the video using keyframes or timestamps. In actual applications, the app in the handheld controller that comes with the FID detector will record information such as the test start and end time, the type of sealing point component, the sealing point size, the device, the area, the current leakage concentration, etc., so that the above time period can be determined based on the test start and end time.

[0030] When using the keyframe method, the FFmpeg open-source computer program is used to locate and extract keyframe data from the detection video. When using the timestamp method, images are extracted and saved frame by frame from the video based on the OpenCV2 computer vision algorithm library, enabling targeted extraction based on timestamp information. In practical applications, the number of images extracted for a single detection and the timeline distribution of extracted images can be adjusted based on detection requirements, achieving the conversion from detection video to detection images. Image formats include BMP and jpg. Testing has shown that the lossless BMP format can improve the accuracy of the image recognition model by approximately 6%, but the recognition time for each BMP image is approximately 30 times that of a jpg image. For cost-effectiveness, jpg can be used as the default format for extracted images.

[0031] The front of the probe is 12 cm away from the camera lens, and the camera is set to a fixed close-range focal length (8 cm). Therefore, when the probe is far away from the sealing point component (i.e., the sealing point component is not within the close-range focal length of the camera), the extracted image cannot clearly display the sealing point component. As a result, subsequent steps will not identify the sealing point component from the image, thus avoiding misjudgment. The camera also has explosion-proof features to adapt to special operating scenarios such as chemical plants. Data collected by the camera facilitates traceability of non-compliant inspections.

[0032] S102: Input each leakage detection image into a pre-trained deep learning neural network model to generate a probe front detection frame and a sealing point component detection frame in each leakage detection image.

[0033] The detection box in this article refers to the square box that surrounds the target object during image recognition. For example, the sealing point component detection box is the square box that surrounds the sealing point component.

[0034] In this embodiment, the specific process of S102 is as follows:

[0035] a) Determine the type of sealing point component currently being inspected. For example, according to the HJ1230-2021 standard, the types of sealing point components include flanges, valves, connectors, open valves or open pipelines, pumps, agitators, pressure relief devices, sampling connection systems, compressors, and others.

[0036] In actual applications, the type of the sealing point component currently being detected can be obtained from the above APP.

[0037] b) Input the leak detection image into the deep learning neural network model corresponding to the type to obtain the positions of multiple probe candidate frames and sealing point component candidate frames and the confidence of each candidate frame.

[0038] In this embodiment, different models are trained for different types of sealing point components. Taking the flange as an example, the sealing point component currently being detected is input into the model corresponding to the flange. The model determines the positions of multiple probe candidate frames and flange candidate frames in the image and the confidence of each candidate frame.

[0039] c) determining a probe detection frame and a sealing point component detection frame from a plurality of probe candidate frames and sealing point component candidate frames using a corresponding confidence threshold of the sealing point.

[0040] Specifically, there will only be one probe in an image. If multiple probes are identified, the others must be false positives. Therefore, when determining the probe detection frame, the probe candidate frame with the largest confidence value exceeding the probe confidence threshold is taken as the probe detection frame. For example, if the model identifies three probe candidate frames from the image (with confidence values of 0.5, 0.7, and 0.8, respectively), and the probe confidence threshold is set to 0.6, the candidate frame with a confidence value of 0.8 is taken as the probe detection frame.

[0041] When determining the seal point component detection frame, the seal point component candidate frame with a confidence level exceeding the seal point component confidence threshold is selected as the seal point component detection frame. For example, if the model identifies three flange candidate frames in an image (with confidence levels of 0.4, 0.6, and 0.9, respectively), and the flange confidence threshold is set to 0.5, the candidate frames with confidence levels of 0.6 and 0.9 are selected as the flange detection frame. It can be seen that multiple seal point component detection frames may appear in a single image.

[0042] It should be pointed out that there is no absolute standard for the confidence threshold. In different application scenarios, the settings of the confidence threshold vary greatly. A confidence threshold that is too low may lead to an increase in the false alarm rate, and irrelevant objects may be identified as target components. A confidence threshold that is too high may lead to an increase in the false alarm rate, and the target components may be missed. By calculating the historical Precision-Confidence curve, the confidence at the balance point between the false alarm rate and the accuracy rate is used as the confidence threshold for daily audit tasks. The confidence thresholds for different types of sealing point components are different.

[0043] d) Determine the probe front detection frame using the probe detection frame: The coordinates of the upper left and lower right corners of the probe front detection frame are (x1, y1) and (x2, (y1+y2) / 2), where (x1, y1) and (x2, y2) are the coordinates of the upper left and lower right corners of the probe detection frame, respectively.

[0044] In order to reduce the recognition errors caused by possible perspective errors and increase the recognition tolerance, the coordinates (x3, y3) and (x4, y4) of the upper left and lower right corners of the sealing point component detection frame can be corrected to ((1+a) / 2*x3+(1-a) / 2*x4, (1+a) / 2*y3+(1-a) / 2*y4),

[0045] ((1-a) / 2*x3+(1+a) / 2*x4,(1-a) / 2*y3+(1+a) / 2*y4), where a is the preset scaling ratio, ranging from 0.8 to 2.

[0046] The purpose of modifying the sealing point component detection frame is to:

[0047] 1. Enlarge the judgment area of the sealing point that is too small, for example, 1.1 times, to avoid incorrect judgment due to insufficient overlap due to being much smaller than the detection frame in front of the probe.

[0048] 2. Regulations allow a 1cm distance between the front of the probe and the dynamic sealing point, and no fitting is required. Therefore, the dynamic sealing point is expanded by a certain proportion and the judgment conditions for "compliance testing" are relaxed.

[0049] For example, the above-mentioned deep learning neural network model can adopt the YOLOv8 model, which has the advantages of fast speed and less computing resources compared to other models. It takes 30-35ms to complete the recognition of a detection image (GPU: NVIDIAT415G).

[0050] Before training the model, the data needs to be labeled first: delete the blurred or duplicated images in the extracted images (see S101 for the process of extracting images) to complete the initial screening, and upload the screened images to the Labelstudio platform. The labeling personnel label the images in combination with the relevant regulatory requirements and industry experience of LDAR detection (label the positions of different types of sealing points in the on-site detection images). The number of labeled images for each type of sealing point component is more than 10,000. The labeled images are derived from the actual use environment and are divided into different time periods, different seasons, different weather, different lighting, different angles and different inspectors as the basis for screening images. During the labeling process, all clear objects are labeled, rather than just the detection objects that need to be identified. In addition, 1%-10% of background images are added to the training set, that is, images without any detection objects. By adding background images, the false alarm rate of undetected recognition errors is reduced by an average of 26%.

[0051] During training, each training set file contains at least 1,000 images. After training, the model is placed in a standard test library for performance evaluation. Prior to evaluation, detection videos from four different scenarios and time periods were selected for each type of sealing point component. Keyframe extraction was used to obtain 1,152 original images from each scene. Through discussions with LDAR experts and engineers, the detection status of a total of 5,760 standard images from the test library was confirmed. These images served as the test set for the new model, and the trained model was evaluated using the test set, achieving a fast and accurate model performance evaluation method.

[0052] S103 , determining the overlapping area between the probe front detection frame and the sealing point assembly detection frame in each leakage detection image, and dividing the overlapping area by the area of the probe front detection frame to obtain the overlap degree corresponding to each leakage detection image.

[0053] As mentioned above, multiple sealing point component detection frames may appear in one image. Therefore, in S103, the overlapping areas of multiple sealing point component detection frames in one image need to be calculated respectively with the probe front detection frame. Therefore, one leakage detection image may have multiple overlapping degrees.

[0054] S104. Detection judgment: Compare the overlap of each leakage detection image with the overlap threshold. If the overlap is greater than the overlap threshold, the detection result of the corresponding leakage detection image is compliant. Otherwise, it is questionable. Alarm inference: If the detection result of each leakage detection image meets the preset conditions, the leakage detection process is compliant. Otherwise, it is questionable.

[0055] When making a detection judgment, if a leak detection image has multiple overlaps, the multiple overlaps are compared with the overlap threshold respectively. If any overlap is greater than the overlap threshold, the detection result of the leak detection image is compliant.

[0056] The overlap thresholds for different types of sealing points vary. For smaller sealing points (e.g., connectors), the overlap threshold might be as low as 1%, while for larger sealing points (e.g., flanges), the overlap threshold might be as high as 20%. Alternatively, a unique overlap threshold for each sealing point can be calculated based on the size information of the sealing point.

[0057] The above preset conditions can be configured by considering the balance between hardware fault tolerance, software fault tolerance, detection fault tolerance, computing power cost and accuracy to achieve a high alarm rate and a low false alarm rate.

[0058] For example, the effective detection time can be compared with the standard time required by the regulations. If it exceeds the standard time, the leakage detection process is compliant; otherwise, it is questionable.

[0059] Among them, effective detection time = number of compliant leak detection images * image extraction time interval; standard time = FID detector response time + sealing point circumference / maximum moving speed under regulation. The response time is the calibration time and can be obtained from the database. The sealing point circumference can be calculated based on the sealing point size recorded by the APP. The maximum moving speed under regulation is: 10 cm / s for routine detection and 3 cm / s for retest.

[0060] In principle, the detection position is not allowed to deviate from the position required by regulations. Taking into account the displacement of the probe during the movement of the detection part, a 10% tolerance is given to the standard time. Taking into account possible ambiguity and unrecognition situations, an additional 10% tolerance is given to the standard time.

[0061] Assume that according to the APP records, the target sealing point is a connector with a diameter of 15mm, and the minimum detection time required by the APP is 15s (the difference between the detection start time and the detection end time, and the leakage detection picture is extracted within 15 seconds), and the FID detector response time is 7s, the picture extraction time interval is 1s, and a total of 15 pictures are extracted within 15 seconds, of which 10 are compliant pictures. Then the effective detection time = 10*1 = 10s, the standard time = 7+(15*3.14) / 10 = 11.71s, and then through fault tolerance correction, the standard time = 11.71*(1-0.1-0.1) = 9.36s. It can be seen that the effective detection time exceeds the standard time, and the detection process is compliant.

[0062] The compliance of the testing process is determined by the effective testing time, without considering factors such as walking that affect the testing time.

[0063] In case of doubts about the leak detection process, the corresponding video will be assigned to the auditor, who will conduct further manual review of the video in the review interface. The review results will be classified and counted by sealing point component type, subtype, unidentified reasons, etc. to guide the data annotation and debugging direction. After review, the data that is considered to be false positives will be annotated and used as training set material to improve the model. The non-false positive data will be sent to the LDAR person in charge for location tracing and re-compliance to complete the sealing point retest. Through manual review, the full process of image recognition is closed-loop and all sealing points are tested for compliance. See Figure 5 .

[0064] As can be seen from the above, the embodiment of the present application sets a camera on the probe handle of the FID detector, and obtains multiple leak detection pictures of the target sealing point during leak detection through the camera. The probe front detection frame and the sealing point component detection frame in the picture are obtained based on computer vision technology, and the overlap is calculated. The compliance of the picture is judged according to the overlap. Finally, the compliance of each picture is determined to determine whether the leakage detection process is compliant, which solves the problems of lack of supervision measures for the leakage detection process and difficulty in tracing fraudulent behavior in the prior art.

[0065] The following describes in detail one or more embodiments of the present invention's leakage detection behavior compliance audit device. Those skilled in the art will appreciate that these audit devices can be constructed using commercially available hardware components configured through the steps taught in this solution. Figure 2 A leakage detection behavior compliance audit device provided by an embodiment of the present invention is shown. Figure 2 As shown, the auditing device includes a leakage detection image acquisition module 11 , a detection frame determination module 12 , an overlap calculation module 13 and an auditing module 14 .

[0066] The leakage detection image acquisition module 11 is used to acquire multiple leakage detection images during the leakage detection process of the target sealing point through a camera.

[0067] The camera is located on the probe handle of the FID detector. In actual applications, LDAR inspectors use a handheld FID detector to detect leaks at a certain sealing point component (where a leak may occur). To obtain a high-quality inspection video, the FID detector's moving speed can be controlled below 7 cm / s. At the same time, the probe's inspection angle must be within the range of a 60° angle between the normal N and the cross-section of the sealing point inspection surface. Inspection angles exceeding 90° should be avoided as much as possible (there is a high probability that the sealing point component will be seriously missing). See [Note: The following sentences appear to be unrelated and should be omitted.] Figure 4 .

[0068] During leak detection at sealing points, video is captured by a camera and uploaded to a cloud platform in real time or offline via a wireless network (4G / 5G / WiFi). To ensure the stability of video transmission, videos are uploaded in 10-minute increments. The cloud platform determines the time period for leak detection at the target sealing point and extracts multiple leak detection images within that time period from the video using keyframes or timestamps. In actual applications, the app in the handheld controller that comes with the FID detector will record information such as the test start and end time, the type of sealing point component, the sealing point size, the device, the area, the current leakage concentration, etc., so that the above time period can be determined based on the test start and end time.

[0069] When using the keyframe method, the FFmpeg open-source computer program is used to locate and extract keyframe data from the detection video. When using the timestamp method, images are extracted and saved frame by frame from the video based on the OpenCV2 computer vision algorithm library, enabling targeted extraction based on timestamp information. In practical applications, the number of images extracted for a single detection and the timeline distribution of extracted images can be adjusted based on detection requirements, achieving the conversion from detection video to detection images. Image formats include BMP and jpg. Testing has shown that the lossless BMP format can improve the accuracy of the image recognition model by approximately 6%, but the recognition time for each BMP image is approximately 30 times that of a jpg image. For cost-effectiveness, jpg can be used as the default format for extracted images.

[0070] The front of the probe is 12 cm away from the camera lens, and the camera is set to a fixed close-range focal length (8 cm). Therefore, when the probe is far away from the sealing point component (i.e., the sealing point component is not within the close-range focal length of the camera), the extracted image cannot clearly display the sealing point component. As a result, subsequent steps will not identify the sealing point component from the image, thus avoiding misjudgment. The camera also has explosion-proof features to adapt to special operating scenarios such as chemical plants. Data collected by the camera facilitates traceability of non-compliant inspections.

[0071] The detection frame determination module 12 is used to input each leakage detection image into a pre-trained deep learning neural network model to generate a probe front detection frame and a sealing point component detection frame in each leakage detection image.

[0072] The detection box in this article refers to the square box that surrounds the target object during image recognition. For example, the sealing point component detection box is the square box that surrounds the sealing point component.

[0073] In this embodiment, the specific process of generating the detection frame is as follows:

[0074] a) Determine the type of sealing point component currently being inspected. For example, according to the HJ1230-2021 standard, the types of sealing point components include flanges, valves, connectors, open valves or open pipelines, pumps, agitators, pressure relief devices, sampling connection systems, compressors, and others.

[0075] In actual applications, the type of the sealing point component currently being detected can be obtained from the above APP.

[0076] b) Input the leak detection image into the deep learning neural network model corresponding to the type to obtain the positions of multiple probe candidate frames and sealing point component candidate frames and the confidence of each candidate frame.

[0077] In this embodiment, different models are trained for different types of sealing point components. Taking the flange as an example, the sealing point component currently being detected is input into the model corresponding to the flange. The model determines the positions of multiple probe candidate frames and flange candidate frames in the image and the confidence of each candidate frame.

[0078] c) determining a probe detection frame and a sealing point component detection frame from a plurality of probe candidate frames and sealing point component candidate frames using a corresponding confidence threshold of the sealing point.

[0079] Specifically, there will only be one probe in an image. If multiple probes are identified, the others must be false positives. Therefore, when determining the probe detection frame, the probe candidate frame with the largest confidence value exceeding the probe confidence threshold is taken as the probe detection frame. For example, if the model identifies three probe candidate frames from the image (with confidence values of 0.5, 0.7, and 0.8, respectively), and the probe confidence threshold is set to 0.6, the candidate frame with a confidence value of 0.8 is taken as the probe detection frame.

[0080] When determining the seal point component detection frame, the seal point component candidate frame with a confidence level exceeding the seal point component confidence threshold is selected as the seal point component detection frame. For example, if the model identifies three flange candidate frames in an image (with confidence levels of 0.4, 0.6, and 0.9, respectively), and the flange confidence threshold is set to 0.5, the candidate frames with confidence levels of 0.6 and 0.9 are selected as the flange detection frame. It can be seen that multiple seal point component detection frames may appear in a single image.

[0081] It should be pointed out that there is no absolute standard for the confidence threshold. In different application scenarios, the settings of the confidence threshold vary greatly. A confidence threshold that is too low may lead to an increase in the false alarm rate, and irrelevant objects may be identified as target components. A confidence threshold that is too high may lead to an increase in the false alarm rate, and the target components may be missed. By calculating the historical Precision-Confidence curve, the confidence at the balance point between the false alarm rate and the accuracy rate is used as the confidence threshold for daily audit tasks. The confidence thresholds for different types of sealing point components are different.

[0082] d) Determine the probe front detection frame using the probe detection frame: The coordinates of the upper left and lower right corners of the probe front detection frame are (x1, y1) and (x2, (y1+y2) / 2), where (x1, y1) and (x2, y2) are the coordinates of the upper left and lower right corners of the probe detection frame, respectively.

[0083] In order to reduce the recognition errors caused by possible perspective errors and increase the recognition tolerance, the coordinates (x3, y3) and (x4, y4) of the upper left and lower right corners of the sealing point component detection frame can be corrected to ((1+a) / 2*x3+(1-a) / 2*x4, (1+a) / 2*y3+(1-a) / 2*y4),

[0084] ((1-a) / 2*x3+(1+a) / 2*x4,(1-a) / 2*y3+(1+a) / 2*y4), where a is the preset scaling ratio, ranging from 0.8 to 2.

[0085] The purpose of modifying the sealing point component detection frame is to:

[0086] 1. Enlarge the judgment area of the sealing point that is too small, for example, 1.1 times, to avoid incorrect judgment due to insufficient overlap due to being much smaller than the detection frame in front of the probe.

[0087] 2. Regulations allow a 1cm distance between the front of the probe and the dynamic sealing point, and no fitting is required. Therefore, the dynamic sealing point is expanded by a certain proportion and the judgment conditions for "compliance testing" are relaxed.

[0088] For example, the above-mentioned deep learning neural network model can adopt the YOLOv8 model, which has the advantages of fast speed and less computing resources compared to other models. It takes 30-35ms to complete the recognition of a detection image (GPU: NVIDIAT415G).

[0089] Before training the model, the data needs to be labeled first: delete the blurred or duplicated images in the extracted images (see the leak detection image acquisition module 11 for the process of extracting images) to complete the initial screening, and upload the screened images to the Labelstudio platform. The labeling personnel label the images in combination with the relevant regulatory requirements and industry experience of LDAR detection (label the positions of different types of sealing points in the on-site detection images). The number of labeled images for each type of sealing point component is more than 10,000. The labeled images are derived from the actual use environment and are divided into different time periods, different seasons, different weather, different lighting, different angles and different inspectors as the basis for screening images. During the labeling process, all clear objects are labeled instead of only labeling the detection objects that need to be identified. In addition, 1%-10% of background images are added to the training set, that is, images without any detection objects. By adding background images, the false alarm rate of undetected recognition errors is reduced by an average of 26%.

[0090] During training, each training set file contains at least 1,000 images. After training, the model is placed in a standard test library for performance evaluation. Prior to evaluation, detection videos from four different scenarios and time periods were selected for each type of sealing point component. Keyframe extraction was used to obtain 1,152 original images from each scene. Through discussions with LDAR experts and engineers, the detection status of a total of 5,760 standard images from the test library was confirmed. These images served as the test set for the new model, and the trained model was evaluated using the test set, achieving a fast and accurate model performance evaluation method.

[0091] The overlap calculation module 13 is used to determine the overlapping area between the probe front detection frame and the sealing point component detection frame in each leakage detection image, and divide the overlapping area by the area of the probe front detection frame to obtain the overlap corresponding to each leakage detection image.

[0092] As mentioned above, multiple sealing point component detection frames may appear in a single image. Therefore, the overlapping areas of the multiple sealing point component detection frames in a single image need to be calculated separately with the probe front detection frame. Therefore, a leak detection image may have multiple degrees of overlap.

[0093] The audit module 14 is used for detection and judgment: the overlap of each leakage detection image is compared with the overlap threshold. If the overlap is greater than the overlap threshold, the detection result of the corresponding leakage detection image is compliant; otherwise, it is questionable. The alarm inference is: if the detection result of each leakage detection image meets the preset conditions, the leakage detection process is compliant; otherwise, it is questionable.

[0094] When making a detection judgment, if a leak detection image has multiple overlaps, the multiple overlaps are compared with the overlap threshold respectively. If any overlap is greater than the overlap threshold, the detection result of the leak detection image is compliant.

[0095] The overlap thresholds for different types of sealing points vary. For smaller sealing points (e.g., connectors), the overlap threshold might be as low as 1%, while for larger sealing points (e.g., flanges), the overlap threshold might be as high as 20%. Alternatively, a unique overlap threshold for each sealing point can be calculated based on the size information of the sealing point.

[0096] The above preset conditions can be configured by considering the balance between hardware fault tolerance, software fault tolerance, detection fault tolerance, computing power cost and accuracy to achieve a high alarm rate and a low false alarm rate.

[0097] For example, the effective detection time can be compared with the standard time required by the regulations. If it exceeds the standard time, the leakage detection process is compliant; otherwise, it is questionable.

[0098] Among them, effective detection time = number of compliant leak detection images * image extraction time interval; standard time = FID detector response time + sealing point circumference / maximum moving speed under regulation. The response time is the calibration time and can be obtained from the database. The sealing point circumference can be calculated based on the sealing point size recorded by the APP. The maximum moving speed under regulation is: 10 cm / s for routine detection and 3 cm / s for retest.

[0099] In principle, the detection position is not allowed to deviate from the position required by regulations. Taking into account the displacement of the probe during the movement of the detection part, a 10% tolerance is given to the standard time. Taking into account possible ambiguity and unrecognition situations, an additional 10% tolerance is given to the standard time.

[0100] Assume that according to the APP records, the target sealing point is a connector with a diameter of 15mm, and the minimum detection time required by the APP is 15s (the difference between the detection start time and the detection end time, and the leakage detection picture is extracted within 15 seconds), and the FID detector response time is 7s, the picture extraction time interval is 1s, and a total of 15 pictures are extracted within 15 seconds, of which 10 are compliant pictures. Then the effective detection time = 10*1 = 10s, the standard time = 7+(15*3.14) / 10 = 11.71s, and then through fault tolerance correction, the standard time = 11.71*(1-0.1-0.1) = 9.36s. It can be seen that the effective detection time exceeds the standard time, and the detection process is compliant.

[0101] The compliance of the testing process is determined by the effective testing time, without considering factors such as walking that affect the testing time.

[0102] In case of doubts about the leak detection process, the corresponding video will be assigned to the auditor, who will conduct further manual review of the video in the review interface. The review results will be classified and counted by sealing point component type, subtype, unidentified reasons, etc. to guide the data annotation and debugging direction. After review, the data that is considered to be false positives will be annotated and used as training set material to improve the model. The non-false positive data will be sent to the LDAR person in charge for location tracing and re-compliance to complete the sealing point retest. Through manual review, the full process of image recognition is closed-loop and all sealing points are tested for compliance. See Figure 5 .

[0103] In summary, the leakage detection behavior compliance audit device provided in the above embodiments can execute the leakage detection behavior compliance audit method provided in the above embodiments.

[0104] The same concept as above, Figure 2 The structure of the leakage detection behavior compliance audit device shown can be implemented as an electronic device. Figure 3 A schematic block diagram of the structure of an electronic device provided by an embodiment of the present invention is shown.

[0105] Exemplarily, the electronic device includes a storage module 21 and a processor 22, the storage module 21 includes instructions loaded and executed by the processor 22, and when the instructions are executed, the processor 22 performs the steps of various exemplary embodiments of the present invention described in the above-mentioned leakage detection behavior compliance audit method section of this specification.

[0106] It should be understood that the processor 22 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0107] A computer-readable storage medium is also provided in an embodiment of the present invention. The computer-readable storage medium stores one or more programs. When the one or more programs are executed by a processor, the steps of various exemplary embodiments of the present invention described in the above-mentioned leakage detection behavior compliance audit method are implemented.

[0108] It will be understood by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer-readable storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0109] As is well known to those skilled in the art, the term computer-readable storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically contains computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0110] For example, the computer-readable storage medium may be an internal storage unit of the electronic device of the aforementioned embodiment, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., provided on the electronic device.

[0111] The electronic device and computer-readable storage medium provided in the aforementioned embodiments obtain multiple leak detection images of the target sealing point during leak detection through a camera, obtain the probe front detection frame and the sealing point component detection frame in the image based on computer vision technology, and calculate the overlap. The compliance of the image is judged based on the overlap, and finally, the compliance of the leakage detection process is determined based on the compliance of each image, which solves the problems of lack of supervision measures for the leakage detection process and difficulty in tracing fraudulent behavior in the existing technology.

[0112] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include these modifications and variations.

Claims

1. A leak detection compliance audit method, characterized in that: include: S101, acquiring multiple leak detection images during a leak detection process of a target sealing point through a camera, wherein the camera is provided on a probe handle of an FID detector; S102, inputting each leak detection image into a pre-trained deep learning neural network model to generate a probe front detection frame and a sealing point component detection frame in each leak detection image; S103, determining the overlapping area between the probe front detection frame and the sealing point assembly detection frame in each leak detection image, and dividing the overlapping area by the area of the probe front detection frame to obtain the overlap degree corresponding to each leak detection image; S104. Compare the overlap of each leak detection image with the overlap threshold. If the overlap is greater than the overlap threshold, the detection result of the corresponding leak detection image is compliant; otherwise, it is questionable. If the detection result of each leak detection image meets the preset conditions, the leak detection process is compliant; otherwise, it is questionable.

2. A leak detection behavior compliance audit method according to claim 1, characterized in that: The S101 further includes: Collecting video through the camera; determining a time period for leak detection of the target sealing point; The multiple leakage detection pictures within the time period are extracted from the video in a key frame or time stamp manner.

3. A leak detection behavior compliance audit method according to claim 2, characterized in that: The distance between the front of the probe and the lens of the camera is 12 cm, and the camera is set to a fixed close-range focal length of 8 cm. When the sealing point component is not within the close-range focal length, the sealing point component cannot be clearly displayed in the extracted leakage detection picture.

4. A leak detection behavior compliance audit method according to claim 1 or 3, characterized in that: The S102 further includes: Determine the type of sealing point component currently being inspected; Inputting the leak detection image into a deep learning neural network model corresponding to the type, obtaining the positions of multiple probe candidate frames and sealing point component candidate frames and the confidence level of each candidate frame; Determining a probe detection frame and a sealing point component detection frame from the multiple probe candidate frames and sealing point component candidate frames according to corresponding confidence thresholds; The probe front detection frame is determined by the probe detection frame: the coordinates of the upper left and lower right corners of the probe front detection frame are (x1, y1), (x2, (y1+y2) / 2), where (x1, y1) and (x2, y2) are the coordinates of the upper left and lower right corners of the probe detection frame, respectively.

5. A leak detection compliance audit method according to claim 4, characterized in that: The determining of the probe detection frame and the sealing point component detection frame from the plurality of probe candidate frames and sealing point component candidate frames by using corresponding confidence thresholds further includes: The largest probe candidate frame with a confidence score exceeding the probe confidence threshold is selected as the probe detection frame. The sealing point component candidate frame whose confidence exceeds the sealing point component confidence threshold is taken as the sealing point component detection frame.

6. A leak detection compliance audit method according to claim 5, characterized in that: The S102 further includes: Correct the coordinates (x3, y3) and (x4, y4) of the upper left and lower right corners of the sealing point component detection frame to ((1+a) / 2*x3+(1-a) / 2*x4, (1+a) / 2*y3+(1-a) / 2*y4), ((1-a) / 2*x3+(1+a) / 2*x4,(1-a) / 2*y3+(1+a) / 2*y4), where a is the preset scaling ratio, ranging from 0.8 to 2.

7. A leak detection compliance audit method according to claim 6, characterized in that: The S104 further includes: If a leak detection image has multiple overlaps, the multiple overlaps are compared with the overlap threshold respectively. If any overlap is greater than the overlap threshold, the detection result of the leak detection image is compliant.

8. A leak detection behavior compliance audit device, characterized in that: include: A leak detection image acquisition module is used to acquire multiple leak detection images during the leak detection process of the target sealing point through a camera, wherein the camera is provided on the probe handle of the FID detector; A detection frame determination module is used to input each leak detection image into a pre-trained deep learning neural network model to generate a probe front detection frame and a sealing point component detection frame in each leak detection image; An overlap calculation module is used to determine the overlapping area between the probe front detection frame and the sealing point assembly detection frame in each leak detection image, and divide the overlapping area by the area of the probe front detection frame to obtain the overlap corresponding to each leak detection image; The audit module is used to compare the overlap of each leakage detection image with the overlap threshold. If the overlap is greater than the overlap threshold, the detection result of the corresponding leakage detection image is compliant; otherwise, it is questionable. If the detection result of each leakage detection image meets the preset conditions, the leakage detection process is compliant; otherwise, it is questionable.

9. An electronic device, characterized in that: It comprises a storage module, which includes instructions loaded and executed by a processor, and when the instructions are executed, the processor executes a leakage detection behavior compliance audit method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing one or more programs, characterized in that: When the one or more programs are executed by the processor, the leakage detection behavior compliance audit method according to any one of claims 1 to 7 is implemented.

Images