Air leakage detection method, device, electronic equipment, system and storage medium

Abstract

The application relates to the technical field of artificial intelligence, and discloses a gas leakage detection method and device, electronic equipment, a system and a storage medium. The method comprises the following steps: acquiring a detection image sequence of a to-be-detected object, wherein the detection image sequence is collected under the condition that the to-be-detected object is soaked in liquid; performing optical flow estimation according to the detection image sequence to obtain optical flow vector data corresponding to the detection image sequence; and determining a gas leakage detection result of the to-be-detected object according to optical flow vectors corresponding to bubble pixel points representing bubbles in the optical flow vector data. The application can automatically identify the gas leakage condition of the to-be-detected object by using the detection image sequence, and improves the gas leakage detection efficiency.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a method, apparatus, electronic device, system, and storage medium for detecting air leaks. Background Technology

[0002] Components requiring sealing (such as gas meters) need to undergo leak testing before leaving the factory. In related technologies, this is typically done manually by continuously observing the gas meter in water for bubbles to determine if it is leaking. However, this manual leak detection method is labor-intensive and inefficient, as it requires personnel to observe the gas meter continuously for a considerable period. Summary of the Invention

[0003] In view of the above problems, this application proposes a method, device, electronic device, system and storage medium for leak detection, so as to solve the problem that manual leak detection is labor-intensive and inefficient in related technologies.

[0004] According to one aspect of the embodiments of this application, a leak detection method is provided, comprising: acquiring a detection image sequence of an object to be detected, the detection image sequence being acquired while the object to be detected is immersed in a liquid; performing optical flow estimation based on the detection image sequence to obtain optical flow vector data corresponding to the detection image sequence; and determining a leak detection result of the object to be detected based on the optical flow vector corresponding to the bubble pixel representing a bubble in the optical flow vector data.

[0005] According to one aspect of the embodiments of this application, a leak detection device is provided, comprising: an acquisition module for acquiring a detection image sequence of an object to be detected, the detection image sequence being acquired while the object to be detected is immersed in a liquid; an optical flow estimation module for performing optical flow estimation based on the detection image sequence to obtain optical flow vector data corresponding to the detection image sequence; and a leak detection result determination module for determining a leak detection result of the object to be detected based on the optical flow vector corresponding to the bubble pixel representing a bubble in the optical flow vector data.

[0006] According to one aspect of the embodiments of this application, an electronic device is provided, including: a processor; a memory, the memory storing computer-readable instructions, which, when executed by the processor, implement the leakage detection method as described above.

[0007] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a processor, implement the air leakage detection method as described above.

[0008] According to one aspect of the embodiments of this application, a computer program product is provided, which includes computer instructions that, when executed by a processor, implement the above-described leak detection method.

[0009] In this application, after acquiring a sequence of detection images of the object to be tested while it is immersed in a liquid, optical flow estimation is performed based on the sequence to obtain corresponding optical flow vector data. If there are moving objects in the detection image sequence while the object is immersed in the liquid, these are likely bubbles caused by leakage. The optical flow vector corresponding to the bubble pixels in the optical flow vector data reflects whether bubbles are present in the detection image sequence and their movement. Therefore, the leakage detection result of the object can be determined based on the optical flow vector corresponding to the bubble pixels. This solution eliminates the need for continuous observation of the object while it is immersed in the liquid. It automatically identifies differences in optical flow changes between different detection images based on the acquired sequence, thereby determining the leakage detection result. This significantly reduces the workload of the inspection personnel and greatly improves the efficiency of leakage detection because it does not rely on manual observation. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0011] Figure 1 This is a schematic diagram of a leak detection system according to an embodiment of this application.

[0012] Figure 2 This is a flowchart illustrating a leak detection method according to an embodiment of this application.

[0013] Figure 3 This is an architectural diagram of an optical flow estimation model according to an embodiment of this application.

[0014] Figure 4 This is a flowchart illustrating optical flow estimation based on an embodiment of this application.

[0015] Figure 5 An exemplary schematic diagram is shown, illustrating an optical flow image determined based on optical flow vector data corresponding to an optical flow detection image sequence.

[0016] Figure 6This is a flowchart illustrating step 230 according to an embodiment of this application.

[0017] Figure 7 This is a flowchart illustrating the training of an optical flow estimation model according to an embodiment of this application.

[0018] Figure 8 This is a flowchart illustrating the training and testing of an optical flow estimation model according to an embodiment of this application.

[0019] Figure 9 This is a block diagram of a leak detection device according to an embodiment of this application.

[0020] Figure 10 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of this application is shown. Detailed Implementation

[0021] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0022] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0023] Figure 1 This is a schematic diagram of a leak detection system according to an embodiment of this application, as shown below. Figure 1 As shown, the leak detection system 100 includes a transparent container 110, an image acquisition device 120, and an electronic device 130. The transparent container 110 is used to hold liquid and the object to be detected, so that the object to be detected is immersed in the liquid in the transparent container. The image acquisition device 120 is used to acquire images facing the transparent container to obtain a sequence of detection images of the object to be detected while it is immersed in the liquid. The image acquisition device 120 can be a camera or other device with image acquisition function (such as a smartphone). The electronic device 130 is communicatively connected to the image acquisition device and is used to determine the leak detection result of the object to be detected based on the detection image sequence according to the leak detection method provided in this application.

[0024] In some embodiments, water can be filled in a transparent container, and the object to be tested can be immersed in the water, ensuring that the object is submerged in the liquid. The object to be tested can be a component with certain sealing requirements; because of these requirements, leakage testing is necessary before the component leaves the factory. Components can be gas meters, pipes, tires, etc., and are not specifically limited here.

[0025] In some embodiments, multiple image acquisition devices 120 can be deployed around the transparent container so that the multiple image acquisition devices 120 simultaneously capture images of the transparent container, obtaining multiple sets of detection image sequences showing the object to be tested being immersed in liquid. This allows the detection image sequence with better presentation from the multiple sets of detection image sequences to identify the leakage detection result of the object to be tested.

[0026] In some embodiments, multiple objects to be detected can be placed dispersedly in a transparent container, and multiple image acquisition devices can be deployed. One image acquisition device can face one object to be detected to acquire detection images. In this way, detection image sequences of multiple objects to be detected can be obtained simultaneously, which can improve the efficiency of leak detection.

[0027] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0028] Figure 2 This is a flowchart illustrating a leak detection method according to an embodiment of this application. The method can be executed by an electronic device with processing capabilities, such as a server. (Refer to...) Figure 2 As shown, the method includes at least steps 210 to 230, which are described in detail below:

[0029] Step 210: Obtain the detection image sequence of the object to be detected. The detection image sequence is acquired when the object to be detected is immersed in liquid.

[0030] In this application, the object to be tested can be a gas meter, pipe, tire, sealed container, or other component requiring a certain level of sealing. In some embodiments, before immersing the object to be tested in the liquid, it can be inflated with gas, such as air, nitrogen, or helium. This way, if the object leaks, the internal gas will better reveal the leak when it is immersed in the liquid. In other embodiments, the object to be tested can be inflated with gas through a gas source connected to the transparent container while it is immersed in the liquid.

[0031] When the object under test is immersed in liquid, if it leaks gas, bubbles will appear around it; conversely, if it does not leak, bubbles will not be generated. For example, if the object under test is a gas meter, and the gas meter leaks due to at least one of the following: pinholes in the gas meter casing, leaks in the internal sealing ring, leaks in the output shaft, leaks in the flange face, pinholes in the screw holes, leaks in the terminals, leaks in the cover plate, leaks in the connectors, or perforations in the casing, then when the gas meter is immersed in water, bubbles will be generated around it, and these bubbles will move over time.

[0032] Based on this principle, when the object to be tested is immersed in liquid, if the object leaks air, at least one image in the acquired detection image sequence will show air bubbles around the object; conversely, if the object does not leak air, no air bubbles will be present in any of the acquired detection image sequences. The method of this application identifies whether the object leaks air by recognizing and tracking air bubbles in the detection images. The detection image sequence includes at least two sequentially acquired detection images, each showing the object immersed in liquid.

[0033] Step 220: Perform optical flow estimation based on the detected image sequence to obtain the optical flow vector data corresponding to the detected image sequence.

[0034] In some embodiments, optical flow estimation can be performed on the detected image sequence using an optical flow estimation model. The optical flow estimation model is a model constructed using a neural network for optical flow estimation; such neural networks include convolutional neural networks, fully connected neural networks, recurrent neural networks, etc., without specific limitations. In some embodiments, the optical flow estimation model can be a FlowNet2 model, a Horn-Schunck-based model, a Lucas-Kanade-based model and a Farnback model, a FlowFormer model, PWC-Net (CNNs for Optical Flow Using Pyramid, Warping, and CostVolume), or a RAFT (Recurrent All-PairsField Transforms for Optical Flow) model.

[0035] Optical flow is the instantaneous velocity of pixels moving on the imaging plane of a spatially moving object. Optical flow estimation utilizes the temporal changes of pixels in an image sequence and the correlation between different frames to determine the pixel correspondences between different image frames, thereby determining the motion information of the object between two consecutively acquired images. Optical flow represents the velocity field of pixel (object) motion through a two-dimensional image, and determines the direction and speed of pixel motion in the image by observing the image changes caused by motion within small time intervals.

[0036] As described above, when the object to be detected is immersed in a liquid, if the object leaks air, bubbles will appear in the detection image sequence acquired facing the object. Furthermore, these bubbles will move over time, for example, moving upwards. In this application, the presence of bubbles in the detection images and their movement across multiple detection images are used to determine whether the object is leaking air.

[0037] In some embodiments, to avoid the leakage detection results being affected by objects other than bubbles in the acquired detection image sequence, for example, non-bubble moving objects in the detection image sequence can be identified as bubbles. The detection image sequence can be acquired facing the object to be detected while the object to be detected is still immersed in the liquid, thereby avoiding the leakage detection results being affected by the movement of the object to be detected itself.

[0038] The optical flow vector data corresponding to the detection image sequence includes the optical flow matrix corresponding to any two different detection images in the detection image sequence. The optical flow matrix corresponding to the two different detection images indicates the optical flow vector of a pixel in one detection image relative to the corresponding pixel in the other detection image. This optical flow vector is the velocity of the corresponding pixel.

[0039] In some embodiments, the optical flow vector data includes the optical flow matrix corresponding to two adjacent frames of detected images in the detected image sequence; in this embodiment, step 220 includes the following steps A1 and A2:

[0040] Step A1: Obtain two adjacent detection images from the detection image sequence.

[0041] In this application, it is equivalent to extracting detection images frame by frame from the detection image sequence, and then using two adjacent detection images as the detection image pairs for the current optical flow matrix to be determined. For example, if the detection image sequence includes detection images P1 to P4, then detection images P1 and P2, P2 and P3, and P3 and P4 can be used as the detection image pairs for the optical flow matrix to be determined, respectively.

[0042] Step A2: Optical flow estimation is performed based on two adjacent detection images using an optical flow estimation model, and the optical flow matrix corresponding to the two adjacent detection images is output.

[0043] For any two adjacent detection images of an arbitrary optical flow matrix to be determined, these two adjacent detection images are used as input to the optical flow estimation model. The optical flow estimation model extracts features from the two detection images, and then determines the optical flow vectors of corresponding pixels in the two detection images based on the differences between them. The optical flow matrix corresponding to two adjacent detection images includes the optical flow vectors of all corresponding pixels in the two adjacent detection images.

[0044] In some embodiments, the earlier detection image in two adjacent detection frames is used as the source image, and the later detection image is used as the target image. The optical flow matrix corresponding to the two adjacent detection frames is used to indicate the optical flow vector of a pixel in the target image relative to the corresponding pixel in the source image. During leak detection of the object to be detected, if the leak level is low (e.g., slight leak), the difference between the pixels containing the bubble in two adjacent detection frames in the detection image sequence may be small. This could make it difficult for the optical flow estimation model to distinguish between the two, leading to incorrect estimation of the optical flow vector corresponding to the bubble pixel (e.g., identifying the bubble pixel as motionless), thus affecting the leak detection result of the object to be detected.

[0045] To address this issue, in some embodiments, optical flow estimation can be performed by extracting two non-adjacent detection images from a frame-skipping sequence. In this case, the optical flow vector data includes the optical flow matrices corresponding to two non-adjacent detection images in the detection image sequence. In this embodiment, step 220 includes the following steps B1 and B2:

[0046] Step B1: Obtain two non-adjacent detection images from the detection image sequence.

[0047] In some embodiments, the number of detection images (assumed to be a first number) between two non-adjacent detection images can be set. Based on this, after determining the first detection image in the two detection images to be detected, the detection images that follow the first detection image and are spaced a first number of frames apart are taken as the other detection image. For example, if the first number is 2, and the detection image sequence includes detection images P1 to P5 acquired sequentially, detection images P1 and P4, and detection images P2 and P5 can be taken as the two detection images for which the optical flow matrix is ​​to be determined. The first number can be set according to actual needs and is an integer greater than zero; no specific limitation is made here.

[0048] Step B2: Optical flow estimation is performed based on two non-adjacent detection images using an optical flow estimation model, and the optical flow matrix corresponding to the two non-adjacent detection images is output.

[0049] Similarly, in step B2, the temporally earlier detection image in two non-adjacent detection frames can be used as the source image, and the temporally later detection image can be used as the target image. The optical flow matrix corresponding to the two non-adjacent detection frames indicates the optical flow vector of a pixel in the target image relative to the corresponding pixel in the source image. The implementation process of step B2 is similar to that of step A2, and will not be described again here.

[0050] In other embodiments, optical flow estimation can be performed by combining the method of extracting adjacent detection images frame by frame for optical flow estimation with the method of extracting non-adjacent detection images by skipping frames. In this embodiment, the optical flow vector data includes the optical flow matrix corresponding to two adjacent detection images in the detection image sequence and the optical flow matrix corresponding to two non-adjacent detection images in the detection image sequence. In this embodiment, the process of determining the corresponding optical flow matrix for two adjacent detection images is described in steps A1 to A2 above, and the process of determining the corresponding optical flow matrix for two non-adjacent detection images is described in steps B1 to B2 above, and will not be repeated here. In this embodiment, using the optical flow matrices corresponding to any two adjacent detection images and any two non-adjacent detection images as the basis for determining the leakage detection result provides more reference information compared to using only the optical flow matrices corresponding to any two adjacent detection images or only the optical flow matrices corresponding to any two non-adjacent detection images, ensuring a more accurate leakage detection result.

[0051] Figure 3 This is an architectural diagram of an optical flow estimation model according to an embodiment of this application, such as... Figure 3 As shown, the optical flow estimation model includes a first encoder, a cost volume encoder, and a cost memory encoder. Let's take step B2 as an example to illustrate the use of... Figure 3 The process of optical flow estimation using the optical flow estimation model shown is described in detail.

[0052] In this embodiment, two adjacent detection frames include a source image that comes first in time and a target image that comes later in time; in other words, the detection image that comes first in time in two adjacent detection frames is taken as the source image, and the detection image that comes later in time is taken as the target image. Based on Figure 3 The optical flow estimation model shown is as follows: Figure 4 As shown, step A2 includes steps 410-440:

[0053] Step 410: The first encoder extracts features from the source image and the target image respectively to obtain the first feature map corresponding to the source image and the second feature map corresponding to the target image.

[0054] like Figure 3 As shown, the source image and target image are input into the first encoder for feature extraction. For ease of distinction, the feature map obtained by the first encoder from the source image is called the first feature map, and the feature map obtained by the first encoder from the target image is called the second feature map.

[0055] In some embodiments, the first encoder can be a Transformer feature encoder. During feature extraction, this Transformer feature encoder includes multiple layers of self-attention-based network layers and a fully connected feed-forward network. Furthermore, it utilizes residual connections and layer normalization to progressively transform the original feature representation into richer and more abstract semantic features. Using the Transformer feature encoder to extract features from images (source and target images) effectively learns the relationships and dependencies between pixels at different locations in the image, thereby ensuring the feature representation capability of the extracted feature maps. This facilitates the use of the extracted first and second feature maps for subsequent tasks.

[0056] Step 420: Construct a four-dimensional cost volume feature map based on the first feature map and the second feature map.

[0057] Specifically, the dot product similarity between all pixel pairs in the first and second feature maps can be calculated to obtain a four-dimensional cost volume feature map (4D cost volume). For example, if the first encoder extracts H2×W2×D from a source image (or target image) of size H1×W1×3... f The first feature map and the second feature map are then multiplied by a dot product to obtain a four-dimensional cost volume feature map of size H2×W2×H2×W2. Here, H1 is the height of the source image, W1 is the width of the source image, and 3 represents the three channels of the source image: red (R), green (G), and blue (B); H2 is the height of the feature map, W2 is the width of the feature map, and D... f This represents the number of channels in the feature map.

[0058] The resulting four-dimensional cost volume feature map can be regarded as a series of two-dimensional cost feature maps of size H2×W2. Each two-dimensional cost feature map is used to measure the visual similarity between a pixel in the source image and all pixels in the target image.

[0059] Step 430: Encode the four-dimensional cost volume feature map using a cost volume encoder to obtain the cost storage feature map.

[0060] like Figure 3 As shown, the Cost Volume Encoder is based on a four-dimensional cost volume feature map. It first performs patching on the four-dimensional cost volume feature map, and then performs feature labeling based on the patch of the potential summary.

[0061] Specifically, during the patching process, the cost feature map M of each pixel X in the source image is... X The cost patch embedding sequence is obtained by cross-convolutional concatenation using a stride-2 convolutional network (stride-2conv) and a ReLU network layer. Specifically, this is applied to the cost feature map M of size H2×W2 for pixel X. X First, in the cost feature map M X The right and bottom sides of the cost feature map are padded with 0s to make its width and height multiples of 8. Then, the padded cost feature map is processed through a stride-2 convolutional network to optimize the cost feature map M. X Downsampling is performed, and the features of each pixel are combined with the features of its neighboring pixels to obtain a more comprehensive and accurate cost feature representation. Then, the output of the stride-2 convolutional network is fed into a ReLU network layer for transformation, and this process is repeated until a patch feature map F is finally output. Each feature in patch feature map F represents a cost feature map M. X One of the 8x8 patch features.

[0062] After patching each pixel in the image according to the above process, a series of patch features are obtained for each pixel. However, the large number of patch features affects the efficiency of information propagation between different source pixels. In other words, the resulting patch feature map F is highly redundant. Therefore, in order to obtain more compact cost features, the cost volume encoder transforms each patch feature map F into K D-dimensional latent vectors T through a dot product attention mechanism (i.e., by calculating the query vector, key vectors, and value vectors through a dot product multi-head attention mechanism). Then, the K D-dimensional latent vectors T are input into the Alternate-Group Transform (AGT) layer for feature labeling.

[0063] The Alternating Group Transformer (AGT) layer alternately divides the features in the latent vector T into two groups and encodes the features within each group using a self-attention mechanism. The AGT layer groups the labels in two mutually orthogonal ways and applies attention alternately between the two groups, thus reducing attentional costs while still enabling information propagation across all labels.

[0064] The cost volume encoder can transform the four-dimensional cost volume feature map of H2×W2×H2×W2 into a latent labeled feature map of H2×W2×K with a length of D, and finally use the latent labeled feature map of H2×W2×K as the optical flow encoding of cost memory.

[0065] The specific process of converting the four-dimensional cost volume feature map into a latent label feature map is as follows: For each pixel (i, j), all cost feature values ​​corresponding to it are obtained from the four-dimensional cost volume feature map based on different disparities. Then, the probability distribution function of each cost feature value is calculated, and a vector of length K is constructed according to the position information of the disparity peak represented by the probability distribution function. The k-th element indicates whether the disparity of the pixel is equal to the disparity value k. The above process is repeated for each pixel to obtain a latent label feature map of length D with a length of H2×W2×K. The plane in the i-th row and j-th column of the latent label feature map represents the latent label at pixel (i, j) in the image, and the k-th layer indicates whether the position belongs to the peak with a disparity value of k.

[0066] Step 440: Optical flow decoding is performed on the cost storage feature map using the cost storage decoder to obtain the optical flow matrix corresponding to two adjacent detection images.

[0067] The Cost Memory Decoder is used to accurately decode the optical flow vector of each pixel from the optical flow encoding in the cost memory. The specific process is as follows: extract features from the cost memory, generate a query vector based on the current estimated optical flow value, determine the current estimated flow by the corresponding position of the pixel in the target image, crop the cost map by performing a 9x9 local window cropping on the cost feature map with the corresponding source pixel as the center, construct the cost query vector by embedding the local cost encoding features and the corresponding point position, obtain the optical flow residual by combining the context features and the current optical flow value, iteratively optimize the optical flow, determine the corresponding optical flow matrix, and visualize multiple optical flow matrices to obtain the corresponding optical flow image.

[0068] Through steps 410-440 above, the optical flow matrix corresponding to the two adjacent detection images can be accurately output.

[0069] When the input to the optical flow estimation model is two non-adjacent detection images, the optical flow estimation model can perform optical flow estimation according to a process similar to steps 410-440 above, and output the optical flow matrix corresponding to the two non-adjacent detection images. The specific process will not be described in detail here.

[0070] In some embodiments, a correspondence between the value of the optical flow vector and the color can be preset. Then, according to the correspondence between the value of the optical flow vector and the color, the optical flow vector of a pixel is represented by the corresponding color at the position of the pixel according to the value of the optical flow vector. Furthermore, the direction is described according to the direction of the optical flow vector of the pixel. This makes it easier for users to intuitively determine the movement of the corresponding pixel based on the color and direction of the position point in the optical flow image.

[0071] In this case, after step 220, the method further includes: generating an optical flow image based on the optical flow vector data according to the correspondence between the values ​​of the optical flow vectors and their colors. This optical flow image describes the motion of corresponding pixels in multiple detection images within the detection image sequence. Specifically, each optical flow vector in each optical flow matrix of the optical flow vector data can be mapped to the brightness and hue in the HSV space, allowing users to intuitively distinguish the magnitude of the optical flow vectors through the colors in the optical flow image.

[0072] Figure 5 An exemplary diagram illustrates an optical flow image determined based on optical flow vector data corresponding to an optical flow detection image sequence. Figure 5 The multiple colored points within the dashed box represent the movement of a bubble within the optical flow detection image sequence.

[0073] Step 230: Determine the leakage detection result of the object to be detected based on the optical flow vector corresponding to the bubble pixel in the optical flow vector data.

[0074] As described above, if bubbles appear in the detected images in the detection image sequence, it indicates that the object to be detected is leaking; conversely, if no bubbles appear in the detected images in the detection image sequence, it indicates that the object to be detected is not leaking. Therefore, if the optical flow vector data does not include the optical flow vector of the bubble pixel representing the bubble, the leak detection result of the object to be detected is determined to be a detection result indicating that the object to be detected is not leaking.

[0075] In the case of an air leak being detected, the speed at which bubbles are formed and their movement reflect the degree of leakage. For example, faster bubble formation indicates a more severe leak; upward movement of bubbles after formation also indicates a more severe leak. The optical flow vector corresponding to each bubble pixel in the optical flow vector data indicates the speed and direction of movement of that bubble pixel. Therefore, the leak detection result of the object being tested can be determined based on the optical flow vector corresponding to each bubble pixel. The leak detection result of the object being tested indicates the determined degree of leakage.

[0076] In some embodiments, such as Figure 6 As shown, step 230 includes:

[0077] Step 610: Determine the optical flow vector with the largest value among multiple optical flow matrices.

[0078] Each optical flow matrix includes optical flow vectors corresponding to multiple pixels. Based on this, for each optical flow matrix, the optical flow vector with the largest value in the optical flow matrix can be determined. Then, the optical flow vectors with the largest values ​​in multiple optical flow matrices are compared to determine the optical flow vector with the largest value in multiple optical flow matrices.

[0079] When optical flow estimation is performed by combining the method of extracting adjacent detection images frame-by-frame with the method of extracting non-adjacent detection images by skipping frames, the optical flow vector data includes multiple sets of optical flow matrices corresponding to two adjacent detection images and multiple sets of optical flow matrices corresponding to two non-adjacent detection images. Conversely, if only the method of extracting adjacent detection images frame-by-frame is used for optical flow estimation, the optical flow vector data includes multiple sets of optical flow matrices corresponding to two adjacent detection images; similarly, if only the method of extracting non-adjacent detection images by skipping frames is used, the optical flow vector data includes multiple sets of optical flow matrices corresponding to two non-adjacent detection images.

[0080] Step 620: If the value of the optical flow vector with the largest value is greater than zero, determine that the optical flow vector with the largest value is the optical flow vector corresponding to the bubble pixel.

[0081] If the value of the optical flow vector with the largest value is greater than zero, and during the leak detection process, the object to be detected and the liquid in the container are relatively stationary, only the generated bubbles are moving. Therefore, in the acquired detection image, the pixel where the optical flow vector with a value greater than zero is located is usually the bubble pixel where the bubble is located. Correspondingly, the optical flow vector with a value greater than zero is also the optical flow vector of the bubble pixel.

[0082] Step 630: Determine the target leakage level corresponding to the optical flow vector of the bubble pixel based on the mapping relationship between the optical flow vector and the leakage level.

[0083] The mapping relationship between the optical flow vector and the leakage level can be set according to actual needs. For different types of objects to be detected, the mapping relationship between the optical flow vector and the leakage level can be different. The mapping relationship between the optical flow vector and the leakage level can define the conditions that the optical flow vector must meet under each leakage level.

[0084] Table 1 illustrates the mapping relationship between leakage level and optical flow vector. As shown in Table 1, when the optical flow vector value is 0, it is considered the normal situation, that is, the object under test does not leak air, and no bubbles are generated in this case. When the optical flow vector value is in the range of (0, 0.3), the leakage level is slight leakage, which is manifested as slow bubble generation.

[0085] Table 1

[0086] Leakage rating The value of the optical flow vector Bubble state normal 0 No bubbles were generated Minor leak >0，≤0.3 Bubbles are generated slowly Slight leak >0.3，≤0.8 Bubbles are generated but do not rise Common leak >0.8，≤2 Bubbles are generated and rise Serious leak >2 A large number of bubbles are generated and rise.

[0087] It is worth mentioning that the values ​​of leakage level and optical flow vector shown in Table 1 are merely illustrative examples and should not be considered as a limitation on the scope of this application.

[0088] The target leakage level refers to the leakage level corresponding to the optical flow vector of the bubble pixel. Based on the established mapping relationship between the optical flow vector and the leakage level, and the determined optical flow vector corresponding to the bubble pixel, the leakage level corresponding to the optical flow vector of that bubble pixel can be determined, i.e., the target leakage level.

[0089] Step 640: Determine the leakage detection result of the object to be tested based on the target leakage level.

[0090] Based on steps 610-640 above, the leakage detection result of the object to be detected is determined according to multiple optical flow matrices in the optical flow vector data.

[0091] In this application, when a sequence of detection images is acquired while the object to be detected is statically immersed in a liquid, optical flow estimation is performed based on the image sequence to obtain corresponding optical flow vector data. If a moving object exists in the image sequence while the object is statically immersed in the liquid, this is likely due to air leakage from the object. The optical flow vector corresponding to the bubble pixel in the optical flow vector data reflects whether a bubble is present in the image sequence and its movement. Therefore, the leakage detection result of the object can be determined based on the optical flow vector corresponding to the bubble pixel. This solution eliminates the need for continuous observation of the object while it is immersed in the liquid. It automatically identifies differences in optical flow changes between different detection images based on the acquired image sequence, thereby determining the leakage detection result. This significantly reduces the workload of the inspection personnel and greatly improves the efficiency of leakage detection because it does not rely on manual observation.

[0092] To ensure the accuracy of optical flow estimation, the optical flow estimation model needs to be trained in advance. The training process is as follows: Figure 7 As shown, it specifically includes:

[0093] Step 710: Obtain training data, which includes multiple first sample detection image sequences and labeled optical flow vector data corresponding to each first sample detection image sequence; the first sample detection image sequences are collected when the sample detection object is statically immersed in liquid.

[0094] The first sample detection image sequence includes at least two sample detection images acquired sequentially. The detection object presented in the sample detection images in each first sample detection image sequence is called the sample detection object. In a specific embodiment, to improve the training effect of the optical flow estimation model, the sample detection object and the object to be detected belong to the same type of object. For example, if the object to be detected is a gas meter, detection images can be acquired from other gas meters to determine multiple first sample detection image sequences.

[0095] The labeled optical flow vector data corresponding to the first sample detection image sequence can be determined by optical flow estimation using the first sample detection image sequence through an existing trained optical flow estimation model. This existing trained optical flow estimation model is different from the optical flow estimation model to be trained. For example, if the optical flow estimation model to be trained is the FlowFormer model, the existing trained optical flow estimation model can be a FlowNet2 model, a Horn-Schunck-based model, a Lucas-Kanade-based model, a Farnback-based model, etc., without specific limitations.

[0096] Step 720: For each first sample detection image sequence, optical flow estimation is performed based on the first sample detection image sequence using an optical flow estimation model, and the predicted optical flow vector data corresponding to the first sample detection image sequence is output.

[0097] The predicted optical flow vector data of the first sample detected image sequence indicates the optical flow vector corresponding to the corresponding pixel in the first sample detected image sequence as predicted by the optical flow estimation model.

[0098] During training, the optical flow estimation model can be performed as follows: Figure 4 The process shown is similar to that of optical flow estimation based on two input sample detection images, and outputting the predicted optical flow vector data corresponding to the first sample detection image sequence.

[0099] In some embodiments, to improve the robustness and generalization ability of the optical flow estimation model, before step 720, the method further includes adding noise to each sample detection image in the first sample detection image sequence. Specifically, Gaussian noise can be added to each sample detection image, and then the two sample detection images with added noise can be input into the optical flow estimation model. This can improve the ability of the optical flow estimation model to accurately calculate the optical flow vector from the noisy detection images.

[0100] Step 730: Calculate the loss value based on the labeled optical flow vector data corresponding to the first sample detection image sequence and the predicted optical flow vector data corresponding to the first sample detection image sequence.

[0101] In a specific embodiment, a loss function can be pre-defined for the optical flow estimation model. Then, in step 730, based on the defined loss function, the labeled optical flow vector data corresponding to the first sample detection image sequence and the predicted optical flow vector data corresponding to the first sample detection image sequence are substituted into the loss function to calculate the loss value. The loss function can be a cross-entropy loss function, an absolute value loss function, a squared loss function, a Hinge loss function, etc., and is not specifically limited here.

[0102] Step 740: Adjust the weight parameters of the optical flow estimation model in reverse based on the loss value.

[0103] The calculated loss value reflects the degree of difference between the predicted optical flow vector data and the labeled optical flow vector data. If the difference is large, it indicates that the optical flow estimation accuracy of the optical flow estimation model is low. In this case, the weight parameters of the optical flow estimation model are adjusted in reverse, and the optical flow estimation model is re-evaluated for the first sample detection image sequence until the recalculated loss value meets the set requirements. In step 740, an optimizer (such as AdamW) can be used to update the parameters of the optical flow estimation model slightly along the gradient direction. Then, the forward and backward propagation processes are repeated iteratively until the weight parameters of the optical flow estimation model with the minimum loss value or the highest accuracy are obtained.

[0104] Through the training process described above, the optical flow estimation model can learn to accurately estimate the optical flow from two detected images, thereby improving the accuracy of optical flow estimation.

[0105] In some embodiments, during the training of the optical flow estimation model, to test the training effect, the optical flow estimation model can be cross-tested during training to determine the training effect of the optical flow estimation model. In this case, after step 740, the method further includes the following steps C1-C3:

[0106] Step C1: Obtain test data, which includes multiple second sample detection image sequences and labeled optical flow vector data corresponding to each second sample detection image sequence; the second sample detection image sequences are collected when the sample detection object is statically immersed in liquid.

[0107] In this application, the sample detection images used to test the optical flow estimation model are referred to as the second sample detection image sequence, and each second sample detection image sequence includes at least two sample detection images. The acquisition process of the sample detection images in the second sample detection image sequence is basically similar to the acquisition process of the sample detection images in the first sample detection image sequence.

[0108] The labeled optical flow vector data corresponding to the second sample detection image sequence can be determined by optical flow estimation using the second sample detection image sequence through an existing trained optical flow estimation model. This existing trained optical flow estimation model is different from the optical flow estimation model to be trained. For example, if the optical flow estimation model to be trained is the FlowFormer model, the existing trained optical flow estimation model can be a FlowNet2 model, a Horn-Schunck-based model, a Lucas-Kanade-based model, a Farnback-based model, etc., without specific limitations.

[0109] Step C2: Perform optical flow estimation on the second sample detection image sequence using the trained optical flow estimation model, and output the predicted optical flow vector data corresponding to the second sample detection image sequence.

[0110] The predicted optical flow vector data of the second sample detection image sequence indicates the optical flow vector corresponding to the corresponding pixel in the second sample detection image sequence as predicted by the optical flow estimation model.

[0111] In step C2, the optical flow estimation model can be performed as follows: Figure 4 The process shown is similar to that of optical flow estimation based on two input sample detection images, and outputting the predicted optical flow vector data corresponding to the two sample detection images in the second sample detection image sequence.

[0112] Step C3: If the difference between the predicted optical flow vector data corresponding to the second sample detection image sequence and the labeled optical flow vector data corresponding to the second sample detection image sequence exceeds the difference threshold, the optical flow estimation model is further trained based on the second sample detection image sequence and the labeled optical flow vector data corresponding to the second sample detection image sequence.

[0113] If the difference between the predicted optical flow vector data corresponding to the second sample detected image sequence and the actual optical flow vector data exceeds the difference threshold, it indicates that the optical flow estimation model's prediction of the optical flow vector data for the second sample detected image sequence deviates significantly from the actual optical flow vector data, meaning the optical flow estimation model is incorrect. In this case, the optical flow estimation model should be further trained using the second sample detected image sequence and the labeled optical flow vector data corresponding to it to improve the accuracy of optical flow estimation.

[0114] In practice, if the bubbles generated when the sample object is immersed in liquid are small, the bubbles appearing in the sample detection image acquired facing the sample object will also be small. In this case, the optical flow estimation model may not be able to accurately identify the optical flow vector of the bubble pixel in the two sample detection images. In this case, the difference between the predicted optical flow vector data corresponding to the second sample detection image sequence and the predicted optical flow vector data corresponding to the second sample detection image sequence may exceed the difference threshold. In this case, continuing to train the optical flow estimation model with the labeled optical flow vector data corresponding to the second sample detection image sequence can improve the estimation ability and accuracy of the optical flow vector of the optical flow estimation model for small bubbles.

[0115] Figure 8 This is a flowchart illustrating the training and testing of an optical flow estimation model according to an embodiment of this application, such as... Figure 8 As shown, it includes:

[0116] Step 810, Data Preparation.

[0117] Specifically, in step 810, when the sample detection object is statically immersed in the liquid, a sequence of sample detection images showing the sample detection object being immersed in the liquid is acquired, and optical flow estimation is performed for each sample detection image sequence using the existing trained optical flow estimation model to determine the labeled optical flow vector data corresponding to each sample detection image sequence.

[0118] Step 820, Data Preprocessing.

[0119] In step 820, the obtained data (i.e., multiple sample detection image sequences and the corresponding labeled optical flow vector data) can be divided according to a set ratio to obtain training data, test data, and validation data. The set ratio can be set according to actual needs, such as a ratio of 7:2:1, 6:3:1, etc. Furthermore, the training data, test data, and validation data can be stored separately in designated file directories for easy data retrieval during the training and testing phases. Correspondingly, the sample detection image sequences in the training data are used as the first sample detection image sequences, and the test data in the test data are used as the second sample detection image sequences.

[0120] Step 830: Train and optimize the optical flow estimation model.

[0121] In step 830, Gaussian noise can be added to each sample detection image in the sample detection image sequence. Then, the two sample detection images to be estimated are input into the optical flow estimation model for optical flow estimation. Using a loss function, combined with the predicted optical flow vector data and the labeled optical flow vector data, the loss value is calculated. At the same time, validation data is used to verify the accuracy of the optical flow estimation model. Then, the weight parameters of the optical flow estimation model are updated slightly along the gradient direction based on the loss value and accuracy. The forward propagation and backward propagation processes are repeated iteratively until the loss value of the optical flow estimation model is minimized or the accuracy is maximized. The specific training process is described in steps 710-740 above and will not be repeated here.

[0122] Optical flow estimation models also involve variable hyperparameters, such as learning rate, batch size, regularization, optimization algorithm, and data augmentation methods. Different hyperparameters affect the loss value, accuracy, and recall of the optical flow estimation model. Therefore, selecting appropriate hyperparameters for the optical flow estimation model is crucial. In a specific embodiment, suitable hyperparameter values ​​can be determined through single-valued comparison tests. That is, for each hyperparameter, the optical flow estimation performance of the model is tested under different values ​​to determine the hyperparameter values ​​that provide the best optical flow estimation performance.

[0123] Step 840: Test the optical flow estimation model.

[0124] Based on the parameters of the trained optical flow estimation model and the test data, the optical flow estimation model is tested. In a specific embodiment, a test script can be invoked to automatically test the optical flow estimation model. The specific testing process is described above.

[0125] Step 850: Determine if there is test problem data during the testing process; if so, analyze the test problem data and incorporate it into the training set, and return to execute steps 820-850 above; if there is no test problem data during the testing process, end the training.

[0126] Test problem data refers to the sequence of detected sample images (second sample detected image sequence) and their corresponding labeled optical flow vector data where the difference between the predicted optical flow vector data and the corresponding labeled optical flow vector data exceeds a difference threshold. In other words, test problem data refers to test samples in the test data where the optical flow estimation model failed to accurately estimate the optical flow vector.

[0127] The solution proposed in this application can be executed by an electronic device equipped with a GPU (Graphics Processing Unit), and can be applied to perform water testing on any gas meter to determine the degree of gas leakage, without restrictions on the type, shape, or material of the gas meter. The operating system running in the electronic device can be Windows, Linux, MacOS, etc., and the hardware operation relies on the GPU.

[0128] The following describes an apparatus embodiment of this application, which can be used to perform the methods described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments described in the above embodiments of this application.

[0129] Figure 9 This is a block diagram illustrating a leak detection device according to an embodiment of this application. This leak detection device can be configured in an electronic device to implement the leak detection method provided in this application. Figure 9 As shown, the air leakage detection device includes: an acquisition module 910, used to acquire a detection image sequence of the object to be detected, the detection image sequence being acquired when the object to be detected is immersed in a liquid; an optical flow estimation module 920, used to perform optical flow estimation based on the detection image sequence to obtain optical flow vector data corresponding to the detection image sequence; and an air leakage detection result determination module 930, used to determine the air leakage detection result of the object to be detected based on the optical flow vector corresponding to the bubble pixel point representing the bubble in the optical flow vector data.

[0130] In some embodiments, the optical flow vector data includes optical flow matrices corresponding to two adjacent detection images in the detection image sequence; the optical flow estimation module 920 includes: a first acquisition unit, used to acquire two adjacent detection images from the detection image sequence; and a first optical flow estimation unit, used to perform optical flow estimation based on the two adjacent detection images using an optical flow estimation model, and output the optical flow matrices corresponding to the two adjacent detection images.

[0131] In other embodiments, the optical flow vector data further includes optical flow matrices corresponding to two non-adjacent detection images in the detection image sequence; the optical flow estimation module 920 further includes: a second acquisition unit, used to acquire two non-adjacent detection images from the detection image sequence; and a second optical flow estimation unit, used to perform optical flow estimation based on the two non-adjacent detection images using an optical flow estimation model, and output the optical flow matrices corresponding to the two non-adjacent detection images.

[0132] In other embodiments, the optical flow vector data includes optical flow matrices corresponding to two non-adjacent detection images in the detection image sequence; the optical flow estimation module 920 includes: a second acquisition unit, used to acquire two non-adjacent detection images from the detection image sequence; and a second optical flow estimation unit, used to perform optical flow estimation based on the two non-adjacent detection images using an optical flow estimation model, and output the optical flow matrices corresponding to the two non-adjacent detection images.

[0133] In some embodiments, the leak detection result determination module 930 includes: a first determination unit, configured to determine the optical flow vector with the largest value among multiple optical flow matrices; a second determination unit, configured to determine the optical flow vector with the largest value as the optical flow vector corresponding to the bubble pixel if the value of the optical flow vector with the largest value is greater than zero; a target leak level determination unit, configured to determine the target leak level corresponding to the optical flow vector corresponding to the bubble pixel according to the mapping relationship between the optical flow vector and the leak level; and a leak detection result determination unit, configured to determine the leak detection result of the object to be detected according to the target leak level.

[0134] In some embodiments, the optical flow estimation model includes a first encoder, a cost volume encoder, and a cost storage decoder; two adjacent detection frames include a source image that is temporally earlier and a target image that is temporally later; in this embodiment, the first optical flow estimation unit includes: a feature extraction unit, used to extract features from the source image and the target image respectively through the first encoder to obtain a first feature map corresponding to the source image and a second feature map corresponding to the target image; a construction unit, used to construct a four-dimensional cost volume feature map based on the first feature map and the second feature map; a cost storage feature map determination unit, used to encode the four-dimensional cost volume feature map through the cost volume encoder to obtain a cost storage feature map; and an optical flow decoding unit, used to perform optical flow decoding on the cost storage feature map through the cost storage decoder to obtain the optical flow matrix corresponding to two adjacent detection frames.

[0135] In some embodiments, the leak detection device further includes: a training data acquisition module, used to acquire training data, the training data including multiple first sample detection image sequences and labeled optical flow vector data corresponding to each first sample detection image sequence; the first sample detection image sequences are collected when the sample detection object is statically immersed in liquid; a first optical flow estimation module, used to perform optical flow estimation based on the first sample detection image sequences using an optical flow estimation model for each first sample detection image sequence, and output predicted optical flow vector data corresponding to the first sample detection image sequence; a loss calculation module, used to calculate a loss value based on the labeled optical flow vector data corresponding to the first sample detection image sequence and the predicted optical flow vector data corresponding to the first sample detection image sequence; and a parameter adjustment module, used to adjust the weight parameters of the optical flow estimation model inversely based on the loss value.

[0136] In some embodiments, the leak detection device further includes: a test data acquisition module, configured to acquire test data, the test data including multiple second sample detection image sequences and labeled optical flow vector data corresponding to each second sample detection image sequence; the second sample detection image sequences are acquired when the sample detection object is statically immersed in liquid; a second optical flow estimation module, configured to perform optical flow estimation on the second sample detection image sequences using a trained optical flow estimation model, and output predicted optical flow vector data corresponding to the second sample detection image sequences; and a continued training module, configured to continue training the optical flow estimation model based on the second sample detection image sequences and the labeled optical flow vector data corresponding to the second sample detection image sequences if the difference between the predicted optical flow vector data corresponding to the second sample detection image sequences and the labeled optical flow vector data corresponding to the second sample detection image sequences exceeds a difference threshold.

[0137] In some embodiments, the air leakage detection device further includes: a noise addition module for adding noise to each sample detection image in the first sample detection image sequence.

[0138] Figure 10 This is a schematic diagram of an electronic device according to an embodiment of this application. The electronic device may be a server used to execute the leakage detection method provided in this application. Figure 10As shown, the electronic device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0139] Those skilled in the art will understand that Figure 10 The structure of the electronic device shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0140] like Figure 10 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a program for implementing a leak detection method. Figure 10 In the illustrated electronic device, the network interface 1004 is mainly used for communication connections with other devices, such as terminals. The user interface 1003 is mainly used for connecting to a client (user terminal) and communicating data with the client; while the processor 1001 can be used to call the program storing the leak detection method in the memory 1005 and execute the steps of the leak detection method as described in any of the above method embodiments.

[0141] According to one aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a processor, implement the air leakage detection method as described in any of the above method embodiments.

[0142] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0143] According to one aspect of the embodiments of this application, a computer program product is provided, which includes computer instructions that, when executed by a processor, implement the air leakage detection method as described in any of the above method embodiments.

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0146] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0147] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0148] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method of detecting a gas leak, the method comprising: include: A detection image sequence of the object to be detected is obtained, wherein the detection image sequence is acquired while the object to be detected is immersed in a liquid; Optical flow estimation is performed based on the detected image sequence to obtain optical flow vector data corresponding to the detected image sequence. The optical flow vector data includes the optical flow matrix corresponding to two adjacent detected images in the detected image sequence and the optical flow matrix corresponding to two non-adjacent detected images in the detected image sequence. Based on the optical flow vector corresponding to the bubble pixel in the optical flow vector data, the leakage detection result of the object to be detected is determined, and the optical flow vector indicates the movement speed and direction of the bubble pixel; The step of determining the leakage detection result of the object to be detected based on the optical flow vector corresponding to the bubble pixel in the optical flow vector data includes: Determine the optical flow vector with the largest value among the multiple optical flow matrices; If the value of the optical flow vector with the largest value is greater than zero, the optical flow vector with the largest value is determined to be the optical flow vector corresponding to the bubble pixel. Based on the mapping relationship between optical flow vector and leakage level, the target leakage level corresponding to the optical flow vector of the bubble pixel is determined; The leakage detection result of the object to be tested is determined based on the target leakage level.

2. The method of claim 1, wherein, The step of performing optical flow estimation based on the detected image sequence to obtain optical flow vector data corresponding to the detected image sequence includes: Obtain two adjacent frames of detection images from the detection image sequence; Optical flow estimation is performed based on the two adjacent detection images using an optical flow estimation model, and the optical flow matrix corresponding to the two adjacent detection images is output.

3. The method of claim 2, wherein, The step of performing optical flow estimation based on the detected image sequence to obtain optical flow vector data corresponding to the detected image sequence further includes: Obtain two non-adjacent detection images from the detection image sequence; The optical flow estimation model estimates optical flow based on the two non-adjacent detection images and outputs the optical flow matrix corresponding to the two non-adjacent detection images.

4. The method of claim 2, wherein, The optical flow estimation model includes a first encoder, a cost volume encoder, and a cost storage decoder; the two adjacent detection images include a source image that is temporally earlier and a target image that is temporally later. Optical flow estimation is performed based on the two adjacent detection frames using an optical flow estimation model, and the optical flow matrix corresponding to the two adjacent detection frames is output, including: The first encoder extracts features from the source image and the target image respectively, to obtain a first feature map corresponding to the source image and a second feature map corresponding to the target image; A four-dimensional cost volume feature map is constructed based on the first feature map and the second feature map. The cost volume feature map is obtained by encoding the four-dimensional cost volume feature map using the cost volume encoder. The cost storage feature map is optically decoded by the cost storage decoder to obtain the optical flow matrix corresponding to the two adjacent detection images.

5. The method according to claim 2 or 3, characterized in that, The method further includes: Acquire training data, which includes multiple first sample detection image sequences and labeled optical flow vector data corresponding to each first sample detection image sequence; the first sample detection image sequences are acquired when the sample detection object is statically immersed in liquid; For each first sample detection image sequence, optical flow estimation is performed based on the first sample detection image sequence using the optical flow estimation model, and the predicted optical flow vector data corresponding to the first sample detection image sequence is output. The loss value is calculated based on the labeled optical flow vector data corresponding to the first sample detection image sequence and the predicted optical flow vector data corresponding to the first sample detection image sequence. The weight parameters of the optical flow estimation model are adjusted in reverse based on the loss value.

6. The method of claim 5, wherein, After adjusting the weight parameters of the optical flow estimation model in reverse based on the loss value, the method further includes: The test data includes multiple second sample detection image sequences and labeled optical flow vector data corresponding to each second sample detection image sequence; the second sample detection image sequences are acquired when the sample detection object is statically immersed in liquid. The optical flow estimation model after training is used to estimate the optical flow of the second sample detection image sequence, and the predicted optical flow vector data corresponding to the second sample detection image sequence is output. If the difference between the predicted optical flow vector data corresponding to the second sample detection image sequence and the labeled optical flow vector data corresponding to the second sample detection image sequence exceeds the difference threshold, the optical flow estimation model is further trained based on the second sample detection image sequence and the labeled optical flow vector data corresponding to the second sample detection image sequence.

7. The method of claim 5, wherein, Before performing optical flow estimation based on the first sample detection image sequence using the optical flow estimation model and outputting the predicted optical flow vector data corresponding to the first sample detection image sequence, the method further includes: Noise is added to each sample detection image in the first sample detection image sequence.

8. A leak detection apparatus, characterized by, include: An acquisition module is used to acquire a sequence of detection images of the object to be detected, wherein the sequence of detection images is acquired when the object to be detected is immersed in a liquid; An optical flow estimation module is used to perform optical flow estimation based on the detection image sequence to obtain optical flow vector data corresponding to the detection image sequence. The optical flow vector data includes the optical flow matrix corresponding to two adjacent detection images in the detection image sequence and the optical flow matrix corresponding to two non-adjacent detection images in the detection image sequence. The air leakage detection result determination module is used to determine the air leakage detection result of the object to be detected based on the optical flow vector corresponding to the bubble pixel in the optical flow vector data, wherein the optical flow vector indicates the movement speed and direction of the bubble pixel. The air leakage detection result determination module includes: a first determination unit, used to determine the optical flow vector with the largest value among the multiple optical flow matrices; The second determining unit is used to determine the optical flow vector with the largest value as the optical flow vector corresponding to the bubble pixel if the value of the optical flow vector with the largest value is greater than zero; the target leakage level determining unit is used to determine the target leakage level corresponding to the optical flow vector corresponding to the bubble pixel according to the mapping relationship between the optical flow vector and the leakage level; the leakage detection result determining unit is used to determine the leakage detection result of the object to be detected according to the target leakage level.

9. An electronic device, comprising: include: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A leak detection system characterized by, include: A transparent container for holding liquid and an object to be tested, such that the object to be tested is immersed in the liquid in the transparent container; An image acquisition device is used to acquire images facing the transparent container to obtain a sequence of detection images of the object to be detected when the object to be detected is immersed in the liquid. An electronic device, communicatively connected to the image acquisition device, is used to determine the leakage detection result of the object to be detected based on the detection image sequence according to the method described in any one of claims 1 to 7.

11. A computer-readable storage medium having stored thereon computer-readable instructions, wherein, When the computer-readable instructions are executed by a processor, the method as described in any one of claims 1 to 7 is implemented.

Images