Pipeline liquid residue detection method, system and device

Abstract

The invention discloses a pipeline liquid residue detection method, system and device, and relates to the technical field of chemical production control, and the method comprises the steps: obtaining a video frame of a transfer pipeline; extracting a region of interest in the video frame; performing multi-scale segmentation on the region of interest; extracting statistical characteristics of the image blocks, and calculating according to the statistical characteristics to obtain a first variable coefficient; deep learning features of the image blocks are extracted, and a second variable coefficient is calculated according to the deep learning features; normalizing the first variable coefficient and the second variable coefficient, and taking the normalized variable coefficient as the weight of the statistical feature and the deep learning feature; performing weighted fusion on the statistical features and the deep learning features; and matching the fusion features with an empty pipeline template. According to the method, the multi-feature dynamic weighted fusion method is adopted, the discrete degrees of the features in different scenes are analyzed, the feature weights are constructed in a self-adaptive mode, cross-scene multi-modal feature fusion is achieved, and the accuracy and robustness of the method are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of chemical production control technology, and in particular to a pipeline liquid residue detection method, system and device. Background Art

[0002] In chemical experiments, it is often necessary to transfer liquid from one instrument to another for the next chemical reaction. Because the amount of liquid has been calculated in advance, it is necessary to ensure that the liquid is completely transferred without excessive residue. In order to determine whether the liquid has been completely transferred, it is necessary to observe the transfer pipeline to confirm whether there is any residual liquid. Since the colors of the liquids used in the experiment are different, and the forms of the liquid remaining in the pipeline are also different, the actual observation scene is very complicated.

[0003] Traditional observation methods rely on manual observation, where researchers rely on their experience to determine the presence of liquid residue. However, this approach suffers from inconsistent standards and is prone to misjudgment. To address this issue, some technologies use image recognition to improve the automation of this process. However, in different experimental scenarios, liquids may have different colors, textures, and shapes. Existing detection methods based on image processing often use fixed features and recognition parameters, resulting in insufficient generalization capabilities. Summary of the Invention

[0004] The embodiments of the present application provide a method, system, and device for detecting liquid residue in a pipeline, which are used to solve the problem of insufficient generalization capability of the existing technology using fixed features and recognition parameters.

[0005] In one aspect, an embodiment of the present application provides a method for detecting liquid residue in a pipeline, comprising:

[0006] Get the video frame of the transfer pipeline;

[0007] Extracting regions of interest in video frames;

[0008] Perform multi-scale segmentation on the region of interest to obtain multiple image blocks of different scales;

[0009] Extracting statistical features of the image blocks at each scale, and calculating the corresponding first coefficient of variation based on the statistical features;

[0010] Extract the deep learning features of the image blocks at each scale, and calculate the corresponding second coefficient of variation based on the deep learning features;

[0011] Normalize the first and second coefficients of variation of the same scale, and use the normalized coefficients of variation as the weights of statistical features and deep learning features respectively;

[0012] According to the weights, statistical features and deep learning features are weighted and fused to obtain fused features;

[0013] The fused features are matched with the empty pipe template of the corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe. If the match is unsuccessful, there is liquid residue in the transfer pipe.

[0014] In one possible implementation, the SegFormer semantic segmentation network is used to extract the region of interest.

[0015] In a possible implementation, the region of interest is segmented at multiple scales according to a preset number of segments and an overlap ratio.

[0016] In a possible implementation, the statistical features include HSV histogram features, HOG features, and Gabor edge histogram features.

[0017] In one possible implementation, the DINOv2 model is used to extract deep learning features.

[0018] In one possible implementation, when determining whether there is liquid residue in the transfer pipeline, the statistical features and deep learning features at the same scale are weightedly fused to obtain fused features at the corresponding scale. If the fused features at at least N scales fail to match the empty pipeline template at the corresponding scale, liquid residue exists in the transfer pipeline.

[0019] On the other hand, an embodiment of the present application further provides a pipeline liquid residue detection system, comprising:

[0020] Video acquisition module, used to obtain video frames of the transfer pipeline;

[0021] A region extraction module is used to extract the region of interest in the video frame;

[0022] The region segmentation module is used to perform multi-scale segmentation on the region of interest to obtain multiple image blocks of different scales;

[0023] A first feature extraction module is used to extract statistical features of image blocks at each scale and calculate a corresponding first coefficient of variation based on the statistical features;

[0024] A second feature extraction module is used to extract deep learning features of image blocks at each scale, and calculate a corresponding second coefficient of variation based on the deep learning features;

[0025] A weight determination module is used to normalize the first coefficient of variation and the second coefficient of variation of the same scale, and use the normalized coefficients of variation as weights of statistical features and deep learning features respectively;

[0026] The feature fusion module is used to fuse statistical features and deep learning features according to weights to obtain fused features;

[0027] The residue detection module is used to match the fused features with the empty pipe template of the corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe. If the match is unsuccessful, there is liquid residue in the transfer pipe.

[0028] On the other hand, an embodiment of the present application further provides a pipeline liquid residue detection device, comprising:

[0029] processor;

[0030] a memory for storing processor-executable instructions;

[0031] Wherein, the processor executes the above-mentioned pipeline liquid residue detection method.

[0032] The present invention provides a method, system, and device for detecting residual liquid in a pipeline, which have the following advantages:

[0033] 1. A multi-feature dynamic weighted fusion method analyzes the discreteness of features in different scenarios and adaptively constructs feature weights, achieving multimodal feature fusion across scenarios. This method not only combines statistical features (such as HSV histograms and HOG features) with deep learning features (such as DINOv2 features), but also fully exploits the basic features of shallow images and the semantic features of deep images, significantly improving the accuracy and robustness of the method.

[0034] 2. A multi-scale template matching method is used to perform multi-scale segmentation of the region of interest (ROI) in the video frame by presetting different segmentation weights. This method enables large-scale images to effectively identify larger residues, while small-scale images focus on capturing microscopic residues, ensuring comprehensive and accurate liquid residue detection and avoiding missed and false positives.

[0035] 3. Using the SegFormer semantic segmentation network, the ROI area can be accurately located in real time, maintaining stability and accuracy even when the device is subject to external interference (such as mechanical vibration or external collision). This greatly enhances the reliability and adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0037] Figure 1 A flow chart of a pipeline liquid residue detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0039] Figure 1 This is a flow chart of a pipeline liquid residue detection method provided in an embodiment of the present application. This embodiment of the present application provides a pipeline liquid residue detection method, including:

[0040] S100, obtaining a video frame of a transfer pipeline.

[0041] For example, an industrial camera can be fixedly installed on the outside of the transfer pipe. The industrial camera can continuously capture the video stream of the transfer pipe during the liquid transfer process, and the video frame can be obtained by extracting a certain frame in the video stream.

[0042] S110: extracting a region of interest from a video frame.

[0043] For example, the SegFormer semantic segmentation network can be used to extract regions of interest to remove background information other than the transfer pipe in the video frame, thereby improving the accuracy and processing efficiency of liquid residue detection.

[0044] It should be understood that the SegFormer semantic segmentation network needs to be trained before use.

[0045] S120 , performing multi-scale segmentation on the region of interest to obtain multiple image blocks of different scales.

[0046] For example, the region of interest may be segmented at multiple scales according to a preset number of segments and overlap ratio. Multi-scale segmentation of the region of interest helps capture detailed features of the liquid residue in the transfer pipeline at various scales.

[0047] Specifically, the method for performing multi-scale segmentation on the region of interest according to the preset number of segmentations and overlap ratio is as follows:

[0048] Each scale has a corresponding number of segments and overlap ratio. For example, at a scale of 320×320, the number of segments is 8 and the overlap ratio is 20%, while at a scale of 240×320, the number of segments is 10 and the overlap ratio is 15%. Taking the 320×320 scale as an example, based on the ROI extracted from S110, a 320×320 sliding window is used, with a 20% overlap ratio, to capture image blocks of different scales horizontally. The specific process is as follows: First, the 320×320 pixel image block at the leftmost position of the ROI is selected as the starting image block. Then, the sliding window is moved horizontally rightward with a 20% overlap ratio, capturing a 320×320 pixel image block with each shift. This process continues until the rightmost boundary of the ROI is reached. If the remaining portion is insufficient to form a 320×320 pixel image block at the rightmost boundary, the sliding window is shifted leftward until the image area covered meets the 320×320 pixel requirement.

[0049] It should be understood that for a region of interest, the image blocks of each scale need to be segmented in sequence. To facilitate template matching, the empty pipeline template also extracts image blocks of different scales in the same way, and these image blocks correspond one-to-one with the image blocks extracted from the region of interest.

[0050] S130 , extracting statistical features of the image blocks at each scale, and calculating a corresponding first coefficient of variation based on the statistical features.

[0051] Exemplarily, the statistical features include HSV histogram features, HOG features, and Gabor edge histogram features.

[0052] It should be understood that after performing multi-scale segmentation on the region of interest, image blocks at multiple scales are obtained. Therefore, after extracting statistical features and calculating the first coefficient of variation for each image block, first coefficients of variation at multiple scales will also be obtained. Moreover, for HSV histogram features, HOG features, and Gabor edge histogram features, each feature will also have multiple first coefficients of variation at different scales.

[0053] S140: Extract deep learning features of the image blocks at each scale, and calculate a corresponding second coefficient of variation based on the deep learning features.

[0054] For example, the DINOv2 model can be used to extract deep learning features. The DINOv2 model is pre-trained on large-scale image datasets using self-supervised learning methods, enabling it to learn rich visual representations without the need for additional annotation. This model utilizes an attention mechanism to capture subtle features and complex patterns in images, making it particularly suitable for multi-scale image analysis tasks.

[0055] Corresponding to the first coefficient of variation, after deep learning features are extracted for each image block and the second coefficient of variation is calculated, the second coefficient of variation at multiple scales will also be obtained.

[0056] S150: Normalize the first coefficient of variation and the second coefficient of variation of the same scale, and use the normalized coefficients of variation as weights of statistical features and deep learning features, respectively.

[0057] For example, the normalized coefficient of variation will be in the range of [0, 1], which can be used as the weight of statistical features and deep learning features.

[0058] S160: Weightedly fuse the statistical features and the deep learning features according to the weights to obtain fused features.

[0059] For example, since the weights in S150 are dynamically adjusted according to the actual scene of each frame, the fused features are highly adaptable and flexible.

[0060] S170, matching the fused features with an empty pipe template of a corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe. If the match is unsuccessful, there is liquid residue in the transfer pipe.

[0061] For example, the empty pipe template also has multiple scales identical to the fused features. During matching, the empty pipe template and the fused features at the same scale are matched. If the fused features at a certain scale successfully match the empty pipe template, it indicates that there is no liquid residue in the video frame at that scale. Otherwise, it indicates that there is liquid residue in the video frame at that scale.

[0062] Specifically, the matching of the fusion feature and the empty pipe template can be achieved by calculating the similarity between the two using a matching algorithm. The matching algorithm can use any one of square difference matching SAD, correlation coefficient matching NCC and gradient correlation matching. If the similarity between the two is less than the matching threshold, it can be considered that the two do not match. If the similarity between the two is greater than or equal to the matching threshold, it can be considered that the two match.

[0063] Furthermore, to determine whether liquid residue exists in the transfer pipe, statistical features and deep learning features at the same scale are weighted and fused to obtain fused features at the corresponding scale. If the fused features at at least N scales fail to match the empty pipe template at the corresponding scale, liquid residue exists in the transfer pipe. If the number of scales where the fused features successfully match the empty pipe template is less than N, it is considered that no liquid residue exists in the transfer pipe.

[0064] The above-mentioned N may be a predetermined fixed value, or may be adjusted as required.

[0065] The present invention also provides a pipeline liquid residue detection system, which includes:

[0066] Video acquisition module, used to obtain video frames of the transfer pipeline;

[0067] A region extraction module is used to extract the region of interest in the video frame;

[0068] The region segmentation module is used to perform multi-scale segmentation on the region of interest to obtain multiple image blocks of different scales;

[0069] A first feature extraction module is used to extract statistical features of image blocks at each scale and calculate a corresponding first coefficient of variation based on the statistical features;

[0070] A second feature extraction module is used to extract deep learning features of image blocks at each scale, and calculate a corresponding second coefficient of variation based on the deep learning features;

[0071] A weight determination module is used to normalize the first coefficient of variation and the second coefficient of variation of the same scale, and use the normalized coefficients of variation as weights of statistical features and deep learning features respectively;

[0072] The feature fusion module is used to fuse statistical features and deep learning features according to weights to obtain fused features;

[0073] The residue detection module is used to match the fused features with the empty pipe template of the corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe. If the match is unsuccessful, there is liquid residue in the transfer pipe.

[0074] The present application also provides a device for detecting residual liquid in a pipeline, comprising:

[0075] processor;

[0076] a memory for storing processor-executable instructions;

[0077] Wherein, the processor executes the above-mentioned pipeline liquid residue detection method.

[0078] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0079] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for detecting liquid residue in a pipeline, characterized in that: include: Get the video frame of the transfer pipeline; Extracting a region of interest in the video frame; Performing multi-scale segmentation on the region of interest to obtain multiple image blocks of different scales; Extracting statistical features of the image block at each scale, and calculating a corresponding first coefficient of variation based on the statistical features; Extracting deep learning features of the image block at each scale, and calculating a corresponding second coefficient of variation based on the deep learning features; Normalizing the first coefficient of variation and the second coefficient of variation of the same scale, and using the normalized coefficients of variation as weights of the statistical feature and the deep learning feature, respectively; Weightedly fusing the statistical features and the deep learning features according to the weights to obtain fused features; The fused features are matched with an empty pipe template of a corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe. If the match is unsuccessful, there is liquid residue in the transfer pipe.

2. A pipeline liquid residue detection method according to claim 1, characterized in that: The SegFormer semantic segmentation network is used to extract the region of interest.

3. A pipeline liquid residue detection method according to claim 1, characterized in that: The region of interest is segmented at multiple scales according to a preset number of segments and an overlap rate.

4. A pipeline liquid residue detection method according to claim 1, characterized in that: The statistical features include HSV histogram features, HOG features and Gabor edge histogram features.

5. The method for detecting residual liquid in a pipeline according to claim 1, characterized in that: The DINOv2 model is used to extract the deep learning features.

6. A pipeline liquid residue detection method according to claim 1, characterized in that: When determining whether there is liquid residue in the transfer pipeline, the statistical features and the deep learning features at the same scale are weightedly fused to obtain the fused features at the corresponding scale. If the fused features at at least N scales fail to match the empty pipeline template at the corresponding scale, liquid residue exists in the transfer pipeline.

7. A pipeline liquid residue detection system, characterized in that: include: Video acquisition module, used to obtain video frames of the transfer pipeline; A region extraction module, configured to extract a region of interest from the video frame; A region segmentation module is used to perform multi-scale segmentation on the region of interest to obtain multiple image blocks of different scales; A first feature extraction module is used to extract statistical features of the image block at each scale, and calculate a corresponding first coefficient of variation based on the statistical features; A second feature extraction module is used to extract deep learning features of the image block at each scale, and calculate a corresponding second variation coefficient based on the deep learning features; a weight determination module, configured to normalize the first coefficient of variation and the second coefficient of variation of the same scale, and use the normalized coefficients of variation as weights of the statistical feature and the deep learning feature, respectively; A feature fusion module, configured to perform weighted fusion of the statistical features and the deep learning features according to the weights to obtain fused features; The residue detection module is used to match the fusion feature with an empty pipe template of a corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe. If the match is unsuccessful, there is liquid residue in the transfer pipe.

8. A pipeline liquid residue detection device, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor executes the pipeline liquid residue detection method described in any one of claims 1-6.

Images