A pipeline liquid residue detection method, system and device
Patent Information
- Application Number
- CN202510714993.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-30
- Publication Date
- 2026-10-27
- Estimated Expiration
- 2045-05-30
AI Technical Summary
[0004]本申请实施例提供了一种管道液体残留检测方法、系统及装置，用以解决现有技术中采用固定特征和识别参数存在泛化能力不足的问题
[0033]1.采用多特征动态加权融合方法，通过分析不同场景下特征的离散程度，自适应地构建特征权重，实现了跨场景的多模态特征融合。该方法不仅联合了统计特征(如HSV直方图、HOG特征等)和深度学习特征(如DINOv2特征)，还充分挖掘了浅层图像的基本特征与深层图像的语义特征，从而显著提高了方法的准确性和鲁棒性。
Smart Images
Figure CN120564104B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of chemical production control technology, and in particular to a method, system and device for detecting liquid residues in pipelines. Background Technology
[0002] In chemical experiments, it is often necessary to transfer liquids from one apparatus to another for the next chemical reaction. Since the amount of liquid has been calculated in advance, it is crucial to ensure complete transfer and prevent excessive residue. To determine if the transfer was complete, the transfer tube needs to be observed to confirm the absence of residual liquid. Because the liquids used in experiments vary in color, and the form of liquid residue in the tube also differs, the actual observation scenario is quite complex.
[0003] Traditional observation methods rely on manual observation, with researchers judging liquid residue based on experience. However, this approach lacks standardized criteria and is prone to misjudgment. To address this issue, some technologies employ image recognition to improve the automation of the process. However, in different experimental scenarios, liquids may exhibit different colors, textures, and shapes. Existing image processing-based detection methods often use fixed features and recognition parameters, resulting in insufficient generalization ability. Summary of the Invention
[0004] This application provides a method, system, and apparatus for detecting residual liquid in pipelines, which addresses the problem of insufficient generalization ability in existing technologies that use fixed features and identification parameters.
[0005] On one hand, embodiments of this application provide a method for detecting residual liquid in pipelines, including:
[0006] Acquire video frames from the transfer pipeline;
[0007] Extracting the region of interest from a video frame;
[0008] The region of interest is segmented at multiple scales to obtain multiple image patches of different scales;
[0009] Extract the statistical features of the image patch at each scale, and calculate the corresponding first coefficient of variation based on the statistical features;
[0010] Deep learning features of image patches at each scale are extracted, and the corresponding second coefficient of variation is calculated based on the deep learning features;
[0011] The first and second coefficients of variation at the same scale are normalized, and the normalized coefficients of variation are used as the weights of the statistical features and the deep learning features, respectively.
[0012] The statistical features and deep learning features are weighted and fused according to their respective weights to obtain the fused features.
[0013] The fusion features are matched with empty pipe templates of the corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe; if the match is unsuccessful, there is liquid residue in the transfer pipe.
[0014] In one possible implementation, the SegFormer semantic segmentation network is used to extract the region of interest.
[0015] In one possible implementation, the region of interest is segmented at multiple scales based on a preset number of segments and an overlap rate.
[0016] In one possible implementation, statistical features include HSV histogram features, HOG features, and Gabor edge histogram features.
[0017] In one possible implementation, the DINOv2 model is used to extract deep learning features.
[0018] In one possible implementation, when determining whether there is liquid residue in the transfer pipeline, statistical features and deep learning features at the same scale are weighted and fused to obtain fused features at the corresponding scale. If at least N fused features at different scales fail to match the empty pipeline template at the corresponding scale, then there is liquid residue in the transfer pipeline.
[0019] On the other hand, embodiments of this application also provide a pipeline liquid residue detection system, including:
[0020] The video acquisition module is used to acquire video frames from the transfer pipeline;
[0021] The region extraction module is used to extract regions of interest from video frames;
[0022] The region segmentation module is used to perform multi-scale segmentation of the region of interest, resulting in multiple image patches of different scales.
[0023] The first feature extraction module is used to extract the statistical features of image patches at each scale and calculate the corresponding first coefficient of variation based on the statistical features.
[0024] The second feature extraction module is used to extract deep learning features of image patches at each scale and calculate the corresponding second coefficient of variation based on the deep learning features.
[0025] The weight determination module is used to normalize the first and second coefficients of variation at the same scale, and use the normalized coefficients of variation as the weights of the statistical features and deep learning features, respectively.
[0026] The feature fusion module is used to weight and fuse statistical features and deep learning features according to their weights to obtain fused features;
[0027] The residue detection module is used to match the fused features with an empty pipe template of the corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe; if the match is unsuccessful, there is liquid residue in the transfer pipe.
[0028] On the other hand, embodiments of this application also provide a pipeline liquid residue detection device, including:
[0029] processor;
[0030] Memory used to store processor-executable instructions;
[0031] The processor executes the aforementioned method for detecting residual liquid in pipelines.
[0032] The pipeline liquid residue detection method, system, and apparatus disclosed in this application have the following advantages:
[0033] 1. A multi-feature dynamic weighted fusion method is adopted. By analyzing the dispersion of features under different scenarios, feature weights are adaptively constructed to achieve multi-modal feature fusion across scenarios. This method not only combines statistical features (such as HSV histograms and HOG features) and deep learning features (such as DINOv2 features), but also fully explores the basic features of shallow images and the semantic features of deep images, thereby significantly improving the accuracy and robustness of the method.
[0034] 2. A multi-scale template matching method is employed, which segments the region of interest (ROI) in video frames at multiple scales by pre-setting different segmentation weights. This method enables large-scale images to effectively identify larger residues, while small-scale images focus on capturing tiny residues, ensuring comprehensive and accurate liquid residue detection and avoiding false negatives and false negatives.
[0035] 3. Utilizing the SegFormer semantic segmentation network, the ROI region can be located accurately in real time, maintaining stability and accuracy even when the device is subjected to external interference (such as mechanical vibration or external impact). This greatly enhances the system's reliability and adaptability. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 A flowchart of a pipeline liquid residue detection method provided in an embodiment of this application. Detailed Implementation
[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0039] Figure 1 A flowchart illustrating a pipeline liquid residue detection method provided in this application embodiment. This application embodiment provides a pipeline liquid residue detection method, including:
[0040] S100, acquire video frames from the transfer pipeline.
[0041] For example, an industrial camera can be fixedly installed on the outside of the transfer pipe. The industrial camera can continuously capture video streams of the transfer pipe during the liquid transfer process. A video frame can be obtained by extracting a frame from the video stream.
[0042] S110, extract the region of interest from the video frame.
[0043] For example, the SegFormer semantic segmentation network can be used to extract regions of interest to remove background information outside the transfer pipe in video frames, thereby improving the accuracy and processing efficiency of liquid residue detection.
[0044] It should be understood that the SegFormer semantic segmentation network needs to be trained before use.
[0045] S120 performs multi-scale segmentation on the region of interest to obtain multiple image patches of different scales.
[0046] For example, the region of interest can be segmented at multiple scales according to a preset number of segments and overlap rate. Multi-scale segmentation of the region of interest helps to capture the detailed features of liquid residues in the transfer pipeline at various scales.
[0047] Specifically, the method for multi-scale segmentation of the region of interest based on a preset number of segments and overlap rate is as follows:
[0048] Each scale has a corresponding number of segments and overlap rate. For example, at a scale of 320×320, the number of segments is 8, and the overlap rate is 20%, while at a scale of 240×320, the number of segments is 10, and the overlap rate is 15%. Taking the 320×320 scale as an example, based on the region of interest extracted from S110, a sliding window of size 320×320 is used, and image patches of different scales are cropped horizontally with a 20% overlap rate. The specific process is as follows: First, the 320×320 pixel image at the leftmost position of the region of interest is selected as the starting image patch. Then, the sliding window is moved to the right horizontally with a 20% overlap rate, and a 320×320 pixel image patch is cropped after each move. This process continues until the rightmost boundary of the region of interest is reached. If, when the rightmost boundary is reached, the remaining part is insufficient to form a 320×320 pixel image patch, the sliding window is moved to the left until the image area it covers meets the 320×320 pixel requirement.
[0049] It should be understood that for a region of interest, image patches of each scale need to be segmented sequentially. To facilitate template matching, the empty pipeline template also extracts image patches of different scales in the same way, and these image patches correspond one-to-one with the image patches extracted from the region of interest.
[0050] S130: Extract the statistical features of the image patch at each scale, and calculate the corresponding first coefficient of variation based on the statistical features.
[0051] For example, statistical features include HSV histogram features, HOG features, and Gabor edge histogram features.
[0052] It should be understood that after multi-scale segmentation of the region of interest, multiple image patches at different scales are obtained. Therefore, after extracting statistical features and calculating the first coefficient of variation for each image patch, multiple first coefficients of variation at different scales will also be obtained. Moreover, for HSV histogram features, HOG features, and Gabor edge histogram features, each feature will also have multiple first coefficients of variation at different scales.
[0053] S140: Extract deep learning features of image patches at each scale, and calculate the corresponding second coefficient of variation based on the deep learning features.
[0054] For example, the DINOv2 model can be used to extract deep learning features. The DINOv2 model is pre-trained on large-scale image datasets using a self-supervised learning method, enabling it to learn rich visual representations without additional annotations. This model leverages an attention mechanism to capture subtle features and complex patterns in images, making it particularly suitable for multi-scale image analysis tasks.
[0055] Corresponding to the first coefficient of variation, after extracting deep learning features for each image patch and calculating the second coefficient of variation, the second coefficient of variation at multiple scales will also be obtained.
[0056] S150 normalizes the first and second coefficients of variation at the same scale, and uses the normalized coefficients of variation as the weights of the statistical features and deep learning features, respectively.
[0057] For example, the normalized coefficient of variation will be in the range of [0,1], which can be used as the weight of statistical features and deep learning features.
[0058] S160, the statistical features and deep learning features are weighted and fused according to their weights to obtain the fused features.
[0059] For example, since the weights in S150 are dynamically adjusted based on the actual scene of each frame, the fused features have a high degree of adaptability and flexibility.
[0060] S170, the fused features are matched with the empty pipe template of the corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe. If the match is unsuccessful, there is liquid residue in the transfer pipe.
[0061] For example, the empty pipe template also has multiple scales, the same as the fusion feature. During matching, empty pipe templates and fusion features of the same scale are matched. If the fusion feature and empty pipe template at a certain scale match successfully, it means that there is no liquid residue in the video frame at that scale; otherwise, it means that there is liquid residue in the video frame at that scale.
[0062] Specifically, the matching of fused features and empty pipeline templates can be achieved by calculating their similarity using a matching algorithm. This matching algorithm can be any one of squared difference matching (SAD), correlation coefficient matching (NCC), and gradient correlation matching. If the similarity between the two is less than the matching threshold, they can be considered mismatched. If the similarity between the two is greater than or equal to the matching threshold, they can be considered matched.
[0063] Furthermore, when determining whether liquid residue exists in the transfer pipeline, statistical features and deep learning features at the same scale are weighted and fused to obtain fused features at the corresponding scale. If at least N fused features at different scales fail to match the empty pipeline template at the corresponding scale, then liquid residue exists in the transfer pipeline. If the number of scales where the fused features successfully match the empty pipeline template is less than N, then it is considered that no liquid residue exists in the transfer pipeline.
[0064] The N mentioned above can be a predetermined fixed value, or it can be adjusted according to needs.
[0065] This application embodiment also provides a pipeline liquid residue detection system, the system comprising:
[0066] The video acquisition module is used to acquire video frames from the transfer pipeline;
[0067] The region extraction module is used to extract regions of interest from video frames;
[0068] The region segmentation module is used to perform multi-scale segmentation of the region of interest, resulting in multiple image patches of different scales.
[0069] The first feature extraction module is used to extract the statistical features of image patches at each scale and calculate the corresponding first coefficient of variation based on the statistical features.
[0070] The second feature extraction module is used to extract deep learning features of image patches at each scale and calculate the corresponding second coefficient of variation based on the deep learning features.
[0071] The weight determination module is used to normalize the first and second coefficients of variation at the same scale, and use the normalized coefficients of variation as the weights of the statistical features and deep learning features, respectively.
[0072] The feature fusion module is used to weight and fuse statistical features and deep learning features according to their weights to obtain fused features;
[0073] The residue detection module is used to match the fused features with an empty pipe template of the corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe; if the match is unsuccessful, there is liquid residue in the transfer pipe.
[0074] This application also provides a pipeline liquid residue detection device, including:
[0075] processor;
[0076] Memory used to store processor-executable instructions;
[0077] The processor executes the aforementioned method for detecting residual liquid in pipelines.
[0078] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.
[0079] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A method for detecting residual liquid in pipelines, characterized in that, include: Acquire video frames from the transfer pipeline; Extract the region of interest from the video frame; The region of interest is segmented at multiple scales to obtain multiple image patches of different scales; Statistical features of the image patch at each scale are extracted, and the corresponding first coefficient of variation is calculated based on the statistical features; the statistical features include HSV histogram features, HOG features, and Gabor edge histogram features; Deep learning features of the image patch at each scale are extracted, and the corresponding second coefficient of variation is calculated based on the deep learning features; The first and second coefficients of variation at the same scale are normalized, and the normalized coefficients of variation are used as the weights of the statistical feature and the deep learning feature, respectively. The statistical features and the deep learning features are weighted and fused according to the weights to obtain the fused features; The fusion feature is matched with an empty pipe template of the corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe. If the match is unsuccessful, there is liquid residue in the transfer pipe.
2. The method for detecting residual liquid in a pipeline according to claim 1, characterized in that, The region of interest is extracted using the SegFormer semantic segmentation network.
3. The method for detecting residual liquid in a pipeline according to claim 1, characterized in that, The region of interest is segmented at multiple scales based on a preset number of segments and overlap rate.
4. The method for detecting residual liquid in a pipeline according to claim 1, characterized in that, The deep learning features were extracted using the DINOv2 model.
5. The method for detecting residual liquid in a pipeline according to claim 1, characterized in that, When determining whether there is liquid residue in the transfer pipeline, the statistical features and deep learning features at the same scale are weighted and fused to obtain the fused features at the corresponding scale. If the fused features at at least N scales fail to match the empty pipeline template at the corresponding scale, then there is liquid residue in the transfer pipeline.
6. A pipeline liquid residue detection system, characterized in that, include: The video acquisition module is used to acquire video frames from the transfer pipeline; A region extraction module is used to extract the region of interest in the video frame; The region segmentation module is used to perform multi-scale segmentation on the region of interest to obtain multiple image blocks of different scales; The first feature extraction module is used to extract statistical features of the image patch at each scale and calculate the corresponding first coefficient of variation based on the statistical features; the statistical features include HSV histogram features, HOG features and Gabor edge histogram features; The second feature extraction module is used to extract deep learning features of the image patch at each scale and calculate the corresponding second coefficient of variation based on the deep learning features. The weight determination module is used to normalize the first coefficient of variation and the second coefficient of variation at the same scale, and use the normalized coefficient of variation as the weights of the statistical feature and the deep learning feature, respectively. The feature fusion module is used to weight and fuse the statistical features and the deep learning features according to the weights to obtain fused features; The residue detection module is used to match the fused features with an empty pipe template of the corresponding scale. If the match is successful, there is no liquid residue in the transfer pipe; if the match is unsuccessful, there is liquid residue in the transfer pipe.
7. A pipeline liquid residue detection device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor executes the pipeline liquid residue detection method according to any one of claims 1-5.
Citation Information
Patent Citations
Video processing method and device based on deep learning
CA3164081A1
Pipeline defect image multi-modal feature fusion method and device
CN119904727A