Visual detection system for rubber pipe joint buckling and pressing and detection method thereof

Through industrial camera capture and grayscale processing of the crimped state image of the rubber tube joint, combined with multi-scale feature extraction and feature interactive fusion, the problem that traditional detection methods cannot cover unknown defects and lighting effects is solved, and higher detection accuracy and Intelligent.

CN119048505AInactive Publication Date: 2024-11-29ZHEJIANG JINHAO PIPE IND CO LTD

Patent Information

Application Number
CN202411528303.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-30
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional rubber tube joint buckle visual inspection relies on manual design features and fixed thresholds, cannot effectively cover unknown or variant defects, and is susceptible to light changes and background interference, resulting in limited detection accuracy and misjudgment.

Method used

An industrial camera is used to capture the crimped state image of the rubber tube joint, perform grayscale processing and multi-scale feature extraction, and through combined implicit analysis and feature interaction fusion, a shallow-structure significant interactive fusion representation is generated to automatically judge surface defects.

Benefits of technology

Reduce dependence on manual engineering, dynamically adjust detection standards, improve the adaptability and generalization capabilities of the detection system, reduce the misjudgment rate under light conditions, and enhance the intelligence and accuracy of rubber pipe joint buckle detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048505A_ABST
    Figure CN119048505A_ABST
Patent Text Reader

Abstract

The invention relates to the field of intelligent detection, and particularly discloses a visual detection system and a detection method for buckling a rubber pipe joint, which are characterized in that an industrial camera is used for capturing a buckling state image after the rubber pipe joint is buckled, and gray processing is carried out to obtain a buckling state gray image; and carrying out target detection, multi-scale feature extraction and context association on the pressing and buckling state grayscale image by adopting an image recognition and analysis technology based on computer vision, so as to automatically judge whether surface defects exist or not according to interactive fusion representation between context association features of a pressing and buckling state shallow layer structure and a pressing and buckling state deep layer structure. According to the method, features of different levels can be automatically extracted from the image, the state of the joint is comprehensively reflected, the dependence of traditional manual operation is reduced, and meanwhile, the generalization ability and the intelligent degree of a visual detection system in the aspect of rubber pipe joint buckling detection are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent detection, and more specifically, to a visual detection system and a detection method thereof for crimping of rubber pipe joints. Background Art

[0002] With the rapid development of the manufacturing industry, the requirements for product quality on automated production lines are getting higher and higher. As a key component widely used in many fields such as automobile manufacturing, aerospace, petrochemicals, etc., the quality of rubber pipe joints directly affects the performance and safety of the final product. In these industries, any tiny defect may lead to serious consequences such as leakage, failure or even accident. Therefore, it is crucial to ensure the quality of rubber pipe joints.

[0003] Traditional visual inspection of rubber hose fitting crimping relies on manually designed features and thresholds. Specifically, the traditional method relies on manually designed features, such as edge detection, texture analysis, etc., which are usually pre-defined based on known defect types. However, in actual production, rubber hose fittings may have a variety of unknown or variant defect types, and these manual features may not be able to fully cover all types of defects, resulting in limited detection accuracy. In addition, the use of fixed thresholds to determine the presence or absence of defects is easily affected by factors such as lighting changes and background interference. For example, under different lighting conditions, the same defect may present different grayscale values ​​or contrasts. In this case, fixed thresholds can easily lead to misjudgment, resulting in qualified products being judged as defective or defective products not being identified.

[0004] Therefore, an optimized visual inspection system for crimping of rubber pipe joints is needed to solve the above technical problems. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. According to one aspect of the present application, a visual inspection system for crimping of rubber pipe joints is provided, which comprises: A rubber pipe joint buckle state acquisition module is used to capture a buckle state image of the rubber pipe joint after buckling using an industrial camera; A buckle state grayscale capture module is used to perform grayscale processing on the buckle state image to obtain a buckle state grayscale image, and then perform joint area target detection to obtain a focused local grayscale image of the joint area; A buckle state multi-scale extraction module is used to perform buckle state multi-scale feature extraction and context association on the local grayscale image focused on the joint area to obtain buckle state shallow context association features and buckle state structural context association features; A buckle state feature interaction module is used to perform significant guided interaction between buckle state features on the buckle state shallow context association features and the buckle state structural context association features to obtain a buckle state shallow-structural significant interaction fusion representation, wherein the buckle state feature interaction module includes: a joint implicit analysis activation unit, used to perform joint implicit feature analysis and activation processing on the buckle state shallow context association features and the buckle state structural context association features to obtain a buckle state conditional feature; a buckle state feature fusion unit, used to perform feature interactive fusion on the buckle state shallow context association features and the buckle state structural context association features with the buckle state conditional feature as conditional information to obtain the buckle state shallow-structural significant interaction fusion representation; The detection result generation module is used to obtain the detection result based on the buckle state shallow-structure significant interaction fusion representation.

[0006] In the above-mentioned visual inspection system for crimping of rubber pipe joints, the crimping state grayscale capture module includes: a crimping state image grayscale processing unit, which is used to perform grayscale processing on the crimping state image to obtain the crimping state grayscale image; a crimping state image joint area target detection unit, which is used to input the crimping state grayscale image into a joint area target detection network based on the YOLO model to obtain a focused local grayscale image of the joint area.

[0007] In the above-mentioned visual inspection system for crimping of rubber pipe joints, the crimping state multi-scale extraction module includes: a crimping state multi-scale processing unit, which is used to input the local grayscale image focused on the joint area into a crimping state multi-scale feature extractor based on a hollow pyramid network model to obtain a crimping state shallow feature map and a crimping state structural feature map; a crimping state structural context association unit, which is used to input the crimping state shallow feature map and the crimping state structural feature map into a feature context association encoder based on a converter structure to obtain a crimping state shallow feature context association feature vector as the crimping state shallow context association feature and a crimping state structural context association feature vector as the crimping state structural context association feature.

[0008] In the above-mentioned visual inspection system for crimping of rubber pipe joints, the joint implicit analysis activation unit includes: a crimping state shallow-structural feature joint subunit, which is used to input the crimping state shallow feature context-associated feature vector and the crimping state structural context-associated feature vector into a joint implicit feature capture network to obtain a crimping state shallow-structural context joint implicit feature vector; a crimping state shallow-structural joint implicit feature activation subunit, which is used to perform feature activation based on the Sigmoid function on the crimping state shallow-structural context joint implicit feature vector to obtain a crimping state conditional feature vector as the crimping state conditional feature.

[0009] In the above-mentioned visual inspection system for crimping of rubber pipe joints, the crimping state shallow-structural feature joint subunit is used to: add the crimping state shallow feature context-associated feature vector and the crimping state structural context-associated feature vector by position point, multiply the obtained crimping state shallow-structural sum vector by a weight matrix, and then add it by position with a bias vector to obtain a crimping state shallow-structural joint interaction vector; use a tanh function to process the crimping state shallow-structural joint interaction vector to obtain the crimping state shallow-structural context joint implicit feature vector.

[0010] In the above-mentioned visual inspection system for crimping of rubber pipe joints, the crimping state feature fusion unit includes: a crimping state shallow semantic contribution calculation subunit, which is used to calculate the crimping state shallow semantic contribution of the crimping state shallow feature context association feature vector relative to the crimping state conditional feature vector; a crimping state structural semantic contribution calculation subunit, which is used to calculate the crimping state structural semantic contribution of the crimping state structural context association feature vector relative to the crimping state conditional feature vector; a crimping state shallow structure feature modulation subunit, which is used to normalize the crimping state shallow semantic contribution and the crimping state structural semantic contribution, and use the normalized crimping state shallow semantic contribution and the normalized crimping state structural semantic contribution to modulate the crimping state shallow feature context association feature vector and the crimping state conditional feature vector. The pressed-button state structural context-associated feature vector is weighted modulated to obtain a post-modulation pressed-button state shallow feature context-associated feature vector and a post-modulation pressed-button state structural context-associated feature vector; a pressed-button state shallow-structure significant interaction subunit is used to use the post-modulation pressed-button state shallow feature context-associated feature vector as a query vector, the post-modulation pressed-button state structural context-associated feature vector as a key vector and the pressed-button state conditional feature vector as a value vector, and input the post-modulation pressed-button state shallow feature context-associated feature vector, the post-modulation pressed-button state structural context-associated feature vector and the pressed-button state conditional feature vector into a converter structure-based feature-to-feature significant guided interaction module to obtain a pressed-button state shallow-structure significant interaction fusion representation vector as the pressed-button state shallow-structure significant interaction fusion representation.

[0011] In the above-mentioned visual inspection system for crimping of rubber pipe joints, the crimping state shallow semantic contribution calculation subunit is used to: calculate the positional division of the crimping state shallow feature context-associated feature vector and the crimping state conditional feature vector to obtain the crimping state shallow semantic contribution vector; calculate the base-two logarithmic function value of the absolute value of each eigenvalue of the crimping state shallow semantic contribution vector to obtain the crimping state shallow semantic contribution logarithmic vector; calculate the positional dot product of the crimping state shallow feature context-associated feature vector and the crimping state shallow semantic contribution logarithmic vector, and perform position-by-position multiplication on the obtained dot product vector to obtain the crimping state shallow semantic contribution value; calculate the exponential function of the crimping state shallow semantic contribution value with the natural constant e as the base to obtain the crimping state shallow semantic contribution.

[0012] In the above-mentioned visual inspection system for crimping of rubber pipe joints, the crimping state shallow structural feature modulation subunit is used to: calculate the sum of the crimping state shallow semantic contribution and the crimping state structural semantic contribution to obtain the total sum of the crimping state semantic contribution; divide the crimping state shallow semantic contribution and the crimping state structural semantic contribution by the total sum of the crimping state semantic contribution respectively to obtain the normalized crimping state shallow semantic contribution and the normalized crimping state structural semantic contribution; perform position point multiplication on the crimping state shallow feature context-associated feature vector and the normalized crimping state shallow semantic contribution to obtain the modulated crimping state shallow feature context-associated feature vector; perform position point multiplication on the crimping state structural context-associated feature vector and the normalized crimping state structural semantic contribution to obtain the modulated crimping state structural context-associated feature vector.

[0013] In the above-mentioned visual inspection system for crimping of rubber pipe joints, the inspection result generation module is used to: input the shallow-structure significant interaction fusion representation vector of the crimping state into the classifier-based inspection result generation module to obtain the inspection result, and the inspection result is used to indicate whether there is a surface defect.

[0014] According to another aspect of the present application, a visual inspection method for crimping of a rubber pipe joint is provided, comprising: Use an industrial camera to capture the image of the crimping status of the rubber pipe joint after crimping; Performing grayscale processing on the buckle state image to obtain a buckle state grayscale image, and then performing joint area target detection to obtain a focused local grayscale image of the joint area; Performing buckle state multi-scale feature extraction and context association on the focused local grayscale image of the joint area to obtain buckle state shallow context association features and buckle state structural context association features; The buckle state shallow context association feature and the buckle state structural context association feature are subjected to significant guided interaction between buckle state features to obtain a buckle state shallow-structural significant interaction fusion representation, and the buckle state feature interaction module includes: performing joint implicit feature analysis and activation processing on the buckle state shallow context association feature and the buckle state structural context association feature to obtain a buckle state conditional feature; taking the buckle state conditional feature as conditional information, performing feature interactive fusion on the buckle state shallow context association feature and the buckle state structural context association feature to obtain the buckle state shallow-structural significant interaction fusion representation; Based on the buckle state shallow-structure significant interaction fusion representation, the detection result is obtained.

[0015] The present application has at least the following technical effects: Compared with the prior art, the present application provides a visual inspection system and inspection method for crimping of rubber pipe joints, which uses an industrial camera to capture the crimping state image of the rubber pipe joint after crimping, and grayscale processes the crimping state image to obtain a crimping state grayscale image, and uses image recognition and analysis technology based on computer vision to perform target detection, multi-scale feature extraction and context association on the crimping state grayscale image, so as to automatically judge whether there are surface defects based on the interactive fusion representation between the shallow and deep structural context association features of the crimping state, and can automatically extract features of different scales and different levels from the image to more comprehensively reflect the state of the joint, reducing the traditional method's dependence on manual engineering, and at the same time can dynamically adjust the adaptive ability of the detection system, reduce the traditional fixed threshold under different lighting conditions. Misjudgment, and improve the generalization ability and intelligence of the visual inspection system in rubber pipe joint crimping detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 4 is a block diagram of a visual inspection system for crimping of rubber pipe joints according to an embodiment of the present application.

[0018] Figure 2 The figure is a data flow diagram of a visual inspection system for crimping rubber pipe joints according to an embodiment of the present application.

[0019] Figure 3 It is a block diagram of a buckle state grayscale capture module in a visual inspection system for buckling a rubber pipe joint according to an embodiment of the present application.

[0020] Figure 4 It is a block diagram of a buckle state feature interaction module in a visual inspection system for buckling a rubber pipe joint according to an embodiment of the present application.

[0021] Figure 5 The figure is a flow chart of a visual inspection method for crimping of a rubber pipe joint according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, but rather these embodiments are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0023] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0024] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0025] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0026] With the rapid development of the manufacturing industry, the requirements for product quality on automated production lines are getting higher and higher. As a key component widely used in many fields such as automobile manufacturing, aerospace, petrochemicals, etc., the quality of rubber pipe joints directly affects the performance and safety of the final product. In these industries, any tiny defect may lead to serious consequences such as leakage, failure or even accident. Therefore, it is crucial to ensure the quality of rubber pipe joints.

[0027] In response to the problems in the background technology, the technical concept of this application is to use an industrial camera to capture the buckle state image of the rubber pipe joint after buckling, and grayscale process the buckle state image to obtain the buckle state grayscale image, and use image recognition and analysis technology based on computer vision to perform target detection, multi-scale feature extraction and context association on the buckle state grayscale image, so as to automatically judge whether there are surface defects based on the interactive fusion representation between the shallow and deep structural context association features of the buckle state, and automatically extract features of different scales and different levels from the image to more comprehensively reflect the state of the joint, reducing the traditional method's reliance on manual engineering. At the same time, it can dynamically adjust the adaptive ability of the detection system, reduce the misjudgment of the traditional fixed threshold under different lighting conditions, and improve the generalization ability and intelligence of the visual detection system in the buckle detection of rubber pipe joints.

[0028] Figure 1 4 is a block diagram of a visual inspection system for crimping of rubber pipe joints according to an embodiment of the present application. Figure 2 FIG. 1 is a data flow diagram of a visual inspection system for crimping a rubber pipe joint according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to an embodiment of the present application, a visual inspection system 100 for crimping a rubber pipe joint includes: a rubber pipe joint crimping state acquisition module 110, which is used to use an industrial camera to capture a crimping state image of the rubber pipe joint after crimping; a crimping state grayscale capture module 120, which is used to grayscale process the crimping state image to obtain a crimping state grayscale image and then perform joint area target detection to obtain a focused local grayscale image of the joint area; a crimping state multi-scale extraction module 130, which is used to perform crimping state multi-scale feature extraction and context association on the focused local grayscale image of the joint area to obtain a crimping state shallow context association feature and a crimping state structural context association feature; a crimping state feature interaction module 140, which is used to perform significant guided interaction between crimping state features on the crimping state shallow context association feature and the crimping state structural context association feature to obtain a crimping state shallow-structural significant interaction fusion representation; a detection result generation module 150, which is used to obtain a detection result based on the crimping state shallow-structural significant interaction fusion representation.

[0029] In an embodiment of the present application, the rubber tube joint buckle state acquisition module 110 is used to capture the buckle state image of the rubber tube joint after buckling using an industrial camera. Specifically, considering that the buckle state image contains a variety of information, which is crucial for evaluating the quality of the joint, such as the overall shape and size of the joint, whether there is any damage or defect on the surface, etc., therefore, in order to be able to more finely and accurately evaluate the surface defects of the rubber tube joint buckle in the future, in the technical solution of the present application, an industrial camera is used to capture the buckle state image of the rubber tube joint after buckling. In particular, the industrial camera is a high-performance camera designed for industrial applications, mainly used in the fields of automated detection, measurement, identification and monitoring. Compared with ordinary consumer-grade cameras, industrial cameras have higher performance and reliability, and are suitable for various harsh conditions in industrial environments. Industrial cameras usually have high resolution, and in this application, they can capture fine details by using them, which is very critical for detecting the state of the rubber tube joint after buckling.

[0030] In the embodiment of the present application, the buckle state grayscale capture module 120 is used to perform grayscale processing on the buckle state image to obtain the buckle state grayscale image and then perform joint area target detection to obtain a focused local grayscale image of the joint area. Figure 3 is a block diagram of a buckle state grayscale capture module in a visual inspection system for buckling a rubber pipe joint according to an embodiment of the present application. Specifically, in the embodiment of the present application, Figure 3 As shown, the buckle state grayscale capture module 120 includes: a buckle state image grayscale processing unit 121, which is used to grayscale process the buckle state image to obtain the buckle state grayscale image; a buckle state image joint area target detection unit 122, which is used to input the buckle state grayscale image into the joint area target detection network based on the YOLO model to obtain the joint area focused local grayscale image. Accordingly, considering that the buckle state image contains information such as the shape of the joint, surface defects such as cracks, bubbles, and whether the buckle is in place, and in order to more centrally analyze the buckle shape and defects of the rubber hose joint, in the technical solution of the present application, the buckle state image is grayscale processed to obtain the buckle state grayscale image. Then, in order to effectively locate the position of the rubber hose joint and further focus on the key position of the joint area to provide accurate position information for subsequent defect detection, in the technical solution of the present application, the buckle state grayscale image is input into the joint area target detection network based on the YOLO model to capture and refine the key grayscale area to obtain the joint area focused local grayscale image. In particular, the YOLO model can provide a more accurate bounding box to mark the location of the joint, which allows the system to focus more on the details of the joint area, thereby improving the accuracy and reliability of defect detection.

[0031] In an embodiment of the present application, the buckle state multi-scale extraction module 130 is used to perform buckle state multi-scale feature extraction and context association on the local grayscale image focused on the joint area to obtain buckle state shallow context association features and buckle state structural context association features. Specifically, in an embodiment of the present application, the buckle state multi-scale extraction module includes: a buckle state multi-scale processing unit, which is used to input the local grayscale image focused on the joint area into a buckle state multi-scale feature extractor based on a hollow pyramid network model to obtain a buckle state shallow feature map and a buckle state structural feature map; a buckle state structural context association unit, which is used to input the buckle state shallow feature map and the buckle state structural feature map into a feature context association encoder based on a converter structure to obtain a buckle state shallow feature context association feature vector as the buckle state shallow context association feature and a buckle state structural context association feature vector as the buckle state structural context association feature. Accordingly, it is considered that the local grayscale image focused on the joint area contains feature information of different scales, such as shallow defect information such as edges and textures, and deep structural information inside the body, such as whether there are gaps or cracks, and the alignment between the joint and the pipe. Considering that the hollow pyramid network model can use the hollow convolution technology to expand the receptive field without increasing the network depth, so as to extract feature information of different scales from the image. Based on this, in order to capture feature information of different scales, so as to more comprehensively understand and analyze the buckle state of the rubber pipe joint, in the technical solution of the present application, the local grayscale image focused on the joint area is input into the buckle state multi-scale feature extractor based on the hollow pyramid network model to obtain the buckle state shallow feature map and the buckle state structural feature map, which can reflect the appearance changes and structural properties of the joint surface, and provide a solid foundation for subsequent defect judgment. Further, considering that both the buckle state shallow feature map and the buckle state structural feature map contain associations between contexts of different scales, the converter structure can capture the long-distance context dependencies in the feature map through the self-attention mechanism. Therefore, in order to further process and integrate the information in these feature maps so as to better understand the contextual relationship in the image, in the technical solution of the present application, the pressed state shallow feature map and the pressed state structural feature map are input into a feature context association encoder based on a converter structure to extract and mine the mutual correlation dependencies between different parts, thereby obtaining a pressed state shallow feature context association feature vector and a pressed state structural context association feature vector.

[0032] In the embodiment of the present application, the button state feature interaction module 140 is used to perform significant guided interaction between button state features on the button state shallow context-related features and the button state structural context-related features to obtain a button state shallow-structural significant interaction fusion representation. Figure 4 is a block diagram of a buckle state feature interaction module in a visual inspection system for buckling a rubber pipe joint according to an embodiment of the present application. Specifically, in the embodiment of the present application, Figure 4 As shown, the button state feature interaction module 140 includes: a joint implicit analysis activation unit 141, which is used to perform joint implicit feature analysis and activation processing on the button state shallow context association feature and the button state structural context association feature to obtain the button state conditional feature; a button state feature fusion unit 142, which is used to perform feature interactive fusion on the button state shallow context association feature and the button state structural context association feature with the button state conditional feature as conditional information to obtain the button state shallow-structural significant interactive fusion representation. Specifically, considering that the pressed state shallow feature context-associated features and the pressed state structural context-associated features each carry associated information about the pressed state at different scales, and in order to create a closer connection between these features, thereby enhancing their correlation and complementarity, so that the model can better focus on the key salient areas in the image, in the technical solution of the present application, the pressed state shallow feature context-associated features and the pressed state structural context-associated features are subjected to significant guided interaction between the pressed state features to obtain a pressed state shallow-structural significant interaction fusion representation vector as a pressed state shallow-structural significant interaction fusion representation.

[0033] In detail, first, the shallow feature context-associated feature vector of the buckled state and the structural context-associated feature vector of the buckled state are subjected to joint implicit feature analysis and activation processing to obtain the buckled state conditional feature vector. That is, the joint implicit feature means that in the feature fusion process, a common abstract representation is used to capture the commonalities of the two features. Such an abstract representation can better reflect the essential attributes of the buckled state while also retaining their respective unique information. Specifically, the shared implicit structure and the original features of each between the shallow feature context-associated feature vector of the buckled state and the structural context-associated feature vector of the buckled state are captured and extracted through a joint implicit feature capture network to obtain the buckled state shallow-structural context joint implicit feature vector, and the sigmoid function is used to perform feature activation on the joint implicit feature vector to extract key feature information to obtain the buckled state conditional feature vector.

[0034] Then, taking the buckle state conditional feature as conditional information, the buckle state shallow feature contextual associated feature vector and the buckle state structural contextual associated feature vector are subjected to feature interactive fusion to obtain the buckle state shallow-structural significant interactive fusion representation vector. Specifically, first, the semantic similarity between different scale features and conditional common features is determined by respectively calculating the semantic contribution between the buckle state conditional feature vector and the buckle state shallow feature contextual associated feature vector and the buckle state structural contextual associated feature vector, so as to determine the importance of each feature for the final representation, and obtain the buckle state shallow semantic contribution and the buckle state structural semantic contribution. Then, the two semantic contributions are normalized so that all their weights are on the same scale, and based on the normalized buckle state shallow semantic contribution and the normalized buckle state structural semantic contribution, the buckle state shallow feature contextual associated feature vector and the buckle state structural contextual associated feature vector are feature modulated to obtain the modulated buckle state shallow feature contextual associated feature vector and the modulated buckle state structural contextual associated feature vector. Finally, the post-modulation buckle state shallow feature context-associated feature vector is used as the query vector, the post-modulation buckle state structural context-associated feature vector is used as the key vector, and the buckle state conditional feature vector is used as the value vector, and the converter-based feature significant guided interaction module is input to achieve interaction and fusion between different features, so that the model can better understand the contextual relationship between features and adaptively focus on the most important features, and finally the buckle state shallow-structural significant interaction fusion representation vector is obtained.

[0035] Specifically, in the embodiment of the present application, the joint implicit analysis activation unit 141 includes: a pressed state shallow-structural feature joint subunit, which is used to input the pressed state shallow feature context-associated feature vector and the pressed state structural context-associated feature vector into a joint implicit feature capture network to obtain a pressed state shallow-structural context joint implicit feature vector; and a pressed state shallow-structural joint implicit feature activation subunit, which is used to perform feature activation based on the Sigmoid function on the pressed state shallow-structural context joint implicit feature vector to obtain a pressed state conditional feature vector as the pressed state conditional feature.

[0036] More specifically, the pressed state shallow-structure feature joint subunit is used to: add the pressed state shallow feature context-associated feature vector and the pressed state structural context-associated feature vector point by point, multiply the obtained pressed state shallow-structure sum vector with the weight matrix, and then add it with the bias vector by position to obtain the pressed state shallow-structure joint interaction vector; use the tanh function to process the pressed state shallow-structure joint interaction vector to obtain the pressed state shallow-structure context joint implicit feature vector.

[0037] Specifically, in the embodiment of the present application, the button state feature fusion unit 142 includes: a button state shallow semantic contribution calculation subunit, which is used to calculate the button state shallow semantic contribution of the button state shallow feature context-related feature vector relative to the button state conditional feature vector; a button state structural semantic contribution calculation subunit, which is used to calculate the button state structural semantic contribution of the button state structural context-related feature vector relative to the button state conditional feature vector; a button state shallow structure feature modulation subunit, which is used to normalize the button state shallow semantic contribution and the button state structural semantic contribution, and use the normalized button state shallow semantic contribution and the normalized button state structural semantic contribution to modulate the button state shallow feature context-related feature vector and the button state conditional feature vector. The state structure context-associated feature vector is weighted modulated to obtain a post-modulation pressed-button state shallow feature context-associated feature vector and a post-modulation pressed-button state structure context-associated feature vector; a pressed-button state shallow-structure significant interaction subunit is used to use the post-modulation pressed-button state shallow feature context-associated feature vector as a query vector, the post-modulation pressed-button state structure context-associated feature vector as a key vector and the pressed-button state conditional feature vector as a value vector, and input the post-modulation pressed-button state shallow feature context-associated feature vector, the post-modulation pressed-button state structure context-associated feature vector and the pressed-button state conditional feature vector into a converter structure-based feature-to-feature significant guided interaction module to obtain a pressed-button state shallow-structure significant interaction fusion representation vector as the pressed-button state shallow-structure significant interaction fusion representation.

[0038] More specifically, the pressed state shallow semantic contribution calculation subunit is used to: calculate the positional division of the pressed state shallow feature context-associated feature vector and the pressed state conditional feature vector to obtain the pressed state shallow semantic contribution vector; calculate the logarithmic function value with base two of the absolute value of each eigenvalue of the pressed state shallow semantic contribution vector to obtain the pressed state shallow semantic contribution logarithmic vector; calculate the positional dot product of the pressed state shallow feature context-associated feature vector and the pressed state shallow semantic contribution logarithmic vector, and perform positional multiplication on the obtained dot product vector to obtain the pressed state shallow semantic contribution value; calculate the exponential function of the pressed state shallow semantic contribution value with the natural constant e as the base to obtain the pressed state shallow semantic contribution.

[0039] More specifically, the pressed-button state shallow structural feature modulation subunit is used to: calculate the sum of the pressed-button state shallow semantic contribution and the pressed-button state structural semantic contribution to obtain the total sum of the pressed-button state semantic contributions; respectively divide the pressed-button state shallow semantic contribution and the pressed-button state structural semantic contribution by the total sum of the pressed-button state semantic contributions to obtain the normalized pressed-button state shallow semantic contribution and the normalized pressed-button state structural semantic contribution; perform positional point multiplication of the pressed-button state shallow feature context-associated feature vector and the normalized pressed-button state shallow semantic contribution to obtain the modulated pressed-button state shallow feature context-associated feature vector; perform positional point multiplication of the pressed-button state structural context-associated feature vector and the normalized pressed-button state structural semantic contribution to obtain the modulated pressed-button state structural context-associated feature vector.

[0040] More specifically, the pressed-state shallow-structure significant interaction subunit is used to: perform vector multiplication on the modulated pressed-state shallow feature context-associated feature vector and the transposed vector of the modulated pressed-state structural context-associated feature vector, divide the obtained modulated pressed-state association matrix by the square root of the length of the modulated pressed-state structural context-associated feature vector by position to obtain the modulated pressed-state association scaling matrix; perform activation processing on the modulated pressed-state association scaling matrix using a softmax function, perform matrix-vector multiplication on the obtained modulated pressed-state association scaling activation matrix and the pressed-state conditional feature vector to obtain the pressed-state shallow-structure significant interaction fusion representation vector.

[0041] In the embodiment of the present application, specifically, the buckle state shallow contextual association feature and the buckle state structural contextual association feature are significantly guided to interact with each other between buckle state features, which can be expressed by the formula: in, and are respectively the shallow feature context associated feature vector of the buckle state and the structural context associated feature vector of the buckle state, To add by location, and are the weight matrix and bias vector respectively, for function, is the joint implicit feature vector of the buckled state shallow-structural context, for function, represents the buckle state condition feature vector, is the feature value of each position in the context-related feature vector of the buckle state shallow feature, is the feature value of each position in the context-related feature vector of the buckle state structure, is the eigenvalue of each position in the buckle state condition eigenvector, is the number of eigenvalues ​​in the buckle state shallow feature context associated feature vector and the buckle state structural context associated feature vector, represents the logarithmic function value with base 2, represents an exponential function with the natural constant e as the base, is the shallow semantic contribution of the buttoned state, is the semantic contribution of the buckled state structure, and They are respectively the normalized shallow semantic contribution of the buckled state and the normalized structural semantic contribution of the buckled state. and They are respectively the shallow feature context associated feature vector of the post-modulation buckle state and the structural context associated feature vector of the post-modulation buckle state, yes The transposed vector of yes Length, represents vector multiplication, yes function, is the shallow-structure significant interaction fusion representation vector of the buckled state.

[0042] In an embodiment of the present application, the detection result generation module 150 is used to obtain a detection result based on the shallow-structure significant interaction fusion representation of the buckle state. Specifically, in an embodiment of the present application, the detection result generation module is used to: input the shallow-structure significant interaction fusion representation vector of the buckle state into the detection result generation module based on the classifier to obtain the detection result, and the detection result is used to indicate whether there is a surface defect. That is, the shallow-structure significant interaction fusion representation of the buckle state obtained by significantly interacting with the buckle state shallow feature context association feature vector and the buckle state structural context association feature vector is used for classification processing, so as to automatically determine whether there is a surface defect, and can automatically extract features of different scales from the image, comprehensively reflect the state of the joint, and reduce the reliance on manual engineering. At the same time, it can dynamically adjust the standard for judging defects, reduce the misjudgment of the traditional fixed threshold under different lighting conditions, and significantly improve the accuracy of the buckle state detection of the rubber pipe joint.

[0043] Specifically, in the technical solution of the present application, the buckle state shallow feature context-associated feature vector and the buckle state structural context-associated feature vector respectively represent the image shallow features and image deep semantic features of the local grayscale image focused on the joint area. In this way, when the buckle state features are subjected to significant guided interaction, considering the difference in feature order and feature receptive field between the buckle state shallow feature context-associated feature vector and the buckle state structural context-associated feature vector, this will make the extraction of the joint implicit feature insufficient and inaccurate, resulting in the feature manifold of the buckle state shallow-structure significant interaction fusion representation vector obtained by the significant guided interaction between features with the joint implicit feature as conditional information having expression sparsity and uneven distribution in the high-dimensional feature space, thereby affecting the accuracy of the detection result obtained by the classifier-based detection result generation module.

[0044] Based on this, in a preferred embodiment, the buckle state shallow-structure significant interaction fusion representation vector is input into a detection result generation module based on a classifier to obtain a detection result, including: performing L random sampling on the buckle state shallow-structure significant interaction fusion representation vector to obtain L buckle state random perturbation eigenvalues; arranging the L buckle state random perturbation eigenvalues ​​in order from large to small to obtain a buckle state random perturbation ordered vector; calculating the inner product of the buckle state shallow-structure significant interaction fusion representation vector to obtain a buckle state energy modulation eigenvalue; and performing L random perturbation on the buckle state randomly perturbed ordered vector based on the buckle state energy modulation eigenvalue. The semantic intensity is scaled to obtain a semantically enhanced randomly perturbed ordered vector of the pressed state; the product of the semantically enhanced randomly perturbed ordered vector of the pressed state and the transposed vector of the pressed state shallow-structure significant interaction fusion representation vector is calculated to obtain the pressed state best sample prompt orthogonal modulation matrix; the matrix product of the pressed state best sample prompt orthogonal modulation matrix and the pressed state shallow-structure significant interaction fusion representation vector is calculated to obtain an optimized pressed state shallow-structure significant interaction fusion representation vector; the optimized pressed state shallow-structure significant interaction fusion representation vector is input into the classifier-based detection result generation module to obtain the detection result.

[0045] The above optimization process is expressed as follows: in, The shallow-structure significant interaction fusion representation vector representing the buckled state, It means to calculate the mean of the vector. represents the mean of the shallow-structure significant interaction fusion representation vector of the buckled state, The variance of the shallow-structure significant interaction fusion representation vector representing the buckled state, represents the Gaussian density probability function, The number of eigenvalues ​​in the shallow-structure significant interaction fusion representation vector representing the buckled state, Indicates the Middle The eigenvalues ​​at the positions, and Expressing the The first The random perturbation eigenvalues ​​of the buckle state, Express Arrange them in order from large to small. represents the randomly disturbed ordered vector of the buckle state, represents the square of the vector's two-norm, represents the energy modulation characteristic value of the buckle state, It means point multiplication by position. represents vector multiplication, express The transposed vector of The best example of the buckling state indicates the orthogonal modulation matrix, The optimized buckled state shallow-structure significant interaction fusion representation vector is represented.

[0046] Specifically, by randomly sampling the buckled state shallow-structure significant interaction fusion representation vector L times to obtain a set of new feature values ​​based on the original features but containing slight changes, randomness is introduced to explore different aspects of the feature space, and by simulating the noise or variation in the actual data, it helps the model to better generalize to unseen data. Then, these buckled state random perturbation feature values ​​are arranged in order from large to small to form a buckled state random perturbation ordered vector to identify which perturbation directions may be the most important for the current problem. Such an ordering not only helps the algorithm focus on the feature differences that can best distinguish different categories, but also helps to reveal hidden data structures or patterns. Subsequently, the inner product of the buckled state shallow-structural significant interaction fusion representation vector is calculated to obtain the buckled state energy modulation eigenvalue, which reflects the overall strength or "energy" of the original feature vector and provides a benchmark for measuring the importance of the feature. Using this buckled state energy modulation eigenvalue as a reference point, the previously formed buckled state random perturbation ordered vector is semantically scaled to construct a semantically enhanced buckled state random perturbation ordered vector. In this way, those disturbances that appear to be more important in the original feature space are amplified, thereby enhancing the model's sensitivity to changes in key features, while also retaining relatively small but potentially useful disturbance information.

[0047] In summary, a visual inspection system 100 for crimping of rubber pipe joints based on an embodiment of the present application is explained, which uses an industrial camera to capture the crimping state image of the rubber pipe joint after crimping, and grayscale processes the crimping state image to obtain a crimping state grayscale image, and uses image recognition and analysis technology based on computer vision to perform target detection, multi-scale feature extraction and context association on the crimping state grayscale image, so as to automatically judge whether there are surface defects based on the interactive fusion representation between the shallow and deep structural context association features of the crimping state, and can automatically extract features of different scales and different levels from the image to more comprehensively reflect the state of the joint, reducing the traditional method's reliance on manual engineering. At the same time, it can dynamically adjust the adaptive ability of the detection system, reduce the misjudgment of the traditional fixed threshold under different lighting conditions, and improve the generalization ability and intelligence of the visual inspection system in the crimping detection of rubber pipe joints.

[0048] As described above, the visual inspection system 100 for crimping rubber pipe joints according to the embodiment of the present application can be implemented in various wireless terminals, such as a server having a visual inspection algorithm for crimping rubber pipe joints. In a possible implementation, the visual inspection system 100 for crimping rubber pipe joints according to the embodiment of the present application can be integrated into a wireless terminal as a software module and / or a hardware module. For example, the visual inspection system 100 for crimping rubber pipe joints can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the visual inspection system 100 for crimping rubber pipe joints can also be one of the many hardware modules of the wireless terminal.

[0049] Alternatively, in another example, the visual inspection system 100 for crimping rubber pipe joints and the wireless terminal may also be separate devices, and the visual inspection system 100 for crimping rubber pipe joints may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0050] Figure 5 FIG. 1 is a flow chart of a visual inspection method for crimping a rubber pipe joint according to an embodiment of the present application. Figure 5As shown, according to the embodiment of the present application, the visual inspection method for crimping of a rubber pipe joint includes: S110, using an industrial camera to capture a crimping state image of the rubber pipe joint after crimping; S120, performing grayscale processing on the crimping state image to obtain a crimping state grayscale image and then performing joint area target detection to obtain a focused local grayscale image of the joint area; S130, performing multi-scale feature extraction and context association on the focused local grayscale image of the joint area to obtain a crimping state shallow context association feature and a crimping state structural context association feature; S140, performing multi-scale feature extraction and context association on the crimping state shallow context association feature and the crimping state structural context association feature. The context-related features are used to significantly guide the interaction between the buckle state features to obtain the buckle state shallow-structural significant interaction fusion representation, including: performing joint implicit feature analysis and activation processing on the buckle state shallow context-related features and the buckle state structural context-related features to obtain the buckle state conditional features; taking the buckle state conditional features as conditional information, performing feature interaction fusion on the buckle state shallow context-related features and the buckle state structural context-related features to obtain the buckle state shallow-structural significant interaction fusion representation; S150, obtaining the detection result based on the buckle state shallow-structural significant interaction fusion representation.

[0051] The specific operations of each step in the above-mentioned visual inspection method for crimping rubber pipe joints have been referred to above. Figures 1 to 4 The description of the visual inspection system for crimping of rubber pipe joints has been described in detail, and therefore, its repeated description will be omitted.

[0052] The above descriptions of various implementations of the present disclosure are exemplary and not exhaustive. The disclosure is not limited to the disclosed implementations, and many modifications and variations are obvious to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The terms used herein are selected to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various practical methods disclosed herein.

Claims

1. A visual inspection system for crimping rubber pipe joints, characterized in that: include: A rubber pipe joint buckle state acquisition module is used to capture a buckle state image of the rubber pipe joint after buckling using an industrial camera; A buckle state grayscale capture module is used to perform grayscale processing on the buckle state image to obtain a buckle state grayscale image, and then perform joint area target detection to obtain a focused local grayscale image of the joint area; A buckle state multi-scale extraction module is used to perform buckle state multi-scale feature extraction and context association on the local grayscale image focused on the joint area to obtain buckle state shallow context association features and buckle state structural context association features; A buckle state feature interaction module, used for performing significant guided interaction between buckle state features on the buckle state shallow contextual association features and the buckle state structural contextual association features to obtain a buckle state shallow-structural significant interaction fusion representation; The buckle state feature interaction module includes: a joint implicit analysis activation unit, which is used to perform joint implicit feature analysis and activation processing on the buckle state shallow context association feature and the buckle state structural context association feature to obtain the buckle state conditional feature; a buckle state feature fusion unit, which is used to perform feature interactive fusion on the buckle state shallow context association feature and the buckle state structural context association feature with the buckle state conditional feature as conditional information to obtain the buckle state shallow-structural significant interactive fusion representation; The detection result generation module is used to obtain the detection result based on the buckle state shallow-structure significant interaction fusion representation.

2. The visual inspection system for crimping of rubber pipe joints according to claim 1, characterized in that: The buckle state grayscale capture module includes: A buckle state image grayscale processing unit, used for performing grayscale processing on the buckle state image to obtain the buckle state grayscale image; The buckle state image joint area target detection unit is used to input the buckle state grayscale image into the joint area target detection network based on the YOLO model to obtain the focused local grayscale image of the joint area.

3. The visual inspection system for crimping of rubber pipe joints according to claim 2, characterized in that: The buckle state multi-scale extraction module comprises: A buckle state multi-scale processing unit, used for inputting the focused local grayscale image of the joint area into a buckle state multi-scale feature extractor based on a hollow pyramid network model to obtain a buckle state shallow feature map and a buckle state structural feature map; A button state structure context association unit is used to input the button state shallow feature map and the button state structure feature map into a feature context association encoder based on a converter structure to obtain a button state shallow feature context association feature vector as the button state shallow context association feature and a button state structure context association feature vector as the button state structure context association feature.

4. The visual inspection system for crimping of rubber pipe joints according to claim 3, characterized in that: The joint implicit analysis activation unit comprises: A buckle state shallow-structure feature joint subunit, used for inputting the buckle state shallow feature context associated feature vector and the buckle state structural context associated feature vector into a joint implicit feature capture network to obtain a buckle state shallow-structure context joint implicit feature vector; The buckle state shallow-structure joint implicit feature activation subunit is used to perform feature activation based on the Sigmoid function on the buckle state shallow-structure context joint implicit feature vector to obtain a buckle state conditional feature vector as the buckle state conditional feature.

5. The visual inspection system for crimping of rubber pipe joints according to claim 4, characterized in that: The buckle state shallow layer-structural feature combined subunit is used for: After adding the buckle state shallow feature contextual associated feature vector and the buckle state structural contextual associated feature vector by position point, the obtained buckle state shallow-structural sum vector is multiplied by the weight matrix and then added by position with the bias vector to obtain the buckle state shallow-structural joint interaction vector; The pressed state shallow-structure joint interaction vector is processed using a tanh function to obtain the pressed state shallow-structure context joint implicit feature vector.

6. The visual inspection system for crimping of rubber pipe joints according to claim 5, characterized in that: The buckle state feature fusion unit comprises: A buckle state shallow semantic contribution calculation subunit, used to calculate the buckle state shallow semantic contribution of the buckle state shallow feature context association feature vector relative to the buckle state condition feature vector; A buckle state structure semantic contribution calculation subunit, used to calculate the buckle state structure semantic contribution of the buckle state structure contextual association feature vector relative to the buckle state condition feature vector; A buckle state shallow structure feature modulation subunit, used for normalizing the buckle state shallow semantic contribution and the buckle state structure semantic contribution, and using the normalized buckle state shallow semantic contribution and the normalized buckle state structure semantic contribution to weighted modulate the buckle state shallow feature context-associated feature vector and the buckle state structure context-associated feature vector, so as to obtain a modulated buckle state shallow feature context-associated feature vector and a modulated buckle state structure context-associated feature vector; The button state shallow-structure significant interaction subunit is used to use the post-modulation button state shallow feature context-associated feature vector as a query vector, the post-modulation button state structural context-associated feature vector as a key vector and the button state conditional feature vector as a value vector, and input the post-modulation button state shallow feature context-associated feature vector, the post-modulation button state structural context-associated feature vector and the button state conditional feature vector into a feature-to-feature significant guided interaction module based on a converter structure to obtain a button state shallow-structure significant interaction fusion representation vector as the button state shallow-structure significant interaction fusion representation.

7. The visual inspection system for crimping of rubber pipe joints according to claim 6, characterized in that: The button state shallow semantic contribution calculation subunit is used to: Calculating the positional division of the buckle state shallow feature contextual association feature vector and the buckle state conditional feature vector to obtain a buckle state shallow semantic contribution vector; Calculating the logarithmic function value with base 2 of the absolute value of each eigenvalue of the pressed state shallow semantic contribution vector to obtain the pressed state shallow semantic contribution logarithmic vector; Calculate the position-by-position dot product of the context-related feature vector of the pressed state shallow feature and the logarithmic vector of the pressed state shallow semantic contribution, and perform position-by-position multiplication on the obtained dot product vector to obtain the shallow semantic contribution value of the pressed state; An exponential function of the shallow semantic contribution value of the pressed state with the natural constant e as the base is calculated to obtain the shallow semantic contribution degree of the pressed state.

8. The visual inspection system for crimping of rubber pipe joints according to claim 7, characterized in that: The buckled state shallow structure characteristic modulation subunit is used for: Calculating the sum of the shallow semantic contribution of the buckled state and the structural semantic contribution of the buckled state to obtain a total sum of the semantic contributions of the buckled state; Respectively dividing the shallow semantic contribution of the buckled state and the structural semantic contribution of the buckled state by the total sum of the semantic contributions of the buckled state to obtain the normalized shallow semantic contribution of the buckled state and the normalized structural semantic contribution of the buckled state; Multiplying the buckle state shallow feature contextual association feature vector by the normalized buckle state shallow semantic contribution by position point to obtain the modulated buckle state shallow feature contextual association feature vector; The buckle state structure context-related feature vector is multiplied by the normalized buckle state structure semantic contribution according to the position point to obtain the modulated buckle state structure context-related feature vector.

9. The visual inspection system for crimping of rubber pipe joints according to claim 8, characterized in that: The detection result generation module is used to: input the buckle state shallow-structure significant interaction fusion representation vector into the classifier-based detection result generation module to obtain the detection result, and the detection result is used to indicate whether there is a surface defect.

10. A visual inspection method for crimping of rubber pipe joints, characterized in that: include: Use an industrial camera to capture the image of the crimping status of the rubber pipe joint after crimping; Performing grayscale processing on the buckle state image to obtain a buckle state grayscale image, and then performing joint area target detection to obtain a focused local grayscale image of the joint area; Performing buckle state multi-scale feature extraction and context association on the focused local grayscale image of the joint area to obtain buckle state shallow context association features and buckle state structural context association features; The buckle state shallow context association feature and the buckle state structural context association feature are subjected to significant guided interaction between buckle state features to obtain a buckle state shallow-structural significant interaction fusion representation, and the buckle state feature interaction module includes: performing joint implicit feature analysis and activation processing on the buckle state shallow context association feature and the buckle state structural context association feature to obtain a buckle state conditional feature; taking the buckle state conditional feature as conditional information, performing feature interactive fusion on the buckle state shallow context association feature and the buckle state structural context association feature to obtain the buckle state shallow-structural significant interaction fusion representation; Based on the buckle state shallow-structure significant interaction fusion representation, the detection result is obtained.

Citation Information

Patent Citations

  • Air conditioner soft and hard pipe connection sealing device and method thereof

    CN116045104A

  • Online visual detection system for high-speed dispensing

    CN117392134A

  • Drunk driving detection method and system based on sensor and machine vision

    CN118457218A

  • Automatic rubber sealing element production line and method thereof

    CN118688206A

  • Deep-learning-based target detection method for defects of image of inner side of commutator

    WO2024208102A1

Cited By

  • Supply relation management method based on artificial intelligence

    CN119417411A

  • Intelligent construction hoist control system

    CN119429885A