A method of detecting the completion of a pipe fitting accessory

Abstract

The application discloses a pipe accessory completion degree detection system and method, wherein the detection system comprises: a detection platform for placing a pipe to be detected; an image acquisition module located above the detection platform, comprising a light source unit and an image acquisition unit, the image acquisition unit being used for acquiring image information of the pipe to be detected; an image processing module in communication connection with the image acquisition module for receiving the image information, and further used for identifying accessory features of the pipe to be detected according to a detection model; wherein the image processing module is further used for saving accessory standard features of the pipe to be detected, and outputting a completion degree detection conclusion according to a comparison result of the accessory standard features and the accessory features. The application realizes detection of general accessories on the pipe, improves the detection efficiency and the detection accuracy of the finished pipe completion degree, and is suitable for pipes of different specifications and different materials.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual detection, and in particular to a detection system and method for the completion degree of a pipe accessory. BACKGROUND

[0002] With the development of the aviation and automobile industries, higher requirements are placed on the quality and production efficiency of pipe processing and assembly, and the automatic and rapid detection of various special-shaped pipe accessories has become a new direction for pipe measurement.

[0003] In a traditional automatic pipe detection process, a process personnel adjusts the processing parameters (such as the bending point and bending radius) of a pipe bending machine according to the 3D measurement data (sheath tolerance) obtained by a pipe measurement software, to ensure the smooth completion of subsequent pipe assembly work. The obtained 3D detection data and the process parameters that need to be adjusted are fed back to the pipe bending machine of various types for compensation. For a traditional pipe measurement device, due to the performance of the data acquisition camera and the function of the measurement analysis software, only the edge of the pipe can be analyzed through the gray value (black and white) of the photo, so as to extract the 3D center line of the pipe. The following information of the pipe accessory cannot be obtained, such as the type, quantity and relative position of the different types of straps, the label information (quantity and relative position), the color marking information (color, quantity and relative position information) of different colors, and the position and quantity of the protective net.

[0004] However, the detection data of these accessories is the key information for determining whether a finished pipe is qualified. At present, the pipe manufacturers detect in the following way: a traditional pipe measurement device (3D sheath tolerance inspection) and manual visual inspection of the accessories, which are combined. The detection of the completion degree of the pipe accessory has become a bottleneck in the automatic detection of the pipe.

[0005] In summary, there is a need to design a detection system and method for the completion degree of a pipe accessory to solve the above problems in the prior art. SUMMARY

[0006] To solve the above problems in the prior art, the present application provides a detection system and method for the completion degree of a pipe accessory, which solves the problem that the traditional pipe accessory detection relies on complex tooling fixtures and manual visual detection, and improves the automation degree and accuracy of the completion degree of the finished pipe.

[0007] To achieve the above purpose, the present application adopts the following technical solutions:

[0008] A detection system for the completion degree of a pipe accessory, comprising:

[0009] a detection platform for placing a pipe to be detected;

[0010] An image acquisition module is located above the detection platform and comprises a light source unit and an image acquisition unit, the image acquisition unit being configured to acquire image information of the pipe fitting to be detected;

[0011] An image processing module is in communication connection with the image acquisition module and configured to receive the image information and identify accessory features of the pipe fitting to be detected according to a detection model;

[0012] The image processing module is further configured to save standard features of the accessories of the pipe fitting to be detected and output a completion degree detection conclusion according to a comparison result of the standard features of the accessories and the accessory features.

[0013] In some embodiments of the present application, the light source unit is a parallelly arranged bar light source, and the image acquisition unit is located in the middle of the bar light source; the light source unit is a hierarchical adjustment light source.

[0014] In some embodiments of the present application, the accessory completion degree detection method of the pipe fitting to be detected comprises the following steps:

[0015] S1. The image processing module establishes a detection model of each type of accessory on the pipe fitting to be detected according to a training picture set;

[0016] S2. The image acquisition module acquires an accessory picture of the pipe fitting to be detected and transmits it to the image processing module;

[0017] S3. The image processing module identifies accessory features in the accessory picture of step S2 by using the detection model in step S1;

[0018] S4. The image processing module retrieves corresponding accessory standard features according to the accessory features in step S3, and outputs a completion degree detection conclusion according to a comparison result of the accessory standard features and the accessory features.

[0019] In some embodiments of the present application, the training picture set in step S1 is an accessory training picture of the pipe fitting to be detected in multiple postures, and each type of accessory feature is marked in the accessory training picture.

[0020] In some embodiments of the present application, step S1 further comprises evaluation of the detection model: the detection rate of the accessory features is tested by using the accessory training picture, and when the detection rate is greater than a threshold value, the image processing module saves the detection model.

[0021] In some embodiments of the present application, step S3 comprises the following steps:

[0022] S31. The accessory picture is segmented into a to-be-detected picture according to the size of the accessory training picture;

[0023] S32, identifying feature information in the to-be-tested picture according to the detection model, the feature information including a feature name, a feature quantity, and a feature position;

[0024] S33, filtering the feature information in the step S32 according to a data structure and a filtering algorithm to obtain an accessory feature of the to-be-tested pipe fitting.

[0025] In some embodiments of the present application, the data structure includes a plurality of containers, and the containers are used to classify and record the feature information.

[0026] In some embodiments of the present application, the filtering algorithm includes the following steps:

[0027] (1) searching for a quantity container according to a position of the accessory feature in a name container to obtain a feature quantity, and then searching for a position container according to the feature quantity to obtain respective feature positions of the accessory feature and storing the feature positions in a temporary feature container;

[0028] (2) traversing elements in the temporary feature container and calculating a coincidence rate between any element and other elements;

[0029] (3) judging whether the coincidence rate is less than a threshold value, if yes, saving the element and then traversing a next element; if no, deleting a coincident element and returning to step 2;

[0030] (4) when the coincidence rates are all less than the threshold value, updating the quantity container and the position container of the accessory feature.

[0031] In some embodiments of the present application, a calculation formula of the coincidence rate is:

[0032] (min(Xi+Wi,Xj+Wj)-max(Xi,Xj))*(min(Yi+Hi,Yj+Hj)–max(Yi,Yj)) / (Wi*Hi+Wj*Hj-(min(Xi+Wi,Xj+Wj)-max(Xi,Xj))*(min(Yi+Hi,Yj+Hj)–max(Yi,Yj)));

[0033] wherein (Xi, Yi, Wi, Hi) is an i-th feature position of the same accessory feature, and (Xj, Yj, Wj, Hj) is a j-th feature position of the same accessory feature.

[0034] In some embodiments of the present application, the feature position is a coordinate (X, Y, W, H) of the accessory feature in a pixel coordinate system of the accessory picture, wherein X and Y are two-dimensional coordinates of a left upper corner of a rectangular region where the accessory feature is located in the pixel coordinate system, and W and H are width and height of the rectangular region, respectively.

[0035] The technical scheme of the present application has the following technical effects compared with the prior art:

[0036] The present application detects the accessories of the pipe to be detected by establishing a detection model of the accessory features of the pipe, improves the detection efficiency of the finished pipe, and eliminates the shortcomings of the traditional detection tool and manual visual identification of the pipe accessories, greatly reducing the labor intensity of the on-site inspection personnel; meanwhile, the detection accuracy of the finished pipe is improved, and the detection result is irrelevant to the proficiency and fatigue degree of the operator.

[0037] Obviously, the cost of special detection tools and the comprehensive material and labor consumption of long-term management of the detection tools are saved. The detection system of the present application is suitable for pipes of different specifications and different materials. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical scheme of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0039] Figure 1 The figure is a structural diagram of the detection system.

[0040] Figure 2 The figure is a flowchart of the establishment of the detection model.

[0041] Figure 3 The figure is a schematic diagram of the data structure.

[0042] Figure 4 The figure is a schematic diagram of the feature position container and the temporary feature container.

[0043] Figure 5 The figure is a schematic diagram of the accessory features in the coincident state.

[0044] Reference signs: 100-detection platform; 210-light source unit; 220-image acquisition unit; 300-shade; 400-pipe to be detected. DETAILED DESCRIPTION

[0045] The technical scheme of the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0046] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting" should be understood in a broad sense, for example, can be fixedly connected, or detachably connected, or integrally connected. For those skilled in the art, the specific meanings of the above terms in the present application can be understood in specific circumstances. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0047] Embodiment 1, refer to Figure 1 As shown in the figure, a pipe fitting accessory completion degree detection system comprises:

[0048] A detection platform 100 is used to place a pipe fitting to be tested 400; during the detection process, the pipe fitting to be tested 400 can be placed on the detection platform 100 in any posture;

[0049] An image acquisition module is located above the detection platform 100, comprising a light source unit 210 and an image acquisition unit 220, the image acquisition unit is used to acquire image information of the pipe fitting to be tested 400;

[0050] An image processing module (not shown in the figure) is in communication connection with the image acquisition module for receiving the image information, and is also used to identify accessory features of the pipe fitting to be tested 400 according to a detection model;

[0051] The image processing module is also used to save standard accessory features of the pipe fitting to be tested 400, and output a completion degree detection conclusion according to a comparison result of the standard accessory features and the accessory features.

[0052] Specifically, the light source unit 210 is a parallel strip light source, and the image acquisition unit 220 is located in the middle of the strip light source; because it is necessary to adapt to the change of ambient light during on-site detection in the workshop, the light source unit 210 adopts a hierarchical adjustable type, for example: adjustable brightness range (0-255 levels).

[0053] For the image acquisition unit 220, a high-definition color CCD camera is adopted, and the basic principle for selection of the CCD camera and the lens is to image the size of the accessory features on the pixel plane according to the size of the pipe fitting 400 to be measured, the working distance, the field of view range, and the size of the accessory features. That is, the test field formed by the image acquisition unit 220 can clearly identify the accessory features (the minimum size) to be measured. For example, assuming that the resolution of a known CCD camera is 4000pix*3000pix, the working distance is 1200mm, and the field of view range (pipe fitting size) to be detected is 2500mm*1200mm. According to the pinhole imaging principle, the minimum feature size M that can be detected is (1200mm / 3000pix)X2=0.8mm / 2pix. In actual application, according to the size of the pipe fitting accessory features, N times (test experience value) of the above theoretical calculation of the minimum feature is taken. Assuming that the accessory feature size to be detected is 10mm, the minimum feature size that can be identified by the optical imaging system needs to be ensured: N*M<10mm.

[0054] In some embodiments of the present application, an interactive interface for automatic measurement is further included in the detection system, which can realize data fusion or report output with a conventional pipe fitting 3D size detection system.

[0055] For data fusion with the conventional pipe fitting 3D size detection system, the XML detection command and parameters, version information, etc. can be transmitted in the XML instruction format of the conventional pipe fitting measuring machine or any specified instruction format, and the return result for the command parameters can be received. The data is transmitted in the form of encoded text byte stream by communicating with the client through the TCP / IP protocol.

[0056] The interface with the high-performance color CCD is realized by using a fixed IP in the same network segment as the host. Through the TCP / IP protocol, all available CCD cameras in the network segment are enumerated to determine the port to be opened. The camera parameters (such as exposure rate, frame rate, gain, etc.) after training are called to capture continuous images. After the soft trigger command arrives, a frame of image is saved in the specified directory for subsequent analysis.

[0057] In some embodiments of the present application, the accessory completeness detection method of the pipe fitting to be measured includes the following steps:

[0058] Referring to Figure 2 , as shown in

[0059] S1, the image processing module establishes a detection model for each type of pipe fitting to be measured according to a training picture set;

[0060] In order to correctly identify the accessory features in variously placed pipe fittings, a deep learning detection model is established.

[0061] Specifically, first, a training picture set is collected, i.e. collecting accessory training pictures of the pipe fitting in various poses, pictures of the accessory of the pipe fitting 400 in different positions in the field of view, and correctly labeling the accessory features to be detected on the pictures (e.g. collet, cable tie, adhesive tape, label, etc.) as the training picture set; and a detection model is obtained through deep learning.

[0062] Then, the completed training detection model is used to detect the existing training pictures, i.e. to evaluate the detection model and test the detection rate of each accessory feature, i.e. whether there is an error. After the training pictures can be correctly detected, or when the detection rate is greater than a threshold, the image processing module saves the detection model and can use the pipe fitting 400 in any free state in the field of view to detect the accessory features.

[0063] The detection process of the accessory features will be described below:

[0064] S2, the image acquisition module acquires the accessory picture of the pipe fitting and transmits it to the image processing module;

[0065] S3, the image processing module uses the detection model in step S1 to identify the accessory features in the accessory picture in step S2;

[0066] S31, the accessory picture is segmented into a test picture according to the size of the accessory training picture;

[0067] When the format and size of the on-site collected accessory picture are different from those of the training picture, the on-site collected accessory picture needs to be segmented according to the size of the training picture, and the segmented picture is used for detection. For example, the size of the training picture is M pixel (width) and N pixel (height), and in the pixel coordinate system of the accessory picture, the starting point coordinates of the segmented picture are X=0 pixel and Y=0 pixel. The accessory picture is segmented with M steps (width direction) and N steps (height direction).

[0068] S32, the feature information in the test picture is identified according to the detection model, and the feature information includes feature name, feature quantity and feature position; wherein the feature position is the position of the accessory feature in the pixel coordinate system of the accessory picture;

[0069] S33, the feature information in step S32 is filtered to obtain the final accessory feature information according to the data structure and filtering algorithm.

[0070] Since the pose of the to-be-detected pipe fitting 400 in the field of view is arbitrary, it is possible that the accessory features to be recognized are cut off when the picture is segmented, which may result in a difference between the number of accessory features obtained through deep recognition and the number of real pipe fitting accessories. In the embodiment, a data structure and a filtering algorithm are used to perform intersection calculation on the rectangular frame range of the accessory features initially recognized, and whether filtering is needed is determined according to the calculated coincidence degree.

[0071] The principle of determining the coincidence degree of the accessory features only needs to consider whether the position regions of the accessory features of the same type in the pixel coordinate system of the accessory picture are coincident (intersected). Even if the accessory features of different types intersect in the pixel coordinate system, they are not repeated features. The expression of the accessory features in the pixel coordinate system is a rectangular region, that is, the coordinates (X, Y, W, H) of the accessory features in the pixel coordinate system are the feature positions, where X and Y are the two-dimensional coordinates of the upper left corner of the rectangular region where the accessory features are located in the pixel coordinate system, and W and H are the width and height of the rectangular region, respectively, as shown in Figure 5 .

[0072] Specifically, the data structure includes a plurality of containers for recording accessory feature information. In this embodiment, as shown in Figure 3 , three containers are used, which are a name container for storing feature names, a quantity container for storing feature quantities, and a position container for storing feature positions.

[0073] The filtering algorithm includes the following steps:

[0074] (1) Starting from the head of the name container, the feature quantity corresponding to the feature name is searched in the quantity container, and then the corresponding feature position is searched in the position container according to the feature quantity, and all pixel coordinate system positions of this type of feature are taken out and stored in a temporary feature container; for example, the feature quantity of the adhesive tape is 3, and three pixel coordinate system XYWH positions are taken out from the head of the position container and put into the temporary feature container, as shown in Figure 4 .

[0075] (2) The elements in the temporary feature container are traversed, and the coincidence rate between any element and other elements is calculated according to the feature position data of each rectangular region in the pixel coordinate system.

[0076] The calculation formula of the coincidence rate is:

[0077] (min(Xi+Wi,Xj+Wj)-max(Xi,Xj))*(min(Yi+Hi,Yj+Hj) -max(Yi,Yj)) / (Wi*Hi+Wj*Hj-(min(Xi+Wi,Xj+Wj)-max(Xi,Xj))*(min(Yi+Hi,Yj+Hj) -max(Yi,Yj))) ;

[0078] wherein (Xi, Yi, Wi, Hi) is the i-th feature position of the same accessory feature, and (Xj, Yj, Wj, Hj) is the j-th feature position of the same accessory feature.

[0079] The calculation formula of the coincidence rate is based on that the position (X, Y, W, H) of the accessory feature is located at the upper left corner of the rectangular region.

[0080] Specifically, when the position (X, Y, W, H) of the accessory feature is located at the lower left corner of the rectangular region, the calculation formula of the coincidence rate is: (min(Xi+Wi,Xj+Wj)-max(Xi,Xj))*(max(Yi-Hi,Yj-Hj) -max(Yi,Yj)) / (Wi*Hi+Wj*Hj-(min(Xi+Wi,Xj+Wj)-max(Xi,Xj))*(max(Yi-Hi,Yj-Hj) -max(Yi,Yj))) ;

[0081] When the position (X, Y, W, H) of the accessory feature is located at the upper right corner of the rectangular region, the calculation formula of the coincidence rate is: (min(Xi,Xj)-max(Xi-Wi,Xj-Wj))*(min(Yi+Hi,Yj+Hj) -min(Yi,Yj)) / (Wi*Hi+Wj*Hj-(min(Xi,Xj)-max(Xi-Wi,Xj-Wj))*(min(Yi+Hi,Yj+Hj) -min(Yi,Yj))) ;

[0082] When the position (X, Y, W, H) of the accessory feature is located at the lower right corner of the rectangular region, the calculation formula of the coincidence rate is:

[0083] (min(Xi,Xj)-max(Xi-Wi,Xj-Wj))*(min(Yi,Yj)-max(Yi-Hi,Yj-Hj)) / (Wi*Hi+Wj*Hj-((min(Xi,Xj)-max(Xi-Wi,Xj-Wj))*(min(Yi,Yj)-max(Yi-Hi,Yj-Hj))).

[0084] 3, judge whether the coincidence rate is less than a threshold, if yes, save the element and proceed to the next element traversal, if not, delete the coincident element and return to step 2, that is, when the coincidence rate is not zero or the coincidence rate is greater than the threshold, it is considered that the two features are coincident, one of the coincident elements needs to be deleted, and the traversal calculation is restarted until all elements are traversed.

[0085] (4) when the coincidence rate is less than the threshold, update the number container and the position container of the accessory feature, that is, according to the number of filtered features (that is, the size of the temporary feature container) and the position data (X, Y, W, H), update the number container and the position container, and the name container will not change.

[0086] S4, the image processing module retrieves the corresponding accessory standard feature according to the accessory feature in step S3, and outputs the completion degree detection conclusion according to the comparison result of the accessory standard feature and the accessory feature.

[0087] In some embodiments of the application, the exclusion method of the fixed interference factor in the field of view is also involved:

[0088] The first method is: for the misjudgment caused by the background environment light or the fixed tool in the field of view, the relative position in the picture pixel coordinate system is calculated to realize the elimination of the fixed interference factor in the environment, that is, the interference shielding of the fixed area can be realized;

[0089] The second method is: based on the accessory feature recognition of deep learning, for each recognized accessory feature, there is a corresponding confidence score, which can also be filtered by the confidence score, and the features less than a certain confidence score can not be considered in the statistical result.

[0090] The technical scheme of the application has the following technical effects relative to the prior art:

[0091] The application establishes a detection model of accessory features to detect the accessories of the pipe to be measured, improves the detection efficiency of the completion degree of the finished pipe, gets rid of the shortcomings of traditional gauges and manual visual identification of pipe accessories, greatly reduces the labor intensity of on-site inspection personnel, and improves the detection accuracy of the completion degree of the finished pipe, and the detection result is independent of the proficiency and fatigue degree of the operator.

[0092] In addition, the special gauge cost and the comprehensive material and labor consumption of long-term management of the gauge are saved. The detection system of the application is suitable for pipe fittings of different specifications and different materials, that is, the completion degree of hard pipes and soft pipes can be detected and can be made into a pipe accessory completion degree detection device alone, or can be used as a functional expansion of the traditional pipe 3D size measurement system. Fill the functional defects of the existing pipe 3D size detection equipment, which cannot check the accessory completion degree.

[0093] In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0094] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for detecting the completion degree of pipe fittings, characterized in that, Includes the following steps: S1. Establish detection models for various attachments on the pipe fittings under test based on the training image set; the training image set consists of attachment training images of the pipe fittings under test in various postures, and the attachment training images are marked with various attachment features; S2. Collect images of the accessories of the pipe fitting to be tested; S3. Use the detection model in step S1 to identify the attachment features in the attachment image in step S2; S4. Retrieve the corresponding standard features of the attachments based on the attachment features in step S3, and output the completion detection conclusion based on the comparison result between the standard features of the attachments and the attachment features. Step S3 includes the following steps: S31. Segment the attachment image according to the size of the attachment training image to obtain the test image; S32. Identify feature information in the image to be tested according to the detection model, wherein the feature information includes feature name, feature quantity, and feature location; S33. Filter the feature information in step S32 according to the data structure and filtering algorithm to obtain the accessory features of the pipe fitting to be tested; The filtering algorithm includes the following steps: (1) Find the number of features in the name container according to the position of the attachment features, then find the position container according to the number of features to obtain the position of each feature of the attachment features, and store them in a temporary feature container; (2) Traverse the elements in the temporary feature container and calculate the overlap rate between any element and other elements; (3) Determine whether the overlap rate is less than the threshold. If so, save the element and proceed to the next element. Otherwise, delete the overlapping element and return to step 2. (4) When the overlap rate is less than the threshold, update the quantity container and location container of the attachment features; The formula for calculating the overlap rate is: (min(Xi + Wi, Wherein, (Xi, Yi, Wi, Hi) is the i-th feature position of the same attachment feature, and (Xj, Yj, Wj, Hj) is the j-th feature position of the same attachment feature; the feature position is the coordinate (X, Y, W, H) of the attachment feature in the pixel coordinate system of the attachment image, where X and Y are the two-dimensional coordinates of the upper left corner of the rectangular area where the attachment feature is located in the pixel coordinate system, and W and H are the width and height of the rectangular area, respectively.

2. The detection method according to claim 1, characterized in that, Step S1 also includes evaluating the detection model: testing the detection rate of the attachment features using attachment training images; when the detection rate is greater than a threshold, the detection model is saved.

3. The detection method according to claim 1, characterized in that, The data structure includes multiple containers, which are used to classify and record the feature information.

Images