A visual detection method for installation state of aerospace pipe joint anti-loose fuse

Abstract

A kind of visual detection method of aerospace pipe joint anti-loose fuse installation state, it is related to aerospace engine assembly detection field, including deep learning dataset production;Deep learning dataset annotation processing;Pipe joint fuse missing detection;Pipe joint fuse installation angle and tightening direction detection;Pipe joint fuse spiral edge line identification: get fuse contour;On fuse contour, watershed algorithm is carried out, and the separated single spiral contour of fuse is obtained;Pipe joint fuse spiral number detection: find the separated single spiral contour, the contour area that does not meet is rejected by area screening method, then all contours of the whole image are traversed and counted, the number of spiral of fuse is obtained, whether the number of spiral of fuse is required is judged.The problem of low detection efficiency of existing aerospace pipe joint fuse is solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of aerospace engine assembly detection, and particularly relates to a visual detection method for the installation state of an aerospace pipe joint anti-loose fuse. BACKGROUND

[0002] A conduit is an important component for liquid rocket engines to transport liquid or gas, and is generally assembled with a combined component through a threaded connection mode, and the connection sealing performance is an important factor affecting the reliability of the engine. In order to ensure the sealing performance of the conduit installation position, torque quantitative control is usually performed on the connection position of the conduit and the component, but there are factors such as vibration and temperature change in the working state of the engine, and the threaded connection position is prone to loosening. Therefore, the fuse is needed to lock and prevent loosening of the conduit end and the connected component.

[0003] At present, the process inspection requirements after the fuse of the liquid rocket engine pipe joint is locked mainly include: (1) checking whether the fuse is missing; (2) checking whether the winding direction of the fuse is consistent with the tightening direction of the pipe joint thread; (3) checking whether the number of spiral turns of the fuse is qualified. At present, the detection of the fuse of the liquid rocket engine pipe joint is mainly manual detection, and due to the large number of detections and the detection content covering the entire engine surface, the risk of missing and misloading is easy to occur, especially for large liquid rocket engines, the activity range of human visual angle is limited, and the manual detection efficiency is relatively low. SUMMARY

[0004] The technical problem solved by the application is to overcome the shortcomings of the prior art and provide a visual detection method for the installation state of an aerospace pipe joint anti-loose fuse, so as to solve the problem of low detection efficiency of the existing aerospace pipe joint fuse.

[0005] The technical solution of the application is:

[0006] A visual detection method for the installation state of an aerospace pipe joint anti-loose fuse, comprising:

[0007] S1, obtaining an image of the pipe joint fuse to be detected, and judging whether the fuse exists or not;

[0008] S2, pipe joint fuse installation angle and tightening direction detection:

[0009] S21, image processing is performed on the image with the detection result of step S1 that the fuse exists, a plurality of line segments corresponding to the fuse are obtained, and the coordinates of the line segments are stored in Lines_Points dynamic array;

[0010] S22, the line segment does not meet the requirement of the angle between the line segment and the longitudinal axis of the pipe joint, the line segment is removed from the Lines_Points dynamic array, and if the end point data of the remaining line segment in the Lines_Points dynamic array is not zero, the fuse installation angle is qualified;

[0011] S23, multi-line segment fitting: calling the Lines_Points dynamic array in step 22, fitting the line segments to obtain a unique line segment If b>0, the fuse tightening direction is correct, the number of fuse spirals of the pipe joint is detected, and if b<0, the tightening direction is incorrect, the detection process is ended, and the detection result that the tightening direction of the pipe joint fuse is incorrect is output.

[0012] S3, identification of the fuse spiral edge line of the pipe joint fuse: the fuse contour is obtained;

[0013] S4, processing the fuse contour to obtain a single separated spiral contour of the fuse;

[0014] S5, finding the single separated spiral contour in step S4, removing the contour with an area that does not meet the requirement by area screening, then traversing and counting all contours of the entire image to obtain the number of fuse spirals, and determining whether the number of fuse spirals meets the requirement.

[0015] In step S1, determining whether the fuse exists includes the following steps

[0016] S11, deep learning data set making: making positive samples and negative samples of the fuse manually according to the actual pipe joint fuse process as a data set;

[0017] S12, deep learning data set annotation processing: dividing the data set into a training set and a validation set, and performing pipe joint fuse data annotation on the training and validation images of the input data set, the annotation types including two detection result types of fuse existence and fuse nonexistence;

[0018] S13, pipe joint fuse missing detection: taking the training set in step S2 as the training data of the YOLOv5 model, performing feature learning on the pipe joint fuse, and performing target detection on the pipe joint fuse to be detected.

[0019] In step S11, the correct installation of the fuse indicates a positive sample, and the nonexistence of the fuse, the incorrect direction of the fuse, the installation angle of the fuse, and the number of spirals of the fuse that do not meet the requirement indicate a negative sample;

[0020] After the positive samples and negative samples in step S11 are made, an industrial camera is used, different intensity illuminations are simulated by using a CCS ring light source and a light source controller to simulate the real environment of the factory, the direction of the pipe joint in the image is kept vertical upward, and image acquisition is performed to obtain the data set.

[0021] The steps following step S1 include: creating a traditional algorithm dataset: using the judgment results from step S1, cropping the area where the YOLOv5 model judges the presence of the fuse and using it as a traditional algorithm dataset; and performing image processing on the traditional algorithm dataset in step S2.

[0022] In step S21, edge extraction is performed on the fuse outline, and then Hough line detection is performed to obtain multiple line segments corresponding to the fuse.

[0023] The multi-segment fitting in step S23 includes:

[0024] The pixel points of each line segment are calculated using the endpoints of the line segments stored in the Lines_Points dynamic array, and the pixel coordinates of all line segments are recorded as follows: v n The direction is parallel to the axis of the pipe connector. A dynamic array named Points_1 stores the coordinates of all pixels. The same calculation is performed. Pixels in coordinates The mean of the coordinates, denoted as A dynamic array named Points_2 stores the mean coordinates of all pixels. These coordinates are then substituted into the least-squares fitted line equation. These are the coordinates of the pixels in the Points_2 dynamic array. The sequence number of the coordinate values ​​( (Values ​​1, 2, 3...n) Indicates the number of coordinate values. It is a constant. The slope of the fitted line

[0025]

[0026]

[0027] Solving the system of equations yields a unique line segment. .

[0028] Step S3 includes performing grayscale processing on the pipe connector image, then performing median filtering, and binarizing the filtered image to obtain the fuse outline.

[0029] In step S4, the watershed algorithm is used to process the fuse profile.

[0030] In the step S5, whether the fuse number of turns is required is determined according to the fuse number of turns, which comprises: obtaining the number of turns per unit centimeter range according to the conversion relationship between the pixel points of the image photographed by the converted industrial camera and the actual length, comparing with the fuse number of turns of the spaceflight pipe joint, and finally outputting the determination result of whether the number of turns is qualified or unqualified.

[0031] In the step S1, the image of the pipe joint fuse to be detected satisfies: the shooting direction is perpendicular to the pipe joint axis, and the shooting direction is parallel to one plane of the hexagon nut on the pipe joint.

[0032] In summary, the present application has at least the following beneficial technical effects:

[0033] Through the method combining deep learning and traditional algorithm, the detection efficiency and intelligent level are improved, the fuse missing and misinstallation can be effectively prevented, and the safety of the spaceflight engine is ensured. According to the YOLOv5 algorithm in the one-stage target detection algorithm YOLO (You Only Look Once) series, the yolov5s.pt model is adopted to detect the engine image collected by the pre-processing, the fuse position information and confidence are detected, the candidate frame target is classified, whether the fuse exists is judged, after the judgment is finished, the region where the fuse exists is cut, as the data set of the subsequent traditional method. Through the traditional method, the interference of scratches, light spots, shadows and edge contours is excluded, the fuse is recognized, and the consistency of the winding direction and the tightening direction of the fuse and the number of turns of the fuse are judged. The detection efficiency of the spaceflight engine fuse is improved, the machine is used instead of manual work, and the intelligent level of the fuse detection in the engine assembly is improved. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 It is a pipe joint fuse missing detection schematic diagram in the embodiment of the present application;

[0035] Figure 2 It is a pipe joint fuse installation angle and tightening direction detection schematic diagram in the embodiment of the present application;

[0036] Figure 3 It is a pipe joint fuse near angle schematic diagram in the embodiment of the present application;

[0037] Figure 4 It is a watershed schematic diagram in the embodiment of the present application;

[0038] Figure 5 It is a pipe joint fuse number of turns detection schematic diagram in the embodiment of the present application;

[0039] Figure 6 It is a pipe joint fuse sample schematic diagram in the embodiment of the present application;

[0040] Figure 7 This is a schematic diagram of the pipe joint fuse dataset in an embodiment of the present invention;

[0041] Figure 8 This is a schematic diagram of the fuse data marking for the pipe joint in an embodiment of the present invention;

[0042] Figure 9 This is a schematic diagram of the target detection of the pipe joint fuse in an embodiment of the present invention;

[0043] Figure 10 This is a schematic diagram of the traditional algorithm dataset for pipe joint fuses in an embodiment of the present invention;

[0044] Figure 11 This is a schematic diagram of the Hough line detection of the pipe joint fuse in an embodiment of the present invention;

[0045] Figure 12 This is a schematic diagram illustrating the rejection of pipe joint locking screws with incorrect installation angles in an embodiment of the present invention.

[0046] Figure 13 This is a schematic diagram of the fitting of the fuse line segment of the pipe joint in an embodiment of the present invention;

[0047] Figure 14 This is a schematic diagram of the outline of the fuse line segment of the pipe joint in an embodiment of the present invention;

[0048] Figure 15 This is a schematic diagram of the twisted markings on the pipe joint fuse in an embodiment of the present invention. Detailed Implementation

[0049] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0050] This invention provides a visual inspection method for the installation status of anti-loosening fuses on aerospace pipe joints, comprising the following steps:

[0051] Step S1: Creating a Deep Learning Dataset

[0052] Here, positive and negative samples of fuses are manually made based on the actual pipe fitting fuse process. A positive sample indicates that the fuse is installed correctly, while a negative sample indicates that the fuse is missing, the fuse is in the wrong direction, the fuse is installed at the wrong angle, or the number of twists in the fuse does not meet the requirements.

[0053] After the sample is completed in step S1, a camera acquisition device is set up. Here, an industrial camera is used, and a CCS ring light source and its light source controller are used to provide different intensity light to the above-mentioned mechanism to simulate the real environment of the factory area.

[0054] Step S2: Deep learning dataset annotation processing

[0055] The data set is divided into a training set and a validation set, and the structure ratio of the former to the latter should be greater than 7:1. After naming the inlet and outlet of the pipeline, it is placed in the directory framework required by the YOLOv5 algorithm, and the labelme label tool is used to perform data labeling on the input training and verification images. The negative sample of the pipe joint fuse is also data labeled. The labeling type is divided into two types: fuse and no_fuse, which correspond to the existence of the fuse and the non-existence of the fuse, respectively. The label format here is xml format. In order to adapt to the YOLOv5 interface, the xml label format is converted into txt format, and finally the txt format data labeling is placed in the required directory framework.

[0056] Step S2 is specifically to divide the data set into a training set and a validation set, and the structure ratio of the former to the latter is greater than 7:1. After naming the inlet and outlet of the pipeline, it is placed in the directory framework required by the YOLOv5 algorithm, and the labelme label tool is used to perform data labeling on the input training and verification images. The negative sample of the pipe joint fuse is also data labeled. The labeling type is divided into two types: fuse and no_fuse, which correspond to the existence of the fuse and the non-existence of the fuse, respectively. The label format here is xml format. In order to adapt to the YOLOv5 interface, the xml label format is converted into txt format, and finally the txt format data labeling is placed in the required directory framework.

[0057] Step S3, pipe joint fuse missing detection

[0058] The detection process is as shown in Figure 1 The prepared training set in step 2 is cited as the training data of the YOLOv5 model, and the characteristics of the pipe joint fuse are learned. The learned YOLOv5 model is used to detect the pipe joint fuse in real time. If the fuse exists, the installation angle and tightening direction of the fuse in the current image are judged. If the fuse does not exist, the detection process is ended, and the detection result is output, i.e. the pipe joint fuse is missing.

[0059] The specific method of step S3 is to cite the prepared training set in step 2 as the training data of the YOLOv5 model, learn the characteristics of the pipe joint fuse, and shoot the pipe joint fuse image to be detected. The principles of shooting the pipe joint fuse image to be detected include: the shooting direction is perpendicular to the pipe joint axis, the direction of the pipe joint in the image is kept vertical upward during image acquisition, and the length of the fuse obtained by the shooting plane is not less than 50% of the length of the whole fuse.

[0060] In combination with the requirements of aerospace pipe joint fuse: the fuse needs to span 1~3 facets, which refers to the six planes of the hexagonal head pipe joint. Considering the uncertainty of the final position of the pipe joint torque tightening and the requirements of fuse image acquisition, in this embodiment, one shooting position is set every 90°, and four images of the same fuse are reserved for missing loading, direction and number of twists determination. If one image has three correct results, the detection result is qualified.

[0061] In the above images, there must be one image that can meet the requirement that the length of the fuse obtained by the shooting plane is not less than 50% of the length of the whole fuse. Other images cannot be shot or do not meet the 50% length requirement, and feedback incorrect results in missing loading or number of twists determination. If one image has three correct results, the detection result is qualified. If all image calculation results are unqualified, it proves that the fuse is incorrect.

[0062] According to the image of the pipe joint fuse to be detected, the learned YOLOv5 model is used to detect the pipe joint fuse in real time. If the fuse exists, the installation angle and tightening direction of the fuse in the current image are judged. If the fuse does not exist, the detection process is ended, and the detection result is output, i.e. the pipe joint fuse is missing.

[0063] Step S4, traditional algorithm data set making

[0064] Because the traditional algorithm has high requirements for the quality of the input image, directly using the image collected in step 1 as the input data of the traditional algorithm will cause too much image information and cannot accurately identify the characteristic information of the fuse, resulting in subsequent detection failure. Therefore, the judgment result in step 2 is used to crop the area of the pipe joint fuse image to be detected where the YOLOv5 model judges the existence of the fuse, remove the redundant information of the image to be detected, and name it according to “pipe joint figure number + position” as the traditional algorithm data set.

[0065] Step S5, pipe joint fuse installation angle and tightening direction detection

[0066] The flow is as shown in Figure 2 The data set in step 4 is imported into the image processing program, the edge of the fuse contour is extracted, and then Hough straight line detection is performed, and the Lines_Points dynamic array is named to store the endpoint coordinates of all detected line segments.

[0067] Using Hough line detection will produce redundant line segments, so it is necessary to fit multiple line segments into one line segment, and two judgment rules are set:

[0068] Rule 1: Installation angle inconsistency elimination principle. For example Figure 3As shown, when the fuse crosses the pipe fitting body, the near angle between the fuse and the pipe fitting's longitudinal axis must not be less than 30°. Therefore, the near angle between all line segments and the pipe fitting's longitudinal axis is calculated using pixel coordinates. When the near angle between the pipe fitting's longitudinal axis and the pipe fitting's longitudinal axis is within 30°, the line segment is considered irrelevant and is removed from the Lines_Points dynamic array. After removing line segments with incorrect angles, if the data of the line endpoints in the Lines_Points dynamic array is 0, it is determined that the fuse installation angle does not meet the requirements, and the detection ends, meaning that the pipe fitting is missing a fuse. If the data of the line endpoints in the Lines_Points dynamic array is not 0, then proceed to rule 2 for further judgment.

[0069] Rule 2: Multi-segment fitting principle.

[0070] The `Lines_Points` dynamic array is called to calculate the pixel points of each line segment using the stored endpoints. The pixel coordinates of all line segments are then recorded as follows: v n The direction is parallel to the axis of the pipe connector. A dynamic array named Points_1 stores the coordinates of all pixels. The same calculation is performed. Pixels in coordinates The mean of the coordinates, denoted as A dynamic array named Points_2 stores the mean coordinates of all pixels. These coordinates are then substituted into the least-squares fitted line equation. These are the coordinates of the pixels in the Points_2 dynamic array. The sequence number of the coordinate values ​​( (Values ​​1, 2, 3...n) Indicates the number of coordinate values. It is a constant. The slope of the fitted line:

[0071] (1),

[0072] (2),

[0073] Solving the system of equations yields a unique line segment. ,like If the fuse is tightened in the right-hand position, the direction is correct. Perform a fuse twist count test on the pipe fitting; otherwise... If the fuse is tightened in the right-hand state, the test process ends and the test result is output, indicating that the fuse of the pipe fitting is tightened in the wrong direction; if the thread state is left-hand, the judgment logic is reversed.

[0074] In some embodiments, the present invention further includes a method for detecting the number of twists in a pipe fitting fuse, specifically implemented according to the following steps:

[0075] Step S6: Preprocessing before watershed segmentation. After detecting the fuse tightening direction, the individual twisted edge lines of the pipe joint fuse are identified, and the dataset from Step 4 is imported into the pipe joint fuse twist count detection program. The specific process is as follows: Under combined lighting, the features of the aerospace pipe joint fuse are well highlighted, making it different from the dark yellow or dark blue pipe joint background. Based on this, grayscale processing is performed on the three-channel pipe connector image. At this time, the grayscale values ​​of the fuses with better feature performance are all in the range of 150-230, which is much larger than the grayscale value of the background. At this time, due to the reflective characteristics of the metal parts of the pipe connector, there are still brightness noises of non-fuse structures in the whole image. Therefore, before binarizing the image, a median filter operation with a convolution kernel size of [3,3] is performed on the image to make the whole image smoother and reduce brightness noise. Then, a suitable threshold is selected to perform binarization operation on the median filtered image. Here, the threshold is selected as b = 150, that is, the grayscale value of the pixels with a grayscale value greater than 150 is changed to 255, and the grayscale value of the pixels with a grayscale value less than 150 is changed to 0. Thus, the outline of the entire fuse can be obtained.

[0076] Step S7: Based on step S6, perform the watershed algorithm to obtain the edge line of each fuse twist.

[0077] In some embodiments, step S7 is as follows: classify all pixels in the fuse image according to their gray values ​​and set a geodesic distance threshold; then find the minimum gray value point and let the threshold increase from the minimum value. During the increase, calculate the geodesic distance between the minimum value point and the pixel point. If it is less than the set threshold, then these pixels are submerged; otherwise, use the dilation method to set image edges on these pixels.

[0078] Specifically, in step S7, the fuse image is treated as the ground surface, and all pixels in the gradient image are classified according to their grayscale values. A geodesic distance threshold is set. Then, the minimum grayscale value point (hmin) is found, and the threshold is increased from the minimum value (hmin + 1, ..., hmax). During the increase, the geodesic distance between hmin and the pixel is calculated. If it is less than the set threshold, these pixels are submerged; otherwise, an image edge (i.e., a "watershed") is set on these pixels using a dilation method. Figure 4 As shown), this classifies the pixels in the neighborhood of these fuse twist adhesion points. The specific recursive "water-filling" process to submerge the pixels is as follows: Let M1, M2, ..., Mn represent the set of coordinates of local minima in the image; C(M i ) represents the set of points within the catchment basin (background of aerospace pipe joints with low grayscale values) associated with the local minimum Mi; Indicates that it is located in a plane The set of points below; represent a set of connected components; representing the first phase catchment basin a set of midpoints; representing the first phase catchment basin being flooded by water , the recursive process is to find where when the connected component and is empty or contains one connected component in is merged into to form ; when contains more than one connected component in , a "watershed" is built in , and the termination condition is: At this point, the fuse frit feature is separated to lay the foundation for subsequent calculation of its density.

[0079] Step S8, pipe joint fuse frit number detection, find the single frit profile separated by step 7, and exclude the profile whose area does not meet the requirement by area screening method, then traverse all the profiles in the image to count and determine whether the number of fuse frits meets the process requirements of space pipe joint fuse assembly.

[0080] In some embodiments, the specific process of step S8 is as follows: find all the profiles existing on the image, exclude the profiles whose area does not meet the single frit area of the fuse by area screening method, ensure that the counted profiles are all separated single frits of the fuse; then traverse all the profiles that meet the requirements to count and obtain the number of fuse frits; finally, convert the conversion relationship between the image pixel points captured by the industrial camera and the actual length to obtain the number of frits per unit centimeter, compare it with the space pipe joint fuse frit number specification, and determine that the number of frits per unit centimeter should be greater than 3 according to the space pipe fuse process, and finally output the comparison and determination result: the number of frits of this pipe joint is qualified or unqualified, and the process ends, as shown in the flowchart of Figure 5 .

[0081] Embodiment

[0082] Step S1, according to the actual pipe joint fuse process, manually make positive and negative samples of the fuse, where the correct installation of the fuse represents the positive sample, and the absence of the fuse, the wrong direction of the fuse, the installation angle of the fuse, and the number of frits of the fuse that do not meet the requirements represent the negative sample, and the samples made are as shown in Figure 5 ​The camera acquisition device is arranged after the sample is made, an industrial camera is used, and a CCS ring light source and a light source controller thereof are used to provide different intensity light for the mechanism to simulate the real environment of a factory. The direction of the pipe joint in the image is kept vertical upward during the image acquisition process. A total of 600 sample images are acquired. The sample images are used as a data set. The acquired data set sample images are as shown in Figure 6 ;

[0083] In step S2, the data set is divided into a training set and a verification set. The structural ratio of the former to the latter should be greater than 7:1. After the pipe inlet and outlet are named, they are placed in the required directory framework of the YOLOv5 algorithm. The labelme label tool is used to perform data labeling on the input training and verification images. The pipe joint fuse negative samples are also labeled. The labeling type is divided into two types: fuse and no_fuse, which correspond to the existence of the fuse and the non-existence of the fuse, respectively. The label format is xml format. In order to adapt to the YOLOv5 interface, the xml label format is converted into a txt format. Finally, the txt format data labeling is placed in the required directory framework. The pipe joint fuse data labeling is as shown in Figure 7 ;

[0084] In step S3, the training set prepared in step 2 is cited as the training data of the YOLOv5 model. The characteristics of the pipe joint fuse are learned. The pipe joint fuse data set and the label thereof are loaded into the input end through the yolov5s.pt network model to perform Mosaic data enhancement, adaptive anchor box calculation, and adaptive picture scaling. The Mosaic data enhancement randomly scales, randomly crops, and randomly arranges the selected fuse target to improve the robustness of the network model to different prediction samples after training. Then, the model enters the Backbone main network. The Focus structure of yolov5s uses 32 convolution kernels. Two CSP structures are designed: the CSP1_X structure is applied to the Backbone main network, and the CSP2_X structure is applied to the Neck. The ability of network feature fusion is strengthened. The CIOU_Loss is used as the loss function of the Bounding box of the model. In the post-processing process of the fuse detection, many target boxes need to be screened through the nms non-maximum suppression operation. The weighted nms, i.e., DIOU_nms, is used in the model, so that the pipe joint fuses with occlusion relationship can also be detected. The learned YOLOv5 model is used to perform real-time detection on the pipe joint fuses to be detected. If the fuse exists, the installation angle and the tightening direction of the fuse of the current image are judged. If the fuse does not exist, the detection process is ended, and the detection result is output, i.e., the pipe joint fuse is missing. As shown in Figure 8 , the YOLOv5 detects that the fuse in the image exists, and the position information is selected and displayed.

[0085] Step S4: Using the judgment result from step S3, crop the area where the YOLOv5 model detects the presence of the fuse, remove redundant information from the image to be detected, and name it according to the inlet and outlet of the pipe connector, using it as the traditional algorithm dataset. A schematic diagram of the traditional algorithm dataset is shown below. Figure 9 and Figure 10 As shown;

[0086] Step S5: Import the traditional algorithm dataset from Step S4 into the image processing program, extract the edges of the fuse outline, and then perform Hough line detection. Name the array Lines_Points, which stores the endpoint coordinates of all detected line segments. A schematic diagram of Hough line detection is shown below. Figure 11 As shown.

[0087] Using Hough lines for detection will generate redundant line segments. Therefore, when determining the tightening direction, multiple line segments need to be fitted into one line segment, and two judgment rules need to be set:

[0088] Rule 1: Removal Principle for Incorrect Installation Angles. When a fuse crosses the pipe fitting body, the near angle between the fuse and the pipe fitting's longitudinal axis must not be less than 30°. Therefore, the near angle between all line segments and the pipe fitting's longitudinal axis is calculated using pixel coordinates. If the near angle is within 30°, the line segment is considered irrelevant and removed from the Lines_Points dynamic array. After removing line segments with incorrect angles, the following applies: Figure 12 As shown, if the line endpoint data in the Lines_Points dynamic array is 0, it is determined that the fuse installation angle does not meet the requirements; if the line endpoint data in the Lines_Points dynamic array is not 0, proceed to rule 2 to continue the judgment.

[0089] Rule 2: Multi-segment fitting principle. Call the dynamic array `Lines_Points`, calculate the pixel points of each segment using the stored endpoints, and record the pixel coordinates of all segments as follows: The Points_1 dynamic array stores the coordinates of all pixels. If multiple pixel coordinates are the same, they are stored as a single pixel, and the same coordinates are calculated. Pixels in coordinates The mean of the coordinates, denoted as A dynamic array named Points_2 stores the mean coordinates of all pixels. These coordinates are then substituted into the least-squares fitted line equation. These are the coordinates of the pixels in the Points_2 dynamic array. The sequence number of the coordinate values ​​( (Values ​​1, 2, 3...n) Indicates the number of coordinate values. It is a constant. To fit the slope of the straight line:

[0090] (1)

[0091] (2)

[0092] Solving the resulting system of equations, we get the unique line segment , the fitting line segment of the pipe joint in the pixel coordinate system is shown in Figure 13 , it can be seen that , the fuse tightening direction is correct, and the number of turns of the pipe joint fuse is detected;

[0093] Step S6, after the fuse tightening direction detection is completed, the data set in step 4 is imported into the pipe joint fuse turn number detection program, and the specific process is as follows: under the irradiation of the combination lamp, the characteristics of the spaceflight pipe joint fuse are well highlighted, and the background difference is differentiated with the dark yellow or dark blue pipe joint. Based on this, the gray scale processing is performed on the pipe joint fuse image, at this time, the gray scale value of the fuse with better characteristics is in the range of 150-230, which is much larger than the gray scale value of the background; At this time, due to the reflection characteristics of the pipe joint metal parts, there are still brightness noise points of non-fuse structure in the whole image, so before the image is binarized, the median filter operation with a kernel size of [3, 3] is performed on the image, so that the whole image is smoother and the brightness noise points are reduced; Then, a suitable threshold is selected to perform a binarization operation on the median filtered image, and the threshold value is b = 150, that is, the gray scale value of the pixel point with a gray scale value greater than 150 is changed to 255, and vice versa. At this time, the fuse contour can be obtained, and the fuse contour is shown in Figure 14 .

[0094] Step S7, on the basis of step S6, the fuse contour is subjected to a watershed algorithm, the edge line of a single turn of the fuse is obtained, the fuse image is regarded as a ground surface, all pixels in the gradient image are classified according to the gray scale value, and a geodesic distance threshold is set; Then find the minimum value point (hmin) of the gray scale value, let the threshold start to grow from the minimum value (hmin +1, …, hmax), calculate the geodesic distance between hmin and the pixel point in the growing process, if it is less than the set threshold, the pixels are flooded, otherwise, the image edge (i.e. "watershed" as shown in Figure 7 ) is set on these pixels by means of inflation, so that the neighborhood pixels of the fuse turn adhesion are classified. The specific recursive "water injection" process is as follows: let M1, M2, …, Mn represent the set of coordinates of local minimum points; represent the set of points in the catchment basin (low gray scale value spaceflight pipe joint background) associated with the local minimum value Mi; represent the set of points in the catchment basin (low gray scale value spaceflight pipe joint background) associated with the local minimum value Mi; The set of points below; represent The set of connected components in the middle; Indicates the first Staged catchment basin The set of midpoints; Indicates the first The set of submerged portions of the catchment basin; recursive initialization conditions: The recursive process is based on beg , where when connected components and Empty or containing When a connected component is in one of the components, Merging constitute ;when Include If there is more than one connected component in the matrix, then... The construction of the "watershed" will be terminated under the following conditions: At this point, the twisted features of the fuse are separated, and the edge line diagram of a single twisted fuse is shown below. Figure 15 As shown, this lays the foundation for subsequent calculations of its density.

[0095] Step S8: In the individual twisted outlines separated in step 7, outlines whose areas do not meet the requirements are removed by area screening. Then, all outlines in the entire image are traversed and counted to determine whether the number of twisted fuses meets the process requirements for aerospace pipe connector fuse assembly.

[0096] Specifically, the OpenCV function library is used to find all contours existing in the image. Contours with areas outside the range [25, 50] are filtered out using an area filtering method, ensuring that all counted contours are separated fuse-shaped braids. Then, all eligible contours are iterated and counted to obtain the fuse-shaped braid count. The braid count is as follows: Figure 15 As shown; finally, the conversion relationship between the pixel points of the image captured by the industrial camera and the actual length is calculated, and the number of twists per unit centimeter is compared with the standard number of twists of the fuse in the aerospace pipe joint. Here, it is stipulated that the number of twists of the fuse should not be less than 3 per unit centimeter. The final output comparison judgment result is: the number of twists of the pipe joint is qualified or unqualified.

[0097] Machine vision technology has been successfully applied in many fields, and is divided into deep learning and traditional algorithm two categories in method. Among them, the method of deep learning has strong adaptability and recognition accuracy under the support of sufficient effective image data, and its disadvantage is also very prominent, that is, poor interpretability, and it is difficult to obtain a large number of effective image data. In addition, the traditional method has clear calculation logic and good detection accuracy under the condition of clear features and scenes, and the shortcoming is that the algorithm is complex and the multi-scene adaptability is poor. In view of the above background, the algorithm combining deep learning and traditional method has strong interpretability, and can be applied to target detection under complex scene through a large amount of data support.

[0098] The method combining deep learning and traditional algorithm improves the detection efficiency and intelligent level, can effectively prevent the fuse from missing and misloading, and guarantees the safety of the aerospace engine. According to the YOLOv5 algorithm in the YOLO (You Only Look Once) series of one-stage target detection algorithm, the yolov5s.pt model is used to detect the engine image collected by pre-processing, the fuse position information and confidence are detected, the candidate frame target detected is classified, whether the fuse exists is judged, and after the judgment is finished, the region where the fuse exists is cut, as the data set of the subsequent traditional method. The traditional method is used to exclude the interference of scratches, light spots, shadows and edge contours, recognize the fuse, and judge the consistency of the winding direction and the tightening direction of the fuse, and the number of the fuse screw. The detection efficiency of the fuse of the aerospace engine is improved, the machine is used instead of the manual work, and the intelligent level of the fuse detection in the engine assembly is improved.

[0099] Although the present application is disclosed with the preferred embodiments as above, it is not intended to limit the present application, and any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present application, therefore, the protection scope of the present application should be limited by the scope defined by the claims of the present application.

Claims

1. A method for visually detecting the installation state of a spaceflight pipe joint anti-loosening fuse, characterized in that: Comprising S1, obtaining the image of the pipe joint fuse to be detected, judging the existence or nonexistence of the fuse; S2, pipe joint fuse installation angle and tightening direction detection: S21, image processing is performed on the image whose detection result in step S1 is that the fuse exists, a plurality of line segments corresponding to the fuse are obtained, and the coordinates of the line segments are stored in Lines_Points dynamic array; S22, the line segments that do not meet the requirements of the angle between the line segments and the longitudinal axis of the pipe joint are removed from the Lines_Points dynamic array, and if the endpoint data of the remaining line segments in the Lines_Points dynamic array is not zero, the installation angle of the fuse is qualified; S23, multi-line segment fitting: call Lines_Points dynamic array in step 22, fit the line segment to get a unique line segment If Then the right-handed state fuse tightening direction is correct, proceed to the pipe joint fuse screw thread number detection, if Then the right-handed state tightening direction is incorrect, end the detection process, output the detection result that the pipe joint fuse tightening direction is incorrect, if the thread state is left-handed, the judgment logic is opposite; S3, pipe joint fuse spiral edge line identification: obtaining the fuse contour; S4, processing the fuse contour to obtain the separated single spiral contour of the fuse; S5, finding the separated single spiral contour in step S4, removing the contours whose areas do not meet the requirements through area screening, then traversing and counting all contours of the entire image to obtain the number of fuse spirals, and determining whether the number of fuse spirals meets the requirements.

2. The method of claim 1, wherein: In step S1, judging whether the fuse exists includes the following steps S11, deep learning data set making: making positive samples and negative samples of the fuse according to the actual pipe joint fuse process as a data set; S12, deep learning data set annotation processing: dividing the data set into a training set and a validation set, performing pipe joint fuse data annotation on the training and validation images of the input data set, and the annotation types include two detection result types: fuse exists and fuse does not exist; S13, pipe joint fuse missing detection: taking the training set in step S12 as the training data of the YOLOv5 model, performing feature learning on the pipe joint fuse, and performing target detection on the pipe joint fuse to be detected.

3. The method of claim 2, wherein: In step S11, the correct installation of the fuse represents a positive sample, and the nonexistence, incorrect direction, installation angle, and number of spirals of the fuse do not meet the requirements represent negative samples; After the positive samples and negative samples in step S11 are made, an industrial camera is used to simulate the real environment of the factory by using a CCS ring light source and its light source controller to provide different intensity of light, keep the direction of the pipe joint vertical upward in the image, and perform image acquisition to obtain the data set.

4. The method of claim 2 or 3, wherein the method further comprises: determining whether the space between the two ends of the aerospace pipe joint is less than a predetermined value; and if the space between the two ends of the aerospace pipe joint is less than the predetermined value, determining that the aerospace pipe joint is in the installed state. After step S1, traditional algorithm data set making: referring to the judgment result in step S1, cutting the area where the YOLOv5 model judges that the fuse exists, taking it as a traditional algorithm data set, and performing image processing on the traditional algorithm data set in step S2.

5. The method of claim 1, wherein: In step S21, edge extraction is performed on the fuse contour, and then Hough straight line detection is performed to obtain a plurality of line segments corresponding to the fuse.

6. The method of claim 1, wherein: In step S23, the plurality of line segments fitting includes: The pixel points of the line segment are calculated using the end points of the line segment stored in the Lines_Points dynamic array, and the pixel point coordinate values of all the line segments are recorded as (u n ,v n ) The direction of v n is parallel to the axial direction of the pipe joint, a Points_1 dynamic array is named to store all the pixel point coordinate values, the average value of the u coordinates of the pixel points under the same v coordinate is calculated and recorded as (x n ,y n ), a Points_2 dynamic array is named to store all the average value coordinate values of the pixel points, and the average value coordinate values are brought into the least square fitting straight line equation, wherein x and y are the coordinate values of the pixel points in the Points_2 dynamic array, i is the serial number of the coordinate values, i takes values of 1, 2, 3…n, N represents the number of coordinate values, is a constant, is the slope of the fitting straight line solving the resulting system of equations yields a unique line segment 7. The method of claim 1, wherein: In step S3, the image of the pipe joint is subjected to grayscale processing, then subjected to median filtering operation, and then subjected to binarization operation on the image subjected to median filtering to obtain the fuse contour.

8. The method of claim 1, wherein: In step S4, the fuse contour is processed by using a watershed algorithm.

9. The method of claim 1, wherein: In the step S5, whether the number of the twisted wires meets the requirement is determined according to the number of the twisted wires, comprising: obtaining the number of the twisted wires per unit centimeter according to the conversion relationship between the pixel points of the image shot by the industrial camera and the actual length, comparing with the number of the twisted wires of the aerospace pipe joint fuse to determine, and finally outputting the determination result of whether the number of the twisted wires is qualified or unqualified.

10. The method of claim 1, wherein: In the step S1, the image of the pipe joint fuse to be detected meets: the shooting direction is perpendicular to the pipe joint axis, the direction of the pipe joint in the image is kept vertical upward, and the length of the fuse obtained by the shooting plane is not less than 50% of the length of the whole fuse; In the step S1, considering the multiple possibilities of the final tightening position of the pipe joint cover nut, multiple shooting angles are reserved according to the possibilities, the images are determined for the missing installation, the direction and the number of the twisted wires, and if the determination results of the three items of one image are all correct, the detection result is output as qualified.

Images