Method and system for detecting the absence of a cross nut of a catenary arm base

Abstract

The application discloses a catenary cantilever base transverse nut loss detection method and system, comprising: acquiring a catenary original image, preprocessing the catenary original image to obtain an image containing a flat and inclined cantilever base area; adopting a YOLO algorithm to perform first-level positioning on the flat and inclined cantilever base area to obtain a first-level positioning result; the first-level positioning result comprises a flat cantilever base and an inclined cantilever base; adopting the YOLO algorithm to perform second-level positioning on the first-level positioning result to obtain a second-level positioning result; the second-level positioning is positioning on a transverse nut area, and the second-level positioning result comprises a front nut area, a back nut area and a cantilever base back; adopting a folding positioning method combined with a YOLO algorithm processing method and / or a convolutional neural network classification processing method to judge whether the transverse nut of the cantilever base in the second-level positioning result is lost, and outputting a judgment result. The application has high recognition accuracy and low misrecognition rate.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of detection of missing transverse nuts of a catenary 4C arm base, and in particular to a detection method and system for missing transverse nuts of a catenary arm base. BACKGROUND

[0002] A "high-speed railway catenary suspension state detection monitoring device (4C)" system detects defects by photographing and collecting key components such as a catenary support device, a positioning device and additional suspension, and then manually checking and analyzing the original collected 4C pictures. The current manual interpretation method is time-consuming and labor-intensive, is greatly affected by personal emotions and responsibility, has high risk of missed detection, a long detection cycle and other shortcomings, and especially if a 4C first-level defect is not discovered and handled in time, it can seriously affect the safety of train operation.

[0003] The transverse nut of the catenary 4C arm base is located at the connection between the arm base and the support column, and when the transverse nut is missing, if the split pin is also missing, the entire arm is prone to falling off, which greatly affects the structural integrity of the catenary. As a first-level defect, if it is not discovered and handled in time, it can seriously affect the safety of train operation.

[0004] In the prior art, there are few technical solutions for missing 4C arm base transverse nuts, and the existing manual detection method and simple convolutional neural network (CNN) classification method for missing 4C arm base transverse nuts have high missed detection rate and high misidentification rate. SUMMARY

[0005] The application solves the technical problem of the existing manual detection method and simple convolutional neural network (CNN) classification method for missing 4C arm base transverse nuts, which have high missed detection rate and high misidentification rate, and aims to provide a detection method and system for missing transverse nuts of a catenary arm base, which has high identification accuracy and low misidentification rate.

[0006] The application is implemented by the following technical solutions:

[0007] In a first aspect, the application provides a detection method for missing transverse nuts of a catenary arm base, as shown in the accompanying drawings, which comprises the following steps: Figure 1

[0008] Obtaining a catenary original image, preprocessing the catenary original image to obtain an image containing a flat and inclined arm base region;

[0009] Using a YOLO algorithm to perform first-level positioning on the flat and inclined arm base region to obtain a first-level positioning result; the first-level positioning result includes a flat arm base and an inclined arm base;

[0010] ​The YOLO algorithm is used for secondary positioning on the first positioning result to obtain a secondary positioning result; the secondary positioning is positioning on a transverse nut area, and the secondary positioning result includes a front nut area, a back nut area, and a back of a wrist arm base;

[0011] The folding positioning method combined with the YOLO algorithm and / or the convolutional neural network classification method is used to determine whether the transverse nut of the wrist arm base in the secondary positioning result is missing, and a determination result is output.

[0012] The YOLO algorithm (You Only Look Once, specifically, the YOLO v5 algorithm) is an existing algorithm.

[0013] The working principle is that, based on the problems of high missing detection rate and high misrecognition rate of the existing 4C wrist arm base transverse nut missing artificial detection method and simple convolutional neural network (CNN) classification method, the present application designs a catenary wrist arm base transverse nut missing detection method, which is based on the YOLO algorithm, two-stage positioning, and then expanding and folding-90 degrees on the secondary positioning result, and then three-stage positioning, and not directly positioning the nut but positioning the screw rod, and then determining the image proportion occupied by the screw rod height to inversely determine whether the nut is missing. Specifically, the present application uses a two-stage two-class positioning method, which considers the back nut area in the positioning stage and the classification stage, which can effectively reduce the false positives of the back nut and reduce the false positive rate. At the same time, based on the secondary positioning, the present application uses the folding positioning method, which expands the nut area based on the secondary positioning, then folds-90 degrees on the expanded area, and then performs three-stage YOLO positioning on the nut screw rod area, and uses the image height proportion occupied by the screw rod area in the entire expanded nut area to determine whether the nut is missing. This method can control the recognition rate and false positive rate according to the set threshold size.

[0014] The folding positioning method and the convolutional neural network classification method can also be combined to detect the transverse nut missing, and when the transverse nut missing is important, the nut missing is considered missing when either of the two methods determines the nut missing; when the transverse nut missing is not very important, the nut missing is considered missing only when both methods determine the nut missing.

[0015] Compared with the CNN classification method with an identification accuracy of less than 80%, the positioning method combined with the geometric parameter determination method of the present application has an identification accuracy of more than 92%, and the accuracy can be further improved by setting the parameter threshold size. The present application has high identification accuracy and low misrecognition rate.

[0016] Further, the folding positioning method combined with the YOLO algorithm is used to determine whether the transverse nut of the wrist arm base in the secondary positioning result is missing, specifically:

[0017] The nut region is expanded by using the folding positioning method, to obtain an expanded nut region; the expanded nut region is folded, and the YOLO algorithm is used to perform three-level positioning on the screw rod region (the screw nut corresponding screw rod region) to obtain a three-level positioning result.

[0018] The ratio of the image height of the three-level positioning result to the image height of the two-level positioning result is calculated, and whether the transverse nut of the wrist arm base is missing is determined according to whether the ratio is greater than a set threshold value, to obtain a first missing determination result.

[0019] Further, the specific steps of the folding positioning method are as follows:

[0020] The two-level positioning result is filtered to filter out the back nut region.

[0021] The up, down, left and right of the front nut region are respectively expanded to obtain an expanded back nut region; wherein the left and right are expanded by one fifth of the image width of the two-level positioning result, and the up and down are expanded by one tenth of the image height of the two-level positioning result.

[0022] The expanded back nut region is folded by 90 degrees to obtain a folded back nut region.

[0023] The YOLO algorithm is used to perform three-level positioning on the screw rod region to obtain a three-level positioning result; the three-level positioning result is the folded screw rod region positioning result.

[0024] Further, whether the transverse nut of the wrist arm base is missing according to whether the ratio is greater than a set threshold value includes:

[0025] If the ratio is greater than the set threshold value, the transverse nut of the wrist arm base is missing.

[0026] If the ratio is less than or equal to the set threshold value, the transverse nut of the wrist arm base is not missing.

[0027] Further, the processing method of using a convolutional neural network classification is used to determine whether the transverse nut of the wrist arm base is missing in the two-level positioning result, specifically as follows:

[0028] The two-level positioning result is directly classified by using the convolutional neural network classification method to obtain a direct classification result.

[0029] According to the direct classification result, whether the transverse nut of the wrist arm base is missing is determined to obtain a second missing determination result.

[0030] Further, the processing method of folding positioning combined with YOLO algorithm is adopted to judge whether the transverse nut of the wrist arm base in the secondary positioning result is missing, and a first missing judgment result is obtained.

[0031] The processing method of convolutional neural network classification is adopted to judge whether the transverse nut of the wrist arm base in the secondary positioning result is missing, and a second missing judgment result is obtained.

[0032] According to the first missing judgment result and / or the second missing judgment result, whether the transverse nut of the wrist arm base is truly missing is judged, and a final missing judgment result is obtained.

[0033] Further, the processing method of folding positioning combined with YOLO algorithm is adopted to judge whether the transverse nut of the wrist arm base in the secondary positioning result is missing, and a first missing judgment result is obtained.

[0034] If the first missing judgment result is missing, and the second missing judgment result is missing, the transverse nut of the wrist arm base is missing; or,

[0035] If the first missing judgment result is missing, or the second missing judgment result is missing, the transverse nut of the wrist arm base is missing.

[0036] Further, the preprocessing of the catenary original image comprises:

[0037] According to the camera number corresponding to the suffix name of the photographed image, the catenary original image is filtered to obtain a filtered image;

[0038] According to the suffix name of the photographed image, the filtered image is folded: when the filtered image is to the left, the filtered image is horizontally folded by 180 degrees to obtain a wrist arm image horizontally to the right.

[0039] The above technical scheme, after preprocessing, adopts the method of judging the camera number according to the picture number to identify only the wrist arm area photographed image when judging the missing of the transverse nut of the wrist arm base, effectively avoids interference and reduces false positives; then, according to the suffix name of the original image, it is judged whether the original image needs to be folded, and all large images are folded to the same direction, ensuring that the small images are in the same direction during convolutional neural network CNN classification and three-level positioning, which can improve the recognition rate. Specifically, the original image is filtered according to the camera number corresponding to the suffix name, and the image is folded left and right to ensure that the small images of the secondary positioning nut are in the same direction, and the back nut is considered in the secondary positioning stage and the convolutional neural network CNN classification stage, effectively reducing false positives. Test results show that the false positive rate is less than 15%.

[0040] Further, the YOLO algorithm adopts a YOLOv5 network, and the YOLOv5s network has a depth of 0.33 and a width of 0.50.

[0041] The convolutional neural network classification method adopts a VGGNet-D network structure.

[0042] In a second aspect, the application further provides a catenary arm base transverse nut loss detection system supporting the catenary arm base transverse nut loss detection method.

[0043] An acquisition unit is configured to acquire a catenary original image.

[0044] A preprocessing unit is configured to preprocess the catenary original image to obtain an image containing a flat and inclined arm base region.

[0045] A first-level positioning unit is configured to perform first-level positioning on the flat and inclined arm base region by using a YOLO algorithm to obtain a first-level positioning result, wherein the first-level positioning result includes a flat arm base and an inclined arm base.

[0046] A second-level positioning unit is configured to perform second-level positioning on the first-level positioning result by using the YOLO algorithm to obtain a second-level positioning result, wherein the second-level positioning is performed on a transverse nut region, and the second-level positioning result includes a front nut region, a back nut region, and an arm base back.

[0047] A judgment unit is configured to judge whether the transverse nut of the arm base is lost in the second-level positioning result by using a folding positioning method combined with a YOLO algorithm processing method and / or a convolutional neural network classification processing method, and output a judgment result.

[0048] Further, the judgment unit includes a first judgment unit, a second judgment unit, and a final judgment unit.

[0049] The first judgment unit is configured to judge whether the transverse nut of the arm base is lost in the second-level positioning result by using the folding positioning method combined with the YOLO algorithm processing method, including:

[0050] The folding positioning method is used to expand the nut region in the second-level positioning result to obtain an expanded nut region, the expanded nut region is folded, and the YOLO algorithm is used to perform third-level positioning on the screw rod region to obtain a third-level positioning result.

[0051] The ratio of the image height of the third-level positioning result to the image height of the second-level positioning result is calculated, and whether the transverse nut of the arm base is lost is judged according to whether the ratio is greater than a set threshold value to obtain a first loss judgment result.

[0052] The second judging unit is configured to judge whether the transverse nut of the wrist arm base is missing in the secondary positioning result by using a convolutional neural network classification method, and includes the following steps:

[0053] The secondary positioning result is directly classified by using the convolutional neural network classification method to obtain a direct classification result.

[0054] According to the direct classification result, it is judged whether the transverse nut of the wrist arm base is missing to obtain a second missing judgment result.

[0055] The final judging unit is configured to judge whether the transverse nut of the wrist arm base is truly missing according to the first missing judgment result and / or the second missing judgment result to obtain a final missing judgment result.

[0056] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0057] 1. The contact net wrist arm base transverse nut missing detection method and system adopts a folding positioning method, expands the nut area on the basis of secondary positioning, then folds the expanded area by 90 degrees, and then performs three-level YOLO positioning on the nut screw area, and uses the proportion of the screw area image height to the entire expanded nut area to judge whether the nut is missing. This method can control the recognition rate and false alarm rate according to the set threshold size. Compared with the CNN classification method, the recognition rate of the present application is higher than 92%, the recognition accuracy is high, and the accuracy can be further improved by setting the parameter threshold size.

[0058] 2. The contact net wrist arm base transverse nut missing detection method and system judges the picture camera number according to the picture serial number in the preprocessing process, only identifies the wrist arm area image when the wrist arm base transverse nut is missing, effectively avoids interference and reduces false alarms; then judges whether the original image needs to be folded according to the picture suffix, folds all large images to the same direction, ensures that the small image directions are consistent during CNN classification and three-level positioning, and can improve the recognition rate.

[0059] 3. The contact net wrist arm base transverse nut missing detection method and system adopts a two-level positioning method, considers the back nut area in the positioning stage and the classification stage, can effectively reduce the false alarm of the back nut, reduces the false alarm rate, has low misrecognition rate, and the test results show that the false alarm rate is less than 15%. BRIEF DESCRIPTION OF DRAWINGS

[0060] The drawings described herein are used to provide further understanding of the embodiments of the present application, constitute a part of the present application, and do not constitute a limitation on the embodiments of the present application. In the drawings:

[0061] Figure 1 A flow chart of a contact net arm base transverse nut missing detection method is disclosed.

[0062] Figure 2 A flow chart of a contact net arm base transverse nut missing detection method is disclosed.

[0063] Figure 3 A "one pole one file" original collection 4C one pole image of embodiment 1 of the application is disclosed.

[0064] Figure 4 A first level positioning result schematic diagram of the arm base of embodiment 1 of the application is disclosed.

[0065] Figure 5 A flat arm base transverse nut area of embodiment 1 of the application is disclosed.

[0066] Figure 6 An inclined arm base transverse nut area of embodiment 1 of the application is disclosed.

[0067] Figure 7 A transverse nut back area of the arm base of embodiment 1 of the application is disclosed.

[0068] Figure 8 A second level positioning result schematic diagram of the transverse nut area of the arm base of embodiment 1 of the application is disclosed.

[0069] Figure 9 A third level positioning result schematic diagram of the transverse nut screw rod area after folding when missing of embodiment 1 of the application is disclosed.

[0070] Figure 10 A third level positioning result schematic diagram of the transverse nut screw rod area after folding when not missing of embodiment 1 of the application is disclosed.

[0071] Figure 11 A geometric parameter calculation schematic diagram of embodiment 1 of the application is disclosed.

[0072] Figure 12 A VGGNet-D network structure schematic diagram of embodiment 2 of the application is disclosed.

[0073] Figure 13 Three kinds of sample schematic diagrams of the convolutional neural network CNN classification of embodiment 2 of the application are disclosed.

[0074] Figure 14 A structure schematic diagram of a contact net arm base transverse nut missing detection system of embodiment 3 of the application is disclosed.

[0075] Figure 15 A structure schematic diagram of a contact net arm base transverse nut missing detection system of embodiment 4 of the application is disclosed.

[0076] Figure 16 A flow chart of a contact net cantilever base transverse nut missing detection method of embodiment 2 of the present application. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical scheme and advantages of the present application clearer, further detailed description will be given below in combination with embodiments and drawings, the illustrative embodiments of the present application and the description thereof are only used to explain the present application, and do not limit the present application.

[0078] Embodiment 1

[0079] Based on the problems of high missed detection rate and high false recognition rate of the existing manual detection method and simple convolutional neural network (CNN) classification method for 4C cantilever base transverse nut missing, the present application designs a contact net cantilever base transverse nut missing detection method. The present application is aimed at 4C cantilever base transverse nut missing defects, based on two-stage positioning of YOLO algorithm, the secondary positioning results are enlarged and then folded-90 degrees, and then three-stage positioning is performed, and instead of directly positioning the nut, the screw rod is positioned, and then the proportion of the height of the screw rod in the image is used to inversely judge whether the nut is missing. Specifically, the present application adopts a two-stage two-class positioning method, and the back nut area is considered in the positioning stage and the classification stage, which can effectively reduce the false alarm of the back nut and reduce the false alarm rate. At the same time, on the basis of two-stage positioning, the present application adopts a folding positioning method, which enlarges the nut area on the basis of two-stage positioning, then folds-90 degrees on the enlarged area, and then performs three-stage YOLO positioning on the nut screw rod area, and uses the proportion of the height of the screw rod area image to the entire enlarged nut area to judge whether the nut is missing. This method can control the recognition rate and false alarm rate according to the set threshold size.

[0080] As shown in Figure 2 , the present application is a contact net cantilever base transverse nut missing detection method, which comprises:

[0081] Step 1: obtaining a contact net original image, pre-processing the contact net original image to obtain an image containing a flat and inclined cantilever base area; the pre-processing of the contact net original image comprises:

[0082] According to the camera number corresponding to the suffix of the photographed image, the contact net original image is filtered to obtain a filtered image;

[0083] According to the suffix of the photographed image, the filtered image is folded: when the filtered image is to the left, the filtered image is folded horizontally by 180 degrees to obtain a cantilever image with the horizontal to the right. At this time, all the input contact net original images are cantilevered to the right. The "one rod and one row" image photographed by 4C is as shown in Figure 3 .

[0084] As shown in Figure 3 , according to the camera layout and the characteristics of the imaging pictures, only the four images with the suffixes “05.jpg”, “06.jpg”, “22.jpg” and “23.jpg” are processed when detecting the missing transverse nut of the wrist arm base, and other images with different suffixes are automatically filtered, effectively reducing false positives and saving identification time. Among them, “05.jpg” and “06.jpg” are rightward, and “22.jpg” and “23.jpg” are leftward, so the processing method is to fold 180 degrees horizontally when the input picture is the original picture with the suffixes “22.jpg” and “23.jpg” and the wrist arm direction is leftward, and no folding is needed when the input picture is the original picture with the suffixes “05.jpg” and “06.jpg” and the wrist arm direction is rightward.

[0085] The above step 1 is preprocessed by judging the camera number according to the picture serial number, and only the wrist arm area image is identified when the transverse nut of the wrist arm base is missing, effectively avoiding interference and reducing false positives; then, whether the original image needs to be folded is judged according to the picture suffix, and all large images are folded to the same direction, ensuring that the small images are in the same direction during convolutional neural network (CNN) classification and three-level positioning, which can improve the recognition rate.

[0086] Step 2: using YOLO algorithm to perform first-level positioning on the flat and inclined wrist arm base area to obtain first-level positioning result; the first-level positioning is performed on the flat and inclined wrist arm base area, and the first-level positioning result includes flat wrist arm base and inclined wrist arm base; wherein the first-level area class of flat wrist arm base is 0, and the first-level area class of inclined wrist arm base is 1, and the first-level positioning result is as shown in Figure 4 .

[0087] YOLO algorithm (You Only Look Once, specifically YOLO v5 algorithm) is an existing algorithm.

[0088] Step 3: using YOLO algorithm to perform second-level positioning on the first-level positioning result to obtain second-level positioning result; the second-level positioning is performed on the transverse nut area, and the second-level positioning result includes front nut area, back nut area and wrist arm base back; wherein the class of front nut area is 0, the class of back nut area is 1, and the class of wrist arm base back is 2, and the second-level positioning result is as shown in the white boxed area in Figure 5 、 Figure 6 、 Figure 7 .

[0089] After successfully positioning the second-level wrist arm base nut area, the second-level positioning wrist arm base transverse nut area is as shown in Figure 8 .

[0090] Step 4, using the folding positioning method, expanding the nut area of the secondary positioning result to obtain an expanded nut area; folding the expanded nut area and using the YOLO algorithm to perform tertiary positioning on the screw rod area (the screw nut corresponding screw rod area) to obtain a tertiary positioning result;

[0091] Specifically, the steps of the folding positioning method are as follows:

[0092] Filtering the secondary positioning result to filter out the back nut area;

[0093] Respectively expanding the up, down, left and right of the front nut area to obtain an expanded back nut area; wherein the left and right are expanded by one fifth of the width of the secondary positioning result image, and the up and down are expanded by one tenth of the height of the secondary positioning result image;

[0094] Folding the expanded back nut area by -90 degrees to obtain a folded back nut area;

[0095] Using the YOLO algorithm to perform tertiary positioning on the screw rod area (the screw nut corresponding screw rod area, not the nut area) to obtain a tertiary positioning result; the tertiary positioning result is the folded screw rod area positioning result.

[0096] The tertiary positioning result of the folded screw rod area is as shown in the white box area in Figure 9 and Figure 10 , Figure 9 is the secondary positioning folded -90 degree screw rod area tertiary result of the missing horizontal nut of the wrist arm base, Figure 10 is the secondary positioning folded -90 degree screw rod area tertiary result of the non-missing horizontal nut of the wrist arm base.

[0097] Step 5, calculating the ratio of the image height of the tertiary positioning result to the image height of the secondary positioning result, i.e. finding the ratio r of the screw rod positioning rectangular frame height h to the image vertical coordinate y of the right lower corner point of the rectangle, as shown in Figure 11 ; according to whether the ratio is greater than a set threshold value, judging whether the horizontal nut of the wrist arm base is missing to obtain a first missing judgment result.

[0098] Specifically, according to whether the ratio r is greater than a set threshold value, judging whether the horizontal nut of the wrist arm base is missing, includes:

[0099] If the ratio r is greater than the set threshold value, the horizontal nut of the wrist arm base is missing;

[0100] If the ratio r is less than or equal to the set threshold value, i.e. the exposed screw rod area is shorter, the horizontal nut of the wrist arm base is not missing.

[0101] After a large number of test verifications, when r >= 0.8, it is judged that the transverse nut of the wrist arm base is missing.

[0102] The YOLO algorithm in the above step adopts a YOLOv5 network, and the depth of the YOLOv5s network is 0.33 and the width is 0.50;

[0103] The present application has the following innovative points:

[0104] 1. In the pretreatment process, the picture camera number is determined according to the picture serial number, and when the wrist arm base transverse nut is missing, only the wrist arm area image is identified, which effectively avoids interference and reduces false positives; then, whether the original image needs to be folded is determined according to the picture suffix, and all large images are folded to the same direction, ensuring that the small images are in the same direction during CNN classification and three-level positioning, which can improve the recognition rate.

[0105] 2. The folding positioning method is used to expand the nut area on the basis of two-level positioning, then the expanded area is folded by-90 degrees, and then the three-level YOLO positioning is performed on the nut screw area, and the proportion of the screw area image height to the entire expanded nut area is used to determine whether the nut is missing, and this method can control the recognition rate and false positive rate according to the set threshold size.

[0106] 3. The two-level positioning method is used, and the back nut area is considered in the positioning stage and the classification stage, which can effectively reduce the false positives of the back nut and reduce the false positive rate.

[0107] The present application has the following beneficial effects:

[0108] 1. The present application has high recognition accuracy: compared with the CNN classification method with an identification accuracy of less than 80%, the present application method has an identification accuracy of more than 92% by using positioning combined with geometric parameter judgment, and the accuracy can be further improved by setting the parameter threshold size.

[0109] 2. Low misidentification rate: according to the corresponding camera number of the original image suffix, the image is folded left and right to ensure that the small image nut in the two-level positioning is in the same direction, and the back nut is considered in the two-level positioning stage and the convolutional neural network CNN classification stage, which effectively reduces the false positives, and the test results show that the false positive rate is less than 15%.

[0110] Example 2

[0111] As shown in Figure 12 , Figure 13 and Figure 16 , the difference between the present embodiment and example 1 is that, as shown in Figure 16 , the method further comprises:

[0112] The two-stage positioning result is directly classified by using a convolutional neural network classification method, and whether the transverse nut of the wrist arm base is missing is judged to obtain a second missing judgment result.

[0113] According to the first missing judgment result and / or the second missing judgment result, whether the transverse nut of the wrist arm base is truly missing is judged to obtain a final missing judgment result.

[0114] The above technical scheme combines the folding positioning method and the convolutional neural network classification method to detect the missing of the transverse nut. When the missing of the transverse nut is important, the missing of the nut is considered to be missing by either of the two methods. When the missing of the transverse nut is not important, the missing of the nut is considered to be missing only when the missing of the nut is judged by both methods.

[0115] Specifically, the two-stage positioning result is directly classified by using a convolutional neural network classification method, and whether the transverse nut of the wrist arm base is missing is judged to obtain a second missing judgment result, including:

[0116] The two-stage positioning result is directly classified by using a convolutional neural network classification method to obtain a direct classification result.

[0117] According to the direct classification result, whether the transverse nut of the wrist arm base is missing is judged to obtain a second missing judgment result.

[0118] Specifically, according to the first missing judgment result and / or the second missing judgment result, whether the transverse nut of the wrist arm base is truly missing is judged to obtain a final missing judgment result, including:

[0119] If the first missing judgment result is missing, and the second missing judgment result is missing, the transverse nut of the wrist arm base is missing; or,

[0120] If the first missing judgment result is missing, or the second missing judgment result is missing, the transverse nut of the wrist arm base is missing.

[0121] Specifically, the convolutional neural network classification method CNN uses a VGGNet-D network structure, as shown in Figure 12 .

[0122] In specific implementation, the VGGNet-D network structure is used to divide the transverse nut region into three categories: 0, 1, and 2, where 0 represents the missing of the transverse nut, 1 represents the existence of the transverse nut, and 2 represents the back region of the transverse nut. Since all large images are folded to the same direction in the preprocessing stage, all positioning small images have the same orientation when classified by CNN, avoiding interference from different orientations and improving the detection rate. The CNN sample is shown in Figure 13 .

[0123] When the CNN is used to predict that the transverse nut region classification category belongs to the 0 category, it is judged that the transverse nut is missing.

[0124] When any one of the folding positioning method or the CNN method judges that the transverse nut is missing, it is judged that the transverse nut is missing.

[0125] The present application is aimed at the defect of missing transverse nut of 4C cantilever base. First, YOLO v5 algorithm is used to position the flat and inclined cantilever base region for first-level positioning, and first-level positioning result is obtained. Considering the real-time performance of the algorithm, YOLOv5s network structure is used, with network depth of 0.33 and width of 0.50. On the basis of the first-level positioning result, second-level transverse nut region positioning is performed, and then two methods (folding positioning method and convolutional neural network classification method) are combined to detect the missing transverse nut. The first method is the folding positioning method, which enlarges the nut region on the basis of the second-level positioning, then folds the enlarged region by-90 degrees, and then performs third-level YOLO positioning on the nut screw region. The proportion of the screw region image height to the entire enlarged nut region is used to judge whether the nut is missing, but if the proportion exceeds a certain threshold, it is judged that the transverse nut is missing. The second method is the convolutional neural network classification method, which directly classifies the second-level positioning result using convolutional neural network (CNN) classification method to judge whether the transverse nut of the cantilever base is missing. Since the missing transverse nut defect is very important, in order to reduce false negatives, the missing nut is considered missing by any one of the two methods.

[0126] Embodiment 3

[0127] As shown in Figure 14 The difference between the present embodiment and embodiment 1 is that the present embodiment provides a catenary cantilever base transverse nut missing detection system, which supports the catenary cantilever base transverse nut missing detection method described in embodiment 1. The system comprises:

[0128] An acquisition unit is configured to acquire a catenary original image.

[0129] A preprocessing unit is configured to preprocess the catenary original image to obtain a preprocessed image.

[0130] A first-level positioning unit is configured to perform first-level positioning on the preprocessed image using YOLO algorithm to obtain first-level positioning result. The first-level positioning is performed on the flat and inclined cantilever base region, and the first-level positioning result includes flat cantilever base and inclined cantilever base.

[0131] A secondary positioning unit is configured to perform secondary positioning on the primary positioning result by using a YOLO algorithm to obtain a secondary positioning result; the secondary positioning is to position a lateral nut region, and the secondary positioning result includes a front nut region, a back nut region, and a back base of a wrist arm;

[0132] A tertiary positioning unit is configured to perform tertiary positioning on the secondary positioning result by using a folding positioning method to expand a nut region, to obtain an expanded nut region; fold the expanded nut region, and perform tertiary positioning on a screw rod region (a screw nut corresponding screw rod region) by using a YOLO algorithm to obtain a tertiary positioning result;

[0133] A first nut absence judgment unit is configured to calculate a ratio of an image height of the tertiary positioning result to an image height of the secondary positioning result, and determine whether a lateral nut of the base of the wrist arm is absent according to whether the ratio is greater than a set threshold to obtain a first absence judgment result.

[0134] The execution process of each unit can be performed according to the process steps of the contact net base of the wrist arm lateral nut absence detection method in Embodiment 1, and will not be repeated here.

[0135] Embodiment 4

[0136] As shown in the figure, the difference between this embodiment and Embodiment 3 is that the system further includes a convolutional neural network classification unit, a second nut absence judgment unit, and a final nut absence judgment unit. Figure 15

[0137] The convolutional neural network classification unit is configured to directly classify the secondary positioning result by using a convolutional neural network classification method to obtain a direct classification result.

[0138] The second nut absence judgment unit is configured to determine whether the lateral nut of the base of the wrist arm is absent according to the direct classification result to obtain a second absence judgment result.

[0139] The final nut absence judgment unit is configured to determine whether the lateral nut of the base of the wrist arm is truly absent according to the first absence judgment result and / or the second absence judgment result to obtain a final absence judgment result.

[0140] The execution process of each unit can be performed according to the process steps of the contact net base of the wrist arm lateral nut absence detection method in Embodiment 1, and will not be repeated here.

[0141] ​Those skilled in the art will appreciate that embodiments of the application can be readily used as software, hardware, or a combination of software and hardware. In one

[0142] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 an apparatus to perform the functions specified in the flowchart block or blocks.

[0143] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 an apparatus to perform the functions specified in the flowchart block or blocks.

[0144] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks in the flowcharts and / or a combination thereof. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 an apparatus to perform the functions specified in the flowchart block or blocks.

[0145] The above detailed description has shown, described, and pointed out the aspects of the application in sufficient detail that others skilled in the art can practice the application. It is understood that various modifications, substitutions, and changes can be made to the techniques and methods specifically without departing from the scope of the present application.

Claims

1. A method for detecting the absence of a cross nut of a catenary arm base, characterized in that, The method comprises: obtaining a catenary original image, preprocessing the catenary original image to obtain an image containing a flat and inclined cantilever base area; using a YOLO algorithm to perform first-level positioning on the flat and inclined cantilever base area to obtain a first-level positioning result; the first-level positioning result comprises a flat cantilever base and an inclined cantilever base; using the YOLO algorithm to perform second-level positioning on the first-level positioning result to obtain a second-level positioning result; the second-level positioning is positioning on a transverse nut area, and the second-level positioning result comprises a front nut area, a back nut area and a cantilever base back surface; using a folding positioning method combined with a YOLO algorithm processing method and / or a convolutional neural network classification processing method to judge whether the transverse nut of the cantilever base in the second-level positioning result is missing, and output a judgment result; using the folding positioning method combined with the YOLO algorithm processing method to judge whether the transverse nut of the cantilever base in the second-level positioning result is missing, specifically as follows: using the folding positioning method to expand the transverse nut area in the second-level positioning result to obtain an expanded nut area; folding the expanded nut area and using the YOLO algorithm to perform third-level positioning on a screw rod area to obtain a third-level positioning result; calculating the ratio of the image height of the third-level positioning result to the image height of the second-level positioning result, and judging whether the transverse nut of the cantilever base is missing according to whether the ratio is greater than a set threshold to obtain a first missing judgment result; the specific steps of the folding positioning method are as follows: filtering the second-level positioning result to filter out the back nut area; expanding the upper, lower, left and right of the front nut area respectively to obtain an expanded back nut area; wherein the left and right are expanded by one fifth of the image width of the second-level positioning result, and the upper and lower are expanded by one tenth of the image height of the second-level positioning result; folding the expanded back nut area by -90 degrees to obtain a folded back nut area; using the YOLO algorithm to perform third-level positioning on the screw rod area to obtain a third-level positioning result; the third-level positioning result is a positioning result of the folded screw rod area.

2. The method for detecting the missing of the cross nut of the catenary arm base according to claim 1, characterized in that, judging whether the transverse nut of the cantilever base is missing according to whether the ratio is greater than a set threshold, comprising: if the ratio is greater than the set threshold, the transverse nut of the cantilever base is missing; if the ratio is less than or equal to the set threshold, the transverse nut of the cantilever base is not missing.

3. The method for detecting the missing of the cross nut of the catenary arm support base according to claim 1, characterized in that, using the convolutional neural network classification processing method to judge whether the transverse nut of the cantilever base in the second-level positioning result is missing, specifically as follows: using the convolutional neural network classification method to directly classify the second-level positioning result to obtain a direct classification result; judging whether the transverse nut of the cantilever base is missing according to the direct classification result to obtain a second missing judgment result.

4. The catenary cantilever base transverse nut missing detection method according to claim 1, wherein the folding positioning method combined with the YOLO algorithm processing method is used to judge whether the transverse nut of the cantilever base in the second-level positioning result is missing to obtain a first missing judgment result. ​ The processing method of the convolutional neural network classification is used to determine whether the transverse nut of the wrist arm base in the secondary positioning result is missing, and a second missing determination result is obtained. According to the first missing determination result and / or the second missing determination result, it is determined whether the transverse nut of the wrist arm base is actually missing, and a final missing determination result is obtained.

5. A method for detecting the absence of a cross nut of a catenary arm base according to claim 4, characterized in that, According to the first missing determination result and / or the second missing determination result, it is determined whether the transverse nut of the wrist arm base is actually missing, and a final missing determination result is obtained. If the first missing determination result is missing and the second missing determination result is missing, the transverse nut of the wrist arm base is missing; or, If the first missing determination result is missing or the second missing determination result is missing, the transverse nut of the wrist arm base is missing.

6. The method for detecting the missing of the cross nut of the catenary arm support base according to claim 1, characterized in that, The preprocessing of the catenary original image includes: According to the camera number corresponding to the suffix name of the photographed image, the catenary original image is filtered to obtain a filtered image; According to the suffix name of the photographed image, the filtered image is judged to be folded: when the filtered image is to the left, the filtered image is horizontally folded by 180 degrees to obtain a horizontal right wrist arm image.

7. A system for detecting the absence of a cross nut of a catenary arm base, characterized in that, The system supports a catenary wrist arm base transverse nut missing detection method as claimed in any one of claims 1 to 6; The system includes: An acquisition unit is configured to acquire a catenary original image; A preprocessing unit is configured to preprocess the catenary original image to obtain an image containing a flat and inclined wrist arm base region; A first-level positioning unit is configured to use a YOLO algorithm to perform first-level positioning on the flat and inclined wrist arm base region to obtain a first-level positioning result; the first-level positioning result includes flat and inclined wrist arm bases; A second-level positioning unit is configured to use a YOLO algorithm to perform second-level positioning on the first-level positioning result to obtain a second-level positioning result; the second-level positioning is positioning on a transverse nut region, and the second-level positioning result includes a front nut region, a back nut region, and a wrist arm base back; A judging unit uses a processing method of folding positioning combined with a YOLO algorithm and / or a processing method of convolutional neural network classification to determine whether the transverse nut of the wrist arm base in the second-level positioning result is missing, and outputs a determination result; The judging unit includes a first judging unit, which is configured to use the processing method of folding positioning combined with the YOLO algorithm to determine whether the transverse nut of the wrist arm base in the second-level positioning result is missing, specifically as follows: An enlarged nut region is obtained by using a folding positioning method on the second-level positioning result; the enlarged nut region is folded, and a YOLO algorithm is used to perform third-level positioning on a screw rod region to obtain a third-level positioning result; The ratio of the image height of the third-level positioning result to the image height of the second-level positioning result is calculated, and according to whether the ratio is greater than a set threshold, it is determined whether the transverse nut of the wrist arm base is missing, and a first missing determination result is obtained; The specific steps of the folding positioning method are as follows: Filtering the secondary positioning result, filtering out the back nut area; Respectively enlarging the upper, lower, left and right of the front nut area to obtain an enlarged back nut area; wherein the left and right are enlarged by one fifth of the width of the secondary positioning result image, and the upper and lower are enlarged by one tenth of the height of the secondary positioning result image; Folding the enlarged back nut area by -90 degrees to obtain a folded back nut area; Using a YOLO algorithm to perform tertiary positioning on the screw rod area to obtain a tertiary positioning result; the tertiary positioning result is a folded screw rod area positioning result.

8. A system for detecting the absence of a cross nut of a catenary arm base according to claim 7, characterized in that, The judgment unit further includes a second judgment unit and a final judgment unit; The second judgment unit is configured to use a convolutional neural network classification method to judge whether the transverse nut of the wrist arm base is missing in the secondary positioning result, including: Directly classifying the secondary positioning result using a convolutional neural network classification method to obtain a direct classification result; According to the direct classification result, judging whether the transverse nut of the wrist arm base is missing to obtain a second missing judgment result; The final judgment unit is configured to judge whether the transverse nut of the wrist arm base is truly missing according to the first missing judgment result and / or the second missing judgment result to obtain a final missing judgment result.

Images