Screw anti-loose detection method based on improved yolo3

Abstract

The invention discloses a screw anti-loosening detection method based on improved yolo3. The method comprises the following steps that an offline modeling module collects training pictures; labeling positions and categories of the pictures by adopting labeling software; segmenting the picture to obtain sub-pictures; all the sub-graphs are substituted into a yo3 training model; the online detection module collects a to-be-detected picture; segmenting the picture to be detected; the segmented sub-graphs are substituted into the trained model for detection, and detection results of the sub-graphs are obtained; and integrating and mapping the detection results of the sub-images into the to-be-detected large image, and finally obtaining the detection result of the large image. According to the invention, automatic detection of locking of the screw of the brake control device is realized, so that the completion detection efficiency and precision are improved, and greater benefits are brought to enterprises.

Description

Technical Field

[0001] The present invention relates to the technical field of object detection, in particular to a method for detecting screw loosening prevention based on improved YOLO3. Background Art

[0002] For the braking control devices produced by the existing technology, in the final completion inspection item, it is necessary to manually check whether each screw has a loosening prevention mark and whether the loosening prevention mark is loose. For each production line, generally dozens of braking control devices are included, and each device contains dozens or even hundreds of screws. Manual inspection not only consumes a large amount of manpower, but also is prone to missed inspection and misjudgment. Therefore, it is considered to use an object detection method based on deep learning to achieve automatic screw detection.

[0003] Existing object detection algorithms have achieved good results in some common data sets. However, it is difficult to meet the requirements for the pictures collected in actual industry. Summary of the Invention

[0004] In view of the problems existing in the prior art, the present invention provides a method for detecting screw loosening prevention based on improved YOLO3, which realizes automatic detection of screw loosening prevention in braking control devices, thereby improving the efficiency and accuracy of completion inspection and bringing greater benefits to enterprises.

[0005] The purpose of the present invention is achieved through the following technical solutions.

[0006] A method for detecting screw loosening prevention based on improved YOLO3 includes the following steps:

[0007] 1) The offline modeling module collects training pictures;

[0008] 2) Use annotation software to annotate the positions and categories of the pictures;

[0009] 3) Cut the pictures to obtain sub-pictures;

[0010] 4) Input all the sub-pictures into the YOLO3 training model;

[0011] 5) The online detection module collects the pictures to be detected;

[0012] 6) Cut the pictures to be detected;

[0013] 7) Input the cut sub-pictures into the trained model for detection to obtain the detection results of the sub-pictures;

[0014] 8) Integrate and map the detection results of the sub-pictures into the large picture to be detected, and finally obtain the detection results of the large picture.

[0015] The training pictures in step 1) include pictures of screws with different angles and different types.

[0016] Step 2) Use the lableImg annotation software to annotate the pictures, use a rectangular box to annotate the position information of the screws, and at the same time label the screws with normal anti-loosening marks as 1, and label the screws without anti-loosening marks or with misaligned anti-loosening marks as 0.

[0017] In step 3), the screw coordinate positions in the sub-pictures after segmentation need to be transformed accordingly; for the case where the screws in the sub-pictures after segmentation are separated, expand the sub-pictures outwards so that the sub-pictures contain the entire part of the screws.

[0018] Step 5) When collecting the pictures to be detected, take pictures with a fixed camera position.

[0019] In step 6), for the case where the screws in the sub-pictures after segmentation are separated, expand the sub-pictures outwards so that the sub-pictures contain the entire part of the screws.

[0020] The detection results of the sub-pictures in step 7) include , , , which respectively represent the category (1 or 0) of the screw, the probability belonging to this category, and the position of the screw , representing the upper left corner and lower right corner coordinates.

[0021] Step 8) Screen and integrate the detection results of the sub-pictures: Screen the detection results of the sub-pictures obtained in step 7), and the screening rule is: Whether it is greater than the threshold, represents a detection result in the sub-picture. If it is greater than the threshold, keep this detection result; otherwise, delete this result. Then map all the detection results retained in the sub-pictures to the picture to be detected, and the mapping method is , remains unchanged, and it is necessary to transform from the coordinate position of the sub-picture to the coordinate of the picture to be detected.

[0022] Step 8) Remove duplicates from the screened and integrated pictures to be detected: Judge the overlap rate between the detection results pairwise , is the ratio of the overlapping area of the two detection frames ( ) to the area of the smallest frame among the two detection frames; if it is less than the maximum ratio threshold, no change is made; otherwise, merge the results of the two detection frames, and the new detection result is: take the larger frame's , as the new , , and the new coordinate is

[0023] Compared with the prior art, the advantages of the present invention are as follows: The present invention realizes the automatic detection of the anti-loosening of the screws of the braking control device, and can accurately and automatically judge whether the screws are marked with anti-loosening marks and tightened, thereby improving the efficiency and accuracy of the completion inspection and bringing greater benefits to the enterprise. Description of the Drawings

[0024] Figure 1 is the flow chart of the improved YOLO3 screw anti-loosening detection algorithm;

[0025] Figure 2 is the screw manually marked as 1;

[0026] Figure 3 is the screw manually marked as 0;

[0027] Figure 4 is the cutting method of the sub-graph. Detailed Embodiment

[0028] The present invention will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0029] 1), Offline training model

[0030] Step 1, Acquisition of training pictures

[0031] In order to make the trained model have better generalization performance, the training pictures should contain pictures of screws with different angles and different types as much as possible.

[0032] Step 2, Manual annotation of pictures

[0033] The present invention uses the labelImg annotation software to annotate the pictures, uses a rectangular box to annotate the position information of the screws, and at the same time marks the screws with normal anti-loosening marks as 1, and marks the screws without anti-loosening marks or misaligned anti-loosening marks as 0. The annotation example is as Figure 2 , Figure 3 shown.

[0034] Step 3, Cutting of pictures

[0035] Cut a large picture into dozens or even hundreds of small pictures. The cutting method is as Figure 4 , where the width and height of the sub-graph are used as parameters and given manually; and there is partial overlap between adjacent sub-graphs, and the purpose is to expand the training set. The width and height pixels of the overlapping part are also used as parameters; m represents the number of sub-graphs cut out in a certain row. Two points need to be noted here:

[0036] a) The coordinate positions of the screws in the small pictures after cutting need to be transformed accordingly

[0037] b) The sub - graphs after cutting may split some screws in half. For such cases, the sub - graphs need to be expanded outwards so that the sub - graphs contain the entire part of the screws.

[0038] Step 4: Train the model

[0039] Bring the cut sub - graphs into yolo3 for training (the yolo3 algorithm is a mature and well - known object detection algorithm, which will not be specifically described here).

[0040] 2) Online detection

[0041] Step 5: Collect the pictures to be detected

[0042] The requirements for collecting test pictures are that the external conditions such as the shooting angle position and lighting should be as consistent as possible each time. The purpose is to reduce the decrease in model accuracy caused by external factors. For the subsequent fixture making, it is easy to meet this condition by fixing the camera position.

[0043] Step 6: Cut the pictures to be detected

[0044] The picture cutting method is similar to that in Step 3. The difference is that the test pictures have no manual annotation, so coordinate transformation is not required.

[0045] Step 7: Model detection

[0046] Bring the cut sub - graphs into the trained yolo3 model for detection to obtain the detection results (this part of the content is open - source and well - known, and will not be described in detail). The detection results include 、 、 which respectively represent the category (1 or 0) of the screw, the probability of belonging to this category, and the position of the screw , representing the upper - left and lower - right coordinates.

[0047] Step 8: Integration of detection results

[0048] Integrate and deduplicate the detection results of the sub - graphs. This step specifically includes two steps:

[0049] a) Screening and integration

[0050] First, screen the detection results of the sub - graphs obtained in Step 7. The screening rule is: ( represents a detection result in the sub - graph) is greater than the threshold (a parameter, default 0.3, set manually). If it is greater, keep the detection result; otherwise, delete the result. Then map the detection results retained by all sub - graphs to the picture to be detected. The mapping method is 、 unchanged, and it is necessary to transform from the coordinate position of the sub - graph to the coordinate of the picture to be detected.

[0051] b) Maximum ratio of deduplication

[0052] De-duplicate the images to be tested after screening and integration, and determine the overlap rate between the test results , For two detection boxes ( ) and the ratio of the area of the smallest box in the two detection boxes. If it is less than the maximum ratio threshold (parameter, default 0.5, manually set), no change is made, otherwise the two detection box results are merged, and the new detection result is: Larger frame , As new , , new The coordinates are .

[0053] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope required by the present invention.

Claims

1. A method for detecting screw loosening prevention based on improved YOLO3, characterized in that It includes the following steps: 1) The offline modeling module collects training pictures; 2) Use annotation software to annotate the position and category of the pictures; 3) Cut the pictures to obtain sub-pictures; 4) Input all the sub-pictures into the yolo3 training model; 5) The online detection module collects the pictures to be detected; 6) Cut the pictures to be detected; 7) Input the cut sub-pictures into the trained model for detection to obtain the detection results of the sub-pictures; 8) Integrate and map the detection results of the sub-pictures into the large picture to be detected, and finally obtain the detection results of the large picture.

2. The screw loosening detection method based on improved YOLO3 according to claim 1, wherein The training pictures in step 1) include screw pictures at different angles and of different types.

3. A method for detecting screw loosening based on improved YOLO3 according to claim 1, characterized in that In step 2), use the lableImg annotation software to annotate the pictures. Use a rectangular box to annotate the position information of the screws. At the same time, label the screws with normal anti-loosening marks as 1, and label the screws without anti-loosening marks or with misaligned anti-loosening marks as 0.

4. A screw loosening prevention detection method based on improved YOLO3 according to claim 1, characterized in that In step 3), the screw coordinate positions in the cut sub-pictures need to be transformed accordingly; for the case where the screws in the cut sub-pictures are segmented, expand the sub-pictures outwards so that the sub-pictures contain the entire part of the screws.

5. A method for detecting screw loosening prevention based on improved YOLO3 according to claim 1, characterized in that In step 5), when collecting the pictures to be detected, take pictures with a fixed camera position.

6. A method for detecting screw loosening based on improved YOLO3 according to claim 1, characterized in that In step 6), for the case where the screws in the cut sub-pictures are segmented, expand the sub-pictures outwards so that the sub-pictures contain the entire part of the screws.

7. A screw loosening detection method based on improved YOLO3 according to claim 1, characterized in that The detection results of the sub - graphs include , , , respectively representing the category of the screw (1 or 0), the probability of belonging to this category, and the position of the screw , representing the upper - left and lower - right coordinates.

8. A method for detecting screw loosening based on improved YOLO3 according to claim 1, characterized in that Step 8) Screening and integrating the detection results of sub - graphs: Screen the detection results of sub - graphs obtained in step 7), and the screening rule is: Whether it is greater than the threshold, which represents a detection result in the sub - graph. If it is greater than the threshold, keep the detection result; otherwise, delete the result. Then map all the retained detection results of sub - graphs to the image to be measured, and the mapping method is and unchanged, and it is necessary to transform from the coordinate position of the sub - graph to the coordinate of the image to be measured.

9. A method for detecting screw loosening based on improved YOLO3 according to claim 8, characterized in that Step 8) Remove duplicates from the screened and integrated test images: Determine the overlap rate between pairwise detection results , is the ratio of the overlapping area of two detection frames ( ) to the area of the smallest frame among the two detection frames; if it is less than the maximum ratio threshold, no change is made, otherwise the two detection frame results are merged, and the new detection result is: take the larger frame's , as the new , , and the new coordinates are .

Images