A lightweight part recognition and positioning method based on YOLOv5

Abstract

The application discloses a kind of light weight part identification positioning method based on YOLOv5, including through industrial camera acquisition part sample image;Image sample is carried out data enhancement, and data set is made, and is divided into training set and verification set;Part identification deep learning model based on light weight YOLOv5 is constructed, and data set is loaded to train algorithm model, and target recognition result and prediction frame are obtained;From prediction frame, target area is extracted from image, edge extraction is carried out to pre-processing area, and target frame is obtained, feature point extraction is carried out to target frame, and the position information of feature point is obtained, and gravity center is calculated as the positioning point of part;Based on camera calibration parameter and pinhole imaging principle, the pixel coordinates of part gravity center are converted into actual physical coordinates, and the actual coordinates of part gravity center are obtained.The application improves the robustness of target detection, and greatly improves the efficiency of target identification.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of part recognition positioning, and in particular relates to a lightweight part recognition positioning method based on YOLOv5, which has high target detection robustness and high target recognition efficiency. BACKGROUND

[0002] With the wide application and research of machine vision, part recognition technology as a key link in the industrial manufacturing process, especially deploying part recognition technology in industrial robots, in order to enable the industrial robot to classify target parts faster and more accurately, a lightweight part recognition method has become the main content of research and development. In the industrial production process, due to the large variety of parts, complex classification environment, large quantity, existence of small targets and other factors, the detection and recognition of parts face many challenges.

[0003] Due to the rapid development of deep learning technology, there are currently two types of part target recognition methods. One is a single-stage (One-stage) recognition algorithm based on regression composed of SSD and YOLO series, and the other is a two-stage (Two-stage) recognition algorithm based on candidate frame, such as R-CNN, Faster R-CNN, etc. The recognition algorithm of part targets in the industrial production and manufacturing process must guarantee the accuracy and recognition rate at the same time, and in the actual environment, small target parts exist in the case of false detection and missed detection, and the parameter quantity and volume of the model are too large, so it is difficult to deploy in the embedded platform, which cannot meet the high-precision demand in industrial production. SUMMARY

[0004] The purpose of the present application is to provide a lightweight part recognition positioning method based on YOLOv5, which has high target detection robustness and high target recognition efficiency.

[0005] Technical scheme: the present application comprises the following steps:

[0006] (1) collecting part sample images through an industrial camera;

[0007] (2) data augmentation is performed on the collected image samples, a data set is made, and the data set is divided into a training set and a verification set;

[0008] (3) a part recognition deep learning model based on lightweight YOLOv5 is constructed, the data set is loaded to train the algorithm model, and a target recognition result and a prediction frame are obtained;

[0009] (4) the target region is cut out from the image from the prediction frame, the target frame is obtained by performing edge extraction on the preprocessed region, the position information of the feature points is obtained by performing feature point extraction on the target frame, and the center of gravity is calculated as the positioning point of the part;

[0010] (5) Based on the camera calibration parameters and the principle of small aperture imaging, the pixel coordinates of the center of gravity of the part are converted into actual physical coordinates to obtain the actual coordinates of the center of gravity of the part.

[0011] Further, the step (1) controls the height of the industrial camera at 30 mm from the desktop, sets the shooting angle of the camera perpendicular to the desktop, and then collects the part image by using the industrial camera.

[0012] Further, the step (2) comprises:

[0013] (21) The sample image obtained in step (1) is flipped, cropped, and randomly adjusted in tone, brightness, and saturation to enhance the data set;

[0014] (22) The image processing photos are labeled by using the YOLO format of the labelimg labeling tool, and are classified and labeled according to the type of the part;

[0015] (23) The training set is randomly selected according to 80% of the total data set, and the remaining 20% is used as the test set.

[0016] Further, the step (3) comprises:

[0017] (31) The original YOLOv5 model is built, and the input end, the main structure, the Neck structure, and the output end of the model are built in sequence and connected according to the algorithm propagation direction;

[0018] (32) The backbone of the YOLOv5 model is replaced by ghostnet;

[0019] (33) A new type of feature pyramid SPP F is constructed;

[0020] (34) The detection layer for large target scale is removed;

[0021] (35) The GIOU loss function in the YOLOv5 model is replaced by the Alpha-SIOU loss function;

[0022] (36) The average precision mAP of the overall sample detection, the target detection average precision P, and the target average recall rate R are obtained, and the specific formula is as follows:

[0023]

[0024]

[0025]

[0026] Wherein, TP refers to the number of correctly identified parts; FN refers to the number of parts not identified; FP refers to the number of incorrectly identified parts; N represents the number of categories of parts.

[0027] Further, the step (35) comprises:

[0028] (35-1) Angle loss

[0029]

[0030]

[0031]

[0032]

[0033] Wherein, c h is the height difference between the center points of the real frame and the predicted frame, and σ is the distance between the center points of the real frame and the predicted frame, is the coordinate of the center point of the real frame, is the coordinate of the center point of the predicted frame;

[0034] (35-2) Distance loss

[0035]

[0036]

[0037]

[0038] γ=2-Λ

[0039] Wherein (c w , c h ) is the width and height of the minimum circumscribed rectangle of the real frame and the predicted frame;

[0040] (35-3) Shape loss

[0041]

[0042]

[0043]

[0044] Wherein w, h, w gt , h t are the width and height of the predicted frame and the real frame respectively; θ controls the degree of attention to the shape loss, usually taken as [2, 6];

[0045] (35-4) IOU loss

[0046]

[0047]

[0048] Wherein, IOU represents the ratio of the intersection of the area of the image real frame and the predicted frame to the union of the area, and delta is the distance loss, and omega is the shape loss.

[0049] Further, the step (4) comprises:

[0050] (41) performing gray processing on the identified image;

[0051] (42) performing Gaussian smoothing processing on the image processed in the step (41);

[0052] (43) performing median filtering on the image processed in the step (42);

[0053] (44) extracting the edge contour of the part by using Canny edge extraction;

[0054] (45) calculating the barycenter.

[0055] Beneficial effects: compared with the prior art, the mAP value in the test set reaches 99.4%, the Precision and Recall are only reduced by 0.1% and 0.2% than the YOLOv5 model, but the inference time is 35% faster than before, the calculation amount and volume are only 35.62% and 14.58% of the YOLOv5 model; the part recognition performance is greatly improved, and the robustness of target detection is also improved. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 It is an algorithm structure diagram of the present application;

[0057] Figure 2 It is a YOLOv5 algorithm structure diagram;

[0058] Figure 3 It is a comparison diagram of traditional convolution and Ghost convolution;

[0059] Figure 4 It is an SPP and SPPF network structure diagram;

[0060] Figure 5 It is an SPP F network structure diagram;

[0061] Figure 6 It is a comparison diagram of the width and height of the mark frame in the part data set;

[0062] Figure 7The effect diagram for part recognition, (a) is a YOLOv5 detection result; (b) is a YOLOv5-ghost detection result, and (c) is a detection result of the application. DETAILED DESCRIPTION

[0063] The technical solutions of the application are further described below with reference to the drawings.

[0064] The application provides a lightweight part recognition and positioning method based on improved YOLOv5, which comprises the following steps:

[0065] (1) collecting part sample images through an industrial camera, specifically comprising:

[0066] Six types of parts, namely flange couplings, gears, bolts, nuts, bearings and shaft sleeves, are selected as objects, and the six types of parts are widely used in actual industrial production due to their details and various types. The height of the industrial camera is controlled to be 30 mm away from the desktop, the shooting angle of the camera is set to be perpendicular to the desktop, and then the part images are collected by using the industrial camera.

[0067] (2) data augmentation is performed on the collected image samples to make a data set, and the data set is divided into a training set and a validation set, specifically comprising: the sample images obtained in S1 are flipped, cropped, and randomly adjusted in tone, brightness and saturation to enhance the data set, and 3797 part images are obtained; the images are labeled by using a YOLO format of a labelimg labeling tool, and the images are labeled according to the types of the parts; the training set is randomly selected according to 80% of the total data set, and the remaining 20% is used as a test set.

[0068] (3) as shown in Figure 1 , a YOLO v5 model structure, a part recognition deep learning model based on lightweight YOLOv5 is constructed, a data set is loaded to train the algorithm model, and target recognition results and prediction boxes are obtained, specifically comprising:

[0069] (31) as shown in Figure 2 , an original YOLOv5 model is built, and the input end, the main structure, the Neck structure and the output end of the model are built in sequence and connected in the algorithm propagation direction;

[0070] (32) the main network backbone of the YOLO v5 model is replaced by a ghostnet;

[0071] as Figure 3As shown, the original intention of ghostnet is to reduce the amount of calculation as much as possible to generate these redundant feature maps. The operation of traditional convolution and ghost module is shown, the ghost module divides the traditional convolution into two steps, first uses the operation of traditional convolution to get a small amount of traditional feature map, then generates ghost feature map through linear operation, finally combines the traditional feature map and ghost feature map for output, where the linear operation replaces a large number of calculations in traditional convolution, greatly reducing the model calculation amount and volume.

[0072] (33) Construct a new feature pyramid SPP_F:

[0073] As shown in Figure 4 , initially, SPP (Spatial Pyramid Pooling) module is used in YOLOv5 as the model's feature pyramid to separate the main context features. SPP uses kernel size = {1*1, 5*5, 9*9, 13*13} size sampling area and Maxpool (maximum pooling) operation, and then Concat (fusion) processes the feature maps of different scales. The original YOLOv5 model used in this paper uses SPPF module as the model's feature pyramid. The SPPF structure is a series of input through multiple 5x5 MaxPool layers, that is, two 5*5 convolutions in series replace the 9*9 convolution in SPP, and three 5*5 convolutions in series replace the 13*13 convolution in SPP. SPPF reduces the calculation time while keeping the SPP receptive field unchanged.

[0074] The large sampling area of SPP and SPPF reduces the number of neurons, resulting in the loss of some information. Therefore, the SPP_F module in this paper uses kernel size = {1*1, 5*5, 9*9, 13*13} size sampling area and Avgpool (average pooling operation). Average pooling can effectively avoid the excessive fluctuation of individual information and ignore the distribution of most values in the sampling area. SPP_F uses Conv (conv+BN+ReLU) module, then does 3*3 average pooling, and then does 5*5 and 7*7 average pooling based on 3*3 pooling. This can effectively expand the receptive field of the backbone network features. Finally, the pooled results are Concat operated with the data for the pooling operation. The SPP_F network structure is shown in Figure 5 .

[0075] (34) Remove the detection layer for large target scale;

[0076] In the YOLOv5 model, there are three detection layers, and when the input image size is 640x640, the neck network is processed accordingly. The 8, 16, and 32 times downsampling correspond to the sizes of the detection layer feature maps of 80x80, 40x40, and 20x20, respectively, which are used to identify small, medium, and large targets. In this paper, the images in the data set are analyzed, as shown in Figure 5 , it is found that most of the parts in the data set are small and medium-sized targets, so the 20x20 feature layer for detecting large targets is removed to reduce the parameter quantity and volume of the model, making it more suitable for part target detection in this paper.

[0077] (35) Replace the GIOU loss function in the YOLOv5 model with the Alpha-SIOU loss function. GIOU does not consider the direction between the real box and the predicted box, which leads to slow convergence speed. To this end, SIOU introduces the angle between the real box and the predicted box, and redefines the related loss function. To address the above issues, a Power parameter a is introduced in the loss function SIOU, which adjusts the a hyper coefficient to meet different levels of Bounding box regression accuracy. The Alpha-SIOU loss function is used to replace the original loss function, where the SIOU loss function is composed of four functions:

[0078] (35-1) Angle loss (Anglecost)

[0079]

[0080]

[0081]

[0082]

[0083] where c h is the height difference between the center points of the real box and the predicted box, and σ is the distance between the center points of the real box and the predicted box, is the coordinate of the center point of the real box, is the coordinate of the center point of the predicted box,

[0084] (35-2) Distance loss (Distancecost)

[0085]

[0086]

[0087]

[0088] γ=2-Λ

[0089] wherein (c w , c h ) are the width and height of the minimum bounding rectangle of the real box and the predicted box;

[0090] (35-3) Shape cost

[0091]

[0092]

[0093]

[0094] wherein w, h, w gt , h gt are the width and height of the predicted box and the real box respectively; theta controls the degree of attention to the shape loss, usually taking [2, 6];

[0095] (35-4) IOU loss

[0096]

[0097]

[0098] IoU represents the ratio of the intersection of the area of the real box and the predicted box of the image to the union of the area.

[0099] (36) The recognition effect diagram of the part as shown in Figure 6 , the average precision mAP of the overall sample detection, the target detection average precision P, and the target average recall rate R are obtained, and the specific formula is as follows:

[0100]

[0101]

[0102]

[0103] In the formula: TP refers to the number of correctly recognized parts; FN refers to the number of unrecognized parts; FP refers to the number of incorrectly recognized parts; and N represents the number of part categories.

[0104] (4) The target region is cut out from the image from the predicted box, the edge extraction is performed on the preprocessed region to obtain the target frame, the feature point extraction is performed on the target frame to obtain the position information of the feature points, and the center of gravity is calculated as the positioning point of the part, and specifically includes:

[0105] (41) The identified image is subjected to gray scale processing: YUV brightness gray scale is used, according to the color space of YUV, the physical meaning of the component Y is the brightness of the point, the value reflects the brightness level, according to the change relationship of RGB and YUV color space, the corresponding of brightness Y and R, G, B three color components can be established, and the gray value of the image is expressed by the brightness value. The formula is:

[0106] Y(i, j) = 0.299R(i, j) + 0.578G(i, j) + 0.114B(i, j)

[0107] Where (i, j) represents the coordinate point in the two-dimensional image.

[0108] (42) The image processed by S4.1 is subjected to Gaussian smoothing processing, Gaussian filtering can eliminate Gaussian noise with normal distribution, which belongs to linear filtering. The Gaussian template scans each pixel in the image, and the value of the center pixel point of the template is replaced by the weighted average gray value of the pixels in the domain determined by the template.

[0109] (43) The image processed by S4.2 is subjected to median filtering, which replaces the gray value of the pixel point with the median value of the gray values of the pixel points in the neighborhood, which can effectively remove impulse noise and salt and pepper noise while preserving image edge details.

[0110] (44) Canny edge extraction of part edge profile. Canny edge detection algorithm has strong noise resistance, Gaussian filtering is used to smooth the image, the amplitude and direction of the gradient are calculated by using the first-order partial derivative of the finite difference, the non-maximum suppression is performed on the gradient amplitude, and the double threshold detection and edge connection are adopted.

[0111] (45) The center of gravity of the part edge extraction graph is calculated, and the formula is as follows:

[0112]

[0113]

[0114] (5) Based on the camera calibration parameters and the principle of small hole imaging, the pixel coordinates of the center of gravity of the part are converted into actual physical coordinates to obtain the actual coordinates of the center of gravity of the part, specifically including: Zhang Zhengyou plane calibration method is used for camera calibration to obtain the internal and external parameters of the camera;

[0115] According to the internal and external parameters of the camera, the vertical distance between the camera and the desktop where the part is located, and the pixel coordinates of the center of gravity of the part, based on the principle of small hole imaging, the actual coordinates of the center of gravity of the part are determined.

[0116] In order to verify the part recognition and positioning method based on YOLOv5 lightweight proposed in the example of the application, the following experiments are carried out.

[0117] The performance of the detection model is measured using precision, recall, and mean average precision (mAP). The calculation formula is as follows:

[0118]

[0119]

[0120]

[0121] In the formula, TP refers to the number of correctly identified parts; FN refers to the number of parts that are not identified; FP refers to the number of incorrectly identified parts; and N represents the number of part categories.

[0122] To better test the performance of the improved algorithm in identifying part targets, the current popular lightweight target recognition algorithm is compared with the improved method in this paper under the premise of using a unified data set and consistent division method. The evaluation indicators are mAP value (%), inference time (ms), computational complexity (GFLOPs), and model size (MB). The final results of the comparison test are shown in Table 1.

[0123] Table 1

[0124]

[0125] From the data comparison in the table, the mAP value of the algorithm in this paper reaches 0.994, and the Precision and Recall are only 0.1% and 0.2% lower than the YOLOv5 model, but the inference time is 35% faster than before, and the computational complexity and size are only 35.62% and 14.58% of the YOLOv5 model. Compared with the current popular lightweight recognition algorithm, not only the Precision and Recall are increased, but also the inference time (ms) is faster, the computational complexity (GFLOPs) and size (MB) are smaller. From this analysis, it can be seen that the lightweight part detection algorithm proposed in this paper has achieved good results in average precision (mAP), inference time (ms), computational complexity (GFLOPs), and size (MB), and is therefore more suitable for deployment on low-power devices. To more intuitively display the detection effect difference before and after the model improvement, part of the test comparison is shown in Figure 7 As shown in the figure, the misidentification before the improvement is improved after the improvement, and the classification confidence score is also improved, further demonstrating the feasibility of the improved method in this paper.

[0126] The ablation experiment is to verify the performance of each improved measure, and the experimental results are shown in Table 2, ① represents using SPP F feature pyramid module, ② represents removing the large target detection layer, and ③ represents modifying the loss function. From the data in the table, when the large target detection layer is removed, the Precision and Recall are reduced by 1.0% compared with the YOLOv5 model, but the volume is reduced to 14.58% of the original. It can be seen that the model after removing the large target detection layer greatly reduces the model volume with slight loss of accuracy. After using the SPP F feature pyramid, the volume and calculation amount of the model do not increase, and the Precision, Recall and mAP are all improved. In summary, after improving the three measures at the same time under the condition of using Ghost module, the comprehensive performance of the algorithm is also the best.

[0127] Table 2

[0128]

Claims

1. A YOLOv5-based lightweight part recognition and positioning method, characterized in that, Includes the following steps: (1) Acquire images of part samples using an industrial camera; (2) Perform data augmentation on the collected image samples, create a dataset, and divide it into a training set and a validation set; (3) Construct a deep learning model for part recognition based on lightweight YOLOv5, load the dataset to train the algorithm model, and obtain the target recognition results and prediction boxes; (4) Extract the target area from the image from the prediction box, perform edge extraction on the preprocessed area to obtain the target frame, extract the feature points of the target frame to obtain the position information of the feature points, and calculate the center of gravity as the positioning point of the part. (5) Based on the camera calibration parameters and the pinhole imaging principle, the pixel coordinates of the center of gravity of the part are converted into actual physical coordinates to obtain the actual coordinates of the center of gravity of the part. Step (3) includes: (31) Build the original YOLOv5 model, and build the input end, backbone structure, Neck structure and output end of the model in sequence and connect them according to the direction of algorithm propagation; (32) Replace the backbone network of the YOLOV5 model with Ghostnet; (33) Construct a new feature pyramid SPP_F; The SPP_F module uses a sampling region of kernel size = {1*1, 5*5, 9*9, 13*13} and performs average pooling operation Avgpool. SPP_F uses the Conv(conv+BN+ReLU) module, and then performs 3*3 average pooling. On the basis of 3*3 pooling, it performs 5*5 and 7*7 average pooling. Finally, the pooled result is concatted with the data that has not been pooled. (34) Remove the detection layer used for large target scale; (35) Replace the GIOU loss function in the YOLOv5 model with the Alpha-SIOU loss function; (36) Obtain the average precision mAP of the overall sample detection, the average precision P of the target detection, and the average recall R of the target detection. The specific formulas are as follows: Where TP refers to the number of correctly identified parts; FN refers to the number of unidentified parts; FP refers to the number of incorrectly identified parts; and N represents the number of part categories. Step (35) includes: (35-1) Angle loss wherein c h is the height difference of the center points of the real frame and the predicted frame, and σ is the distance of the center points of the real frame and the predicted frame, is the coordinate of the center point of the real frame, is the coordinate of the center point of the predicted frame. (35-2) Distance loss γ=2-Λ where (c w , c h ) are the width and height of the minimum bounding box of the real and predicted boxes, respectively. (35-3) Shape loss where w, h, w gt , h gt are the width and height of the predicted and ground truth boxes, respectively; 0 controls the degree of attention to shape loss, typically taken from [2, 6]; (35-4) IOU loss Where IOU represents the ratio of the intersection of the areas of the ground truth bounding boxes and the union of the areas of the predicted bounding boxes, Δ is the distance loss, and Ω is the shape loss.

2. The YOLOv5-based lightweight part recognition and positioning method according to claim 1, characterized in that, In step (1), the height of the industrial camera is controlled at 30mm from the table, and the shooting angle of the camera is set to be perpendicular to the table. Then, the industrial camera is used to capture images of the parts.

3. The YOLOv5-based lightweight part recognition and positioning method of claim 1, wherein, Step (2) includes: (21) The sample images obtained in step (1) are randomly flipped, cropped, and their hue, brightness, and saturation are randomly adjusted to enhance the dataset. (22) Use the labelimg annotation tool in YOLO format to annotate the image processing photos and classify them according to the type of parts; (23) The training set is randomly selected by 80% of the total data set, and the remaining 20% is the test set.

4. The YOLOv5-based lightweight part recognition and positioning method of claim 1, wherein, The step (4) comprises: (41) The identified image is processed by grayscale; (42) The image processed in step (41) is processed by Gaussian smoothing; (43) The image processed in step (42) is processed by median filtering; (44) The edge contour of the part is extracted by Canny edge extraction; (45) The center of gravity is calculated.

Images