Workpiece recognition and counting method based on target detection and template matching in complex scene

Abstract

The present application relates to the technical field of target detection and template matching, in particular to a workpiece recognition and counting method based on target detection and template matching in a complex scene, comprising the following steps: obtaining a standard workpiece original data set by using a high-definition industrial camera; obtaining an enhanced data set by pre-processing, data augmentation, cropping and classification labeling of the workpiece image set; constructing a target detection network model based on an improved YOLOv5 and performing data set training; deploying the trained target detection model to read the industrial camera in real time to detect the workpieces on the production line, crop and save the recognized workpiece images after rotating the angle. The present application has the advantages that: firstly, the workpieces are recognized from the complex scene image by the deep learning target detection method, and the position information of the workpieces is obtained, which has better robustness for the sensitivity of environmental information. It can handle multi-target and multi-angle recognition tasks, and can excellently complete workpiece recognition and counting in a relatively complex production scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection and template matching, and particularly to a workpiece recognition and counting method based on target detection and template matching in a complex scene. BACKGROUND

[0002] With the rapid development of technological revolution and industrial transformation, traditional manufacturing is facing comprehensive transformation and upgrading, so it is of great significance to deeply promote intelligent manufacturing. Many traditional manufacturing industries still rely on a large number of manpower in production, resulting in low production efficiency, low adaptability of supply to market demand, and challenges to the stability of industrial chain and industrial chain. Taking the home manufacturing industry as an example, various workpieces need to be matched and counted by workers when they circulate on the production line. When the order quantity of goods is large and the number of production lines is large, efficiency and cost problems will be caused.

[0003] For example, Chinese application number CN2202210090650.4 discloses a workpiece recognition and positioning method based on template matching. The coordinates of feature points on the imaging plane are obtained, and the coordinates of the corresponding points in the camera image coordinate system are combined to obtain the coordinates of the feature points in three-dimensional space. A three-dimensional space geometric model is established according to the imaging point coordinate values, and a shape-based template matching method is used to search and complete matching within the camera field of view.

[0004] For example, in "Workpiece Recognition Based on Improved YOLOv4", Zhang Jianhua, Zhao Wei, etc. proposed a workpiece recognition method based on improved YOLOv4. Simplified Moblienetv3 is used to replace the feature extraction network of the YOLOv4 framework, and the standard convolution layer of the deep network of YOLOv4 is replaced by a depth separable convolution layer. Finally, a 104*104 feature detection scale is added, which is fused with the other three scales (52*52, 26*26, 13*13). An image data acquisition platform for typical workpieces is built, a training sample set is constructed, and network training is performed.

[0005] The environment involved in the industrial manufacturing site is relatively complex. The workpiece recognition technology based on the traditional machine vision method usually performs pixel-by-pixel traversal search on an image, uses an image segmentation method such as threshold value to segment the workpiece from the image, and then uses a feature extraction algorithm to extract features of the workpiece, which include color features (HSV), texture features, shape features and the like. This kind of method has obvious shortcomings in the actual industrial scene: large amount of calculation, low real-time performance, poor effect when dealing with many scene sundries, many workpiece types and close or even overlapping positions. At present, the image processing means based on deep learning has become the mainstream. The target object image of the industrial site is collected by a high-definition camera to make a data set for model training, and then the trained network model is deployed to the production site to realize real-time detection. This kind of method has good universality and portability and high precision. However, in a complex production scene, the recognition method based on deep learning still has shortcomings when facing multiple targets, multiple angles and noise interference, and is prone to miss detection. In order to realize the automatic production line, a technical method capable of realizing real-time, efficient and accurate recognition and counting of workpiece types in a complex scene is urgently needed. SUMMARY

[0006] In order to solve the above technical problems, the application provides a workpiece recognition and counting method based on target detection and template matching in a complex scene. The target detection algorithm is used to automatically recognize and extract the workpiece image area on the production line, and then the template matching algorithm is used to finely match the workpiece types and count, so as to liberate a large amount of manpower and realize intelligent production. The improved YOLOv5 lightweight rotating target detection network is used to quickly realize end-to-end workpiece detection and positioning, and the workpiece region image to be classified is coarsely extracted. Then the SURF algorithm is used to obtain the key points of the workpiece region to be classified and the template to realize the matching of the two, realize fine classification and counting. It can better handle the problems of scale change, view angle change and light change. More importantly, it can handle the multi-target multi-angle recognition task, and can excellently complete the workpiece recognition and counting in a relatively complex production scene.

[0007] The technical problems to be solved by the application are realized by the following technical solutions:

[0008] The workpiece recognition and counting method based on target detection and template matching in a complex scene comprises the following steps:

[0009] Step (1) obtaining a standard workpiece original data set by using a high-definition industrial camera;

[0010] Step (2) obtaining an enhanced data set by pre-processing, data augmentation, cropping and classification labeling of the workpiece image set;

[0011] Step (3) constructs an object detection network model based on improved YOLOv5 and performs dataset training:

[0012] The specific process of constructing an object detection network model based on improved YOLOv5 is as follows:

[0013] Step (31) redesigns the anchor box of the traditional YOLOv5, increases the anchor box parameter θ, and uses the K-means++ clustering method to regenerate the anchor box to fit the real box of the workpiece target. The positioning method of the anchor box is selected as the long side definition method. The angle regression problem is converted into a classification problem by using CSL. The annotation file format is [classid, x, y, longside, shortside, θ];

[0014] Step (32) replaces the backbone network in the traditional YOLOv5 with MobileNetv3. MobileNetv3 includes three parts: the first part extracts feature layers through convolution layers, the second part is a large number of inverted residual structure blocks, and the third part outputs categories;

[0015] Step (33) improves the Neck module in the traditional YOLOv5 by replacing the original PAN structure with a BiFPN structure;

[0016] Step (34) modifies the Head part in the traditional YOLOv5. The Detect class constructor is modified for the increased θ dimension. The angle_classes parameter is added to each anchor box prediction parameter;

[0017] Step (35) modifies the loss function in the traditional YOLOv5. The classification loss of angle θ is added, and other loss functions remain unchanged. The loss function of θ uses the BCE loss function;

[0018] Step (4) deploys the trained object detection model to read the industrial camera in real time to detect the workpieces on the production line, rotates the angle, and crops and saves the recognized workpiece images;

[0019] Step (5) annotates and crops the workpiece matching area in the rotated image through the object detection network. Using the prepared workpiece template image, the SURF algorithm is used to perform template matching on the workpiece to be classified extracted by the object detection network and set a threshold. When the similarity is greater than the threshold, the workpiece class in the matching area is determined as the workpiece class in the template, and the number is counted;

[0020] The steps of the SURF algorithm are as follows:

[0021] Step (51) feature point detection: a Hessian matrix is constructed for a pixel point x = I(x, y) in the image, the feature points in the image are extracted using the Hessian matrix, a transformed image is obtained by transforming the original image to find the feature points, each pixel point is processed using the Hessian matrix judgment formula, each processed pixel point is compared with all adjacent points in its image domain and scale domain, when it is greater or less than all adjacent points, the point is an extreme point, thereby preliminarily locating the feature points, and then filtering out the key points with weak energy and the key points wrongly positioned to screen out the final stable feature points;

[0022] Step (52) feature point direction confirmation: in the circular neighborhood of the feature point, the horizontal and vertical harr wavelet feature sums of all points in a 60-degree sector are counted, then the sector is rotated at an interval of 0.2 radian and the harr wavelet feature values in the region are counted again, and finally the direction of the sector with the maximum value is taken as the main direction of the feature point;

[0023] Step (53) feature description: a 4*4 rectangular region block is taken around the feature point along the main direction of the feature point, the horizontal and vertical direction haar wavelet features of 25 pixels in each sub-region are counted, the haar wavelet feature is the sum of the horizontal direction values d y , the sum of the absolute values |d y | of the horizontal direction, the sum of the vertical direction values d x , and the sum of the absolute values |d x | of the vertical direction, each small region has 4 values, each feature point is a 16*4 = 64-dimensional vector, and the formula is: V sub =(∑d x ,∑|d x |,∑d y ,∑|d y |);

[0024] Step (54) feature matching: the Euclidean shortest distance Dis i and the Euclidean second shortest distance Dis i ' of the feature points m mini and n mini on the workpiece image to be classified and the workpiece template image are calculated, if the result satisfies the judgment formula of the judgment threshold, the matching point pair of the feature point m i on the workpiece image to be classified and the feature point n i on the workpiece template image is reserved to the matrix M1, and the feature point m i on the workpiece image to be classified and the feature point n iThe Euclidean distance of the result is stored in M2, and the result of the preliminary matching feature point pair is recorded as M3. After obtaining M3, the transformation matrix is calculated, and the final registration is realized by mapping. The RANSAC algorithm is used to further purify the matching result, and the inner point matrix that meets the algorithm is obtained, that is, the accurate matching result.

[0025] Preferably, step (1) is specifically: using an industrial high-definition camera deployed on the production line to capture each type of workpiece in the production site at a fixed time interval to obtain a set of original images of the workpiece.

[0026] Preferably, step (2) is specifically: the preprocessing includes eliminating light and noise in the workpiece image, the data augmentation operation includes flipping, mirroring, brightness and contrast transformation on the collected workpiece image to expand the image set, and then the workpiece image is cropped and classified and labeled. The workpiece image is labeled using a labeling software, and the bounding rectangle of the workpiece is marked. Then, the data format is converted into DOTA format with center point coordinates of the target frame, height, width and angle information, and then converted into YOLO network training data format.

[0027] Preferably, the step of K-means++ clustering to generate anchor frames in step (31) is:

[0028] Step (311) randomly selects a sample M0 from the data set M as an initial clustering center;

[0029] Step (312) calculates the distance d i of each sample M i to the current clustering center C i , and the calculation formula is as follows:

[0030] D i = 1-IOU(M i , C i )

[0031] In the formula, IOU is the overlap degree of the anchor frame and the Bounding box;

[0032] Step (313) calculates the probability p of each sample being selected as the next clustering center, and the calculation formula is as follows:

[0033]

[0034] Step (314) repeats steps (311) to (313) to select K clustering centers.

[0035] Preferably, the calculation formula of CSL in step (31) is as follows:

[0036]

[0037] where g(x) is a window function, and r is a radius.

[0038] Preferably, the specific steps of the Block function in step (32) are as follows:

[0039] (321) The input features are first upgraded to exp size by 1*1 convolution, and then pass through a nonlinear activation function NL;

[0040] (322) The output features are passed through a 3*3 Dwise convolution, and then pass through a nonlinear activation function NL, without changing the channel number expsize;

[0041] (323) The output features are input into the SE block, and first global average pooling is performed on each channel to obtain a 1*1*expsize vector;

[0042] (324) Then the vector is input into the first fully connected layer, and the activation function is ReLU6, to obtain an output of 1*1*(expsize / 4);

[0043] (325) Then the output is input into the second fully connected layer, and the activation function is h-sigmoid, to obtain a 1*1*expsize vector, wherein the elements in the vector take values of (0, 1);

[0044] (326) Each element in the vector is multiplied by the corresponding channel feature matrix of the feature map input into the SE layer to obtain a new feature map with the same size as the feature map; Figure One

[0045] (327) Finally, 1*1 convolution is performed for dimension reduction.

[0046] Preferably, the expression of the ReLU6 activation function in step (324) is as follows:

[0047] y = ReLU6(x) = min(max(x, 0), 6);

[0048] The expression of the h-sigmoid activation function is as follows:

[0049]

[0050] The expression of the Hard-swish activation function is as follows:

[0051]

[0052] ​Preferably, step (4) specifically involves: deploying the target detection model trained in step (3), reading the workpieces on the production line detected in real time by the industrial camera, marking the minimum bounding rectangle of the workpiece in the image, using the obtained center coordinates (x, y) and θ angle information to rotate the workpiece in the region to the horizontal and crop and save it, thereby obtaining all the workpieces to be identified in the image.

[0053] Preferably, the Hessian matrix formula in step (51) is as follows:

[0054]

[0055] In the formula, σ is the spatial scale, and L xx (x,σ),L xy (x,σ),L yy (x,σ) is the convolution of the second-order Gaussian derivative;

[0056] The Hessian matrix discrimination formula is:

[0057] Det(H approx ) = L xx L yy -(αL xy ) 2

[0058] In the formula, α is taken as an empirical value of 0.9.

[0059] Preferably, the formula for calculating the Euclidean distance in step (54) is as follows:

[0060]

[0061] The formula for determining the threshold is:

[0062]

[0063] The beneficial effects of this invention are:

[0064] Compared with existing technologies, the advantages of this invention are as follows: First, it identifies workpieces from complex scene images using deep learning object detection to obtain the workpiece's position information, exhibiting better robustness to environmental information. Then, it performs further feature matching operations on the workpieces within the rectangular boxes in the identification results, using template matching to achieve accurate classification and counting of each workpiece. It can handle multi-target, multi-angle recognition tasks and can excellently complete workpiece identification and counting in relatively complex production scenarios. Attached Figure Description

[0065] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0066] Figure 1This is a diagram of the improved YOLOv5 network structure of this invention;

[0067] Figure 2 This is a schematic diagram of the workpiece fine classification process of the present invention;

[0068] Figure 3 This is a flowchart of the present invention. Detailed Implementation

[0069] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0070] like Figures 1 to 3 As shown, the workpiece recognition and counting method based on target detection and template matching in complex scenarios specifically includes the following steps:

[0071] Step (1) Use industrial high-definition cameras deployed on the production line to capture images of various types of workpieces at fixed time intervals to obtain the original image set of the workpieces.

[0072] Step (2) involves preprocessing and data augmentation of the workpiece image set to obtain an enhanced dataset. Image preprocessing eliminates the effects of lighting and noise in the images, while data augmentation expands the image set by flipping, mirroring, and performing brightness and contrast transformations on the acquired workpiece images. The workpiece images are then cropped and labeled using annotation software to mark the bounding rectangle of the workpiece. The data is then converted to DOTA format, which includes the center point coordinates, height, width, and angle information of the target bounding box, and finally to the format of YOLO network training data.

[0073] Step 3: Build an object detection network model based on the improved YOLOv5 and train it on the dataset.

[0074] Step 3 involves making some improvements to YOLOv5 to build a target detection network that enables coarse extraction of the workpiece.

[0075] Step (31) First, since the workpieces on the production line are placed on the ground rail frame or grinding machine, reciprocating machine, etc. by workers, the randomness of the workers' placement will cause the workpiece position to be not regular horizontal or vertical from the camera's top view. The traditional YOLOv5 target detection result is a horizontal rectangular detection box, which inevitably contains some irrelevant background areas. A large amount of redundant information will increase the computational load of the network model. Therefore, YOLOv5 is improved to adapt to workpieces with rotation angles: the anchor box is redesigned, the anchor box parameter θ is increased, and the K-means++ clustering method is used to regenerate the anchor box to fit the true box of the workpiece target. Compared with the original YOLOv5 using the K-means clustering method, its advantage is that it reduces the problem that the clustering result is greatly affected by the initial value selection, and can select a better initial cluster center.

[0076] The steps for generating anchor boxes using K-means++ clustering are as follows:

[0077] Step (311) Randomly select a sample M0 from the dataset M as the initial cluster center;

[0078] Step (312) Calculate M for each sample i With the current cluster center C i distance d i The calculation formula is as follows:

[0079] D i =1-IOU(M i C i )

[0080] In the formula, IOU represents the degree of overlap between the anchor frame and the bounding box;

[0081] Step (313) calculates the probability p of each sample being selected as the next cluster center, using the following formula:

[0082]

[0083] Step (314) Repeat steps (311) to (313) to select K cluster centers.

[0084] After introducing the angle parameter, the loss function needs to add a loss for θ. In order to deal with the boundary problem caused by the angle, the definition method of the rotating box is to use the long side definition method, that is, the angle θ between the long side of the box and the x-axis is ∈ [-90, 9). The circular smooth label (CSL) method is used to transform the angle regression problem into a classification problem. Therefore, the annotation file format is [classid, x, y, longside, shortside, θ].

[0085] The formula for calculating CSL is:

[0086]

[0087] g(x) is a window function used to estimate the angular distance between the predicted bounding box and the ground truth bounding box. The closer the predicted value is to the true value, the smaller the loss of angular information. The window function is chosen to be a Gaussian function, and r is the radius.

[0088] The data loading section ensures that the annotation file format conforms to the long-side definition method and adds the angle θ dimension to the original label data.

[0089] Step (32) In order to facilitate model deployment, consider replacing the backbone network of YOLOv5 with the more lightweight MobileNetv3 to achieve a balance between speed and accuracy while making the network model lighter.

[0090] MobileNetv3 can be divided into three parts: the first part extracts features through convolutional layers; the second part consists of numerous inverse residual structure blocks; and the third part outputs the categories. The specific steps of the Block function are as follows: the input features are first enlarged to exp size using 1*1 convolutions and then passed through a non-linear activation function (NL). Next, the output features are passed through 3*3 Dwise convolutions and then through a non-linear activation function, keeping the number of channels (exp size) unchanged. Then, the output features are fed into the SE block, where each channel undergoes global average pooling to obtain a vector of size 1*1*exp size. This vector is then fed into the first fully connected layer with ReLU6 activation, resulting in an output of size 1*1*(exp size / 4). The output is then fed into the second fully connected layer with h-sigmoid activation, resulting in a vector of size 1*1*exp size, where each element is (0,1). Each element of this vector is multiplied by the corresponding channel feature matrix of the feature map input to the SE layer to obtain a sum feature. Figure One A new feature map of the same size is generated. Finally, 1*1 convolutional dimensionality reduction is performed.

[0091] The expression for the ReLU6 activation function is:

[0092] y=ReLU6(x)=min(max(x,0),6);

[0093] The expression for the h-sigmoid activation function is:

[0094]

[0095] The expression for the hard-swish activation function is:

[0096]

[0097] Step (33) improves the traditional YOLOv5 Neck module by replacing the original PAN structure with a BiFPN structure, and strengthens feature fusion through bidirectional cross-scale connection and weighted feature fusion, while improving detection speed.

[0098] Step (34) Modify the Head part of the traditional YOLOv5, modify the constructor of the Detect class for the added θ dimension, and add the parameter angle_classes to the parameters of each anchor box prediction.

[0099] Step 3.5: Finally, modify the loss function θ. Simply add the classification loss for the angle θ, and keep the other loss functions unchanged. The loss function for θ is the BCE loss function.

[0100] The above are a series of improvements to YOLOv5, which complete the construction of the object detection network.

[0101] Step (4) Deploy the target detection model trained in step (3), read the workpieces on the production line in real time from the industrial camera, mark the minimum bounding rectangle of the workpiece in the image, use the obtained center coordinates (x, y) and θ angle information to rotate the workpiece in the region to the horizontal and crop and save it, so as to obtain all the workpieces to be identified in the image, which is convenient for subsequent feature matching to achieve fine classification and counting of workpieces.

[0102] Step (5) After obtaining the workpiece's position information through the target detection network, fine-grained feature matching of the workpiece is performed based on the improved template matching method:

[0103] The workpiece regions to be matched in the image are obtained by labeling, cropping and rotating the object detection network. Then, using the prepared workpiece template image, the similarity between the workpiece regions to be matched and the workpiece regions in the template image is calculated one by one and a threshold is set. When the similarity is greater than the threshold, the workpiece category in the workpiece region to be matched is determined to be the workpiece category in the template. The specific method is to use the SURF algorithm to perform template matching on the workpieces to be classified extracted by the object detection network.

[0104] The steps of the SURF algorithm are as follows:

[0105] Step (51) Feature point detection:

[0106] Feature points in an image are extracted and described using the Hessian matrix. The Hessian matrix is ​​constructed for each pixel x = I(x, y) in the image.

[0107]

[0108] σ is the spatial scale, Lxx (x,σ),L xy (x,σ),L yy (x,σ) is the convolution of the second-order Gaussian derivative:

[0109]

[0110] In the formula, g(σ) is a two-dimensional Gaussian function. For two-dimensional convolution calculation, L xy (x,σ),L yy (x,σ) can be derived similarly. Transforming the original image yields the transformed image from which we need to find feature points. The Hessian matrix discrimination formula is:

[0111] Det(H approx ) = L xx L yy -(αL xy ) 2 , where α is taken as an empirical value of 0.9.

[0112] Each pixel processed by the Hessian matrix discriminant is compared with all its neighboring points in the image domain and scale domain. When it is greater than (or less than) all its neighboring points, the point is the extreme point, thus initially locating the feature point. Then, after filtering out key points with weak energy and incorrectly located key points, the final stable feature points are selected.

[0113] Step (52) Feature point orientation confirmation:

[0114] Within the circular neighborhood of the feature point, the sum of the horizontal and vertical Harr wavelet features of all points within a 60-degree sector is calculated. Then, the sector is rotated at intervals of 0.2 radians, and the Harr wavelet feature values ​​within the region are calculated again. Finally, the direction of the sector with the largest value is taken as the main direction of the feature point.

[0115] Step (53) Feature Description:

[0116] Take a 4x4 rectangular region around the feature point along its main direction. For each sub-region, calculate the horizontal and vertical Haar wavelet features of 25 pixels. The horizontal Haar wavelet feature is represented by the value d. y The sum of the absolute values ​​in the horizontal direction |d y The sum of | and the perpendicular direction d x The sum of the absolute values ​​in the vertical direction |d x The sum of these values ​​results in 4 values ​​for each small region, and each feature point is a 16*4=64 dimensional vector.

[0117] V sub =(Σd x , Σ|dx |,Σd y , Σ|d y |).

[0118] Step (54) Feature matching:

[0119] Calculate the Euclidean distance between feature points on the workpiece image to be classified and feature points on the workpiece template image:

[0120]

[0121] The feature points m of the workpiece image to be classified are calculated using the above formula. i Feature point n on the workpiece template image i European shortest distance Dis mini and European-style shortest distance Dis mini ', If satisfied:

[0122]

[0123] Then retain the feature points m of the workpiece image to be classified. i To the feature point n of the workpiece template image i This matching point is mapped to matrix M1, which will then be the feature point m of the workpiece image to be classified. i With feature points n on the template image i The Euclidean distance ratio result is stored in M2, and the result of the initial matching of feature point pairs is recorded as M3:

[0124] M3 = M1 ∩ M2;

[0125] After obtaining M3, the transformation matrix is ​​calculated, and the mapping achieves the final registration. The matching result is further refined using the RANSAC algorithm to obtain the interior point matrix that conforms to the algorithm, which is the exact matching result.

[0126] The coordinates (x, y) of the feature point m in the workpiece image to be classified satisfy the homography matrix constraint with the coordinates (x', y') of the feature point n in the template image.

[0127]

[0128] Then, the data in M3 is sorted in ascending order of Euclidean distance. Assuming that the first four pairs of points that are not on the same straight line satisfy the above matrix constraints, they are exactly matched pairs, and the specific transformation matrix H can be calculated from the coordinates of the point pairs.

[0129]

[0130] In M3, other feature point pairs that meet the threshold conditions and matrix constraints can be stored in the interior point matrix; otherwise, they can be stored in the exterior point matrix.

[0131] If the number of elements I in the interior point matrix is ​​less than the number of elements W in the exterior point matrix, then four pairs of points that are not on the same straight line are selected again according to the sorting, and the above calculation is repeated. If the number of elements in the interior point matrix is ​​greater than the number of elements in the exterior point matrix, then it is denoted as the optimal interior point set I. best =I, and simultaneously update the iteration count K. If the iteration count is greater than K, exit the calculation; otherwise, increment the iteration count by 1. Note that during each iteration, if the number of elements in the interior point matrix is ​​greater than the number of elements in the exterior point matrix, update the optimal interior point set I. best =I k The number of iterations K is:

[0132] p is the confidence level of 0.995, and w is the proportion of inliers;

[0133] In summary, the system ultimately achieves the matching of workpieces to be classified with template workpieces, and counts and stores the count results of workpieces of the same type that are identified.

[0134] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely prisms of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A workpiece recognition and counting method based on target detection and template matching in complex scenarios, characterized by: Includes the following steps: Step (1) Use a high-definition industrial camera to acquire the original dataset of a standard workpiece; Step (2) involves preprocessing, data augmentation, cropping, and classification labeling of the workpiece image set to obtain an augmented dataset; Step (3) Construct an object detection network model based on the improved YOLOv5 and train it on the dataset: The specific process of constructing an object detection network model based on the improved YOLOv5 is as follows: Step (31) redesign the anchor frame of traditional YOLOv5 and add anchor frame parameters. θ K-means++ clustering was used to regenerate anchor boxes to fit the true bounding boxes of the workpiece targets. The anchor box localization method was chosen to be the long-side definition method. The circular smooth labeling method (CSL) was used to transform the angle regression problem into a classification problem. The annotation file format was [classid, x, y, longside, shortside, ...]. θ ]; Step (32) replaces the backbone network in the traditional YOLOv5 with MobileNetv3. MobileNetv3 consists of three parts: the first part extracts features through convolutional layers, the second part consists of a large number of inverted residual structure blocks, and the third part outputs the categories. Step (33) improves the Neck module in the traditional YOLOv5 by replacing the original PAN structure with a BiFPN structure; Step (34) Modify the Head section in the traditional YOLOv5, specifically for the added... θ Modify the constructor of the Detect class to add the parameter `angle_classes` to the parameters of each anchor box prediction; Step (35) modifies the loss function in the traditional YOLOv5 by adding an angle. θ The classification loss remains unchanged, while other loss functions remain the same. θ The loss function used is the BCE loss function; Step (4) Deploy the trained target detection model to read the industrial camera and detect the workpieces on the production line in real time. After rotating the angle, crop and save the images of each identified workpiece. Step (5) The workpiece to be matched region in the image is obtained by labeling and cropping and rotating through the target detection network. Using the prepared workpiece template image, the SURF algorithm is used to perform template matching on the workpiece to be classified extracted by the target detection network and a threshold is set. When the similarity is greater than the threshold, the workpiece category in the to be matched region is determined to be the workpiece category in the template, and the number is counted. The steps of the SURF algorithm are as follows: Step (51) Feature point detection: Construct a Hessian matrix for the pixel x = I(x, y) in the image, extract the feature points in the image using the Hessian matrix, transform the original image to obtain the transformed image for which the feature points need to be found, process each pixel using the Hessian matrix discrimination formula, compare each processed pixel with all its neighboring points in the image domain and scale domain, and when it is greater than or less than all its neighboring points, the point is the extreme point, thus initially locating the feature points, and then filtering out key points with weak energy and incorrectly located key points to select the final stable feature points; Step (52) Feature point direction confirmation: In the circular neighborhood of the feature point, the sum of the horizontal and vertical Harr wavelet features of all points in the 60-degree sector is calculated. Then, the sector is rotated at intervals of 0.2 radians and the Harr wavelet feature values ​​in the region are calculated again. Finally, the direction of the sector with the largest value is taken as the main direction of the feature point. Step (53) Feature Description: Take a 4*4 rectangular region around the feature point along the main direction of the feature point. For each sub-region, count the horizontal and vertical Haar wavelet features of 25 pixels. The Haar wavelet feature is the horizontal value d. y The sum of the absolute values ​​in the horizontal direction |d y The sum of | and the perpendicular direction d x The sum of the absolute values ​​in the vertical direction |d x The sum of these values ​​results in 4 values ​​for each small region, and each feature point is a 16*4=64 dimensional vector. The formula is: V sub =(Σd x , Σ|d x |,Σd y , Σ|d y |); Step (54) Feature matching: Calculate the feature points m on the workpiece drawing to be classified. i With workpiece template image feature points n i European shortest distance Dis mini and European-style shortest distance Dis mini If the result satisfies the judgment formula for the judgment threshold, then retain the feature points m of the workpiece image to be classified. i To the feature point n of the workpiece template image i This matching point is mapped to matrix M1, which will then be the feature point m of the workpiece image to be classified. i With feature points n on the template image i The Euclidean distance ratio result is stored in M2, and the result of the initial matching of feature point pairs is recorded as M3. After obtaining M3, the transformation matrix is ​​calculated, and the mapping is used to achieve the final registration. The matching result is further refined using the RANSAC algorithm to obtain the interior point matrix that conforms to the algorithm, which is the accurate matching result.

2. The workpiece identification and counting method based on target detection and template matching in complex scenarios according to claim 1, characterized in that: Step (1) specifically involves using industrial high-definition cameras deployed on the production line to capture images of various types of workpieces at fixed time intervals to obtain the original image set of the workpieces.

3. The workpiece identification and counting method based on target detection and template matching in complex scenarios according to claim 1, characterized in that: Step (2) specifically involves: preprocessing including eliminating illumination and noise in the workpiece image; data augmentation operations including flipping, mirroring, brightness and contrast transformation of the acquired workpiece image to expand the image set; then cropping and classifying the workpiece image; using annotation software to annotate the workpiece image and mark the bounding rectangle of the workpiece; then converting the data format into DOTA format with the center point coordinates of the target box and the height, width and angle information, and then converting it into the format of YOLO network training data.

4. The workpiece identification and counting method based on target detection and template matching in complex scenarios according to claim 1, characterized in that: The steps in step (31) for generating anchor boxes using K-means++ clustering are as follows: Step (311) Randomly select a sample M0 from the dataset M as the initial cluster center; Step (312) Calculate M for each sample i With the current cluster center C i distance d i The calculation formula is as follows: d i =1-IOU(M) i C i In the formula, IOU represents the degree of overlap between the anchor frame and the bounding box; Step (313) calculates the probability p of each sample being selected as the next cluster center, using the following formula: ； Step (314) Repeat steps (311) to (313) to select K cluster centers.

5. The workpiece identification and counting method based on target detection and template matching in complex scenarios according to claim 1, characterized in that: The formula for calculating CSL in step (31) is as follows: In the formula, g(x) is the window function and r is the radius.

6. The workpiece identification and counting method based on target detection and template matching in complex scenarios according to claim 1, characterized in that: The specific steps for implementing the Block function in step (32) are as follows: (321) The input features are first increased in dimension to exp size using 1*1 convolution, and then passed through a non-linear activation function NL; (322) The output features are passed through a 3*3 Dwise convolution and then through a non-linear activation function NL, without changing the number of channels exp size; (323) Pass the output features into the SE block, first perform global average pooling on each channel to obtain a vector of size 1*1*exp; (324) Then pass the vector into the first fully connected layer with ReLU6 as the activation function, and the output is 1*1*(expsize / 4). (325) The output is then fed into the second fully connected layer with h-sigmoid as the activation function, resulting in a vector of size 1*1*exp, where the elements of the vector are (0,1). (326) Multiply each element of the vector by the corresponding channel feature matrix of the feature map input to the SE layer to obtain a new feature map of the same size as the feature map; (327) Finally, perform 1*1 convolution dimensionality reduction.

7. The workpiece identification and counting method based on target detection and template matching in complex scenarios according to claim 6, characterized in that: The expression for the ReLU6 activation function in step (324) is: ； The expression for the h-sigmoid activation function is: ； The expression for the Hard-swish activation function is: 。 8. The workpiece identification and counting method based on target detection and template matching in complex scenarios according to claim 1, characterized in that: Step (4) specifically involves: deploying the target detection model trained in step (3), reading the workpieces on the production line in real time from the industrial camera, marking the minimum bounding rectangle of the workpiece in the image, and using the obtained center coordinates (x, y) and θ The angle information is used to rotate the workpiece in the area to a horizontal position and then crop and save the image to obtain all the workpieces to be identified.

9. The workpiece identification and counting method based on target detection and template matching in complex scenarios according to claim 1, characterized in that: The formula for the Hessian matrix in step (51) is as follows: In the formula, σ is the spatial scale, and L xx (x,σ),L xy (x,σ),L yy (x,σ) is the convolution of the second-order Gaussian derivative; The Hessian matrix discrimination formula is: In the formula, α is taken as an empirical value of 0.

9.

10. The workpiece identification and counting method based on target detection and template matching in complex scenarios according to claim 1, characterized in that: The formula for calculating the Euclidean distance in step (54) is as follows: ； The formula for determining the threshold is: 。

Images