Workpiece piecework model training method, piecework method, and related device

Abstract

The application provides a workpiece piecework model training method, a piecework method and related devices; an anchor frame is determined through multi-scale features of an image; a threshold of an intersection over union is determined according to a mean value and a standard deviation of the intersection over union of a target anchor frame and a real boundary frame, the target anchor frame is divided into a positive sample anchor frame and a negative sample anchor frame according to the threshold of the intersection over union, and a backbone network is retrained by using the positive sample anchor frame and the negative sample anchor frame; a candidate frame and a second feature are determined according to a first feature through the backbone network; a predicted boundary frame and a third feature are determined according to the candidate frame and the second feature; a predicted category is obtained through clustering according to the predicted boundary frame and the third feature; then, parameters of a workpiece piecework model are adjusted; an intelligent real-time workpiece counting method is realized, workpiece detection is completed through a class-independent detector, workpieces are classified and counted through clustering, piecework operation can be effectively performed on multiple categories of workpieces, and workpiece counting efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to a training method, a counting method, and related apparatus for a workpiece counting model. Background Technology

[0002] During the production process, it is necessary to count the number of workpieces for subsequent material management processes such as warehousing, logistics, and sales orders. Workpiece classification and counting rely on methods such as manual visual counting, scanning one by one, and entering the data into the system. Due to the large volume and variety of workpieces, the workpiece counting process requires a significant investment of manpower and resources, resulting in extremely low efficiency.

[0003] Currently, image processing techniques such as density estimation are commonly used for workpiece counting. However, the density map contains inconspicuous Gaussian spots, and significant overlap is found in dense density map regions, resulting in inaccurate workpiece localization. Additionally, self-attention mechanisms are used for workpiece counting, but this method is limited to performing well when identifying workpieces of the same category; its effectiveness is poor when counting multi-category workpieces. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] The purpose of this application is to at least partially solve one of the technical problems existing in the related technologies. The embodiments of this application provide a training method, a counting method and related devices for a workpiece counting model, which can realize a real-time workpiece statistics method.

[0006] An embodiment of the first aspect of this application provides a method for training a piece counting model, comprising:

[0007] Acquire training workpiece images and input the training workpiece images into the workpiece counting model. The training workpiece images are labeled with true bounding boxes and true categories.

[0008] Extract multiple first features at different scales from the training workpiece image;

[0009] The anchor frame is determined based on the first feature through the backbone network of the workpiece counting model, and a target anchor frame is selected from the anchor frames based on the distance between the anchor frame and the real bounding box.

[0010] The threshold of the cross-union ratio is determined based on the mean and standard deviation of the cross-union ratio between the target anchor box and the real bounding box. The target anchor box is divided into positive sample anchor boxes and negative sample anchor boxes based on the threshold of the cross-union ratio. The backbone network is trained using the positive sample anchor boxes and the negative sample anchor boxes to obtain a trained backbone network.

[0011] The trained backbone network determines candidate boxes and second features based on the first feature.

[0012] Based on the candidate box and the second feature, a predicted bounding box and a third feature are determined;

[0013] Clustering is performed based on the predicted bounding box and the third feature to obtain the predicted category;

[0014] The parameters of the workpiece counting model are adjusted based on the predicted bounding box, the true bounding box, the predicted category, and the true category to obtain a trained workpiece counting model.

[0015] According to certain embodiments of the first aspect of this application, the step of determining a threshold for the intersection-union ratio (IU) based on the mean and standard deviation of the IU between the target anchor box and the ground truth bounding box, and classifying the target anchor box into positive sample anchor boxes and negative sample anchor boxes based on the IU threshold, includes:

[0016] The threshold value of the cross-union ratio (CUR) between the target anchor box and the real bounding box is obtained by adding the mean of the CUR and the standard deviation of the CUR between the target anchor box and the real bounding box.

[0017] Target anchor boxes whose absolute value is greater than the threshold of the intersection-union ratio and whose center is located within the target true bounding box are regarded as positive sample anchor boxes;

[0018] The target anchor boxes outside the positive sample anchor boxes are used as negative sample anchor boxes;

[0019] Wherein, the target true bounding box is the true bounding box with the largest intersection-union ratio with the target anchor box.

[0020] According to certain embodiments of the first aspect of this application, determining the predicted bounding box and the third feature based on the candidate box and the second feature includes:

[0021] The region of interest at the first level is determined based on the second feature and the candidate box. The region of interest at the first level is then passed through a fully connected layer and an activation function layer to obtain the predicted bounding box and the third feature at the first level.

[0022] The region of interest (ROI) of the current level is determined based on the second feature and the predicted bounding box of the previous level. The ROI of the current level is then passed through a fully connected layer and an activation function layer to obtain the predicted bounding box and the third feature of the current level.

[0023] According to certain embodiments of the first aspect of this application, the first loss function for determining the predicted bounding box and the third feature based on the candidate box and the second feature is as follows: Where, x t Let L(x) be the second feature of level t, g be the candidate box, and L(x) be the second feature of level t.t (g) is the first loss function, L cls For the classification loss function, h t Let y be the classifier at level t. t For x t Background labels at the crossover-union ratio threshold, where λ is the tradeoff coefficient, and F t For the regressor at level t, L loc For the regression loss function, b t is the bias value for level t.

[0024] According to certain embodiments of the first aspect of this application, the step of clustering based on the predicted bounding box and the third feature to obtain the predicted category includes:

[0025] Deep semantic information of the feature map containing the predicted bounding box and the third feature is embedded into a low-dimensional space to determine the coordinate array of the category data points;

[0026] K-means clustering is performed on the coordinate array of the data points of the categories to obtain the number of target categories;

[0027] The feature map containing the predicted bounding box and the third feature is mapped from the high-dimensional space to the low-dimensional space according to the basis vectors to obtain the dimensionality-reduced feature map.

[0028] Cluster the reduced-dimensional feature map based on the number of target categories to obtain the predicted categories;

[0029] The basis vectors are selected based on the covariance matrix of the predicted bounding box and the third feature.

[0030] According to certain embodiments of the first aspect of this application, the step of performing K-means clustering based on the coordinate array of the category data points to obtain the target number of categories includes:

[0031] The clustering score is calculated based on the coordinate array of the data points in the category.

[0032] The number of target categories is determined based on the clustering scores;

[0033] The clustering score is represented as: s(k) is the cluster score, B k W is the covariance matrix between categories. k Let tr(B) be the covariance matrix of the samples within the class. k ) is B k trace, tr(W) k ) for W k The trace, N is the number of samples in the coordinate array of class data points, k is the number of clusters, x is the number of samples in the coordinate array of class data points, cq Let n be the sample set of cluster q. q Let be the number of samples in cluster q, and c be the number of samples in cluster q.

[0034] According to certain embodiments of the first aspect of this application, the step of clustering the dimensionality-reduced feature map based on the number of target categories to obtain predicted categories includes:

[0035] Clustering is performed on the reduced-dimensional feature map to determine the weight, mean, and covariance matrix of each category, where the number of categories is the number of target categories;

[0036] The probability density function is calculated based on the weights, means, and covariance matrices of each category;

[0037] The predicted category is obtained based on the probability density function.

[0038] According to a second aspect of this application, a method for counting workpieces includes:

[0039] Obtain images of the workpieces to be counted;

[0040] The workpiece image to be counted is input into the trained workpiece counting model to count the number of workpieces.

[0041] The trained workpiece counting model is obtained by training the workpiece counting model according to the training method of the workpiece counting model described in the first aspect of this application.

[0042] According to a third aspect of this application, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the workpiece counting model training method as described in the first aspect of this application and the workpiece counting method as described in the second aspect of this application.

[0043] According to a fourth aspect of this application, a computer storage medium stores computer-executable instructions for performing a training method for a workpiece counting model as described in an embodiment of the first aspect of this application and a workpiece counting method as described in an embodiment of the second aspect of this application.

[0044] The above scheme has at least the following beneficial effects: Anchor boxes are determined based on first features at different scales of the training workpiece image; target anchor boxes are selected from the anchor boxes based on the distance between the anchor boxes and the true bounding boxes; a threshold for the intersection-union ratio (IU) is determined based on the mean and standard deviation of the IU between the target anchor boxes and the true bounding boxes; the target anchor boxes are divided into positive and negative sample anchor boxes based on the IU threshold; a backbone network is trained using the positive and negative sample anchor boxes to obtain a trained backbone network; candidate boxes and second features are determined using the trained backbone network based on the first feature; predicted bounding boxes and third features are determined based on the candidate boxes and second features; clustering is performed based on the predicted bounding boxes and third features to obtain the predicted category; the parameters of the workpiece counting model are adjusted based on the predicted bounding boxes, true bounding boxes, predicted categories, and true categories to obtain a trained workpiece counting model; an intelligent real-time workpiece statistics method is realized, which uses a class-independent detector to detect workpieces and uses clustering to classify and count workpieces, effectively performing counting operations on multi-category workpieces and improving workpiece counting efficiency. Attached Figure Description

[0045] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0046] Figure 1 This is a step diagram of the workpiece counting method provided in the embodiments of this application;

[0047] Figure 2 This is a flowchart illustrating the steps of the training method for the workpiece counting model provided in the embodiments of this application;

[0048] Figure 3 This is a diagram of the sub-steps of step S600 provided in the embodiments of this application;

[0049] Figure 4 This is a sub-step diagram of step S620 provided in the embodiments of this application;

[0050] Figure 5 This is a sub-step diagram of step S640 provided in the embodiments of this application;

[0051] Figure 6 This is a flowchart illustrating steps S300, S400, and S500 provided in the embodiments of this application.

[0052] Figure 7 This is a schematic diagram of the cascaded RCNN network provided in the embodiments of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0055] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0056] An embodiment of this application provides a method for counting workpieces.

[0057] Reference Figure 1 The piece counting method includes the following steps:

[0058] Step S110: Obtain the image of the workpiece to be counted;

[0059] Step S120: Input the image of the workpiece to be counted into the trained workpiece counting model to count the number of workpieces and obtain the number of workpieces.

[0060] Among them, the well-trained workpiece counting model is obtained by training the workpiece counting model using the same method.

[0061] Reference Figure 2 The training method for the piece counting model includes the following steps:

[0062] Step S200: Obtain the training workpiece image and input the training workpiece image into the workpiece counting model;

[0063] Step S300: Extract multiple first features at different scales from the training workpiece image;

[0064] Step S400: Anchor boxes are determined based on the first feature using the backbone network of the workpiece counting model; target anchor boxes are selected from the anchor boxes based on the distance between the anchor boxes and the real bounding boxes; a threshold for the cross-union ratio (CUI) is determined based on the mean and standard deviation of the CUI between the target anchor boxes and the real bounding boxes; target anchor boxes are divided into positive sample anchor boxes and negative sample anchor boxes based on the CUI threshold; the backbone network is trained using the positive and negative sample anchor boxes to obtain a trained backbone network; candidate boxes and a second feature are determined based on the first feature using the trained backbone network.

[0065] Step S500: Determine the predicted bounding box and the third feature based on the candidate box and the second feature;

[0066] Step S600: Cluster the predicted bounding boxes and the third feature to obtain the predicted categories;

[0067] Step S700: Adjust the parameters of the workpiece counting model based on the predicted bounding box, the true bounding box, the predicted category, and the true category to obtain the trained workpiece counting model.

[0068] In step S200 of some embodiments, various workpiece images in a real environment are collected, and the dataset of workpiece images is cleaned. Labeling software such as Labelme is used to manually annotate the workpiece images in the dataset, labeling the workpiece images with true bounding boxes and true category labels, obtaining a JSON format annotation file, and thus obtaining training workpiece images. Where H is the height of the image, W is the width of the image, and C is the number of channels in the image.

[0069] The workpiece images are input into the workpiece counting model to train the workpiece counting model.

[0070] In step S300 of some embodiments, multiple first features at different scales of the training workpiece image are extracted using a pyramid network.

[0071] Reference Figure 6 The pyramid network extracts features through a series of convolutional and pooling layers, forming feature maps at multiple scales. As network depth increases, the resolution of the feature maps gradually decreases, but the number of feature channels increases, thus capturing features from higher levels. This allows the network to capture both local details and global contextual information simultaneously. Lower-level feature maps have higher resolution but fewer feature channels; higher-level feature maps have lower resolution but more feature channels. The pyramid network merges feature maps from different levels using feature fusion techniques, such as skip connections or feature upsampling. This helps integrate information at different scales, enhancing the expressive power of the features.

[0072] The output of the last residual block of each level is used as the feature activation output; for the outputs of conv2, conv3, conv4, and conv5, their last residual block outputs are represented as {F2, F3, F4, F5}, which have strides of 4, 8, 16, and 32 pixels relative to the input image, respectively. Because conv1 has a large memory footprint, it is not included in the pyramid. The number of output channels for all predicted feature layers is set to 256.

[0073] The backbone network is ConvAFPN, a pure convolutional network improved from ResNet50. It utilizes residual blocks to alleviate the gradient vanishing problem and helps improve the model's accuracy and efficiency. Furthermore, it changes the structure from thick at both ends and thin in the middle to an anti-bottleneck structure that is thin at both ends and thick in the middle, which can partially reduce the model's parameter size while slightly improving accuracy, thus improving the overall performance of the model.

[0074] The anchor frame is determined based on the first feature by the backbone network of the workpiece counting model, and the target anchor frame is selected from the anchor frames according to the distance between the anchor frame and the real bounding box.

[0075] Reference Figure 6 For example, in each of the Levels 1, 2, 3, and 4 of the backbone network ConvAFPN, k anchor boxes whose centers are closest to the true bounding boxes are selected based on the L2 distance to form an Anchor set, resulting in Anchor set 1, Anchor set 2, Anchor set 3, and Anchor set 4, respectively. The L2 distance is expressed as: x i Let be the i-th first feature, and let n be the nth feature.

[0076] The threshold for the cross-union ratio (CUR) is determined based on the mean and standard deviation of the CUR between the target anchor boxes and the true bounding boxes. Based on the CUR threshold, the target anchor boxes are divided into positive sample anchor boxes and negative sample anchor boxes.

[0077] For example, Anchor set 1, Anchor set 2, Anchor set 3, and Anchor set 4 are compared with the ground truth labels. The threshold of the intersection-union ratio (IU) is obtained by adding the mean of the IU of the target anchor boxes and the ground truth bounding boxes to the standard deviation of the IU of the target anchor boxes and the ground truth bounding boxes; wherein, the IU of the target anchor boxes and the ground truth bounding boxes is expressed as: A0 represents the intersection of the target anchor box and the true bounding box. U This represents the union of the target anchor box and the ground truth bounding box. The threshold for the intersection-union ratio (IU) is expressed as: t g =m g +v g ;m g v represents the mean of the intersection-union ratio (IU) between the target anchor box and the ground truth bounding box. g The standard deviation of the intersection-union ratio (IU) between the target anchor box and the true bounding box.

[0078] Target anchor boxes whose absolute value is greater than the cross-union ratio threshold and whose center is located within the target ground truth bounding box are designated as positive sample anchor boxes; target anchor boxes outside the positive sample anchor boxes are designated as negative sample anchor boxes; where the target ground truth bounding box is the ground truth bounding box with the largest cross-union ratio with the target anchor box.

[0079] A backbone network is trained using positive and negative sample anchor boxes, with the same layer's anchor boxes used to train the same layer's network, resulting in a well-trained backbone network. This trained backbone network then determines candidate boxes and a second feature based on a first feature. Candidate boxes are closer to the ground truth boxes than anchor boxes, and the second feature provides a higher-level feature description than the first feature.

[0080] In step S500 of some embodiments, a cascaded RCNN network is used to determine the predicted bounding box and the third feature based on the candidate box and the second feature.

[0081] Reference Figure 7 A cascaded RCNN network consists of T cascaded RCNN networks.

[0082] For the first-level RCNN network, the region of interest in the first level is determined based on the second feature and the candidate box. The region of interest in the first level is then passed through a fully connected layer and an activation function layer to obtain the predicted bounding box and the third feature of the first level.

[0083] For the nth level of the RCNN network, the region of interest (ROI) of the current level is determined based on the second feature and the predicted bounding box of the previous level. The ROI of the current level is then passed through a fully connected layer and an activation function layer to obtain the predicted bounding box and the third feature of the current level.

[0084] In each stage t, the RCNN head includes a classifier h. t And a threshold u for the corresponding crossover ratio t Optimized regressor Where u t >u t-1 .

[0085] The processing of a cascaded R-CNN network can be expressed by the following formula: F(x,b)=F T ·F T-1 *…*F1(x,b); T is the total number of stages in the cascade. The cascaded R-CNN is trained sequentially, using the output of one stage to train the next. For each regression variable F… t Both are based on the regression box distribution generated by the previous regressor {b t} to optimize, instead of the initial distribution {b} 1 This method gradually improves the bounding boxes. 7×7 convolutions are used to predict the foreground bounding boxes and the likelihood scores of the foreground and background, progressively refining the bounding box positions. During training, the positive and negative sample thresholds u of the regressor are set to A{0.5, 0.6, 0.7}, progressively filtering out bounding boxes with low confidence. Initially, all artifact categories are assigned a foreground label of 1, and any unassigned objects are labeled with a background label of 0.

[0086] The first loss function, which determines the predicted bounding box and the third feature based on the candidate box and the second feature, is as follows: Where, x t Let L(x) be the second feature of level t, g be the candidate box, and L(x) be the second feature of level t. t (g) is the first loss function, L cls For the classification loss function, h t Let y be the classifier at level t. t For x t Background labels at the crossover-union ratio threshold, where λ is the tradeoff coefficient, and F t For the regressor at level t, L loc For the regression loss function, b t is the bias value for level t.

[0087] An efficient detector can find all positive samples in an image while suppressing false positive samples. However, directly choosing an excessively low crossover ratio (CUP) threshold will lead to low-quality detection results, while directly choosing an excessively high CUP threshold will also reduce detection performance. In order to achieve high-quality object detection, a cascaded RCNN is used for bounding box prediction. Cascading a deeper detector has higher selectivity for false positive samples.

[0088] Reference Figure 3 In step S600 of some embodiments, clustering is performed based on the predicted bounding box and the third feature to obtain the predicted category, including the following steps:

[0089] Step S610: Embed the deep semantic information of the feature map containing the predicted bounding box and the third feature into a low-dimensional space to determine the coordinate array of the category data points;

[0090] Step S620: Perform K-means clustering based on the coordinate array of the category data points to obtain the number of target categories;

[0091] Step S630: Map the feature map containing the predicted bounding box and the third feature from the high-dimensional space to the low-dimensional space according to the basis vectors to obtain the dimensionality-reduced feature map.

[0092] Step S640: Cluster the dimensionality-reduced feature map according to the number of target categories to obtain the predicted categories.

[0093] For example, after locating the workpiece using predicted bounding boxes, images of each workpiece instance are obtained through coordinate cropping. All workpiece instances are then resized to 64x64 and embedded into the feature space. However, high-dimensional spaces contain redundant information, making data analysis more difficult, and clustering excessively long feature sequences requires a considerable amount of time. Considering the need for high efficiency and high utilization, the entire process is typically required to be completed in a very short time. Therefore, it is necessary to reduce the dimensionality of the feature sequences before clustering.

[0094] Deep semantic information from feature maps containing predicted bounding boxes and third features is embedded into a low-dimensional space based on manifold learning theory to find optimal edge weights in the low-dimensional representation; these optimal weights are obtained by minimizing the cross-entropy function. Stochastic gradient descent is used to find the optimal edge weights. The final result is an array of coordinates containing the category data points for each workpiece in the specified low-dimensional space. The aim is to discover hidden low-dimensional structures in high-dimensional data and map them to a more easily understood and analyzed space. By constructing topological relationships in the data and utilizing these topological relationships to approximate the distribution of data in the original high-dimensional space, both local and global relationships between data points are preserved.

[0095] Because a single prediction bounding box may contain multiple different categories of workpieces, and there are significant differences between the different categories of workpieces within a single prediction bounding box. (Refer to...) Figure 4 The target number of categories is determined by performing K-means clustering based on the coordinate array of the category data points, including the following steps:

[0096] Step S621: Calculate the clustering score based on the coordinate array of the category data points;

[0097] Step S622: Determine the number of target categories based on the clustering scores;

[0098] The clustering score is represented as: s(k) is the cluster score, B k W is the covariance matrix between categories. k Let tr(B) be the covariance matrix of the samples within the class. k ) is B k trace, tr(W) k ) for W k The trace, N is the number of samples in the coordinate array of class data points, k is the number of clusters, x is the number of samples in the coordinate array of class data points, c q Let n be the sample set of cluster q. q Let be the number of samples in cluster q, and c be the number of samples in cluster q.

[0099] The number of target categories is the optimal K value; corresponding cluster labels are generated based on the optimal K value.

[0100] Principal Component Analysis (PCA) is used to reduce the dimensionality of another part of the feature map, which contains the predicted bounding boxes and the third feature. This allows for the extraction of key information from large and complex datasets, simplifying the data analysis process. Dimensionality reduction is achieved by mapping high-dimensional data to a low-dimensional space while minimizing information loss from the original data. It assumes that the main changes in the data can be represented by a few newly constructed features (i.e., principal components), which are linear combinations of the original features and are uncorrelated with each other. The PCA formula is: Y = XV; representing a linear transformation of the original dataset X through new basis vectors V to obtain the dimensionality-reduced dataset Y. Each sample in Y is the coordinate representation of the original sample in the new basis vector space, and the new basis vectors are selected based on the covariance matrix of the predicted bounding boxes and the third feature, effectively representing the main direction of change in the original data. The covariance matrix is ​​calculated as follows: Where X is a matrix composed of the original data column by column, and m is the number of samples. Principal component analysis is used to further compress the dimension to 32, so that the projection of the sample points on the new representation space is as dispersed as possible.

[0101] K-means clustering cannot cluster two classes with the same mean (i.e., the same cluster centroid). Instead, it clusters the dimensionality-reduced feature map based on the number of target classes, maximizing the posterior probability by selecting components to obtain the predicted class. Since the posterior probability of each data point represents the likelihood of it belonging to any class, rather than simply determining whether it completely belongs to a particular class, it provides more accurate classification results than K-means clustering.

[0102] Reference Figure 5 The predicted categories are obtained by clustering the dimensionality-reduced feature maps based on the number of target categories, including the following steps:

[0103] Step S641: Cluster the reduced feature map to determine the weight, mean, and covariance matrix of each category;

[0104] Step S642: Calculate the probability density function based on the weights, means, and covariance matrices of each category;

[0105] Step S643: Obtain the predicted category based on the probability density function.

[0106] Suppose all data points are composed of a finite number of distinct Gaussian distributions mixed in varying proportions, each with its own mean and covariance matrix; these distributions are called components. For a given dataset, we need to determine the number of components, the mean and covariance matrix of each component, and the weight of each component, so that the model can fit the data well. It can be understood that the number of components is equivalent to the number of Gaussian distributions, which is also the number of classes, and the number of classes equals the number of target classes.

[0107] The probability density function of the Gaussian distribution is: Where x is the data point of the dimensionality-reduced feature map, μ is the mean vector of the distribution, Σ is the covariance matrix, and D is the dimension of the data.

[0108] The overall probability density function is: Here, θ is the set of model parameters, including the weights, mean, and covariance matrix of all components. k It is the mean, π k It's the weight.

[0109] After obtaining the clustering results, the prototype of each category is determined based on the cluster centers and the Top-k strategy. Then, all other unknown artifacts are fed into a Siamese network using ResNet18 as the feature extraction network, one by one, along with their prototypes. A soft voting mechanism is introduced: the unknown artifact is assigned to the category of the prototype with which it has the highest similarity, thus obtaining the predicted category. The number of artifacts is then counted based on the predicted categories. This completes the classification and counting of the detected artifacts.

[0110] In step S700 of some embodiments, the parameters of the workpiece counting model are adjusted according to the predicted bounding box, the true bounding box, the predicted category, and the true category to obtain a trained workpiece counting model.

[0111] For example, the loss function of the workpiece counting model is calculated based on the difference between the predicted bounding box and the true bounding box, and the difference between the predicted class and the true class; the parameters of the workpiece counting model are adjusted according to the loss function to obtain the trained workpiece counting model.

[0112] An intelligent real-time workpiece counting method has been implemented. It uses a class-independent detector to detect workpieces and uses clustering to classify and count workpieces, which can effectively perform workpiece counting operations for multiple categories of workpieces and improve workpiece counting efficiency.

[0113] An embodiment of this application provides an electronic device. The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the workpiece counting model training method and the workpiece counting method as described above.

[0114] This electronic device can be any smart terminal, including computers.

[0115] In general, for the hardware structure of electronic devices, the processor can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, to execute relevant programs and implement the technical solutions provided in the embodiments of this application.

[0116] The memory can be implemented in the form of read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and is called and executed by the processor.

[0117] Input / output interfaces are used to implement information input and output.

[0118] The communication interface is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0119] The bus transmits information between various components of a device, such as the processor, memory, input / output interfaces, and communication interfaces. The processor, memory, input / output interfaces, and communication interfaces communicate with each other within the device via the bus.

[0120] An embodiment of this application provides a computer storage medium. The computer storage medium stores computer-executable instructions for executing the workpiece counting model training method and workpiece counting method as described above.

[0121] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0122] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0123] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0125] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms. Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

[0127] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A training method for a workpiece counting model, characterized in that, include: Acquire training workpiece images and input the training workpiece images into the workpiece counting model. The training workpiece images are labeled with true bounding boxes and true categories. Extract multiple first features at different scales from the training workpiece image; The anchor frame is determined based on the first feature through the backbone network of the workpiece counting model, and a target anchor frame is selected from the anchor frames based on the distance between the anchor frame and the real bounding box. The threshold of the cross-union ratio is determined based on the mean and standard deviation of the cross-union ratio between the target anchor box and the real bounding box. The target anchor box is divided into positive sample anchor boxes and negative sample anchor boxes based on the threshold of the cross-union ratio. The backbone network is trained using the positive sample anchor boxes and the negative sample anchor boxes to obtain a trained backbone network. The trained backbone network determines candidate boxes and second features based on the first feature. Based on the candidate box and the second feature, a predicted bounding box and a third feature are determined; Clustering is performed based on the predicted bounding box and the third feature to obtain the predicted category; The parameters of the workpiece counting model are adjusted based on the predicted bounding box, the true bounding box, the predicted category, and the true category to obtain a trained workpiece counting model.

2. The training method for the workpiece counting model according to claim 1, characterized in that, The step of determining the threshold for the intersection-union ratio (IU) based on the mean and standard deviation of the IU between the target anchor boxes and the ground truth bounding boxes, and classifying the target anchor boxes into positive and negative sample anchor boxes based on the IU threshold, includes: The threshold value of the cross-union ratio (CUR) between the target anchor box and the real bounding box is obtained by adding the mean of the CUR and the standard deviation of the CUR between the target anchor box and the real bounding box. Target anchor boxes whose absolute value is greater than the threshold of the intersection-union ratio and whose center is located within the target true bounding box are regarded as positive sample anchor boxes; The target anchor boxes outside the positive sample anchor boxes are used as negative sample anchor boxes; Wherein, the target true bounding box is the true bounding box with the largest intersection-union ratio with the target anchor box.

3. The training method for the workpiece counting model according to claim 1, characterized in that, The step of determining the predicted bounding box and the third feature based on the candidate box and the second feature includes: The region of interest at the first level is determined based on the second feature and the candidate box. The region of interest at the first level is then passed through a fully connected layer and an activation function layer to obtain the predicted bounding box and the third feature at the first level. The region of interest (ROI) of the current level is determined based on the second feature and the predicted bounding box of the previous level. The ROI of the current level is then passed through a fully connected layer and an activation function layer to obtain the predicted bounding box and the third feature of the current level.

4. The training method for the workpiece counting model according to claim 3, characterized in that, Based on the candidate box and the second feature, the first loss function corresponding to the predicted bounding box and the third feature is determined as follows: Where, x t Let L(x) be the second feature of level t, g be the candidate box, and L(x) be the second feature of level t. t (g) is the first loss function, L cls For the classification loss function, h t Let y be the classifier at level t. t For x t Background labels at the crossover-union ratio threshold, where λ is the tradeoff coefficient, and F t For the regressor at level t, L loc For the regression loss function, b t is the bias value for level t.

5. The training method for the workpiece counting model according to claim 1, characterized in that, The step of clustering based on the predicted bounding box and the third feature to obtain the predicted category includes: Deep semantic information of the feature map containing the predicted bounding box and the third feature is embedded into a low-dimensional space to determine the coordinate array of the category data points; K-means clustering is performed on the coordinate array of the data points of the categories to obtain the number of target categories; The feature map containing the predicted bounding box and the third feature is mapped from the high-dimensional space to the low-dimensional space according to the basis vectors to obtain the dimensionality-reduced feature map. Cluster the reduced-dimensional feature map based on the number of target categories to obtain the predicted categories; The basis vectors are selected based on the covariance matrix of the predicted bounding box and the third feature.

6. The training method for the workpiece counting model according to claim 5, characterized in that, The step of performing K-means clustering based on the coordinate array of the category data points to obtain the target number of categories includes: The clustering score is calculated based on the coordinate array of the data points in the category. The number of target categories is determined based on the clustering scores; The clustering score is represented as: s(k) is the cluster score, B k W is the covariance matrix between categories. k Let tr(B) be the covariance matrix of the samples within the class. k ) is B k trace, tr(W) k ) for W k The trace, N is the number of samples in the coordinate array of class data points, k is the number of clusters, x is the number of samples in the coordinate array of class data points, c q Let n be the sample set of cluster q. q Let be the number of samples in cluster q, and c be the number of samples in cluster q.

7. The training method for the workpiece counting model according to claim 5, characterized in that, The step of clustering the dimensionality-reduced feature map based on the number of target categories to obtain predicted categories includes: Clustering is performed on the reduced-dimensional feature map to determine the weight, mean, and covariance matrix of each category, where the number of categories is the number of target categories; The probability density function is calculated based on the weights, means, and covariance matrices of each category; The predicted category is obtained based on the probability density function.

8. A method for calculating workpiece count, characterized in that, include: Obtain images of the workpieces to be counted; The workpiece image to be counted is input into the trained workpiece counting model to count the number of workpieces. The trained workpiece counting model is obtained by training according to the workpiece counting model training method as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the training method for the workpiece counting model as described in any one of claims 1 to 7 and the workpiece counting method as described in claim 8.

10. A computer storage medium, characterized in that, The device stores computer-executable instructions for performing the training method for the workpiece counting model as described in any one of claims 1 to 7 and the workpiece counting method as described in claim 8.

Images