Workpiece counting model training method, workpiece counting method and related device

Abstract

The invention provides a training method of a workpiece counting model, a workpiece counting method and a related device. Determining an anchor frame through the multi-scale features of the image; determining a threshold value of the intersection-to-union ratio according to the mean value and the standard deviation of the intersection-to-union ratio of the target anchor frame and the real bounding box, dividing the target anchor frame into a positive sample anchor frame and a negative sample anchor frame according to the threshold value of the intersection-to-union ratio, and retraining the backbone network by using the positive sample anchor frame and the negative sample anchor frame; determining a candidate frame and a second feature through the backbone network according to the first feature; determining a prediction bounding box and a third feature according to the candidate box and the second feature; clustering according to the prediction bounding box and the third feature to obtain a prediction category; then adjusting parameters of the workpiece counting model; the intelligent real-time workpiece counting method is achieved, the workpieces are detected through the class-independent detector, the workpieces are counted in a classified mode through clustering, the workpieces of multiple classes can be effectively counted, and the workpiece counting efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular to a training method for a workpiece piece counting model, a piece counting method, and related devices. Background Art

[0002] During the workpiece production process, the number of workpieces needs to be counted for subsequent material management processes such as warehouse management, logistics transportation, and sales orders. The classification and statistics of workpieces rely on methods such as manual visual counting, scanning one by one, and entering the system. Due to the large output and various types of workpieces, the workpiece counting process requires a lot of manpower and material resources, and the efficiency is extremely low.

[0003] Currently, image processing techniques such as density estimation methods are commonly used to count artifacts. However, since the density map contains inconspicuous Gaussian spots and significant overlap is found in dense density map areas, it is impossible to provide accurate artifact positioning. In addition, the self-attention mechanism is used to count artifacts, but this method is limited to identifying artifacts of the same category and has a good effect, and the effect of counting artifacts of multiple categories is poor. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The purpose of the present application is to solve one of the technical problems existing in the related art to at least a certain extent. The embodiments of the present application provide a training method for a workpiece piece counting model, a piece counting method and related devices, which can realize a real-time workpiece counting method.

[0006] An embodiment of the first aspect of the present application is a method for training a workpiece counting model, comprising:

[0007] Acquire a training workpiece image, and input the training workpiece image into a workpiece piece counting model, wherein the training workpiece image is provided with a label including a true bounding box and a true category;

[0008] extracting a plurality of first features of different scales of the training workpiece image;

[0009] Determine an anchor frame according to the first feature through the backbone network of the workpiece piece counting model, and select a target anchor frame from the anchor frame according to the distance between the anchor frame and the real bounding box;

[0010] Determine an intersection-and-union (IoU) threshold according to the mean and standard deviation of the IoU between the target anchor box and the true bounding box, divide the target anchor box into a positive sample anchor box and a negative sample anchor box according to the IoU threshold, and train the backbone network using the positive sample anchor box and the negative sample anchor box to obtain a trained backbone network;

[0011] Determine a candidate frame and a second feature according to the first feature through a trained backbone network;

[0012] Determine a predicted bounding box and a third feature according to the candidate box and the second feature;

[0013] Perform clustering according to the predicted bounding box and the third feature to obtain a predicted category;

[0014] The parameters of the workpiece piece counting model are adjusted according to the predicted bounding box, the true bounding box, the predicted category and the true category to obtain a trained workpiece piece counting model.

[0015] According to some embodiments of the first aspect of the present application, determining a threshold of the intersection-and-union ratio according to a mean and a standard deviation of the intersection-and-union ratio between the target anchor box and the true bounding box, and dividing the target anchor box into a positive sample anchor box and a negative sample anchor box according to the threshold of the intersection-and-union ratio, comprises:

[0016] Adding the mean of the intersection-and-union ratio of the target anchor box to the true bounding box and the standard deviation of the intersection-and-union ratio of the target anchor box to the true bounding box to obtain a threshold of the intersection-and-union ratio;

[0017] The target anchor box whose absolute value is greater than the threshold of the intersection-over-union ratio and whose center is located in the target true bounding box is taken as the positive sample anchor box;

[0018] Taking the target anchor frame other than the positive sample anchor frame as the negative sample anchor frame;

[0019] The target real bounding box is a real bounding box with the largest intersection-over-union ratio with the target anchor box.

[0020] According to some embodiments of the first aspect of the present application, determining a predicted bounding box and a third feature according to the candidate box and the second feature includes:

[0021] Determine a region of interest at a first level according to the second feature and the candidate box, and obtain a predicted bounding box and a third feature at a first level by passing the region of interest at the first level through a fully connected layer and an activation function layer;

[0022] The region of interest at the current level is determined according to the second feature and the predicted bounding box at the previous level, and the predicted bounding box and the third feature at the current level are obtained by passing the region of interest at the current level through a fully connected layer and an activation function layer.

[0023] According to some embodiments of the first aspect of the present application, a first loss function corresponding to a predicted bounding box and a third feature is determined according to the candidate box and the second feature: Among them, x t is the second feature of the t-th level, g is the candidate box, L(xt ,g) is the first loss function, L cls is the classification loss function, h t is the classifier at level t, y t For x t Background label under the threshold of intersection-over-union ratio, λ is the weight coefficient, F t is the regressor at level t, L loc is the regression loss function, b t is the bias value of the tth level.

[0024] According to certain embodiments of the first aspect of the present application, clustering according to the predicted bounding box and the third feature to obtain a predicted category includes:

[0025] Embedding deep semantic information of the feature map including the predicted bounding box and the third feature into a low-dimensional space to determine a coordinate array of category data points;

[0026] Perform K-means clustering processing according to the coordinate array of the category data points to obtain the target category number;

[0027] Mapping a feature map including the predicted bounding box and the third feature from a high-dimensional space to a low-dimensional space according to a basis vector to obtain a reduced-dimensional feature map;

[0028] Clustering the dimension reduction feature graph according to the number of target categories to obtain a predicted category;

[0029] The basis vector is selected based on the covariance matrix of the predicted bounding box and the third feature.

[0030] According to certain embodiments of the first aspect of the present application, performing K-means clustering processing on the coordinate array of the category data points to obtain the target category number includes:

[0031] Calculate the clustering score according to the coordinate array of the category data points;

[0032] determining the number of target categories according to the clustering scores;

[0033] Wherein, the clustering score is expressed as: s(k) is the clustering score, B k is the covariance matrix between categories, W k is the covariance matrix of samples within the class, tr(B k ) is B k The trace of tr(W k ) is W k trace, N is the number of samples of the coordinate array of the category data point, k is the number of clusters, x is the sample of the coordinate array of the category data point, cq is the sample set of cluster q, n q is the number of samples in cluster q, and c is the number of samples in cluster q.

[0034] According to certain embodiments of the first aspect of the present application, clustering the reduced-dimensionality feature graph according to the number of target categories to obtain a predicted category includes:

[0035] Clustering the reduced-dimensional feature graph to determine the weight, mean, and covariance matrix of each category, the number of categories being the number of target categories;

[0036] The probability density function is calculated based on the weight, mean and covariance matrix of each category;

[0037] The predicted category is obtained according to the probability density function.

[0038] An embodiment of the second aspect of the present application is a workpiece counting method, comprising:

[0039] Acquire the image of the workpiece to be counted;

[0040] Inputting the workpiece image to be counted into a trained workpiece counting model for piece counting to obtain the number of workpieces;

[0041] The trained workpiece piece counting model is obtained by training according to the training method for the workpiece piece counting model described in the embodiment of the first aspect of the present application.

[0042] An embodiment of the third aspect of the present application is an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method of the workpiece piece counting model as described in the embodiment of the first aspect of the present application and the workpiece piece counting method as described in the embodiment of the second aspect of the present application are implemented.

[0043] An embodiment of the fourth aspect of the present application is a computer storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the training method of the workpiece piece counting model as described in the embodiment of the first aspect of the present application and the workpiece piece counting method as described in the embodiment of the second aspect of the present application.

[0044] The above scheme has at least the following beneficial effects: determining anchor frames according to first features of different scales of training workpiece images, selecting target anchor frames from anchor frames according to the distance between anchor frames and true bounding frames; determining the threshold of intersection-over-union ratio according to the mean and standard deviation of the intersection-over-union ratio between target anchor frames and true bounding frames, dividing target anchor frames into positive sample anchor frames and negative sample anchor frames according to the threshold of intersection-over-union ratio, and training a backbone network using positive sample anchor frames and negative sample anchor frames to obtain a trained backbone network; determining candidate frames and second features according to the first feature through the trained backbone network; determining predicted bounding boxes and third features according to the candidate frames and second features; clustering according to the predicted bounding boxes and third features to obtain predicted categories; adjusting parameters of a workpiece piece counting model according to the predicted bounding boxes, true bounding boxes, predicted categories and true categories to obtain a trained workpiece piece counting model; realizing an intelligent real-time workpiece counting method, completing the detection of workpieces through class-independent detectors, and classifying and counting workpieces using clustering, which can effectively perform piece counting operations on multi-category workpieces and improve workpiece counting efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0046] Figure 1 is a step diagram of a workpiece counting method provided in an embodiment of the present application;

[0047] Figure 2 is a step diagram of a training method for a workpiece piece counting model provided in an embodiment of the present application;

[0048] Figure 3 It is a sub-step diagram of step S600 provided in an embodiment of the present application;

[0049] Figure 4 It is a sub-step diagram of step S620 provided in an embodiment of the present application;

[0050] Figure 5 It is a sub-step diagram of step S640 provided in an embodiment of the present application;

[0051] Figure 6 It is a flowchart corresponding to step S300, step S400 and step S500 provided in an embodiment of the present application;

[0052] Figure 7 It is a flow chart of the cascade RCNN network provided in the embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0054] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0055] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0056] An embodiment of the present application provides a workpiece counting method.

[0057] Reference Figure 1 , a workpiece piece counting method, comprising the following steps:

[0058] Step S110, acquiring an image of a workpiece to be counted;

[0059] Step S120, inputting the workpiece image to be counted into the trained workpiece counting model for piece counting to obtain the number of workpieces.

[0060] The trained workpiece piece counting model is obtained by training the workpiece piece counting model using a training method.

[0061] Reference Figure 2 , the training method of the workpiece piece counting model includes the following steps:

[0062] Step S200, obtaining a training workpiece image, and inputting the training workpiece image into a workpiece piece counting model;

[0063] Step S300, extracting a plurality of first features of different scales of the training workpiece image;

[0064] Step S400, determining an anchor frame according to the first feature through the backbone network of the workpiece piece counting model, selecting a target anchor frame from the anchor frame according to the distance between the anchor frame and the true bounding box; determining an intersection-and-union ratio threshold according to the mean and standard deviation of the intersection-and-union ratio between the target anchor frame and the true bounding box, dividing the target anchor frame into a positive sample anchor frame and a negative sample anchor frame according to the intersection-and-union ratio threshold, training the backbone network with the positive sample anchor frame and the negative sample anchor frame to obtain a trained backbone network; determining a candidate frame and a second feature according to the first feature through the trained backbone network;

[0065] Step S500, determining a predicted bounding box and a third feature according to the candidate box and the second feature;

[0066] Step S600, clustering is performed according to the predicted bounding box and the third feature to obtain a predicted category;

[0067] Step S700 , adjusting the parameters of the workpiece piece counting model according to the predicted bounding box, the true bounding box, the predicted category and the true category to obtain a trained workpiece piece counting model.

[0068] In step S200 of some embodiments, various types of artifact images in a real environment are collected, and data cleaning is performed on the artifact image data set; the artifact images in the data set are manually annotated using labeling software such as Labelme, and the artifact images are annotated with real bounding boxes and real category labels to obtain a Json format annotation file to obtain a training artifact image. Where H is the height of the image, W is the width of the image, and C is the number of channels of the image.

[0069] The training workpiece image is input into the workpiece piece counting model to train the workpiece piece counting model.

[0070] In step S300 of some embodiments, a plurality of first features of different scales of the training workpiece image are extracted through a pyramid network.

[0071] Reference Figure 6 , the pyramid network extracts features through a series of convolutional layers and pooling layers, and these hierarchical structures form feature maps of multiple scales. As the depth of the network increases, the resolution of the feature map gradually decreases, but at the same time the number of feature channels increases, thereby capturing higher-level features. In this way, the network can capture local details and global context information at the same time. The underlying feature map has a higher resolution but fewer feature channels; the high-level feature map has a lower resolution but more feature channels. The pyramid network merges feature maps of different levels through feature fusion techniques such as skip connections or feature upsampling. This helps to integrate information at different scales and enhance the expressiveness of features.

[0072] The last residual block output of each level is used as the feature activation output; for conv2, conv3, conv4, and conv5 outputs, their last residual block outputs are represented as {F2, F3, F4, F5}, which have strides of 4, 8, 16, and 32 pixels relative to the input image, respectively. Because conv1 has a large memory footprint, conv1 is not included in the pyramid. The number of output channels of all prediction feature layers is set to 256.

[0073] The backbone network is ConvAFPN, which is a pure convolutional network based on ResNet50. It uses residual blocks to alleviate the problem of gradient vanishing and helps improve the accuracy and efficiency of the model. In addition, the structure with thick ends and thin middle is changed to an anti-bottleneck structure with thin ends and thick middle, which can partially reduce the parameter scale of the model while slightly improving the accuracy, thereby improving the overall performance of the model.

[0074] An anchor box is determined according to the first feature through the backbone network of the workpiece counting model, and a target anchor box is selected from the anchor box according to the distance between the anchor box and the real bounding box.

[0075] Reference Figure 6 , exemplarily, each layer of Level1, Level2, Level3 and Level4 of the backbone network ConvAFPN selects k anchor boxes whose centers are closest to the true bounding box according to the L2 distance to form an Anchor set, and obtains Anchor set 1, Anchor set 2, Anchor set 3 and Anchor set 4 respectively. Among them, the L2 distance is expressed as: x i is the i-th first feature and is the n-th feature.

[0076] The threshold of the intersection-over-union ratio is determined according to the mean and standard deviation of the intersection-over-union ratio between the target anchor box and the true bounding box, and the target anchor box is divided into a positive sample anchor box and a negative sample anchor box according to the threshold of the intersection-over-union ratio.

[0077] Exemplarily, Anchor Set 1, Anchor Set 2, Anchor Set 3 and Anchor Set 4 are compared with the true label. The mean of the intersection-and-union ratio of the target anchor box and the true bounding box and the standard deviation of the intersection-and-union ratio of the target anchor box and the true bounding box are added to obtain the threshold of the intersection-and-union ratio; wherein the intersection-and-union ratio of the target anchor box and the true bounding box is expressed as: A0 represents the intersection of the target anchor box and the true bounding box, A U Represents the union of the target anchor box and the true bounding box. The threshold of the intersection ratio is expressed as: t g =m g +v g ;m g represents the mean of the intersection-over-union ratio of the target anchor box and the true bounding box, v g Represents the standard deviation of the intersection-over-union ratio between the target anchor box and the true bounding box.

[0078] The target anchor box whose absolute value is greater than the intersection-and-union ratio threshold and whose center is located in the target true bounding box is taken as the positive sample anchor box; the target anchor box outside the positive sample anchor box is taken as the negative sample anchor box; among which, the target true bounding box is the true bounding box with the largest intersection-and-union ratio with the target anchor box.

[0079] The backbone network is trained using the positive sample anchor frame and the negative sample anchor frame, and the positive sample anchor frame and the negative sample anchor frame of the same layer are used to train the network of the same layer to obtain a trained backbone network. The candidate frame and the second feature are determined according to the first feature through the trained backbone network. The candidate frame is closer to the true frame than the anchor frame, and the second feature has a higher level of feature description than the first feature.

[0080] In step S500 of some embodiments, a predicted bounding box and a third feature are determined based on the candidate box and the second feature through a cascaded RCNN network.

[0081] Reference Figure 7 , the cascaded RCNN network consists of T cascaded RCNN networks.

[0082] For the first level RCNN network, the first level region of interest is determined according to the second feature and the candidate box, and the first level region of interest is passed through the fully connected layer and the activation function layer to obtain the first level predicted bounding box and the third feature;

[0083] For the n-th level RCNN network, the region of interest of the current level is determined according to the second feature and the predicted bounding box of the previous level, and the region of interest of the current level is passed through the fully connected layer and the activation function layer to obtain the predicted bounding box of the current level and the third feature.

[0084] In each stage t, the RCNN head consists of a classifier h t and a threshold u for the corresponding intersection-over-union ratio t Optimized Regressor where u t >u t-1 .

[0085] The processing of the cascade RCNN network can be expressed by the following formula: F(x,b)=F T ·F T-1 *…*F1(x,b); T is the total number of cascades. Cascade RCNN is trained sequentially, using the output of one stage to train the next stage. Each regressor F t All of them are for the regression box distribution {b t} for optimization, rather than the initial distribution {b 1}. In this way, the regression box is gradually improved. A 7×7 convolution is used to predict the likelihood score of the regression box of the foreground and the foreground background, and the regression box position is gradually refined. During the training phase, the positive and negative sample thresholds u of the regressor are set to A{0.5, 0.6, 0.7}, respectively, and the regression boxes with low confidence are gradually filtered out. During training, all artifact categories are initially assigned a foreground label of 1, and any unassigned objects are marked with a background label of 0.

[0086] The first loss function corresponding to the predicted bounding box and the third feature is determined based on the candidate box and the second feature: Among them, x t is the second feature of the t-th level, g is the candidate box, L(x t ,g) is the first loss function, L cls is the classification loss function, h t is the classifier at level t, y t For x t Background label under the threshold of intersection-over-union ratio, λ is the weight coefficient, F t is the regressor at level t, L loc is the regression loss function, b t is the bias value of the tth level.

[0087] An efficient detector can find all positive samples in an image while suppressing those false positive samples. However, directly selecting a too low IoU threshold will lead to low-quality detection results. Directly selecting a too high IoU threshold will also lead to reduced detection performance. In order to achieve high-quality target detection, cascade RCNN is used to make regression box predictions. Detectors deeper in the cascade have higher selectivity for false positive samples.

[0088] Reference Figure 3 In step S600 of some embodiments, clustering is performed according to the predicted bounding box and the third feature to obtain the predicted category, including the following steps:

[0089] Step S610, embedding the deep semantic information of the feature map including the predicted bounding box and the third feature into a low-dimensional space, and determining a coordinate array of the category data points;

[0090] Step S620, performing K-means clustering processing according to the coordinate array of the category data points to obtain the target category number;

[0091] Step S630, mapping the feature map including the predicted bounding box and the third feature from the high-dimensional space to the low-dimensional space according to the basis vector to obtain a reduced-dimensional feature map;

[0092] Step S640, clustering the reduced-dimensional feature graph according to the number of target categories to obtain predicted categories.

[0093] For example, after locating the workpiece position by predicting the bounding box, the image of each workpiece instance is obtained by the coordinate clipping method, all workpiece instances are adjusted and extracted to a size of 64x64, and embedded in the feature space. However, the redundant information in high-dimensional space makes data analysis more difficult, and clustering of long feature sequences takes a long time. Considering the demand for high efficiency and high utilization, the entire process is usually required to be completed in a very short time. Therefore, it is necessary to reduce the dimension of the feature sequence before clustering.

[0094] The deep semantic information of a part of the feature map containing the predicted bounding box and the third feature is embedded into the low-dimensional space based on the manifold learning theory, so as to find the optimal weights of the edges in the low-dimensional representation; these optimal weights are obtained by minimizing the cross entropy function. The optimal weights of the edges are found using the stochastic gradient descent method. Finally, the coordinate array containing the category data points of each workpiece in the specified low-dimensional space is obtained. It aims to discover the hidden low-dimensional structure in the high-dimensional data and map it to a space that is easier to understand and analyze. By constructing the topological relationship of the data and using this topological relationship to approximate the distribution of the data in the original high-dimensional space, the local and global relationships between the data points are preserved.

[0095] There may be multiple different types of artifacts within a predicted bounding box, and there are significant differences between artifacts of different categories within a predicted bounding box. Figure 4 , perform K-means clustering processing based on the coordinate array of the category data points to determine the number of target categories, including the following steps:

[0096] Step S621, calculating the clustering score according to the coordinate array of the category data points;

[0097] Step S622, determining the number of target categories according to the clustering score;

[0098] The clustering score is expressed as: s(k) is the clustering score, B k is the covariance matrix between categories, W k is the covariance matrix of samples within the class, tr(B k ) is B k The trace of tr(W k ) is W k trace, N is the number of samples of the coordinate array of the category data point, k is the number of clusters, x is the sample of the coordinate array of the category data point, c q is the sample set of cluster q, n q is the number of samples in cluster q, and c is the number of samples in cluster q.

[0099] The number of target categories is the optimal K value; the corresponding cluster labels are generated according to the optimal K value.

[0100] The other part of the feature map containing the predicted bounding box and the third feature is reduced in dimension by the principal component analysis method, which can extract key information from a large and complex data set, thereby simplifying the data analysis process. By mapping high-dimensional data to low-dimensional space, the purpose of dimensionality reduction is achieved, while minimizing the loss of information in the original data. It is assumed that the main changes in the data can be represented by a few newly constructed features (i.e., principal components), which are linear combinations of the original features and are unrelated to each other. The calculation formula of the principal component analysis method is: Y = XV; it means that the original data set X is linearly transformed through the new basis vector V to obtain the reduced-dimensional data set Y. Among them, each sample of Y is the coordinate representation of the original sample in the new basis vector space, and the new basis vector is selected according to the covariance matrix of the predicted bounding box and the third feature, which can well represent the main change direction of the original data. The formula for calculating the covariance matrix is: Among them, X is the matrix composed of columns of the original data, and m is the number of samples. The dimension is further compressed to 32 using the principal component analysis method to make the projection of the sample points on the new representation space as dispersed as possible.

[0101] The K-means clustering method cannot cluster two classes with the same mean (same cluster center point). It clusters the reduced dimension feature map according to the number of target categories, and completes the clustering by selecting components to maximize the posterior probability to obtain the predicted category. Since the posterior probability of each data point represents the possibility that it belongs to each category, rather than just judging whether it completely belongs to a certain category, it provides more accurate classification results than the K-means clustering method.

[0102] Reference Figure 5 , cluster the reduced dimension feature graph according to the number of target categories to obtain the predicted category, including the following steps:

[0103] Step S641, clustering the reduced-dimensional feature graph to determine the weight, mean, and covariance matrix of each category;

[0104] Step S642, calculating the probability density function according to the weight, mean and covariance matrix of each category;

[0105] Step S643, obtaining a predicted category according to the probability density function.

[0106] Assume that all data points are a mixture of a finite number of different Gaussian distributions in different proportions, each with its own mean and covariance matrix. These distributions are called components. For a given data set, it is necessary to determine the number of components, the mean and covariance matrix of each component, and the weight of each component so that the model can fit the data well. It is understandable that the number of components is the number of Gaussian distributions, that is, the number of categories, and the number of categories is equal to the number of target categories.

[0107] The probability density function of the Gaussian distribution is: Among them, x is the data point of the reduced dimension feature map, μ is the mean vector of the distribution, Σ is the covariance matrix, and D is the dimension of the data.

[0108] The overall probability density function is: Where θ is the set of model parameters, including the weights, means and covariance matrices of all components. k is the mean, π k is the weight.

[0109] After obtaining the clustering results, the artifact prototype of each category is determined according to the cluster center and the Top-k strategy, and then all other unknown artifacts are put into the twin network with ResNet18 as the feature extraction network one by one together with the artifact prototype. The soft voting mechanism is introduced, that is, the unknown artifact is classified into the category of the artifact prototype with which the artifact prototype has the highest similarity, and the predicted category is obtained. The number of artifacts is counted according to the predicted category. In this way, the detected artifacts can be classified and counted.

[0110] In step S700 of some embodiments, the parameters of the workpiece piece counting model are adjusted according to the predicted bounding box, the true bounding box, the predicted category and the true category to obtain a trained workpiece piece counting model.

[0111] Exemplarily, the loss function of the workpiece piece counting model is calculated based on the difference between the predicted bounding box and the true bounding box and the difference between the predicted category and the true category; the parameters of the workpiece piece counting model are adjusted according to the loss function to obtain a trained workpiece piece counting model.

[0112] An intelligent real-time workpiece counting method is implemented. The workpiece detection is completed through class-independent detectors, and clustering is used to classify and count the workpieces. It can effectively count multiple categories of workpieces and improve the workpiece counting efficiency.

[0113] An embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the training method of the workpiece counting model and the workpiece counting method as described above when executing the computer program.

[0114] The electronic device may be any intelligent terminal including a computer.

[0115] In general, for the hardware structure of the electronic device, the processor can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, etc., to execute related programs to implement the technical solutions provided in the embodiments of the present application.

[0116] The memory can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory can store an operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory, and the processor is called to execute the methods of the embodiments of this application.

[0117] The input / output interface is used to realize information input and output.

[0118] The communication interface is used to realize the communication interaction between this device and other devices. Communication can be achieved through wired methods (such as USB, network cable, etc.) or wireless methods (such as mobile network, WIFI, Bluetooth, etc.).

[0119] The bus transmits information between the various components of the device (such as the processor, memory, input / output interface, and communication interface). The processor, memory, input / output interface, and communication interface communicate with each other within the device through the bus.

[0120] An embodiment of the present application provides a computer storage medium storing computer executable instructions for executing the workpiece piece counting model training method and workpiece piece counting method as described above.

[0121] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. In the above description of this specification, the description of reference terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0122] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0123] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0124] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0125] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0126] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above-mentioned units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and purposes of the present application, and the scope of the present application is defined by the claims and their equivalents.

[0127] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Technical personnel familiar with the field can make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A training method for a workpiece counting model, characterized in that: include: Acquire a training workpiece image, and input the training workpiece image into a workpiece piece counting model, wherein the training workpiece image is provided with a label including a true bounding box and a true category; extracting a plurality of first features of different scales of the training workpiece image; Determine an anchor frame according to the first feature through the backbone network of the workpiece piece counting model, and select a target anchor frame from the anchor frame according to the distance between the anchor frame and the real bounding box; Determine an intersection-and-union (IoU) threshold according to the mean and standard deviation of the IoU between the target anchor box and the true bounding box, divide the target anchor box into a positive sample anchor box and a negative sample anchor box according to the IoU threshold, and train the backbone network using the positive sample anchor box and the negative sample anchor box to obtain a trained backbone network; Determine a candidate frame and a second feature according to the first feature through a trained backbone network; Determine a predicted bounding box and a third feature according to the candidate box and the second feature; Perform clustering according to the predicted bounding box and the third feature to obtain a predicted category; The parameters of the workpiece piece counting model are adjusted according to the predicted bounding box, the true bounding box, the predicted category and the true category to obtain a trained workpiece piece counting model.

2. The training method of the workpiece piece counting model according to claim 1, characterized in that: The step of determining the threshold of the intersection-and-union ratio according to the mean and standard deviation of the intersection-and-union ratio between the target anchor box and the true bounding box, and dividing the target anchor box into a positive sample anchor box and a negative sample anchor box according to the threshold of the intersection-and-union ratio comprises: Adding the mean of the intersection-and-union ratio of the target anchor box to the true bounding box and the standard deviation of the intersection-and-union ratio of the target anchor box to the true bounding box to obtain a threshold of the intersection-and-union ratio; The target anchor box whose absolute value is greater than the threshold of the intersection-over-union ratio and whose center is located in the target true bounding box is taken as the positive sample anchor box; Taking the target anchor frame other than the positive sample anchor frame as the negative sample anchor frame; The target real bounding box is a real bounding box with the largest intersection-over-union ratio with the target anchor box.

3. The training method of the workpiece piece counting model according to claim 1, characterized in that: The determining a predicted bounding box and a third feature according to the candidate box and the second feature includes: Determine a region of interest at a first level according to the second feature and the candidate box, and obtain a predicted bounding box and a third feature at a first level by passing the region of interest at the first level through a fully connected layer and an activation function layer; The region of interest at the current level is determined according to the second feature and the predicted bounding box at the previous level, and the predicted bounding box and the third feature at the current level are obtained by passing the region of interest at the current level through a fully connected layer and an activation function layer.

4. The training method of the workpiece piece counting model according to claim 3, characterized in that: The first loss function corresponding to the predicted bounding box and the third feature is determined according to the candidate box and the second feature: Among them, x t is the second feature of the t-th level, g is the candidate box, L(x t ,g) is the first loss function, L cls is the classification loss function, h t is the classifier at level t, y t For x t Background label under the threshold of intersection-over-union ratio, λ is the weight coefficient, F t is the regressor at level t, L loc is the regression loss function, b t is the bias value of the tth level.

5. The training method of the workpiece piece counting model according to claim 1, characterized in that: The step of clustering the predicted bounding box and the third feature to obtain a predicted category includes: Embedding deep semantic information of the feature map including the predicted bounding box and the third feature into a low-dimensional space to determine a coordinate array of category data points; Perform K-means clustering processing according to the coordinate array of the category data points to obtain the target category number; Mapping a feature map including the predicted bounding box and the third feature from a high-dimensional space to a low-dimensional space according to a basis vector to obtain a reduced-dimensional feature map; Clustering the dimension reduction feature graph according to the number of target categories to obtain a predicted category; The basis vector is selected based on the covariance matrix of the predicted bounding box and the third feature.

6. The training method of the workpiece piece counting model according to claim 5, characterized in that: The step of performing K-means clustering processing according to the coordinate array of the category data points to obtain the target category number includes: Calculate the clustering score according to the coordinate array of the category data points; determining the number of target categories according to the clustering scores; Wherein, the clustering score is expressed as: s(k) is the clustering score, B k is the covariance matrix between categories, W k is the covariance matrix of samples within the class, tr(B k ) is B k The trace of tr(W k ) is W k trace, N is the number of samples of the coordinate array of the category data point, k is the number of clusters, x is the sample of the coordinate array of the category data point, c q is the sample set of cluster q, n q is the number of samples in cluster q, and c is the number of samples in cluster q.

7. The training method of the workpiece piece counting model according to claim 5, characterized in that: The clustering of the dimension reduction feature graph according to the number of target categories to obtain a predicted category includes: Clustering the reduced-dimensional feature graph to determine the weight, mean, and covariance matrix of each category, the number of categories being the number of target categories; The probability density function is calculated based on the weight, mean and covariance matrix of each category; The predicted category is obtained according to the probability density function.

8. A workpiece counting method, characterized in that: include: Acquire the image of the workpiece to be counted; Inputting the workpiece image to be counted into a trained workpiece counting model for piece counting to obtain the number of workpieces; The trained workpiece piece counting model is obtained by training according to the training method for a workpiece piece counting model as claimed in any one of claims 1 to 7.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the training method for a workpiece piece counting model according to any one of claims 1 to 7 and the workpiece piece counting method according to claim 8 are implemented.

10. A computer storage medium, characterized in that: Computer-executable instructions are stored, and the computer-executable instructions are used to execute the training method of the workpiece piece counting model according to any one of claims 1 to 7 and the workpiece piece counting method according to claim 8.

Images