A product classification model training method and related device

Abstract

The application discloses a product classification model training method and related device. Two predictions are performed on a plurality of unlabeled sample product image data to obtain a prediction label and a first prediction result corresponding to each unlabeled sample product image data, respectively. The plurality of unlabeled sample product image data is divided into a first sample product image data set and a second sample product image data set based on the consistency of the prediction label and the first prediction result. A label calibration loss function for eliminating noise in the prediction label is constructed based on the first sample product image data set, and a product classification model is trained based on the label calibration loss function. A feature consistency loss function for correcting feature bias is constructed based on the second sample product image data set, and the product classification model is trained based on the feature consistency loss function. Different training strategies are adopted for sample product image data sets with different classification difficulties to improve the distinguishing ability of the product classification model, thereby improving the accuracy of the classification result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of detection, in particular to a product classification model training method and related device. BACKGROUND

[0002] Industrial quality inspection refers to quality detection on products in the production process, so as to distinguish whether the products are normal or defective. Traditional quality inspection is generally performed by quality inspection workers through manual visual inspection. In recent years, with the rise of artificial intelligence (AI) technology, AI quality inspection based on machine vision can greatly improve quality inspection accuracy and save labor costs, and has broad market application prospects.

[0003] In the industrial AI quality inspection scene, a product classification model is usually trained in a positive and unlabeled (PU) learning manner, so as to use the trained product classification model to detect products.

[0004] However, the product classification model trained by this method is difficult to distinguish between some similar products, and thus the classification result is not accurate enough. SUMMARY

[0005] To solve the above technical problems, the present application provides a product classification model training method and related device, which uses different training strategies for sample product image data sets of different classification difficulties, realizes targeted training of sample product image data sets of different classification difficulties, and thus improves the distinguishing ability of the product classification model and the accuracy of the classification result of the product classification model.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In one aspect, the embodiments of the present application provide a product classification model training method, which comprises:

[0008] obtaining training sample product image data, wherein the training sample product image data comprises a plurality of unlabeled sample product image data;

[0009] performing preliminary prediction on the plurality of unlabeled sample product image data to obtain prediction labels corresponding to the plurality of unlabeled sample product image data respectively;

[0010] performing secondary prediction on the plurality of unlabeled sample product image data to obtain first prediction results corresponding to the plurality of unlabeled sample product image data respectively;

[0011] dividing the plurality of unlabeled sample product image data into a first sample product image dataset and a second sample product image dataset based on consistency between the predicted label and the first prediction result, wherein the classification difficulty of the first sample product image data in the first sample product image dataset is lower than the classification difficulty of the second sample product image data in the second sample product image dataset;

[0012] Training a product classification model using the first sample product image dataset and the second sample product image dataset;

[0013] In the process of training the product classification model using the first sample product image dataset, constructing a label calibration loss function based on the first sample product image dataset, and training the product classification model based on the label calibration loss function, wherein the label calibration loss function is used to eliminate noise in the predicted labels;

[0014] In the process of training the product classification model using the second sample product image dataset, a feature consistency loss function is constructed based on the second sample product image dataset, and the product classification model is trained based on the feature consistency loss function. The feature consistency loss function is used to correct the feature deviation of feature extraction of the second sample product image data in the second sample product image dataset.

[0015] In one aspect, an embodiment of the present application provides a training device for a product classification model, the device comprising an acquisition unit, a prediction unit, a division unit, a training unit, and a construction unit:

[0016] The acquisition unit is configured to acquire training sample product image data, wherein the training sample product image data includes a plurality of unlabeled sample product image data;

[0017] The prediction unit is configured to perform preliminary prediction on the plurality of unlabeled sample product image data to obtain prediction labels corresponding to the plurality of unlabeled sample product image data respectively;

[0018] The prediction unit is further configured to perform secondary prediction on the plurality of unlabeled sample product image data to obtain first prediction results corresponding to the plurality of unlabeled sample product image data respectively;

[0019] The dividing unit is configured to divide the plurality of unlabeled sample product image data into a first sample product image dataset and a second sample product image dataset based on consistency between the predicted label and the first prediction result, wherein the classification difficulty of the first sample product image data in the first sample product image dataset is lower than the classification difficulty of the second sample product image data in the second sample product image dataset;

[0020] The training unit is configured to train a product classification model using the first sample product image dataset and the second sample product image dataset;

[0021] The construction unit is configured to construct a label calibration loss function based on the first sample product image dataset in a process of training the product classification model using the first sample product image dataset;

[0022] The training unit is specifically configured to train the product classification model based on the label calibration loss function, wherein the label calibration loss function is configured to eliminate noise in the predicted labels;

[0023] The construction unit is further configured to construct a feature consistency loss function based on the second sample product image dataset in a process of training the product classification model using the second sample product image dataset;

[0024] The training unit is specifically used to train the product classification model based on the feature consistency loss function, and the feature consistency loss function is used to correct feature deviations in feature extraction of the second sample product image data in the second sample product image dataset.

[0025] In one aspect, an embodiment of the present application provides a computer device, comprising a processor and a memory.

[0026] The memory is used to store program code and transmit the program code to the processor;

[0027] The processor is configured to execute the method described in any one of the aforementioned aspects according to instructions in the program code.

[0028] In one aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and when the program code is executed by a processor, the processor executes the method described in any one of the aforementioned aspects.

[0029] In one aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements the method described in any of the aforementioned aspects when executed by a processor.

[0030] It can be seen from the above technical solution that in industrial quality inspection scenarios, the training sample product image data usually obtained includes a large amount of unlabeled sample product image data. In this case, in order to train a product classification model that can accurately classify the products to be inspected, thereby accurately identifying whether the products to be inspected are defective products or normal products, a preliminary prediction can be performed on multiple unlabeled sample product image data to obtain prediction labels corresponding to the multiple unlabeled sample product image data, and a secondary prediction can be performed on the multiple unlabeled sample product image data to obtain first prediction results corresponding to the multiple unlabeled sample product image data. The consistency between the prediction label and the first prediction result can reflect the classification difficulty of the unlabeled sample product image data to a certain extent. Generally, the more consistent the prediction label is with the first prediction result, the lower the classification difficulty. Therefore, the multiple unlabeled sample product image data can be divided based on the consistency of the prediction label and the first prediction result to obtain a first sample product image data set and a second sample product image data set. The classification difficulty of the first sample product image data in the first sample product image data set is lower than the classification difficulty of the second sample product image data in the second sample product image data set. Then, the product classification model is trained using the first sample product image dataset and the second sample product image dataset. During the training process, different training strategies are adopted for sample product image datasets with different classification difficulties. Specifically, in the process of training the product classification model using the first sample product image dataset, since its classification difficulty is very low, noise may occasionally have an impact. Therefore, a label calibration loss function can be constructed based on the first sample product image dataset, and the product classification model is trained based on the label calibration loss function. The label calibration loss function is used to eliminate noise in the predicted labels; in the process of training the product classification model using the second sample product image dataset, since its classification difficulty is greater, the predicted labels may have greater uncertainty. Therefore, a feature consistency loss function can be constructed based on the second sample product image dataset, and the product classification model is trained based on the feature consistency loss function. The feature consistency loss function is used to correct the feature deviation of feature extraction of the second sample product image data in the second sample product image dataset, thereby achieving training for the consistency of its features, mining the inherent feature differences of the second sample product image data in the second sample product image dataset, and improving the product classification model's ability to distinguish between positive and negative samples. It can be seen that this application adopts different training strategies for sample product image data sets with different classification difficulties during the training process, so as to realize targeted training of sample product image data sets with different classification difficulties, thereby improving the distinguishing ability of the product classification model and improving the accuracy of the classification results of the product classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technical members in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0032] Figure 1 An application scenario architecture diagram of a product classification model training method provided in an embodiment of the present application;

[0033] Figure 2 A flowchart of a method for training a product classification model provided in an embodiment of the present application;

[0034] Figure 3 An example diagram of a training method for a basic classification model provided in an embodiment of the present application;

[0035] Figure 4 An example diagram of the training process of a discrimination model provided in an embodiment of the present application;

[0036] Figure 5 This is an example diagram of the training process of a product classification model provided in an embodiment of the present application;

[0037] Figure 6 This is an example diagram of a product classification display interface provided in an embodiment of the present application;

[0038] Figure 7 This is an overall architecture diagram for an AI quality inspection scenario provided by an embodiment of the present application;

[0039] Figure 8 A structural diagram of a product classification model training device provided in an embodiment of the present application;

[0040] Figure 9 A structural diagram of a terminal provided in an embodiment of the present application;

[0041] Figure 10 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The embodiments of the present application are described below with reference to the accompanying drawings.

[0043] In order to ensure the quality of products during the manufacturing process, it is necessary to conduct defect quality inspection on the products. For example, for leather defect recognition, steel plate surface defect recognition, wood surface defect recognition, etc. When performing defect recognition, AI quality inspection based on machine vision is currently the main method. Through AI quality inspection, it is determined whether it has defects, thereby distinguishing normal products from defective products. In the industrial AI quality inspection scenario, there is such a scenario: the customer only provides a small amount of positive sample product image data with annotated labels, as well as a large amount of production line data (i.e., unlabeled sample product image data) and a rough production line yield. At the same time, it is difficult to collect defective products in batches, and the type of defect cannot be specifically specified. For example, for the identification of leather defects, the leather image is input into the product classification model, so that the product classification model can be used to implement AI quality inspection to determine whether there are defects.

[0044] In this scenario, the product classification model used is typically trained using PU learning. This involves training the product classification model based on a small amount of known positive product image data and a large amount of unlabeled product image data. This allows the model to correctly distinguish between positive and negative samples (positive samples are normal products, negative samples are defective products). However, this training method lacks a targeted training strategy for the difficult-to-distinguish positive and negative samples (with similar features) in unlabeled data. This results in the product classification model having difficulty distinguishing between products with similar features, leading to insufficient classification accuracy.

[0045] In order to solve the above technical problems, an embodiment of the present application provides a training method for a product classification model, which divides multiple unlabeled sample product image data into a first sample product image data set and a second sample product image data set with different classification difficulties. During the training process, different training strategies are adopted for the sample product image data sets with different classification difficulties to achieve targeted training of the sample product image data sets with different classification difficulties, thereby improving the distinguishing ability of the product classification model and improving the accuracy of the classification results of the product classification model.

[0046] It should be noted that the training method of the product classification model provided in the embodiment of the present application can be executed by a computer device, which can be, for example, a server or a terminal. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car terminal, a smart TV, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0047] like Figure 1 As shown, Figure 1 An application scenario architecture diagram of a product classification model training method is shown, which takes a server executing the product classification model training method provided in the present application as an example for introduction.

[0048] In this application scenario, the server 100 can be included. In the industrial quality inspection scene, the training sample product image data obtained usually includes a large amount of unlabeled sample product image data. In this case, in order to train a product classification model that can accurately classify the to-be-detected product, so as to accurately identify whether the to-be-detected product is a defective product or a normal product. Therefore, when the server 100 trains the product classification model, it can first perform preliminary prediction on the plurality of unlabeled sample product image data to obtain a plurality of predicted labels corresponding to the plurality of unlabeled sample product image data respectively. The predicted label can be used as a pseudo label of the unlabeled sample product image data, thereby assisting the subsequent training process.

[0049] The server 100 can also perform secondary prediction on the plurality of unlabeled sample product image data to obtain a plurality of first prediction results corresponding to the plurality of unlabeled sample product image data respectively. The consistency of the predicted label and the first prediction result can reflect the classification difficulty of the unlabeled sample product image data to a certain extent. Generally, the more consistent the predicted label and the first prediction result are, the lower the classification difficulty is. Therefore, the server 100 can divide the plurality of unlabeled sample product image data based on the consistency of the predicted label and the first prediction result to obtain a first sample product image data set and a second sample product image data set. The classification difficulty of the first sample product image data in the first sample product image data set is lower than the classification difficulty of the second sample product image data in the second sample product image data set.

[0050] Then the server 100 trains the product classification model using the first sample product image data set and the second sample product image data set. Since the classification difficulty of the sample product image data in different sample product image data sets is different, and in order to distinguish the sample product image data with different classification difficulties, the features relied on may be different. Therefore, the focus of learning required in the training process for different classification difficulty sample product image data sets will also be different. Therefore, in the present application, the server 100 adopts different training strategies for different classification difficulty sample product image data sets.

[0051] Specifically, in the process of training the product classification model using the first sample product image dataset, since the classification difficulty of the first sample product image data in the first sample product image dataset is very low, its predicted label is usually reliable. Occasionally, the predicted label may be unreliable due to the presence of noise. Therefore, when the server 100 uses the first sample product image dataset for training, it focuses on avoiding the situation where the predicted label is unreliable due to noise. Therefore, the server 100 can construct a label calibration loss function based on the first sample product image dataset, and train the product classification model based on the label calibration loss function. The label calibration loss function is used to eliminate noise in the predicted label. In the process of training the product classification model using the second sample product image dataset, since the classification of the second sample product image data in the second sample product image dataset is relatively difficult and the predicted label may have a large uncertainty, the server 100 focuses on training the feature consistency of the second sample product image data when using the second sample product image dataset for training. Therefore, the server 100 can construct a feature consistency loss function based on the second sample product image dataset and train the product classification model based on the feature consistency loss function. The feature consistency loss function is used to correct the feature deviation of feature extraction of the second sample product image data in the second sample product image dataset, thereby realizing training for the consistency of its features, exploring the inherent feature differences of the second sample product image data in the second sample product image dataset, and improving the product classification model's ability to distinguish between positive and negative samples.

[0052] During the training process, this application adopts different training strategies for sample product image data sets with different classification difficulties, so as to achieve targeted training for sample product image data sets with different classification difficulties, thereby improving the distinguishing ability of the product classification model and improving the accuracy of the classification results of the product classification model.

[0053] It should be noted that the methods provided in the embodiments of this application primarily relate to artificial intelligence. Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI also encompasses the study of the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0054] Artificial intelligence technology is a comprehensive discipline covering a wide range of fields, encompassing both hardware and software technologies. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning. The methods provided in the embodiments of this application primarily involve computer vision and machine learning / deep learning.

[0055] Computer vision (CV) is a science that studies how to make machines "see." Specifically, it refers to using cameras and computers to replace human eyes to perform machine vision tasks such as identifying, tracking, and measuring targets, and further performing image processing to make computer-processed images more suitable for human observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, attempting to establish artificial intelligence systems that can obtain information from images or multidimensional data. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, and other technologies. It also includes common biometric recognition technologies such as face recognition and fingerprint recognition. For example, embodiments of the present application may perform image processing on training sample product image data (e.g., unlabeled sample product image data, positive sample product image data) and the image of the product to be detected, extract features through image semantic understanding for classification prediction, etc.; image processing may also be used to perform different degrees of data enhancement on the second sample product image data.

[0056] Machine Learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching techniques. In the embodiment of the present application, the product classification model can be trained by machine learning.

[0057] Next, the training method of the product classification model executed by the server will be taken as an example, and the training method of the product classification model provided by the embodiment of the present application will be described in detail with reference to the accompanying drawings. Figure 2 , Figure 2 A flowchart of a method for training a product classification model is shown, the method comprising:

[0058] S201: Acquire training sample product image data, where the training sample product image data includes a plurality of unlabeled sample product image data.

[0059] In the industrial AI quality inspection scenario, there is a scenario where the customer only provides a small amount of positive sample product image data with annotated labels, as well as a large amount of production line data (i.e., unlabeled sample product image data) and an approximate production line yield. Based on this, in order to use a large amount of sample product image data to train a product classification model, the server can obtain training sample product image data including multiple unlabeled sample product image data, so as to train a product classification model using multiple unlabeled sample product image data. The unlabeled sample product image data can be represented as X u Since the label of the unlabeled sample product image data is unknown, the label Y of the unlabeled sample product image data is u Not assigned, X u and Y u It constitutes an unlabeled sample product image dataset, which can be expressed as {D u ={X u ,Y u}}.

[0060] S202: Perform preliminary prediction on the plurality of unlabeled sample product image data to obtain prediction labels corresponding to the plurality of unlabeled sample product image data.

[0061] After obtaining the unlabeled sample product image data, the server can first perform preliminary predictions on multiple unlabeled sample product image data to obtain prediction labels corresponding to the multiple unlabeled sample product image data. The prediction labels can be used as pseudo labels for the unlabeled sample product image data to assist in the subsequent training process. express.

[0062] In a possible implementation, S202 can be implemented by performing preliminary predictions on multiple unlabeled sample product image data based on the trained basic classification model to obtain prediction labels corresponding to the multiple unlabeled sample product image data. The basic classification model can be represented as g base By g base A preliminary prediction is performed on multiple unlabeled sample product image data to obtain the corresponding predicted labels of multiple unlabeled sample product image data, which can be expressed as

[0063] It is understandable that the basic classification model can be pre-trained. In the industrial AI quality inspection scenario, in addition to providing unlabeled sample product image data, customers also provide a small amount of positive sample product image data with annotated labels and an approximate production line yield. The production line yield can, to a certain extent, reflect the proportion of positive sample product image data in the unlabeled sample product image data. p , then the proportion of negative sample product image data in the unlabeled sample product image data is π n =1-π p In this case, the training sample product image data may also include positive sample product image data with annotated labels and production line yield. In this case, the basic classification model can be trained by using the positive sample product image data, multiple unlabeled sample product image data and production line yield to train the basic classification model. The positive sample product image data is represented by X p , the label of the positive sample product image data is represented by Y p , since the label of the positive sample product image data is known, that is, the corresponding sample product image data is identified as the positive sample product image data, so Y p Typically this can be 1. X p and Y p It constitutes a positive sample product image dataset, which can be expressed as {D p ={X p ,Y p}}.

[0064] Specifically, the initial model g can be trained using non-negative unbiased estimation to obtain a basic classification model g base , in order to use g base To identify the positive and negative sample product image data in the unlabeled sample product image data. The optimization objective formula is as follows:

[0065]

[0066] in, represents expectation, l is an alternative loss function, usually a convex function, and l ensures that the output result is between +1 and -1; It is used to represent the prediction result g(X p ) and 1; It represents the prediction result g(X) of the negative sample product image data in the unlabeled sample product image data using the initial model g u ) and -1; Represents the objective function The value of the variable when it reaches its minimum value.

[0067] When you get Take the minimum value of g * After that, the obtained g * As the final trained basic classification model g base .

[0068] See also Figure 3 As shown, Figure 3 The figure shows an example of a training method for a basic classification model. The basic classification model is obtained by training positive sample product image data with labeled labels and unlabeled sample product image data. During the training process, the basic classification loss function L of the basic classification model is used. base Perform iterative optimization. Among them, L base It can be calculated based on the above optimization objective formula, such as L base It can be expressed as

[0069] It should be noted that the embodiments of the present application do not limit the network structure of the basic classification model, which can be various neural network models, such as convolutional neural networks (CNN), deep neural networks (DNN), etc.

[0070] The embodiment of the present application obtains a basic classification model g by training the PU classification algorithm with non-negative unbiased estimation base , to a certain extent, improve the accuracy of the predicted labels predicted by the basic classification model.

[0071] S203 : Perform secondary prediction on the plurality of unlabeled sample product image data to obtain first prediction results corresponding to the plurality of unlabeled sample product image data.

[0072] Since there may be some unlabeled sample product image data that are difficult to distinguish (with similar features) among the multiple unlabeled sample product images, in order to enable the trained product classification model to accurately distinguish these unlabeled sample product image data, a targeted training strategy can be implemented for the unlabeled sample product image data with different classification difficulties. In order to be able to implement a targeted training strategy for the unlabeled sample product image data with different classification difficulties, it is necessary to divide the unlabeled sample product image data with different classification difficulties, and the basis for division can be the consistency of multiple prediction results. Based on this, the server can also perform secondary predictions on the multiple unlabeled sample product image data to obtain the first prediction results corresponding to the multiple unlabeled sample product image data, so that the multiple unlabeled sample product image data can be divided subsequently according to the consistency between the first prediction result and the prediction label (i.e., the prediction result of the preliminary prediction).

[0073] In a possible implementation, S203 can be implemented by performing secondary predictions on multiple unlabeled sample product image data based on the trained discriminant model to obtain first prediction results corresponding to the multiple unlabeled sample product image data. The discriminant model can be expressed as g dis .

[0074] It is understandable that the discrimination model can be obtained by pre-training. Because in the industrial AI quality inspection scenario, in addition to providing unlabeled sample product image data, the customer will also provide a small amount of positive sample product image data with labeled labels. In this case, the training sample product image data in the embodiment of the present application can also include positive sample product image data with labeled labels. The training method of the discrimination model can be to train the discrimination model using the positive sample product image data with labeled labels, multiple unlabeled sample product image data, and the predicted labels corresponding to multiple unlabeled sample product image data. In the process of training the discrimination model using the positive sample product image data with labeled labels, since the labeled labels of the positive sample product image data are pre-labeled and accurate, supervised training can be performed based on the labeled labels. Specifically, the server can construct a third loss function based on the labeled labels and the positive sample product image data, and train the discrimination model based on the third loss function. Among them, the method of constructing the third loss function based on the labeled labels and the positive sample product image data can be to predict the positive sample product image data through the discrimination model to obtain a sixth prediction result, and then construct a third loss function based on the gap between the sixth prediction result and the labeled labels. In the process of training the discriminant model using the plurality of unlabeled sample product image data and the prediction labels corresponding to the plurality of unlabeled sample product image data, for each unlabeled sample product image data in the plurality of unlabeled sample product image data, the discriminant model outputs a fifth prediction result of the unlabeled sample product image data, and a cross-entropy loss function is constructed based on the fifth prediction result and the corresponding prediction label, so as to train the discriminant model using the cross-entropy loss function. The specific formula of the cross-entropy loss function is as follows:

[0075]

[0076] in, represents the cross entropy loss function, Represents the predicted labels corresponding to multiple unlabeled sample product image data The predicted label corresponding to any unlabeled sample product image data, x u For multiple unlabeled sample product image data X u Any unlabeled sample product image data in g dis (x u) represents the fifth prediction result output by the discriminant model during the training process. When the discriminant model training is completed, the fifth prediction result output by the discriminant model can be used as the first prediction result mentioned above.

[0077] During the training process, a small learning rate is maintained to facilitate the training of the optimal discriminant model. In one possible case, the training end condition of the discriminant model can be set. When the training end condition is met, the training result is determined to obtain a trained discriminant model. For example, during the training process, the discriminant model obtained by training can be verified. When the verification accuracy reaches a threshold value τ, it is determined that the training end condition is met and the training is stopped. dis .

[0078] It should be noted that the embodiments of the present application do not limit the network structure of the discrimination model, which can be various neural network models, such as convolutional neural networks (CNN), deep neural networks (DNN), etc.

[0079] In the embodiment of the present application, the discrimination model is semi-supervisedly trained by combining positive sample product image data with annotated labels, unlabeled sample product image data, and the prediction results (e.g., prediction labels) of the basic classification model, thereby making full use of the information of the unlabeled sample product image data and improving the training performance of the discrimination model. In addition, since the embodiment of the present application uses the prediction results of the basic classification model to train the discrimination model, and the prediction results can distinguish between the positive sample product image data and the negative sample product image data in the unlabeled sample product image data, the negative sample product image data is introduced into the training process of the discrimination model, so that the discrimination model can better learn the characteristics of the negative sample product image data, improve the ability of the discrimination model to distinguish between positive and negative sample product image data, and thus reduce the false detection rate and improve the generalization ability of the discrimination model.

[0080] S204. Divide the multiple unlabeled sample product image data into a first sample product image data set and a second sample product image data set based on the consistency between the predicted label and the first prediction result, wherein the classification difficulty of the first sample product image data in the first sample product image data set is lower than the classification difficulty of the second sample product image data in the second sample product image data set.

[0081] After completing the above two predictions, the consistency of the two prediction results (such as the predicted label and the first prediction result) can reflect the classification difficulty of the unlabeled sample product image data to a certain extent. Generally, the more consistent the predicted label is with the first prediction result, the lower the classification difficulty, and the more inconsistent the predicted label is with the first prediction result, the higher the classification difficulty. Therefore, the server can divide the multiple unlabeled sample product image data into a first sample product image data set and a second sample product image data set according to the consistency of the predicted label and the first prediction result. Among them, the classification difficulty of the first sample product image data in the first sample product image data set is lower than the classification difficulty of the second sample product image data in the second sample product image data set. In this case, the first sample product image data set can be called a simple sample product image data set, which is expressed as The second sample product image dataset can be called a difficult sample product image dataset, expressed as

[0082] The first sample product image data in the first sample product image dataset is relatively easy to classify, meaning it is easy to correctly classify. Each first sample product image data item has only one category, meaning it has only one correct prediction result. This is because, for the first first sample product image data item, its multiple prediction results should be consistent. However, the second sample product image data in the second sample product image dataset is more difficult to classify, meaning it is difficult to correctly classify. Therefore, for each second sample product image data item, its multiple prediction results may be inconsistent, making it difficult to determine its correct classification. Based on this, the method for dividing the multiple unlabeled sample product image data items into the first sample product image dataset and the second sample product image dataset based on the consistency between the predicted label and the first prediction result can be: for each unlabeled sample product image data item, if the first prediction result of the unlabeled sample product image data item is consistent with the corresponding predicted label, the unlabeled sample product image data item is classified into the first sample product image dataset; if the first prediction result of the unlabeled sample product image data item is inconsistent with the corresponding predicted label, the unlabeled sample product image data item is classified into the second sample product image dataset.

[0083] See also Figure 4 As shown, Figure 4An example diagram of the training process of a discrimination model is shown, in which the discrimination model is trained using positive sample product image data with labeled labels, multiple unlabeled sample product image data, and predicted labels corresponding to multiple unlabeled sample product image data. The predicted labels corresponding to the multiple unlabeled sample product image data can be directly output using S202, or a trained basic classification model can be introduced in the process of training the discrimination model, and the basic classification model can be kept unchanged, so as to output the predicted labels corresponding to the multiple unlabeled sample product image data using the basic classification model. Then, a cross-entropy loss function is constructed based on the predicted label and the fifth prediction result output by the discrimination model, and the cross-entropy loss function is used to train the discrimination model. After the discrimination model is trained, the first prediction results corresponding to the multiple unlabeled sample product image data can be input using the trained discrimination model. For each unlabeled sample product image data, if its first prediction result is consistent with the predicted label, it is divided into the first sample product image data set. Otherwise, it is divided into the second sample product image dataset Thus, the separation of difficult and easy sample product image data is achieved. Therefore, the positive sample product image dataset D is finally obtained. p , the first sample product image dataset and the second sample product image dataset

[0084] S205: Train a product classification model using the first sample product image dataset and the second sample product image dataset.

[0085] For the first sample product image dataset and the second sample product image dataset, the embodiment of the present application can use the first sample product image dataset and the second sample product image dataset to train a refined product classification model g fine , the specific details are as follows.

[0086] It should be noted that the embodiments of the present application do not limit the network structure of the product classification model, which can be various neural network models, such as convolutional neural networks (CNN), deep neural networks (DNN), etc.

[0087] S206. In the process of training the product classification model using the first sample product image dataset, a label calibration loss function is constructed based on the first sample product image dataset, and the product classification model is trained based on the label calibration loss function, where the label calibration loss function is used to eliminate noise in the predicted labels.

[0088] Since the embodiment of the present application divides multiple labeled sample product image data into a first sample product image data set and a second sample product image data set with different classification difficulties, and since the classification difficulties of sample product image data in different sample product image data sets are different, and the features used to distinguish sample product image data with different classification difficulties may be different, the focus of learning for sample product image data sets with different classification difficulties during training will also be different. Therefore, in the embodiment of the present application, the server adopts different training strategies for sample product image data sets with different classification difficulties.

[0089] Specifically, in the process of training the product classification model using the first sample product image dataset, since the classification difficulty of the first sample product image data in the first sample product image dataset is very low and the noise influence is small, the predicted label is usually reliable. Occasionally, the predicted label may be unreliable due to the presence of noise. Therefore, in order to avoid the unreliability of the predicted label due to the influence of noise, the server focuses on avoiding the unreliable predicted label caused by noise when using the first sample product image dataset for training. Therefore, the server can construct a label calibration loss function based on the first sample product image dataset, and train the product classification model based on the label calibration loss function. The label calibration loss function is used to eliminate the noise in the predicted label.

[0090] In one possible implementation, the label calibration loss function may be constructed based on the first sample product image dataset by obtaining a second prediction result for each first sample product image data using a product classification model based on each first sample product image data in the first sample product image dataset, and then constructing a label calibration loss function based on the second prediction result and the corresponding predicted label for each first sample product image data. Figure 5 As shown, in Figure 5 The first sample product image data in the first sample product image dataset is directly input into the product classification model, and its label calibration loss function is Figure 5 Not shown in the figure.

[0091] In the process of constructing a label calibration loss function based on the second prediction result and the corresponding prediction label, in order to calibrate the prediction label and minimize the noise influence that may exist in the prediction label, the prediction label can be mixed with the second prediction result output by the product classification model, so that during the training process, the distance between the prediction label and the mixed result is as small as possible, and the distance between the second prediction result and the mixed result is as small as possible, so as to eliminate the noise influence that may exist in the prediction label. Among them, the distance can be various distances, such as cosine distance, KL (Kullback-Leibler) distance, etc. The embodiment of this application mainly introduces KL distance as an example.

[0092] To this end, the formula of the label calibration loss function can be shown as follows:

[0093]

[0094] in, represents the i-th first sample product image data in the first sample product image dataset, g fine () represents the product classification model, Denotes the predicted label corresponding to the i-th first sample product image data in the first sample product image dataset, D KL (A||B) means calculating the KL (Kullback-Leibler) distance between A and B, which can also be called relative entropy, to measure the difference between A and B. Here A is or B is m, and γ is a hyperparameter that can be set according to actual needs.

[0095] In one possible implementation, an adaptive hyperparameter adjustment strategy can be added to the loss function containing hyperparameters during the training process, so that the product classification model can autonomously learn parameters.

[0096] The embodiment of the present application designs a suitable training strategy for the first sample product image data set, which can perform noisy learning on sample product image data with lower classification difficulty, eliminate the noise impact that may exist in the predicted labels, and improve the classification performance of the product classification model.

[0097] S207. In the process of training the product classification model using the second sample product image dataset, a feature consistency loss function is constructed based on the second sample product image dataset, and the product classification model is trained based on the feature consistency loss function. The feature consistency loss function is used to correct feature deviations in feature extraction of the second sample product image data in the second sample product image dataset.

[0098] In the process of training the product classification model using the second sample product image dataset, since the classification of the second sample product image data in the second sample product image dataset is more difficult and the predicted label may have greater uncertainty, the server focuses on training the feature consistency of the second sample product image data when using the second sample product image dataset for training. Therefore, the server can construct a feature consistency loss function based on the second sample product image dataset and train the product classification model based on the feature consistency loss function. The feature consistency loss function is used to correct the feature deviation of feature extraction of the second sample product image data in the second sample product image dataset, thereby realizing training for its feature consistency, exploring the inherent feature differences of the second sample product image data in the second sample product image dataset, and improving the product classification model's ability to distinguish between positive and negative samples.

[0099] It should be noted that S206 and S207 are the specific training processes of S205. S206 and S207 are not executed after S205 is executed, but are executed during the execution of S205. S205 is implemented through the execution of S206 and S207, and the embodiment of this application does not limit the order of S206 and S207.

[0100] When performing feature consistency training on the second sample product image data in the second sample product image dataset, since the product classification model includes multiple convolutional layers, the multiple convolutional layers can serve as the backbone of the product classification model for extracting features. The features extracted by different convolutional layers may be different. Therefore, the consistency training of this application is actually to train the consistency of features of different layers of the second sample product image data. The core idea is to design different loss functions for shallow and deep features. The shallow features are supervised by the cross-consistency loss function (i.e., the first loss function), and the deep features are supervised by the self-supervised consistency loss function (i.e., the second loss function).

[0101] In this case, the method of constructing the feature consistency loss function based on the second sample product image dataset can be to output the first convolution feature through the first convolution layer in the multi-layer convolution layer for each second sample product image data in the second sample product image dataset, and construct the first loss function based on the first convolution feature. The first loss function is the loss function corresponding to the shallow feature, which can be expressed as The deep features use a self-supervised consistency loss function. In order to achieve self-supervision, during the training process, each second sample product image data in the second sample product image data set can be first subjected to different degrees of data enhancement to obtain the first sample enhancement data and the second sample enhancement data corresponding to each second sample product image data. Then, for each second sample product image data, based on the first sample enhancement data and the second sample enhancement data of the second sample product image data, the second convolution layer in the multi-layer convolution layer outputs the second convolution features corresponding to the first sample enhancement data and the second sample enhancement data. In the forward propagation direction of the multi-layer convolution layer, the second convolution layer is located after the first convolution layer, so that the first convolution feature extracted by the first convolution layer can be called a shallow feature, and the second convolution feature extracted by the second convolution layer can be called a deep feature. Afterwards, the second loss function can be constructed based on the second convolution features corresponding to the first sample enhancement data and the second sample enhancement data. Among them, the second loss function is the loss function corresponding to the deep feature, which can be expressed as Next, a feature consistency loss function is constructed based on the first loss function and the second loss function. The construction formula can be shown as follows:

[0102]

[0103] in, represents the feature consistency loss function, represents the first loss function, represents the second loss function.

[0104] It should be noted that the basic classification model can perform basic classification of products based on surface features such as product color and texture. The product classification model needs to be able to perform basic classification of products based on surface features such as product color and texture, and also needs to be able to accurately classify products that are difficult to distinguish based on more complex features. For this reason, the product classification model and the basic classification model have certain similarities. Usually, the backbone of the product classification model and the basic classification model are the same, but the classification layers of the product classification model and the basic classification model are different. Since the backbone of the product classification model and the basic classification model are the same, and both can functionally learn surface features such as product color and texture, and shallow features usually reflect the color and texture of the second sample product image data, the basic classification model has a certain reference significance when designing the loss function for shallow features. Therefore, for shallow features, g base and g fineThe mutual supervision can be performed, that is, the manner of constructing the first loss function based on the first convolutional features can be based on the second sample product image dataset, outputting third convolutional features through a third convolutional layer in the multiple layers of convolutional layers included in the basic classification model, and the relative position of the third convolutional layer in the multiple layers of convolutional layers included in the basic classification model is the same as the relative position of the first convolutional layer in the multiple layers of convolutional layers included in the product classification model. Then, the first loss function is constructed according to the gap between the first convolutional features and the third convolutional features (see Figure 5

[0105] In a possible implementation, the first convolutional layer is the first layer of convolutional layers in the multiple layers of convolutional layers included in the product classification model (see Figure 5

[0106] In this case, the formula of the first loss function can be represented as:

[0107]

[0108] wherein, represents the first loss function, represents the first layer of convolutional layers of the product classification model, represents the first layer of convolutional layers of the basic classification model, represents any second sample product image data in the second sample product image dataset, and ||C||2 represents a norm, where C is

[0109] In the embodiments of the present application, the extraction capability of the shallow features such as color, texture and the like is optimized by the supervision of the basic classification model with better performance, so that the extraction capability of the product classification model for the shallow features of the second sample product image data with greater classification difficulty can be improved.

[0110] It can be understood that, in the process of self-supervision of the deep features, the deep features generally refer to the features extracted through more times of convolution, and in the forward propagation direction in the multiple layers of convolutional layers, the convolutional features output by the convolutional layer at the rear are subjected to more times of convolution. Therefore, in a possible implementation, in order to obtain the deep features, the second convolutional layer can be the last layer of convolutional layers in the multiple layers of convolutional layers included in the product classification model (see Figure 5 ​​502).

[0111] Different degrees of data augmentation can refer to different degrees of data augmentation for the second sample product image data. The degree of data augmentation can include 0, which means that no data augmentation is performed on the second sample product image data. For example, for a second sample product image data E, strong data augmentation is performed on the second sample product image data E to obtain first sample enhanced data, and weak data augmentation is performed on the second sample product image data E to obtain second sample enhanced data. For another example, for a second sample product image data E, data augmentation is performed once on the second sample product image data E to obtain first sample enhanced data, and the second sample product image data E itself is used as the second sample enhanced data.

[0112] Taking different degrees of data enhancement including strong data enhancement and weak data enhancement as an example, in the embodiment of the present application, different degrees of data enhancement are performed on each second sample product image data in the second sample product image data set, which can be respectively performing weak data enhancement and strong data enhancement on each second sample product image data (see Figure 5 As shown). Among them, strong data enhancement can include color change, shearing, blurring, etc., and weak data enhancement can include rotation and mirroring. Strong data enhancement can be done with A s (*) indicates that weak data enhancement can be achieved with A w (*).

[0113] In one possible implementation, for the second sample product image data that is more difficult to classify, the training focus of the product classification model is on training the feature consistency of the second sample product image data. For the second sample product image data that is more difficult to classify, the deep features can better reflect the differences between different second sample product image data, which is conducive to mining the inherent feature differences of the second sample product image data. At the same time, the role of the product classification model is to achieve classification, so its prediction ability should also be the focus of attention during training. Based on this, in a possible implementation, the method of constructing the second loss function based on the second convolutional features corresponding to the first sample enhancement data and the second sample enhancement data can be to construct a feature information loss function based on the gap between the second convolutional features corresponding to the first sample enhancement data and the second sample enhancement data, output a third prediction result through the product classification model based on the first sample enhancement data of the second sample product image data, and output a fourth prediction result through the product classification model based on the second sample enhancement data of the second sample product image data, and then construct a prediction loss function based on the gap between the third prediction result and the fourth prediction result, so as to construct a second loss function based on the feature information loss function and the prediction loss function (see Figure 5 The formula of the second loss function is as follows:

[0114]

[0115] in, represents the second loss function, represents the prediction loss function, Represents the feature information loss function, α and β are harmonic parameters, which can be set according to actual needs.

[0116] In an embodiment of the present application, by training the product classification model through a combination of the prediction loss function and the feature information loss function, it is possible to improve the inherent feature extraction capability of the product classification model for the second sample product image data that is more difficult to classify, and to improve the overall classification capability of the product classification model.

[0117] It should be noted that in the process of constructing the feature information loss function, since the second convolution features corresponding to the first sample enhancement data and the second sample enhancement data may be expressed differently, in order to facilitate the comparison of the gap between the two, the two can be feature converted to obtain an expression that is easy to compare. The gap between the second convolution features corresponding to the first sample enhancement data and the second sample enhancement data can be represented by the distance between the two, such as cosine distance, KL distance, etc. The embodiment of the present application is mainly introduced by taking cosine distance as an example. Based on this, taking the second convolution layer as the last convolution layer in the multi-layer convolution layer included in the product classification model as an example, the formula of the feature information loss function can be as follows:

[0118]

[0119] in, represents the feature information loss function, D(*) represents the cosine distance, A w (*) and A s (*) represents weak data enhancement and strong data enhancement, respectively, f represents feature transformation, represents the last convolutional layer of the product classification model, Represents any second sample product image data in the second sample product image dataset.

[0120] In the process of constructing the prediction loss function, the difference between the third prediction result and the fourth prediction result can be represented by a distance, which can be a cosine distance, a KL distance, etc. The embodiment of this application mainly uses the KL distance as an example. Based on this, the formula of the prediction loss function can be as follows:

[0121]

[0122] in, Denotes the prediction loss function, D KL(A||B) means calculating the KL distance between A and B, where A is B is g fine () represents the product classification model, A w (*) and A s (*) represents weak data enhancement and strong data enhancement, respectively. Represents any second sample product image data in the second sample product image dataset.

[0123] It can be understood that by adopting different training strategies for sample product image data of different classification difficulties through the above steps, the product classification model can better handle the prediction deviations of simple noise and complex sample product image data with greater classification difficulty, thereby improving the accuracy of PU classification.

[0124] In addition, compared with the related art, although the embodiment of the present application adopts the production line yield when training the basic classification model, since the embodiment of the present application only uses the basic classification model as an auxiliary, by dividing the unlabeled sample product image data, sample product image data sets with different classification difficulties are obtained, and then different training strategies are used for training sample product image data sets with different classification difficulties, thereby eliminating possible deviations in the prediction labels output by the basic classification model, and can effectively classify the products to be tested in the industrial AI quality inspection scenario.

[0125] It can be seen from the above technical solution that in industrial quality inspection scenarios, the training sample product image data usually obtained includes a large amount of unlabeled sample product image data. In this case, in order to train a product classification model that can accurately classify the products to be inspected, thereby accurately identifying whether the products to be inspected are defective products or normal products, a preliminary prediction can be performed on multiple unlabeled sample product image data to obtain prediction labels corresponding to the multiple unlabeled sample product image data, and a secondary prediction can be performed on the multiple unlabeled sample product image data to obtain first prediction results corresponding to the multiple unlabeled sample product image data. The consistency between the prediction label and the first prediction result can reflect the classification difficulty of the unlabeled sample product image data to a certain extent. Generally, the more consistent the prediction label is with the first prediction result, the lower the classification difficulty. Therefore, the multiple unlabeled sample product image data can be divided based on the consistency of the prediction label and the first prediction result to obtain a first sample product image data set and a second sample product image data set. The classification difficulty of the first sample product image data in the first sample product image data set is lower than the classification difficulty of the second sample product image data in the second sample product image data set. Then, the product classification model is trained using the first sample product image dataset and the second sample product image dataset. During the training process, different training strategies are adopted for sample product image datasets with different classification difficulties. Specifically, in the process of training the product classification model using the first sample product image dataset, since its classification difficulty is very low, noise may occasionally have an impact. Therefore, a label calibration loss function can be constructed based on the first sample product image dataset, and the product classification model is trained based on the label calibration loss function. The label calibration loss function is used to eliminate noise in the predicted labels; in the process of training the product classification model using the second sample product image dataset, since its classification difficulty is greater, the predicted labels may have greater uncertainty. Therefore, a feature consistency loss function can be constructed based on the second sample product image dataset, and the product classification model is trained based on the feature consistency loss function. The feature consistency loss function is used to correct the feature deviation of feature extraction of the second sample product image data in the second sample product image dataset, thereby achieving training for the consistency of its features, mining the inherent feature differences of the second sample product image data in the second sample product image dataset, and improving the product classification model's ability to distinguish between positive and negative samples. It can be seen that this application adopts different training strategies for sample product image data sets with different classification difficulties during the training process, so as to realize targeted training of sample product image data sets with different classification difficulties, thereby improving the distinguishing ability of the product classification model and improving the accuracy of the classification results of the product classification model.

[0126] In a possible implementation, the size of the product classification model in the training framework can be compressed in a manner of model distillation, so as to improve efficiency. Model distillation provides a training means, which can migrate the knowledge of one or more large models to a small model, facilitate model deployment, and accelerate inference speed. That is, the product classification model adopted by the embodiments of the present application can be a small model, which learns the generalization ability of a large model quickly through model distillation, and the learning of the small model is guided by the large model to a certain extent. The inputs of the large model and the small model are the same, only the model structure of the small model is simpler and lighter than that of the large model, so as to improve efficiency.

[0127] After obtaining the product classification model, it can be used in an industrial AI quality inspection scene. In the industrial AI quality inspection scene, when a product needs to be classified to determine whether it is a normal product or a defective product, the product can be taken as a to-be-detected product, so as to obtain a to-be-detected product image corresponding to the to-be-detected product, input the to-be-detected product image into the product classification model trained by the method provided in the embodiments of the present application, obtain the classification result of the to-be-detected product image through the product classification model, and use the classification result to indicate whether the to-be-detected product image corresponds to a defective product.

[0128] Referring to Figure 6 as shown, Figure 6 Taking the to-be-detected product as leather as an example, Figure 6 Five to-be-detected product images corresponding to the leathers are shown. Through the product classification model, the classification results of each to-be-detected product image are obtained, which are defective product, defective product, defective product, defective product, and normal product in turn.

[0129] Referring to Figure 6 After detecting a large number of leathers, based on the final classification results of each leather, the proportion of defective products and normal products is counted to obtain a statistical result, in which the defective products account for 1%, and the normal products account for 99%.

[0130] When the product classification model is used for product classification, the training method of the product classification model and the product classification method based on the product classification model are often integrated as a sub-module of a defect identification module in the background service of a technical service provider in actual business use. Its upstream modules include a product imaging module and a defect registration module, and its downstream modules include a database module and a data analysis and statistics module. The overall architecture diagram is as follows: Figure 7As shown. Among them, the product imaging module can be used to take pictures of the product to be inspected to obtain the image of the product to be inspected; the defect registration module is mainly used to register the photographed images of the product to be inspected, and align the defects of different images of the product to be inspected to similar positions, so as to obtain an image of the product to be inspected that is easy to identify; the data analysis and statistics module is used to count the number or ratio of defective products and normal products; the database module can be used to record the statistical results. Figure 7 In the figure, the product imaging module shown in 7011 and the defect registration module shown in 7012 are located on the front-end module 701, which can be a client; the defect recognition module shown in 7021, the data analysis and statistics module shown in 7022 and the database module shown in 7023 are located on the back-end module 702.

[0131] It should be noted that, based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods.

[0132] based on Figure 2 In accordance with the training method of the product classification model provided in the embodiment, the present application also provides a training device 800 for the product classification model. Figure 8 The product classification model training device 800 includes an acquisition unit 801, a prediction unit 802, a division unit 803, a training unit 804 and a construction unit 805:

[0133] The acquisition unit 801 is configured to acquire training sample product image data, wherein the training sample product image data includes a plurality of unlabeled sample product image data;

[0134] The prediction unit 802 is configured to perform preliminary prediction on the plurality of unlabeled sample product image data to obtain prediction labels corresponding to the plurality of unlabeled sample product image data;

[0135] The prediction unit 802 is further configured to perform secondary prediction on the plurality of unlabeled sample product image data to obtain first prediction results corresponding to the plurality of unlabeled sample product image data respectively;

[0136] The division unit 803 is configured to divide the plurality of unlabeled sample product image data into a first sample product image dataset and a second sample product image dataset based on the consistency between the predicted label and the first prediction result, wherein the classification difficulty of the first sample product image data in the first sample product image dataset is lower than the classification difficulty of the second sample product image data in the second sample product image dataset;

[0137] The training unit 804 is configured to train a product classification model using the first sample product image dataset and the second sample product image dataset;

[0138] The construction unit 805 is configured to construct a label calibration loss function based on the first sample product image dataset in the process of training the product classification model using the first sample product image dataset;

[0139] The training unit 804 is specifically configured to train the product classification model based on the label calibration loss function, wherein the label calibration loss function is configured to eliminate noise in the predicted labels;

[0140] The construction unit 805 is further configured to construct a feature consistency loss function based on the second sample product image dataset in the process of training the product classification model using the second sample product image dataset;

[0141] The training unit 804 is specifically configured to train the product classification model based on the feature consistency loss function, where the feature consistency loss function is configured to correct feature deviations in feature extraction of the second sample product image data in the second sample product image dataset.

[0142] In a possible implementation, the constructing unit 805 is specifically configured to:

[0143] Based on each first sample product image data in the first sample product image dataset, obtaining a second prediction result for each first sample product image data by using the product classification model;

[0144] A label calibration loss function is constructed according to the second prediction result of each first sample product image data and the corresponding prediction label.

[0145] In a possible implementation, the product classification model includes multiple convolutional layers, and the construction unit 805 is specifically configured to:

[0146] For each second sample product image data in the second sample product image dataset, output a first convolution feature through a first convolution layer in the multi-layer convolution layer, and construct a first loss function based on the first convolution feature;

[0147] performing data enhancement of different degrees on each second sample product image data in the second sample product image dataset to obtain first sample enhanced data and second sample enhanced data corresponding to each second sample product image data;

[0148] For each second sample product image data, based on the first sample enhancement data and the second sample enhancement data of the second sample product image data, output second convolution features corresponding to the first sample enhancement data and the second sample enhancement data, respectively, through a second convolution layer in the multi-layer convolutional layer, where the second convolution layer is located after the first convolution layer in the forward propagation direction of the multi-layer convolutional layer;

[0149] Constructing a second loss function based on the second convolution features corresponding to the first sample enhanced data and the second sample enhanced data respectively;

[0150] The feature consistency loss function is constructed according to the first loss function and the second loss function.

[0151] In a possible implementation, the constructing unit 805 is specifically configured to:

[0152] Based on the second sample product image dataset, outputting a third convolutional feature through a third convolutional layer in the multi-layer convolutional layers included in the basic classification model, where a relative position of the third convolutional layer in the multi-layer convolutional layers included in the basic classification model is the same as a relative position of the first convolutional layer in the multi-layer convolutional layers included in the product classification model;

[0153] The first loss function is constructed according to the gap between the first convolution feature and the third convolution feature.

[0154] In a possible implementation, the first convolutional layer is the first convolutional layer in the multi-layer convolutional layer included in the product classification model, and the third convolutional layer is the first convolutional layer in the multi-layer convolutional layer included in the basic classification model.

[0155] In a possible implementation, the constructing unit 805 is specifically configured to:

[0156] constructing a feature information loss function based on the gap between the second convolution features corresponding to the first sample enhanced data and the second sample enhanced data respectively;

[0157] outputting a third prediction result through the product classification model based on the first sample enhancement data of the second sample product image data, and outputting a fourth prediction result through the product classification model based on the second sample enhancement data of the second sample product image data;

[0158] constructing a prediction loss function based on the gap between the third prediction result and the fourth prediction result;

[0159] The second loss function is constructed according to the feature information loss function and the prediction loss function.

[0160] In a possible implementation, the second convolutional layer is the last convolutional layer in the multi-layer convolutional layers included in the product classification model.

[0161] In a possible implementation, the training sample product image data also includes positive sample product image data with annotated labels and production line yield, and the training unit 804 is further configured to:

[0162] Training a basic classification model using the positive sample product image data, the plurality of unlabeled sample product image data, and the production line yield;

[0163] The prediction unit 802 is specifically configured to:

[0164] Based on the trained basic classification model, preliminary predictions are made on the multiple unlabeled sample product image data to obtain predicted labels corresponding to the multiple unlabeled sample product image data.

[0165] In a possible implementation, the dividing unit 803 is specifically configured to:

[0166] For each unlabeled sample product image data, if the first prediction result of the unlabeled sample product image data is consistent with the corresponding prediction label, the unlabeled sample product image data is divided into the first sample product image data set;

[0167] If the first prediction result of the unlabeled sample product image data is inconsistent with the corresponding prediction label, the unlabeled sample product image data is divided into the second sample product image data set.

[0168] In a possible implementation, the training sample product image data also includes positive sample product image data with annotated labels, and the training unit 804 is further configured to:

[0169] Before performing secondary prediction on the plurality of unlabeled sample product image data to obtain first prediction results respectively corresponding to the plurality of unlabeled sample product image data, the discrimination model is trained using the positive sample product image data with labeled labels, the plurality of unlabeled sample product image data, and the predicted labels respectively corresponding to the plurality of unlabeled sample product image data;

[0170] In the process of training the discrimination model using positive sample product image data with annotated labels, constructing a third loss function based on the annotated labels and the positive sample product image data, and training the discrimination model based on the third loss function;

[0171] In the process of training the discrimination model using the multiple unlabeled sample product image data and the prediction labels respectively corresponding to the multiple unlabeled sample product image data, for each unlabeled sample product image data in the multiple unlabeled sample product image data, outputting a fifth prediction result of the unlabeled sample product image data through the discrimination model, and constructing a cross-entropy loss function based on the fifth prediction result and the corresponding prediction label, so as to train the discrimination model using the cross-entropy loss function;

[0172] The prediction unit 802 is specifically configured to:

[0173] The discriminant model obtained through training is used to perform secondary prediction on the plurality of unlabeled sample product image data to obtain first prediction results corresponding to the plurality of unlabeled sample product image data respectively.

[0174] In a possible implementation, the device further includes a classification unit:

[0175] The acquisition unit 801 is also used to acquire an image of the product to be inspected;

[0176] The classification unit is configured to obtain a classification result of the image of the product to be detected by using the product classification model, wherein the classification result is used to indicate whether the product to be detected corresponding to the image of the product to be detected is a defective product.

[0177] It can be seen from the above technical solution that in industrial quality inspection scenarios, the training sample product image data usually obtained includes a large amount of unlabeled sample product image data. In this case, in order to train a product classification model that can accurately classify the products to be inspected, thereby accurately identifying whether the products to be inspected are defective products or normal products, a preliminary prediction can be performed on multiple unlabeled sample product image data to obtain prediction labels corresponding to the multiple unlabeled sample product image data, and a secondary prediction can be performed on the multiple unlabeled sample product image data to obtain first prediction results corresponding to the multiple unlabeled sample product image data. The consistency between the prediction label and the first prediction result can reflect the classification difficulty of the unlabeled sample product image data to a certain extent. Generally, the more consistent the prediction label is with the first prediction result, the lower the classification difficulty. Therefore, the multiple unlabeled sample product image data can be divided based on the consistency of the prediction label and the first prediction result to obtain a first sample product image data set and a second sample product image data set. The classification difficulty of the first sample product image data in the first sample product image data set is lower than the classification difficulty of the second sample product image data in the second sample product image data set. Then, the product classification model is trained using the first sample product image dataset and the second sample product image dataset. During the training process, different training strategies are adopted for sample product image datasets with different classification difficulties. Specifically, in the process of training the product classification model using the first sample product image dataset, since its classification difficulty is very low, noise may occasionally have an impact. Therefore, a label calibration loss function can be constructed based on the first sample product image dataset, and the product classification model is trained based on the label calibration loss function. The label calibration loss function is used to eliminate noise in the predicted labels; in the process of training the product classification model using the second sample product image dataset, since its classification difficulty is greater, the predicted labels may have greater uncertainty. Therefore, a feature consistency loss function can be constructed based on the second sample product image dataset, and the product classification model is trained based on the feature consistency loss function. The feature consistency loss function is used to correct the feature deviation of feature extraction of the second sample product image data in the second sample product image dataset, thereby achieving training for the consistency of its features, mining the inherent feature differences of the second sample product image data in the second sample product image dataset, and improving the product classification model's ability to distinguish between positive and negative samples. It can be seen that this application adopts different training strategies for sample product image data sets with different classification difficulties during the training process, so as to realize targeted training of sample product image data sets with different classification difficulties, thereby improving the distinguishing ability of the product classification model and improving the accuracy of the classification results of the product classification model.

[0178] The present application also provides a computer device that can execute a product classification model training method. The computer device can be, for example, a terminal, for example, a smartphone:

[0179] Figure 9 The block diagram shows a partial structure of a smart phone provided by an embodiment of the present application. Figure 9 The smartphone includes components such as a radio frequency (RF) circuit 910, a memory 920, an input unit 930, a display unit 940, a sensor 950, an audio circuit 960, a wireless fidelity (WiFi) module 970, a processor 980, and a power supply 990. The input unit 930 may include a touch panel 931 and other input devices 932, the display unit 940 may include a display panel 941, and the audio circuit 960 may include a speaker 961 and a microphone 962. It is understood that Figure 9 The structure of the smartphone shown in the figure does not constitute a limitation on the smartphone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0180] The memory 920 can be used to store software programs and modules. The processor 980 executes the various functional applications and data processing of the smartphone by running the software programs and modules stored in the memory 920. The memory 920 may mainly include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created based on the use of the smartphone (such as audio data, a phone book, etc.). In addition, the memory 920 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0181] The processor 980 is the control center of the smartphone, connecting all components of the smartphone using various interfaces and circuits. It executes the various functions of the smartphone and processes data by running or executing software programs and / or modules stored in the memory 920 and accessing data stored in the memory 920. Optionally, the processor 980 may include one or more processing units; preferably, the processor 980 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 980.

[0182] In this embodiment, the processor 980 in the smartphone may perform the following steps:

[0183] Acquire training sample product image data, wherein the training sample product image data includes a plurality of unlabeled sample product image data;

[0184] Performing preliminary prediction on the plurality of unlabeled sample product image data to obtain prediction labels corresponding to the plurality of unlabeled sample product image data respectively;

[0185] Performing secondary prediction on the plurality of unlabeled sample product image data to obtain first prediction results respectively corresponding to the plurality of unlabeled sample product image data;

[0186] dividing the plurality of unlabeled sample product image data into a first sample product image dataset and a second sample product image dataset based on consistency between the predicted label and the first prediction result, wherein the classification difficulty of the first sample product image data in the first sample product image dataset is lower than the classification difficulty of the second sample product image data in the second sample product image dataset;

[0187] Training a product classification model using the first sample product image dataset and the second sample product image dataset;

[0188] In the process of training the product classification model using the first sample product image dataset, constructing a label calibration loss function based on the first sample product image dataset, and training the product classification model based on the label calibration loss function, wherein the label calibration loss function is used to eliminate noise in the predicted labels;

[0189] In the process of training the product classification model using the second sample product image dataset, a feature consistency loss function is constructed based on the second sample product image dataset, and the product classification model is trained based on the feature consistency loss function. The feature consistency loss function is used to correct the feature deviation of feature extraction of the second sample product image data in the second sample product image dataset.

[0190] The computer device provided in the embodiment of the present application may also be a server, see Figure 10 As shown, Figure 10The server 1000 provided in the embodiments of the present application can have great differences due to different configurations or performances. The server 1000 can include one or more processors, for example, a central processing unit (CPU) 1022, and a memory 1032, one or more storage media 1030 (for example, one or more mass storage devices) storing application programs 1042 or data 1044. The memory 1032 and the storage media 1030 can be temporary storage or persistent storage. The programs stored in the storage media 1030 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the server. Further, the central processing unit 1022 can be configured to communicate with the storage media 1030 and execute the series of instruction operations in the storage media 1030 on the server 1000.

[0191] The server 1000 can also include one or more power supplies 1026, one or more wired or wireless network interfaces 1050, one or more input / output interfaces 1058, and / or one or more operating systems 1041, for example, Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM , and the like.

[0192] In the embodiments, the central processing unit 1022 in the server 1000 can perform the following steps:

[0193] Obtaining training sample product image data, the training sample product image data including a plurality of unlabeled sample product image data;

[0194] Performing preliminary prediction on the plurality of unlabeled sample product image data to obtain predicted labels corresponding to the plurality of unlabeled sample product image data, respectively;

[0195] Performing secondary prediction on the plurality of unlabeled sample product image data to obtain first prediction results corresponding to the plurality of unlabeled sample product image data, respectively;

[0196] Dividing the plurality of unlabeled sample product image data based on consistency of the predicted labels and the first prediction results to obtain a first sample product image data set and a second sample product image data set, a classification difficulty of first sample product image data in the first sample product image data set being lower than a classification difficulty of second sample product image data in the second sample product image data set;

[0197] Training a product classification model using the first sample product image dataset and the second sample product image dataset;

[0198] In the process of training the product classification model using the first sample product image dataset, constructing a label calibration loss function based on the first sample product image dataset, and training the product classification model based on the label calibration loss function, wherein the label calibration loss function is used to eliminate noise in the predicted labels;

[0199] In the process of training the product classification model using the second sample product image dataset, a feature consistency loss function is constructed based on the second sample product image dataset, and the product classification model is trained based on the feature consistency loss function. The feature consistency loss function is used to correct the feature deviation of feature extraction of the second sample product image data in the second sample product image dataset.

[0200] According to one aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the training method of the product classification model described in the aforementioned embodiments.

[0201] According to one aspect of the present application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in various optional implementations of the above-described embodiments.

[0202] The descriptions of the processes or structures corresponding to the above figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0203] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0204] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0205] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0206] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0207] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store computer programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0208] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, ordinary technical members in this field should understand that they can still modify the technical solutions recorded in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A training method for a product classification model, characterized in that: The method comprises: Acquire training sample product image data, wherein the training sample product image data includes a plurality of unlabeled sample product image data, positive sample product image data with labeled labels, and production line yield; Performing preliminary predictions on the plurality of unlabeled sample product image data using a basic classification model to obtain predicted labels corresponding to the plurality of unlabeled sample product image data, wherein the basic classification model is trained using the positive sample product image data, the plurality of unlabeled sample product image data, and the production line yield; Performing secondary prediction on the plurality of unlabeled sample product image data using a discriminant model to obtain first prediction results corresponding to the plurality of unlabeled sample product image data, wherein the discriminant model is trained using the positive sample product image data, the plurality of unlabeled sample product image data, and the prediction labels corresponding to the plurality of unlabeled sample product image data; dividing the plurality of unlabeled sample product image data into a first sample product image dataset and a second sample product image dataset based on consistency between the predicted label and the first prediction result, wherein the classification difficulty of the first sample product image data in the first sample product image dataset is lower than the classification difficulty of the second sample product image data in the second sample product image dataset; Training a product classification model using the first sample product image dataset and the second sample product image dataset; In the process of training the product classification model using the first sample product image dataset, constructing a label calibration loss function based on the first sample product image dataset, and training the product classification model based on the label calibration loss function, wherein the label calibration loss function is used to eliminate noise in the predicted labels; In the process of training the product classification model using the second sample product image dataset, a feature consistency loss function is constructed based on the second sample product image dataset, and the product classification model is trained based on the feature consistency loss function. The feature consistency loss function is used to correct the feature deviation of feature extraction of the second sample product image data in the second sample product image dataset.

2. The method according to claim 1, characterized in that The constructing of a label calibration loss function based on the first sample product image dataset includes: Based on each first sample product image data in the first sample product image dataset, obtaining a second prediction result for each first sample product image data by using the product classification model; A label calibration loss function is constructed according to the second prediction result of each first sample product image data and the corresponding prediction label.

3. The method according to claim 1, characterized in that The product classification model includes multiple convolutional layers, and constructing a feature consistency loss function based on the second sample product image dataset includes: For each second sample product image data in the second sample product image dataset, output a first convolution feature through a first convolution layer in the multi-layer convolution layer, and construct a first loss function based on the first convolution feature; performing data enhancement of different degrees on each second sample product image data in the second sample product image dataset to obtain first sample enhanced data and second sample enhanced data corresponding to each second sample product image data; For each second sample product image data, based on the first sample enhancement data and the second sample enhancement data of the second sample product image data, output second convolution features corresponding to the first sample enhancement data and the second sample enhancement data, respectively, through a second convolution layer in the multi-layer convolutional layer, where the second convolution layer is located after the first convolution layer in the forward propagation direction of the multi-layer convolutional layer; Constructing a second loss function based on the second convolution features corresponding to the first sample enhanced data and the second sample enhanced data respectively; The feature consistency loss function is constructed according to the first loss function and the second loss function.

4. The method according to claim 3, characterized in that The constructing a first loss function based on the first convolutional feature includes: Based on the second sample product image dataset, outputting a third convolutional feature through a third convolutional layer in the multi-layer convolutional layers included in the basic classification model, where a relative position of the third convolutional layer in the multi-layer convolutional layers included in the basic classification model is the same as a relative position of the first convolutional layer in the multi-layer convolutional layers included in the product classification model; The first loss function is constructed according to the gap between the first convolution feature and the third convolution feature.

5. The method according to claim 4, characterized in that The first convolutional layer is the first convolutional layer among the multiple convolutional layers included in the product classification model, and the third convolutional layer is the first convolutional layer among the multiple convolutional layers included in the basic classification model.

6. The method according to claim 3, characterized in that The constructing a second loss function based on the second convolution features corresponding to the first sample enhancement data and the second sample enhancement data respectively includes: constructing a feature information loss function based on the gap between the second convolution features corresponding to the first sample enhanced data and the second sample enhanced data respectively; outputting a third prediction result through the product classification model based on the first sample enhancement data of the second sample product image data, and outputting a fourth prediction result through the product classification model based on the second sample enhancement data of the second sample product image data; constructing a prediction loss function based on the gap between the third prediction result and the fourth prediction result; The second loss function is constructed according to the feature information loss function and the prediction loss function.

7. The method according to any one of claims 3 to 6, characterized in that: The second convolutional layer is the last convolutional layer in the multi-layer convolutional layer included in the product classification model.

8. The method according to any one of claims 1 to 6, characterized in that The dividing the plurality of unlabeled sample product image data based on the consistency between the predicted label and the first prediction result to obtain a first sample product image dataset and a second sample product image dataset includes: For each unlabeled sample product image data, if the first prediction result of the unlabeled sample product image data is consistent with the corresponding prediction label, the unlabeled sample product image data is divided into the first sample product image data set; If the first prediction result of the unlabeled sample product image data is inconsistent with the corresponding prediction label, the unlabeled sample product image data is divided into the second sample product image data set.

9. The method according to any one of claims 1 to 6, characterized in that Before performing secondary prediction on the plurality of unlabeled sample product image data to obtain first prediction results respectively corresponding to the plurality of unlabeled sample product image data, the method further includes: Training the discrimination model using the positive sample product image data with the labeled labels, the plurality of unlabeled sample product image data, and the predicted labels corresponding to the plurality of unlabeled sample product image data; In the process of training the discrimination model using positive sample product image data with annotated labels, constructing a third loss function based on the annotated labels and the positive sample product image data, and training the discrimination model based on the third loss function; In the process of training the discrimination model using the multiple unlabeled sample product image data and the predicted labels corresponding to the multiple unlabeled sample product image data, for each unlabeled sample product image data in the multiple unlabeled sample product image data, the discrimination model outputs a fifth prediction result of the unlabeled sample product image data, and a cross-entropy loss function is constructed based on the fifth prediction result and the corresponding prediction label to train the discrimination model using the cross-entropy loss function.

10. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Obtain an image of the product to be inspected; A classification result of the product image to be inspected is obtained through the product classification model, and the classification result is used to indicate whether the product to be inspected corresponding to the product image to be inspected is a defective product.

11. A training device for a product classification model, characterized in that: The device includes an acquisition unit, a prediction unit, a division unit, a training unit and a construction unit: The acquisition unit is configured to acquire training sample product image data, wherein the training sample product image data includes a plurality of unlabeled sample product image data, positive sample product image data with labeled labels, and production line yield; The prediction unit is configured to perform preliminary prediction on the plurality of unlabeled sample product image data using a basic classification model to obtain prediction labels corresponding to the plurality of unlabeled sample product image data, wherein the basic classification model is trained using the positive sample product image data, the plurality of unlabeled sample product image data, and the production line yield; The prediction unit is further configured to perform secondary prediction on the plurality of unlabeled sample product image data using a discriminant model to obtain first prediction results corresponding to the plurality of unlabeled sample product image data, wherein the discriminant model is trained using the positive sample product image data, the plurality of unlabeled sample product image data, and the prediction labels corresponding to the plurality of unlabeled sample product image data; The dividing unit is configured to divide the plurality of unlabeled sample product image data into a first sample product image dataset and a second sample product image dataset based on consistency between the predicted label and the first prediction result, wherein the classification difficulty of the first sample product image data in the first sample product image dataset is lower than the classification difficulty of the second sample product image data in the second sample product image dataset; The training unit is configured to train a product classification model using the first sample product image dataset and the second sample product image dataset; The construction unit is configured to construct a label calibration loss function based on the first sample product image dataset in a process of training the product classification model using the first sample product image dataset; The training unit is specifically configured to train the product classification model based on the label calibration loss function, wherein the label calibration loss function is configured to eliminate noise in the predicted labels; The construction unit is further configured to construct a feature consistency loss function based on the second sample product image dataset in a process of training the product classification model using the second sample product image dataset; The training unit is specifically used to train the product classification model based on the feature consistency loss function, and the feature consistency loss function is used to correct feature deviations in feature extraction of the second sample product image data in the second sample product image dataset.

12. A computer device, characterized in that: The computer device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1 to 10 according to instructions in the program code.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program codes, which, when executed by a processor, enable the processor to perform the method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images