Training sample checking method and device, equipment, medium and product

Abstract

The application discloses a training sample investigation method and device, equipment, medium and product, and the method comprises the following steps: obtaining a feature vector of a training sample in a training data set by using a classification model and generating a probability vector by mapping the feature vector to each classification label; calculating a quantitative score representing the classification uncertainty of each unlabeled sample based on the probability vector, and screening out the unlabeled samples with the quantitative score exceeding a preset score threshold to form a coarse recall set; screening out relatively dissimilar unlabeled samples from the most similar unlabeled sample set formed by the corresponding labeled samples in the coarse recall set to form a secondary recall set; and screening out relatively dissimilar unlabeled samples from the most similar unlabeled sample set formed by the corresponding class center vectors in the secondary recall set according to the similarity between the feature vector of the unlabeled sample and the class center vector corresponding to each classification label, and taking the relatively dissimilar unlabeled samples as samples to be labeled. The application can realize the selection of training samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data mining, and in particular to a training sample investigation method and a corresponding device, computer equipment, computer readable storage medium, and computer program product. BACKGROUND

[0002] In recent years, due to the rapid development of deep learning neural network technology, artificial intelligence technology has been increasingly applied to practical production in various industries. Before a deep learning model is put into use, a large amount of labeled data needs to be prepared, so a data labeling team may need to be hired to label data samples for training business models. In practice, due to the large amount of data, it is very labor-intensive for the labeling team to label all the data. Although the number of labeling personnel can be increased to solve the problem, the corresponding high labor cost is brought to the enterprise.

[0003] Furthermore, there may be uneven data distribution in business data. The frequency of occurrence of some categories of data may be higher than that of some other categories. At this time, the labeling team spends most of its effort on labeling through sample sampling methods, and as a result, samples with low frequency of occurrence and high contribution to the model are rarely labeled. A typical example is the product data of an e-commerce platform. The e-commerce platform has its own classification system, and the number of products corresponding to the end classification labels such as "clothes", "shoes", "corner code", and "toilet cover" is usually exponential. Uneven distribution of business data makes it difficult for the model to learn the ability to generalize semantic features when using these business data as training samples for the neural network model, thereby adversely affecting the effective and rapid convergence of the model and indirectly increasing the training cost of the model.

[0004] In another aspect, the training method of the neural network model not only includes supervised training, but also includes semi-supervised training and unsupervised training. When semi-supervised training is performed, a relatively small number of labeled samples and a large number of unlabeled samples are usually provided. After different unlabeled samples are represented and classified by the model, the corresponding output information contains the differences between the samples. These differences can be used for data mining to select the best samples. Based on the selected unlabeled samples, further artificial labeling can convert them into labeled samples. Through the selection and reuse of these model-ambiguous unlabeled samples, the generalization ability of the training samples for the model can often be targetedly improved. For this purpose, although there are corresponding ideas in the prior art, effective implementation schemes are rarely seen. In view of this, the present applicant attempts to make corresponding explorations. SUMMARY

[0005] The primary purpose of the present application is to solve at least one of the above problems to provide a training sample investigation method and its corresponding device, computer equipment, computer readable storage medium, computer program product.

[0006] To meet the various purposes of the present application, the present application adopts the following technical solutions:

[0007] A training sample investigation method provided to adapt to one of the purposes of the present application, comprising the following steps:

[0008] The classification model is used to obtain the feature vector of the training sample in the training data set and the probability vector generated by mapping the feature vector to each classification label; the training sample includes labeled samples and unlabeled samples, the classification model is trained to a convergence state in advance with the labeled samples, and the probability vector contains the classification probability corresponding to each classification label;

[0009] Based on the probability vector, the quantitative score representing the classification uncertainty of each unlabeled sample is calculated, and the unlabeled samples with a quantitative score exceeding a preset score threshold are filtered out from the training data set to form a rough recall set;

[0010] According to the similarity between the feature vectors, the relatively dissimilar unlabeled samples are filtered out from the most similar unlabeled sample set consisting of the corresponding labeled samples in the rough recall set to form a secondary recall set;

[0011] According to the similarity between the feature vector of the unlabeled sample and the class center vector corresponding to each classification label, the relatively dissimilar unlabeled samples are filtered out from the most similar unlabeled sample set consisting of the corresponding class center vector in the secondary recall set as the samples to be labeled; the class center vector is the mean vector between the feature vectors of all labeled samples under the same classification label.

[0012] In the specific embodiment, the classification model performs the following steps:

[0013] The feature vector representing the deep semantic information of the training sample input therein is extracted; the training sample is a picture or text;

[0014] The feature vector is mapped to a preset classification space to obtain the corresponding probability vector.

[0015] In the specific embodiment, based on the probability vector, the quantitative score representing the classification uncertainty of each unlabeled sample is calculated, comprising the following steps:

[0016] The information entropy of the unlabeled sample in the training data set is calculated;

[0017] Calculate the probability difference between the two maximum classification probabilities in the probability vector of the unlabeled samples in the training dataset;

[0018] The product of the information entropy and the probability difference for each unlabeled sample in the training dataset is calculated as a quantitative score characterizing the classification uncertainty of that unlabeled sample.

[0019] In a specific embodiment, based on the similarity between feature vectors, relatively dissimilar unlabeled samples are selected from the set of unlabeled samples that are most similar to the corresponding labeled samples in the coarse recall set to form a secondary recall set, including the following steps:

[0020] Calculate the cosine similarity between the feature vector of each unlabeled sample in the coarse recall set and the feature vector of each labeled sample in the training dataset to obtain a single-sample similarity sequence corresponding to each unlabeled sample. The single-sample similarity sequence contains similarity values ​​corresponding to each of the labeled samples to represent similarity.

[0021] For each unlabeled sample, the similarity sequence is filtered, and only the maximum similarity value is retained. The maximum similarity value indicates that the corresponding unlabeled sample is most similar to its corresponding labeled sample. All the retained similarity values ​​are constructed into a summary similarity sequence.

[0022] The similarity values ​​in the aggregated similarity sequence are filtered according to a preset similarity threshold, and only elements with similarity values ​​lower than the similarity threshold are retained. The unlabeled samples corresponding to the retained elements are then used to construct a secondary recall set.

[0023] In a specific embodiment, based on the similarity between the feature vectors of unlabeled samples and the class center vectors corresponding to each classification label, relatively dissimilar unlabeled samples are selected from the set of unlabeled samples that are most similar to the corresponding class center vectors in the secondary recall set as samples to be labeled, including the following steps:

[0024] Determine the labeled samples corresponding to each category label, sum the feature vectors of these labeled samples element by element and calculate the mean value to obtain the corresponding mean vector, which is used as the class center vector corresponding to that category label.

[0025] Calculate the Euclidean distance between the feature vector of each unlabeled sample in the secondary recall set and the class center vector of each classification label to obtain a single-sample distance sequence corresponding to each unlabeled sample. The single-sample distance sequence contains distance values ​​corresponding to each class center vector to characterize similarity.

[0026] For each unlabeled sample, the single-sample distance sequence is filtered, and only the minimum distance value is retained. The minimum distance value indicates that the corresponding unlabeled sample is most similar to its corresponding labeled sample. All retained distance values ​​are constructed into a summary distance sequence.

[0027] The distance values ​​in the aggregated distance sequence are filtered according to a preset distance threshold, and only elements with distance values ​​higher than the distance threshold are retained. The unlabeled samples corresponding to the retained elements are used as samples to be labeled.

[0028] In an extended embodiment, the training sample screening method of this application further includes the following post-step:

[0029] Obtain the artificial labels corresponding to the sample to be labeled, and convert the sample to be labeled into a labeled sample;

[0030] The labeled samples converted from the unlabeled samples and the labeled samples in the training dataset are combined to form the second training dataset;

[0031] The training task is initiated, and the second training dataset is used to restart the fine-tuning training of the classification model, so that the classification model is trained to a convergent state.

[0032] A training sample screening device provided for one of the purposes of this application includes: a basic module, a coarse recall module, a secondary recall module, and a fine recall module. The basic module is used to obtain feature vectors of training samples in the training dataset and probability vectors generated by mapping these feature vectors to various classification labels using a classification model. The training samples include labeled samples and unlabeled samples. The classification model is pre-trained to convergence using the labeled samples, and the probability vectors contain the classification probabilities corresponding to each classification label. The coarse recall module is used to calculate a quantitative score representing the classification uncertainty of each unlabeled sample based on the probability vectors, and to screen samples from the training dataset. Unlabeled samples whose quantitative scores exceed a preset scoring threshold constitute a coarse recall set. The secondary recall module is used to select relatively dissimilar unlabeled samples from the coarse recall set that are most similar to the corresponding labeled samples based on the similarity between feature vectors, thus forming a secondary recall set. The fine recall module is used to select relatively dissimilar unlabeled samples from the secondary recall set that are most similar to the corresponding class center vectors based on the similarity between the feature vectors of the unlabeled samples and the class center vectors corresponding to each classification label, thus forming a secondary recall set. The class center vector is the mean vector among the feature vectors of all labeled samples under the same classification label.

[0033] In a specific embodiment, the classification model includes: a feature extraction network for extracting feature vectors representing deep semantic information from the input training samples; the training samples are images or text; and a classifier for mapping the feature vectors to a preset classification space to obtain corresponding probability vectors.

[0034] In a specific embodiment, the coarse recall module includes: an information entropy calculation unit for calculating the information entropy of unlabeled samples in the training dataset; a probability difference calculation unit for calculating the probability difference between the two maximum classification probabilities in the probability vector of unlabeled samples in the training dataset; and a scoring calculation unit for calculating the product of the information entropy and the probability difference for each unlabeled sample in the training dataset as a quantitative score characterizing the classification uncertainty of the unlabeled sample.

[0035] In a specific embodiment, the coarse recall module includes: a similarity calculation unit, used to calculate the cosine similarity between the feature vector of each unlabeled sample in the coarse recall set and the feature vector of each labeled sample in the training dataset, to obtain a single-sample similarity sequence corresponding to each unlabeled sample, the single-sample similarity sequence containing similarity values ​​corresponding to each labeled sample to characterize similarity; a most similarity filtering unit, used to filter the similarity sequence corresponding to each unlabeled sample, retaining only the maximum similarity value, the maximum similarity value indicating that its corresponding unlabeled sample is most similar to its corresponding labeled sample, and constructing a summary similarity sequence from all retained similarity values; and a low similarity selection unit, used to filter the similarity values ​​in the summary similarity sequence according to a preset similarity threshold, retaining only elements whose similarity values ​​are lower than the similarity threshold, and constructing a secondary recall set from the unlabeled samples corresponding to the retained elements.

[0036] In a specific embodiment, the secondary recall module includes: a classification averaging unit, used to determine the labeled samples corresponding to each classification label, and to sum the feature vectors of these labeled samples element by element to obtain the corresponding mean vector, which serves as the class center vector corresponding to the classification label; a distance calculation unit, used to calculate the Euclidean distance between the feature vector of each unlabeled sample in the secondary recall set and the class center vector of each classification label, to obtain a single-sample distance sequence corresponding to each unlabeled sample, the single-sample distance sequence containing distance values ​​corresponding to each of the class center vectors for characterizing similarity; a closest filtering unit, used to filter the single-sample distance sequence corresponding to each unlabeled sample, retaining only the minimum distance value, the minimum distance value indicating that its corresponding unlabeled sample is most similar to its corresponding labeled sample, and constructing a summary distance sequence from all the retained distance values; and a high similarity selection unit, used to filter the distance values ​​in the summary distance sequence according to a preset distance threshold, retaining only the elements whose distance values ​​are higher than the distance threshold, and using the unlabeled samples corresponding to the retained elements as samples to be labeled.

[0037] In an extended embodiment, the training sample screening method of this application further includes: a label acquisition module, used to acquire the artificial labels corresponding to the sample to be labeled, so that the sample to be labeled is converted into a labeled sample; a dataset reconstruction module, used to merge the labeled samples converted from the sample to be labeled and the labeled samples in the training dataset to form a second training dataset; and a training restart module, used to enable the training task, call the second training dataset to restart the fine-tuning training of the classification model, so that the classification model is trained to a convergent state.

[0038] A computer device provided for one of the purposes of this application includes a central processing unit and a memory, wherein the central processing unit is configured to invoke and run a computer program stored in the memory to perform the steps of the training sample screening method described in this application.

[0039] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the training sample screening method, which, when invoked by a computer, performs the steps included in the method.

[0040] A computer program product provided for another purpose of this application includes a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0041] Compared with existing technologies, the advantages of this application are as follows: In the process of screening and selecting training samples in the training dataset to determine the samples to be labeled, this application extracts the feature vectors and probability vectors of each training sample based on the classification model trained on the training dataset. Based on this information, data mining processing is carried out in three stages: coarse recall, secondary recall, and finally fine recall. This process is carried out step by step to achieve a fine screening of unlabeled samples in the training dataset. Unlabeled samples that can be used to improve the feature generalization ability of the neural network model during training are identified and manually labeled. These samples are then converted into labeled samples by manual labeling and re-input into the training of the neural network model. This makes it easier for the trained neural network model to be trained to a convergent state and effectively avoids excessive manual labeling. This ensures that while achieving high training results, the model training cost remains low. Attached Figure Description

[0042] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0043] Figure 1 This is a flowchart illustrating a typical embodiment of the training sample screening method of this application;

[0044] Figure 2 This is a schematic diagram of the network architecture of an exemplary classification model in this application;

[0045] Figure 3 This is a flowchart illustrating the quantitative scoring process for determining the uncertainty of unlabeled samples in an embodiment of this application.

[0046] Figure 4 This is a flowchart illustrating the secondary recall process in an embodiment of this application.

[0047] Figure 5 This is a flowchart illustrating the implementation of the precise recall process in the embodiments of this application;

[0048] Figure 6 This is a flowchart illustrating the triggering process for restarting fine-tuning training in an embodiment of this application;

[0049] Figure 7 This is a schematic diagram illustrating the application principle of the training sample screening method of this application;

[0050] Figure 8 This is a schematic diagram of the training sample screening device of this application;

[0051] Figure 9 This is a schematic diagram of the structure of a computer device used in this application. Detailed Implementation

[0052] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0053] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0054] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0055] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0056] The hardware referred to by the names "server," "client," and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0057] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0058] One or more of the technical features of this application, unless explicitly specified herein, can be deployed on a server and accessed by a client remotely calling the online service interface provided by the server, or can be directly deployed and run on a client for access.

[0059] Unless otherwise specified, the neural network models referenced or potentially referenced in this application may be deployed on a remote server and invoked remotely on the client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on the client, the corresponding intelligence may be acquired through transfer learning in order to reduce the requirements on the client's hardware resources and avoid excessive consumption of the client's hardware resources.

[0060] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.

[0061] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0062] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0063] The training sample screening method of this application can be programmed into a computer program product and deployed on a client or server. For example, in the e-commerce platform application scenario of this application, it is generally deployed on a server. In this way, the method can be executed by human-computer interaction with the process of the computer program product through a graphical user interface by accessing the interface opened after the computer program product is run.

[0064] One exemplary application scenario of this application is an application in an e-commerce platform based on independent websites. Each independent website is a merchant instance of the e-commerce platform, with its own independent access domain name, and its actual owner is responsible for publishing and updating products.

[0065] Each merchant instance on an independent website lists its products online. After the e-commerce platform obtains information related to the product, it constructs a corresponding product object for data storage. The product object's information mainly includes text and image information. The text information includes the product object's prominent title, detailed product descriptions, and attribute information describing the product's characteristics.

[0066] To address the need for e-commerce platforms to categorize their products, summary text and product images can be collected as training samples in the training dataset. These samples, combined with appropriately manually labeled tags, are used to semi-supervised training of a neural network classification model serving the e-commerce platform. This allows the model to learn to classify products based on their corresponding text or images. The summary text primarily consists of the product's title, content, and one or more attribute information, the composition of which can be flexibly determined by those skilled in the art. Product images can be the main images depicting the product's appearance. Generally, summary text and product images can be obtained according to preset quantity and content requirements. For example, the summary text can be defined as including the product's title and all its attribute information, and the product image can be a single image used for default display. Of course, those skilled in the art can flexibly adjust and modify these settings.

[0067] Furthermore, the application scenarios of the technical solution of this application are not limited to the neural network models required by e-commerce platforms. In fact, in any field where semi-supervised training of neural network models is carried out, as long as it is necessary to select the best unlabeled samples in the training dataset used by the neural network model for further manual annotation, the technical solution of this application can theoretically be used. Therefore, the examples of application scenarios in this application are only provided for the convenience of the reader's understanding, and those skilled in the art should know that the scope of the inventive spirit of this application should not be limited by the exemplary application scenarios of this application.

[0068] Please see Figure 1 The training sample screening method of this application, in its typical embodiment, includes the following steps:

[0069] Step S1100: Use a classification model to obtain the feature vectors of the training samples in the training dataset and the probability vectors generated by mapping the feature vectors to each classification label; the training samples include labeled samples and unlabeled samples, the classification model is pre-trained to convergence using the labeled samples, and the probability vectors contain the classification probabilities corresponding to each classification label.

[0070] The exemplary classification model in this application, as described above, can be a classification model used to classify product objects based on images and / or text on an e-commerce platform. For ease of understanding, please refer to [link / reference needed]. Figure 2 The network architecture shown illustrates that this classification model includes a feature extraction network and a classifier. The feature extraction network is used to learn representations of the input images and / or text, extract their deep semantic information, and obtain corresponding feature vectors. The classifier is used to map the feature vectors to a preset classification space, which contains multiple preset classification labels, thereby obtaining a corresponding classification probability for each classification label. These classification probabilities constitute a probability vector.

[0071] The classification model is pre-trained to convergence using a training dataset. This training dataset contains a large number of training samples, some of which are pre-labeled samples, while the remainder are unlabeled samples. In other words, the training dataset contains a portion with supervised labels and a portion without. Therefore, the classification model can be adapted for semi-supervised training.

[0072] Accordingly, the classification model operates according to the following process when it is being trained, or when it is put into production after being trained to a convergent state:

[0073] First, feature vectors representing the deep semantic information of the input training samples are extracted:

[0074] The feature extraction network can be an image feature extraction network specifically for image processing, such as ResNet, IresNet, and EfficientNet. The feature vector extracted by this network is directly output to the classifier for classification prediction. Alternatively, it can be a basic network specifically for text processing, such as TextCNN, FastText, BERT, Albert, ERNIE, and ELECTRA. Similarly, the feature vector extracted by this network can also be directly output to the classifier for classification prediction. Another option is a network jointly constructed by the image feature extraction network and the text feature extraction model, where the feature vectors of both are concatenated or fused in the last layer to obtain the final feature vector input to the classifier. The specific configuration of the feature extraction network depends on the composition of the training samples in the training dataset of the classification model of this application. That is, the training samples adapted to the classification model can include only images or text, or simultaneously include images and text related to the same task. After feature extraction by the classification model, the hidden layer vector obtained in the last layer of the training samples is the feature vector used for classification prediction.

[0075] Secondly, the feature vector is mapped to a predefined classification space to obtain the corresponding probability vector:

[0076] After the feature vector is output, it is fed into a classifier through a fully connected layer, thereby mapping it to a preset classification space. This classification space contains multiple preset classification labels. Therefore, after the feature vector is mapped to each classification label, a corresponding classification probability is obtained, and all the classification probabilities constitute a probability vector.

[0077] Based on the above description of the working process of the classification model of this application, it can be understood that in the process of processing the training samples using the classification model, two products are obtained: one is the feature vector corresponding to the training sample, and the other is the probability vector obtained by performing a classification mapping based on the feature vector.

[0078] In order to implement the technical solution of this application, after the classification model is trained to convergence using the labeled samples in the training dataset, the classification model is still used to extract features and classify each training sample in the training dataset, including labeled samples and unlabeled samples. Thus, the feature vector and probability vector of each training sample can be obtained accordingly. These data are stored together for subsequent steps.

[0079] For ease of explanation later, the classification model is represented by the formula: M, and the training dataset is represented by X. labelLet c be the number of class labels in the classification space, and pc be the probability that a training sample belongs to class c.

[0080] Calculate the feature vectors of the unlabeled samples. Assume the set of unlabeled samples in the training dataset is X. un X un，i Let M be one of the unlabeled samples in the unlabeled sample set. If this unlabeled sample is input into the classification model M, then the feature vector output by the last hidden layer is:

[0081] V h，i =M(X) un，i )

[0082] The output layer vector of the classification model, i.e., the probability vector, is represented as follows:

[0083]

[0084] Where subsequent embodiments in this article are described using formulas, please refer to the relevant formula representations here for understanding.

[0085] Step S1200: Calculate a quantitative score representing the classification uncertainty of each unlabeled sample based on the probability vector, and select unlabeled samples with quantitative scores exceeding a preset score threshold from the training dataset to form a coarse recall set.

[0086] To achieve coarse recall of unlabeled samples in the training dataset, this application designs an uncertainty-based quantification method. This method quantifies the uncertainty of unlabeled samples that are difficult for the classification model to identify during model classification, and performs initial recall based on this uncertainty. To this end, two uncertainty calculation methods are combined: information entropy and the maximum boundary method. Based on the classification probability in the probability vector of each unlabeled sample, a quantification score representing the uncertainty of unlabeled samples that the model cannot distinguish is calculated. Information entropy represents the degree of disorder in the output of the classification model when classifying each unlabeled sample, i.e., uncertainty. The maximum boundary method also plays a role in uncertainty calculation, focusing only on the difference between the two classification labels with the highest probability values. Combining these two methods calculates the quantification score corresponding to each unlabeled sample, considering the uncertainty of unlabeled samples from multiple dimensions, thereby expanding the range and diversity of samples that the model cannot distinguish.

[0087] After calculating the quantization score of each unlabeled sample, a preset scoring threshold is used for comparison. When the quantization score exceeds the preset scoring threshold, the corresponding unlabeled sample is recalled to construct a coarse recall set. Therefore, the final coarse recall set contains all unlabeled samples in the training dataset whose quantization scores exceed the preset scoring threshold. The preset scoring threshold can be an empirical threshold or an experimental threshold, which can be flexibly set by those skilled in the art.

[0088] Step S1300: Based on the similarity between feature vectors, select the relatively dissimilar unlabeled samples from the set of most similar unlabeled samples that are the corresponding labeled samples in the coarse recall set, and form the secondary recall set:

[0089] To further recall unlabeled samples in the coarse recall set, this application also employs a similarity comparison method. Based on the similarity between the feature vectors of unlabeled samples in the coarse recall set and labeled samples in the training dataset, the most similar labeled sample for each unlabeled sample is first determined, obtaining the corresponding similarity representation data. Then, based on this similarity representation data, some highly similar elements are filtered out from all unlabeled samples in the coarse recall set, leaving some relatively dissimilar elements. The unlabeled samples corresponding to these elements are used to construct a secondary recall set. The obtained secondary recall set contains some unlabeled samples selected from the coarse recall set. These unlabeled samples are semantically most similar to a certain labeled sample in the training dataset, but relatively speaking, they are only the unlabeled samples corresponding to the relatively dissimilar parts of this most similar case.

[0090] Methods for measuring the similarity between feature vectors can be implemented using any known algorithm or similar algorithms, such as cosine similarity algorithm, Euclidean distance algorithm, Jaccard algorithm, Pearson coefficient algorithm, Manhattan distance algorithm, Chebyshev distance algorithm, etc.

[0091] In the secondary recall stage, this application employs a similarity comparison method to recall unlabeled samples with low similarity to labeled samples. Since labeled samples represent the semantic space distribution that the model can focus on, continuously adding samples similar to labeled samples to the training dataset used for subsequent training will undoubtedly only cause the model to move closer to the direction of semantic distribution similarity, thus causing learning bias in the model. Therefore, for the purpose of optimizing unlabeled samples, it is necessary to screen out unlabeled samples with low similarity to labeled samples. The significance lies in selecting important samples that are as dispersed as possible from labeled samples in the semantic space, thereby supplementing the information that the neural network model has not learned in other semantic dimensions during subsequent training, filling the gaps in the model's insufficient learning ability in certain semantic dimensions, and thus improving the model's performance and robustness.

[0092] Step S1400: Based on the similarity between the feature vectors of unlabeled samples and the class center vectors corresponding to each classification label, select the relatively dissimilar unlabeled samples from the set of most similar unlabeled samples with corresponding class center vectors within the secondary recall set as samples to be labeled; the class center vector is the mean vector among the feature vectors of all labeled samples under the same classification label.

[0093] Similar to the previous step, this step also uses a similarity comparison method to further select the samples to be labeled from the unlabeled samples in the secondary recall set. However, this step mainly compares the similarity between the feature vectors in the secondary recall set and the class center vectors of the labeled samples in the training dataset corresponding to each classification label in the classification space.

[0094] The class center vector is obtained for each classification label in the classification space. Specifically, it can be obtained by determining the feature vectors of all labeled samples carrying the same classification label from the training dataset, summing the element-wise feature vectors of all labeled samples under the same classification label, and then averaging them (equivalent to mean pooling) to obtain a mean vector. This mean vector is the class center vector corresponding to that classification label. It is easy to understand that this class center vector is the result of normalized synthesis of the semantic information of all labeled samples under that classification label.

[0095] After determining the class center vectors corresponding to each classification label, the feature vectors of each unlabeled sample in the secondary recall set are compared with the class center vectors of each classification label to determine the similarity between the feature vectors of each unlabeled sample and the class center vectors of each classification label. Based on this, the classification label most similar to each unlabeled sample and the similarity representation data between them are determined. Then, based on this similarity representation data, some highly similar elements are filtered out from all unlabeled samples in the secondary recall set, leaving some relatively dissimilar elements. The unlabeled samples corresponding to these elements are finally determined as samples to be labeled.

[0096] Similarly, the method for measuring the similarity between feature vectors and class center vectors can be any known algorithm or similar algorithm, such as cosine similarity algorithm, Euclidean distance algorithm, Jaccard algorithm, Pearson coefficient algorithm, Manhattan distance algorithm, Chebyshev distance algorithm, etc.

[0097] In a recommended alternative embodiment, steps S1300 and S1400 can select different similarity calculation methods in order to obtain a more dispersed mathematical expectation by utilizing multiple different similarity calculation methods.

[0098] This application employs a similarity comparison method to identify unlabeled samples that are relatively far similar to labeled samples. Similarly, its purpose is to filter important samples that are as semantically dispersed as possible from labeled samples. However, here the calculation focuses on the inter-class similarity between unlabeled and labeled samples, selecting the least similar examples as preferred choices. This further strengthens the intention to filter sample diversity, resulting in labeled samples that are more likely to enable the trained neural network model to generalize features.

[0099] After the above process of screening and selecting samples to be labeled from the training dataset, the samples to be labeled can be marked or saved for backend users to call and manually annotate, so that they can be converted into labeled samples. Based on this, together with other training samples in the training dataset, they can be used to train new neural network models, such as the aforementioned classification model, so that the trained new model can further improve its generalization ability to features, thereby making the training of the new model easier and faster to converge and saving training costs.

[0100] Through the embodiments of this application, it is easy to understand that in the process of screening and selecting training samples in the training dataset to determine the samples to be labeled, the feature vectors and probability vectors of each training sample are extracted based on the classification model trained on the training dataset. Based on this information, data mining processing is carried out in three stages: coarse recall, secondary recall, and finally fine recall. This process is carried out step by step to achieve a fine screening of unlabeled samples in the training dataset. Unlabeled samples that can be used to improve the feature generalization ability of the neural network model during training are identified and manually labeled. These samples are then converted into labeled samples by manual labeling and re-input into the training of the neural network model. This makes it easier for the trained neural network model to be trained to a convergent state and effectively avoids excessive manual labeling. This ensures that while achieving high training results, the model training cost remains low.

[0101] Please see Figure 3 In a specific embodiment, step S1200, calculating a quantitative score representing the classification uncertainty of each unlabeled sample based on the probability vector, includes the following steps:

[0102] Step S1210: Calculate the information entropy of the unlabeled samples in the training dataset:

[0103] For the unlabeled sample set X un Each unlabeled sample X in un，i According to the principles of information science, its information entropy is calculated based on the classification probability in its probability vector, and the formula is expressed as:

[0104]

[0105] Step S1220: Calculate the probability difference between the two maximum classification probabilities in the probability vector of the unlabeled samples in the training dataset:

[0106] For each unlabeled sample, calculate the difference between the two highest classification probabilities in its probability vector. By examining the difference between the two highest confidence levels in the probability vector, i.e., the probability difference, we determine the degree of confusion of the unlabeled sample relative to the model. The formula is expressed as:

[0107]

[0108] Step S1230: Calculate the product of the information entropy and the probability difference for each unlabeled sample in the training dataset as a quantitative score characterizing the classification uncertainty of the unlabeled sample.

[0109] For each unlabeled sample X un，i If we simply use information entropy or the difference in confidence scores to measure the degree of confusion relative to the model, i.e., its uncertainty, it may be biased. Therefore, this embodiment fuses the two. Specifically, for the same unlabeled sample, its information entropy and its probability difference are multiplied and the product is taken. The resulting value is used as a quantitative score representing the classification uncertainty of the unlabeled sample. The formula is expressed as:

[0110]

[0111] After determining the quantization score of each unlabeled sample through the specific process described above in this embodiment, the preset scoring threshold can be used to filter them. Unlabeled samples with quantization scores exceeding the preset scoring threshold are retained, while unlabeled samples with scores below the preset scoring threshold are deleted from the training dataset, thereby obtaining the coarse recall set X. uncer .

[0112] This embodiment employs a method to quantify the uncertainty of unlabeled samples being recognized by the model, thereby obtaining their corresponding quantitative scores. It recalls unlabeled samples that the model struggles to identify, constructing a coarse recall set. During this process, two methods for quantifying uncertainty—information entropy and the maximum boundary method—are used to quantify the uncertainty of samples that the model cannot distinguish. Information entropy represents the degree of disorder in the model's output when classifying a single sample, thus quantifying uncertainty. The maximum boundary method also serves to quantify uncertainty, focusing only on the difference between the two classification labels with the highest probability values ​​in the probability vector. Therefore, combining these two methods comprehensively and effectively reflects the uncertainty of current unlabeled samples from multiple dimensions, thereby expanding the range and diversity of samples that the model struggles to distinguish.

[0113] Please see Figure 4 In a specific embodiment, step S1300, which involves selecting relatively dissimilar unlabeled samples from the set of unlabeled samples that are most similar to the corresponding labeled samples in the coarse recall set based on the similarity between feature vectors, to form a secondary recall set, includes the following steps:

[0114] Step S1310: Calculate the cosine similarity between the feature vector of each unlabeled sample in the coarse recall set and the feature vector of each labeled sample in the training dataset, to obtain a single-sample similarity sequence corresponding to each unlabeled sample. This single-sample similarity sequence contains similarity values ​​corresponding to each labeled sample to represent similarity.

[0115] In order to examine the semantic similarity between the unlabeled samples and the labeled samples, the coarse recall set X obtained through coarse recall is analyzed. uncer Calculate the relationship between each unlabeled sample and the labeled dataset X. label For each labeled sample, the cosine similarity is calculated to obtain the corresponding similarity value. The sample with the highest similarity value is retained, thus obtaining a single-sample similarity sequence composed of the set of similarity values ​​of each unlabeled sample in the coarse recall set. An example formula is shown below:

[0116] S cosine，k =Max i (Cosine(V h，i V h，k ))

[0117] Where k represents X uncer One of the unlabeled samples.

[0118] It is easy to see that the similarity value calculated by the cosine similarity algorithm can realize the quantitative representation of the similarity between the feature vectors of unlabeled samples and the feature vectors of labeled samples. The higher the similarity value, the higher the similarity between the two, and vice versa. Based on this, the quantitative representation of similarity is realized.

[0119] Step S1320: Filter the similarity sequence corresponding to each unlabeled sample, retaining only the maximum similarity value. This maximum similarity value indicates that the corresponding unlabeled sample is most similar to its corresponding labeled sample. Construct a summary similarity sequence from all retained similarity values.

[0120] Following the previous step, each unlabeled sample obtains a corresponding single-sample similarity sequence. This sequence contains the similarity values ​​between the unlabeled sample and each labeled sample in the training dataset. Based on this, the element with the highest similarity value can be selected from the single-sample similarity sequence of each unlabeled sample. All other elements in the same single-sample similarity sequence can be deleted. Thus, each unlabeled sample retains only one corresponding similarity value, which represents the degree of similarity between the unlabeled sample and the labeled sample in the training dataset that is semantically most similar to it. The retained elements of all unlabeled samples are constructed into a summary similarity sequence for ranking and selection.

[0121] Step S1330: Filter the similarity values ​​in the summarized similarity sequence according to a preset similarity threshold, retaining only elements with similarity values ​​lower than the threshold, and constructing the unlabeled samples corresponding to the retained elements as a secondary recall set.

[0122] The summarized similarity sequence includes X uncer Therefore, the elements of the summarized similarity sequence can be sorted from smallest to largest based on their similarity scores. Then, the summarized similarity sequence can be filtered according to a preset similarity threshold, deleting elements with similarity scores higher than the threshold and retaining elements with similarity scores lower than the threshold, thus obtaining X. sim The elements and the corresponding unlabeled samples of these elements can be used to construct a secondary recall set. Based on this, unlabeled samples that do not enter the secondary recall set can be removed from the training dataset to achieve purification.

[0123] As an equivalent substitution, in this step, the top n values ​​can also be selected from the sorted results of the aggregated similarity sequences. sim Each sample serves the same purpose as filtering using a preset similarity threshold, and this should also be understood as being within the scope of the inventive spirit of this application, as those skilled in the art should know.

[0124] This embodiment calculates the cosine similarity with labeled samples and recalls unlabeled samples with low similarity to the labeled samples to construct a secondary recall set. Labeled samples represent the semantic space distribution that the model can focus on. Continuously adding samples similar to labeled samples to the same training dataset will undoubtedly cause the model to move towards a more similar semantic distribution, resulting in learning bias. Therefore, it is necessary to select unlabeled samples with low similarity to labeled samples. The significance lies in selecting important samples that are as dispersed as possible in the semantic space as possible from the labeled samples, thereby supplementing the information that the model has not learned in other semantic dimensions, filling the gaps in the model's learning ability in certain semantic dimensions, and thus improving the model's performance and robustness.

[0125] Please see Figure 5 In a specific embodiment, step S1400, based on the similarity between the feature vectors of the unlabeled samples and the class center vectors corresponding to each classification label, selects relatively dissimilar unlabeled samples as samples to be labeled from the set of unlabeled samples that are most similar to the corresponding class center vectors in the secondary recall set. This includes the following steps:

[0126] Step S1410: Determine the labeled samples corresponding to each classification label, sum the feature vectors of these labeled samples element by element and calculate the mean, obtain the corresponding mean vector, and use it as the class center vector corresponding to that classification label.

[0127] As mentioned earlier, in order to determine the class center vector corresponding to each classification label in the classification space, it is first necessary to determine all labeled samples X carrying that classification label from the training dataset based on each classification label. label，i And call its feature vector, which has been pre-extracted by the classification model:

[0128] V h，i =M(X) label，i )

[0129] Then, for each category label, the feature vectors of all labeled samples under that category are summed element-wise and the mean is calculated to obtain the corresponding mean vector. This mean vector is the class center vector for that category label. This class center vector provides a comprehensive abstract representation of the semantic information of all labeled samples under the same category label. Its formula is as follows:

[0130] V center，c =Avg(V h，c，i )

[0131] Where V h，c，i The feature vector represents the category label c, where c∈{1,2,...,C} represents the specific category label.

[0132] Each category label is assigned a corresponding class center vector, which allows for similarity comparison of unlabeled samples in the secondary recall set.

[0133] Step S1420: Calculate the Euclidean distance between the feature vector of each unlabeled sample in the secondary recall set and the class center vector of each classification label to obtain a single-sample distance sequence corresponding to each unlabeled sample. This single-sample distance sequence contains distance values ​​corresponding to each class center vector for characterizing similarity.

[0134] To calculate the secondary recall set X sim The semantic similarity between the feature vector of each unlabeled sample and the class center vector of each classification label is calculated. The Euclidean distance between the feature vector of each unlabeled sample and the class center vector of each classification label is obtained, and the smallest distance value is retained. This yields the results for each unlabeled sample X in the second recall set. sim，i The single-sample distance sequence, composed of the set of distance values, is illustrated in the following formula example:

[0135] The feature vector of the unlabeled sample is: V sim，i =M(X) sim，i )

[0136] The formula for calculating the Euclidean distance is: d c，i =L2(V sim，i V center，i )

[0137] Based on the above formula, the distance between the feature vector of each unlabeled sample and the class center vector of each classification label is calculated. This allows for a quantitative representation of the similarity between the feature vector of the unlabeled sample and each class center vector. The smaller the distance value, the higher the similarity between the two, and vice versa. This achieves a quantitative representation of similarity.

[0138] Step S1430: Filter the single-sample distance sequence corresponding to each unlabeled sample, retaining only the minimum distance value. This minimum distance value indicates that the corresponding unlabeled sample is most similar to its corresponding labeled sample. Construct a summary distance sequence from all retained distance values.

[0139] Following the previous step, each unlabeled sample obtains a corresponding single-sample distance sequence. This sequence contains the Euclidean distance values ​​between the feature vector of the unlabeled sample and the class center vectors of each classification label. Based on this, for each unlabeled sample, the element most similar to it, i.e., with the smallest distance value, can be selected from its single-sample distance sequence. The formula is expressed as:

[0140] d i =Min c (d c，i )

[0141] All other elements in the same single-sample distance sequence can be deleted. Thus, each unlabeled sample retains only one corresponding distance value, which represents the degree of similarity between the feature vector of the unlabeled sample and the classification label that is most similar to it semantically. The retained elements of all unlabeled samples are constructed into a summary distance sequence for sorting and selection.

[0142] Step S1440: Filter the distance values ​​in the summarized distance sequence according to a preset distance threshold, retaining only elements whose distance values ​​are higher than the threshold, and using the corresponding unlabeled samples of the retained elements as samples to be labeled.

[0143] The summarized distance sequence includes X sim The similarity values ​​are used to sort the elements of the aggregated distance sequence from largest to smallest. Then, the aggregated distance sequence is filtered according to a preset distance threshold. Elements with distance values ​​higher than the preset distance threshold are deleted, while elements with distance values ​​lower than the preset distance threshold are retained, thus obtaining a subset of elements. The unlabeled samples corresponding to these elements can be used as unlabeled samples in the screening results of this application, which can be constructed as a fine recall set. Accordingly, unlabeled samples that do not enter the fine recall set can be deleted from the training dataset to achieve purification.

[0144] As an equivalent replacement, in this step, one could also select the top n from the sorted results of the aggregated distance sequence. r Each sample serves the same purpose as filtering using a preset distance threshold, and this should also be understood as being within the scope of the inventive spirit of this application, as those skilled in the art should know.

[0145] In this embodiment, the feature center distance is determined based on the Euclidean distance algorithm, and unlabeled samples that are relatively far from the labeled sample vectors are selected as samples to be labeled. Similarly, the purpose is to select important samples that are as semantically dispersed as possible from the labeled samples. However, here the calculation focuses on the class center distance between unlabeled and labeled samples, and the distance values ​​of unlabeled samples that are farthest from all class center vectors are selected as ranking scores. Based on this, the selection of samples to be labeled is further optimized, thus strengthening the intention to filter sample diversity.

[0146] Please see Figure 6 In an extended embodiment, the training sample screening method of this application further includes the following post-step:

[0147] Step S1500: Obtain the artificial label corresponding to the sample to be labeled, so that the sample to be labeled is converted into a labeled sample:

[0148] Please combine Figure 7 As shown, the samples to be labeled determined in the foregoing embodiments of this application can be labeled in the training dataset or transferred to another storage space for retrieval.

[0149] Subsequently, experts can manually annotate the samples to be labeled, thereby obtaining corresponding supervisory labels for the samples, making them labeled samples. These re-labeled samples can then be used to train a new neural network model.

[0150] Step S1600: Merge the labeled samples converted from the unlabeled samples with the labeled samples in the training dataset to form the second training dataset.

[0151] After the unlabeled samples are manually labeled, they can be merged with the existing labeled samples in the training dataset to obtain a second training dataset consisting entirely of labeled samples, which is used to train the new neural network model instance. Of course, to adapt to the training method of the new neural network model instance, other training samples can be further included in the second training dataset, such as including other unlabeled samples from the original dataset to meet the needs of semi-supervised learning.

[0152] Step S1700: Enable the training task and call the second training dataset to restart fine-tuning the classification model, so that the classification model is trained to a convergent state.

[0153] After obtaining the second training dataset, a training task for the new neural network model instance can be initiated, and training can be performed on it. The neural network model instance is trained to convergence using training samples from the second training dataset, resulting in a new neural network model. In a typical application example, the new neural network model can be used for fine-tuning the classification model, or it can be used to train a new instance of the classification model. This significantly improves the feature generalization ability of the classification model with the help of the labeled samples prepared in the technical solution of this application. Of course, the second training dataset can also be used to train neural network models for other different tasks.

[0154] In this embodiment, the training samples selected and manually labeled by this application are recycled, which reflects the value of data mining and further demonstrates that the technical solution of this application can achieve efficient screening of training samples and help improve the feature generalization ability of the neural network model trained by it, making the model easier to train to convergence, thereby saving training costs.

[0155] In the e-commerce application scenario of this application, the technical solution of this application is used to select unlabeled samples. After manual labeling, a second training dataset is obtained. Based on the second training dataset, the classification model used to classify product objects is trained, which can obtain a classification model with higher classification accuracy. It can determine the classification labels of product objects in e-commerce platforms more quickly and efficiently based on the images and / or text of product objects, and weaken the adverse effect of the uneven number of product data used as training samples in the lowest level subcategories of different e-commerce platform classification systems.

[0156] Please see Figure 8 A training sample screening device provided to meet one of the purposes of this application is a functional embodiment of the training sample screening method of this application. The device includes: a basic module 1100, a coarse recall module 1200, a secondary recall module 1300, and a fine recall module 1400. The basic module 1100 is used to obtain feature vectors of training samples in the training dataset and probability vectors generated by mapping these feature vectors to various classification labels using a classification model. The training samples include labeled samples and unlabeled samples. The classification model is pre-trained to convergence using the labeled samples, and the probability vectors contain the classification probabilities corresponding to each classification label. The coarse recall module 1200 is used to calculate a representation of the classification of each unlabeled sample based on the probability vectors. The uncertainty is quantified, and unlabeled samples with quantified scores exceeding a preset score threshold are selected from the training dataset to form a coarse recall set. The secondary recall module 1300 is used to select relatively dissimilar unlabeled samples from the set of unlabeled samples most similar to the corresponding labeled samples in the coarse recall set based on the similarity between feature vectors, and form a secondary recall set. The fine recall module 1400 is used to select relatively dissimilar unlabeled samples from the set of unlabeled samples most similar to the corresponding class center vectors in the secondary recall set based on the similarity between the feature vectors of the unlabeled samples and the class center vectors corresponding to each classification label, and form relatively dissimilar unlabeled samples as samples to be labeled. The class center vector is the mean vector among the feature vectors of all labeled samples under the same classification label.

[0157] In a specific embodiment, the classification model includes: a feature extraction network for extracting feature vectors representing deep semantic information from the input training samples; the training samples are images or text; and a classifier for mapping the feature vectors to a preset classification space to obtain corresponding probability vectors.

[0158] In a specific embodiment, the coarse recall module 1200 includes: an information entropy calculation unit for calculating the information entropy of unlabeled samples in the training dataset; a probability difference calculation unit for calculating the probability difference between the two maximum classification probabilities in the probability vector of unlabeled samples in the training dataset; and a scoring calculation unit for calculating the product of the information entropy and the probability difference of each unlabeled sample in the training dataset as a quantitative score characterizing the classification uncertainty of the unlabeled sample.

[0159] In a specific embodiment, the coarse recall module 1200 includes: a similarity calculation unit, used to calculate the cosine similarity between the feature vector of each unlabeled sample in the coarse recall set and the feature vector of each labeled sample in the training dataset, to obtain a single-sample similarity sequence corresponding to each unlabeled sample, the single-sample similarity sequence containing similarity values ​​corresponding to each labeled sample to characterize similarity; a most similarity filtering unit, used to filter the similarity sequence corresponding to each unlabeled sample, retaining only the maximum similarity value, the maximum similarity value indicating that its corresponding unlabeled sample is most similar to its corresponding labeled sample, and constructing a summary similarity sequence from all retained similarity values; and a low similarity selection unit, used to filter the similarity values ​​in the summary similarity sequence according to a preset similarity threshold, retaining only elements whose similarity values ​​are lower than the similarity threshold, and constructing a secondary recall set from the unlabeled samples corresponding to the retained elements.

[0160] In a specific embodiment, the secondary recall module 1300 includes: a classification averaging unit, used to determine the labeled samples corresponding to each classification label, and to sum the feature vectors of these labeled samples element by element to obtain the corresponding mean vector, which serves as the class center vector corresponding to the classification label; a distance calculation unit, used to calculate the Euclidean distance between the feature vector of each unlabeled sample in the secondary recall set and the class center vector of each classification label, to obtain a single-sample distance sequence corresponding to each unlabeled sample, the single-sample distance sequence containing distance values ​​corresponding to each of the class center vectors for characterizing similarity; a closest filtering unit, used to filter the single-sample distance sequence corresponding to each unlabeled sample, retaining only the minimum distance value, the minimum distance value indicating that its corresponding unlabeled sample is most similar to its corresponding labeled sample, and constructing a summary distance sequence from all the retained distance values; and a high similarity selection unit, used to filter the distance values ​​in the summary distance sequence according to a preset distance threshold, retaining only the elements whose distance values ​​are higher than the distance threshold, and using the unlabeled samples corresponding to the retained elements as samples to be labeled.

[0161] In an extended embodiment, the training sample screening method of this application further includes: a label acquisition module, used to acquire the artificial labels corresponding to the sample to be labeled, so that the sample to be labeled is converted into a labeled sample; a dataset reconstruction module, used to merge the labeled samples converted from the sample to be labeled and the labeled samples in the training dataset to form a second training dataset; and a training restart module, used to enable the training task, call the second training dataset to restart the fine-tuning training of the classification model, so that the classification model is trained to a convergent state.

[0162] To address the aforementioned technical problems, embodiments of this application also provide computer equipment. For example... Figure 9 The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When the computer-readable instructions are executed by the processor, the processor can implement a training sample screening method. The processor of the computer device provides computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the training sample screening method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 9The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0163] In this embodiment, the processor is used to execute... Figure 8 The system contains the specific functions of each module and its sub-modules. The memory stores the program code and various data required to execute these modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / sub-modules in the training sample screening device of this application. The server can call the server's program code and data to execute the functions of all sub-modules.

[0164] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the training sample screening method of any embodiment of this application.

[0165] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0166] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0167] In summary, this application can improve the efficiency of training sample screening, select training samples that are conducive to improving the feature generalization ability of the neural network model during the training process, make the model easier to train to a convergent state, facilitate the labeling of product objects on e-commerce platforms, and has broad application prospects.

[0168] Those skilled in the art will understand that the steps, measures, and solutions in the various operations, methods, and processes discussed in this application can be alternated, modified, combined, or deleted. Furthermore, other steps, measures, and solutions in the various operations, methods, and processes discussed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and solutions in the prior art that are similar to those disclosed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted.

[0169] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for training sample troubleshooting, characterized in that, A method for an independent station-based e-commerce platform, comprising the following steps: A classification model is used to obtain a feature vector of a training sample in a training data set and a probability vector generated by mapping the feature vector to each classification label; the training sample includes a labeled sample and an unlabeled sample, the classification model is trained to a converged state with the labeled sample in advance, and the probability vector contains a classification probability corresponding to each classification label, wherein the training sample is text information and image information of a commodity object; A quantitative score representing the classification uncertainty of each unlabeled sample is calculated based on the probability vector, and an unlabeled sample with a quantitative score exceeding a preset score threshold is selected from the training data set to form a coarse recall set; According to the similarity between the feature vectors, relatively dissimilar unlabeled samples are selected from a most similar unlabeled sample set formed with the corresponding labeled samples in the coarse recall set to form a secondary recall set; According to the similarity between the feature vector of the unlabeled sample and the class center vector corresponding to each classification label, relatively dissimilar unlabeled samples are selected from a most similar unlabeled sample set formed with the corresponding class center vector in the secondary recall set as a sample to be labeled, and the classification model for the commodity object is trained with the sample to be labeled, and the classification model is used to classify the commodity object; the class center vector is the mean vector between the feature vectors of all labeled samples under the same classification label.

2. The method of claim 1, wherein, The classification model performs the following steps: A feature vector representing the deep semantic information of the input training sample is extracted; the training sample is a picture or text; The feature vector is mapped to a preset classification space to obtain a corresponding probability vector.

3. The method of claim 1, wherein, A quantitative score representing the classification uncertainty of each unlabeled sample is calculated based on the probability vector, comprising the following steps: Calculate the information entropy of the unlabeled sample in the training data set; Calculate the probability difference between the two largest classification probabilities in the probability vector of the unlabeled sample in the training data set; The product of the information entropy and the probability difference of each unlabeled sample in the training data set is calculated as a quantitative score representing the classification uncertainty of the unlabeled sample.

4. The method of claim 1, wherein, According to the similarity between the feature vectors, relatively dissimilar unlabeled samples are selected from a most similar unlabeled sample set formed with the corresponding labeled samples in the coarse recall set to form a secondary recall set, comprising the following steps: Calculate the cosine similarity between the feature vector of each unlabeled sample in the coarse recall set and the feature vector of each labeled sample in the training data set to obtain a single sample similarity sequence corresponding to each unlabeled sample, which contains a similarity value corresponding to each labeled sample for representing similarity; Filter the similarity sequence corresponding to each unlabeled sample, and only keep the maximum similarity value, which represents that the corresponding unlabeled sample and the corresponding labeled sample form the most similar, and all the retained similarity values are constructed into an aggregated similarity sequence; Filtering the similarity values in the aggregated similarity sequence according to a preset similarity threshold, only keeping elements with similarity values lower than the similarity threshold, and constructing the unlabeled samples corresponding to the kept elements as a secondary recall set.

5. The method of claim 1, wherein, Screening, from the secondary recall set, relatively dissimilar unlabeled samples as the samples to be labeled, from a set of most similar unlabeled samples to corresponding class center vectors, according to the similarity between the feature vectors of the unlabeled samples and the class center vectors corresponding to each classification label, including the following steps: Determining the labeled samples corresponding to each classification label, and obtaining the mean vector corresponding to each classification label by element-wise summing and averaging the feature vectors of the labeled samples; Calculating the Euclidean distance between the feature vector of each unlabeled sample in the secondary recall set and the class center vectors of each classification label, to obtain a single-sample distance sequence corresponding to each unlabeled sample, which contains distance values representing the similarity corresponding to each class center vector; Filtering the single-sample distance sequence corresponding to each unlabeled sample, only keeping the minimum distance value, which represents that the corresponding unlabeled sample and the corresponding labeled sample form the most similar, and constructing all the kept distance values as an aggregated distance sequence; Filtering the distance values in the aggregated distance sequence according to a preset distance threshold, only keeping elements with distance values higher than the distance threshold, and taking the unlabeled samples corresponding to the kept elements as the samples to be labeled.

6. The training sample troubleshooting method according to any one of claims 1 to 5, characterized in that, Further comprising the following post-processing steps: Obtaining the artificial label corresponding to the samples to be labeled, and converting the samples to be labeled into labeled samples; Merging the labeled samples converted from the samples to be labeled and the labeled samples in the training data set to form a second training data set; Enabling a training task, calling the second training data set to restart fine-tuning training of the classification model, and training the classification model to a converged state.

7. A training sample troubleshooting apparatus, characterized by comprising: For use in an independent station-based e-commerce platform, comprising: A basic module for obtaining the feature vector of a training sample in a training data set and the probability vector generated by mapping the feature vector to each classification label using a classification model; the training sample includes labeled samples and unlabeled samples, the classification model is pre-trained to a converged state using the labeled samples, and the probability vector contains the classification probability corresponding to each classification label, wherein the training sample is the text information and image information of a commodity object; A rough recall module for calculating a quantitative score representing the classification uncertainty of each unlabeled sample based on the probability vector, and screening unlabeled samples with a quantitative score exceeding a preset score threshold from the training data set to form a rough recall set; A secondary recall module for screening relatively dissimilar unlabeled samples from a set of most similar unlabeled samples to corresponding labeled samples in the rough recall set according to the similarity between the feature vectors, to form a secondary recall set. The fine recall module is configured to filter out relatively dissimilar unlabeled samples from a set of most similar unlabeled samples to the corresponding class center vector from a secondary recall set as samples to be labeled based on similarity between a feature vector of the unlabeled samples and the class center vector corresponding to each classification label, so as to classify the product objects based on the classification model after the samples to be labeled are used to train the classification model for the product objects; and the class center vector is a mean vector between feature vectors of all labeled samples under the same classification label.

8. The training sample troubleshooting apparatus of claim 7, wherein, The classification model comprises: a feature extraction network configured to extract a feature vector representing deep semantic information of a training sample input thereto; the training sample is a picture or text; a classifier configured to map the feature vector to a preset classification space to obtain a corresponding probability vector.

9. A computer device comprising a central processing unit and a memory, characterized in that The central processing unit is configured to invoke a computer program stored in the memory to execute steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer readable instructions store a computer program implemented according to the method of any one of claims 1 to 6, and the computer program is invoked to execute steps included in the corresponding method when running.

Images