Sample set optimization method and apparatus, device, medium, product

Abstract

The application relates to a sample set optimization method and device, equipment, medium and product. The method comprises the following steps: obtaining an original sample set, including a plurality of training samples and corresponding supervision labels; determining the influence degree of each training sample in the original sample set after the supervision label is changed according to an influence function, removing part of the training samples with relatively high influence degrees, and obtaining a balanced sample set; implementing text enhancement processing based on part of the training samples in the balanced sample set, expanding the training samples through text enhancement, and obtaining an augmented sample set; removing the outlying training samples in the augmented sample set based on the clustering results of the deep semantic information of each training sample in the augmented sample set, and obtaining an optimized sample set. The optimized sample set obtained by the application is rich in sample quantity and excellent in sample quality, is suitable for training a deep learning model corresponding to a related downstream task, makes the trained deep learning model more easily convergent, and can obtain a higher prediction accuracy.

Description

Technical Field

[0001] This application relates to e-commerce information processing technology, and in particular to a sample set optimization method and its apparatus, equipment, medium, and product. Background Technology

[0002] With the advent of the big data era and the enhancement of hardware computing power, deep learning technology has made significant progress. In the field of natural language processing, it has a wide range of applications in e-commerce user intent classification, such as predicting product categories. Accurate category prediction can provide better ranking results for exposed products, resulting in higher total transaction volume.

[0003] On the one hand, as the scale of a store's products continues to expand and the number of product categories increases daily, the corresponding user input corpus is often not rich enough, resulting in insufficient training datasets. In this case, it is often necessary to use text augmentation techniques to generate high-quality data to improve the performance of downstream models, rather than simply using a small number of seed samples. However, traditional text augmentation techniques rely on high-quality seed samples to obtain high-quality samples, and therefore also have limitations.

[0004] On the other hand, with the advent of the era of large-scale models, in engineering implementation, the methodology centered on model improvement has begun to evolve into a data-centered approach. The problem is that there is a shortage of training data and the labeling cost is high, which consumes a lot of human and material resources. Therefore, there is an urgent need for training data augmentation technology.

[0005] In summary, how to organically integrate various technical means to optimize and expand the training samples required for deep learning models is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this application is to solve the above-mentioned problems by providing a sample set optimization method and corresponding apparatus, devices, non-volatile readable storage media, and computer program products.

[0007] According to one aspect of this application, a sample set optimization method is provided, comprising the following steps:

[0008] Obtain the original sample set, which includes multiple training samples and their corresponding supervision labels. The training samples are descriptive text, and the supervision labels are intent labels corresponding to the descriptive text.

[0009] The influence degree of each training sample in the original sample set after changing the supervision label is determined according to the influence function. Then, the training samples with relatively high influence degree are removed to obtain a balanced sample set.

[0010] Text augmentation is performed on a portion of the training samples in the balanced sample set to expand the training samples and obtain an augmented sample set.

[0011] Based on the clustering results of the deep semantic information of each training sample in the augmented sample set, outlier training samples in the augmented sample set are removed to obtain an optimized sample set.

[0012] Optionally, obtain the original sample set, including:

[0013] Obtain the user search text corresponding to product search events on e-commerce platforms;

[0014] Retrieve the product category corresponding to the product item accessed by the user from the product results list obtained based on the user's search text;

[0015] The user search text for each product search event is used as a training sample, and the product category corresponding to the user search text is used as a supervision label, which is stored in the original sample set.

[0016] Optionally, based on the influence function, the influence degree of each training sample in the original sample set after changing the supervision label is determined, and the training samples with relatively high influence degrees are removed to obtain a balanced sample set, including:

[0017] The classification model is trained to convergence using the training samples and their supervision labels from the original sample set.

[0018] The supervision labels of each training sample in the original sample set are changed, and each training sample is input into the classification model one by one to predict the classification result.

[0019] Based on the modified supervision labels, the classification loss value of the corresponding training samples is determined, and the influence function is applied to determine the influence degree of the corresponding training samples according to the classification loss value.

[0020] Remove training samples whose influence exceeds a set threshold from the original sample set, and retain the other training samples to form a balanced sample set.

[0021] Optionally, text augmentation processing is performed on a portion of the training samples in the balanced sample set to expand the training samples and obtain an augmented sample set, including:

[0022] The number of word segments in the training samples of the balanced sample set is detected and determined. Those with a number of word segments greater than a preset threshold are identified as first-type training samples, and the rest are identified as second-type training samples.

[0023] Based on the first type of training samples, new training samples are expanded by performing word segmentation granularity editing operations while keeping the original supervision label correspondence unchanged.

[0024] Based on the second type of training samples, the local description texts of different training samples corresponding to the same original supervision label are randomly concatenated and expanded into new training samples corresponding to the original supervision label.

[0025] Based on the expanded training samples, a text generation model is applied to further expand the training samples. All the newly added training samples are added to the balanced sample set to form an augmented sample set.

[0026] Optionally, based on the clustering results of the deep semantic information of each training sample in the augmented sample set, outlier training samples in the augmented sample set are removed to obtain an optimized sample set, including:

[0027] The feature extraction model is iteratively trained until convergence using training samples from the augmented sample set, making the feature extraction model suitable for representing features from the training samples;

[0028] The converged feature extraction model is used to extract the deep semantic information of each training sample in the augmented sample set one by one, and the deep semantic information is compressed into a low-dimensional vector of the same scale.

[0029] Clustering algorithms are used to cluster the low-dimensional vectors of each training sample, resulting in multiple clusters, each of which includes a portion of the training samples in the augmented sample set.

[0030] Training samples that do not fall into any cluster are taken as outliers. Outliers in the augmented sample set are removed to obtain an optimized sample set.

[0031] Optionally, the step of removing outlier training samples from the augmented sample set based on the clustering results of the deep semantic information of each training sample in the augmented sample set is performed iteratively multiple times so that the final optimized sample set has removed training samples belonging to marginal data with long-tail effects.

[0032] Optionally, based on the clustering results of the deep semantic information of each training sample in the augmented sample set, after removing outlier training samples from the augmented sample set to obtain an optimized sample set, the process includes:

[0033] The search redirection model is trained to convergence using training samples from the optimized sample set, making the search redirection model suitable for predicting the corresponding intent tag based on the input descriptive text.

[0034] In response to a user's search request, the user's search text carried in the request is input into a converged search redirection model to predict the intent tag corresponding to the user's search text.

[0035] Based on the intent tag, retrieve product items carrying the intent tag from the product database of the independent website, and construct a product information list to respond to the user's search request.

[0036] According to another aspect of this application, a sample set optimization apparatus is provided, comprising:

[0037] The sample acquisition module is configured to acquire an original sample set, which includes multiple training samples and their corresponding supervision labels. The training samples are descriptive text, and the supervision labels are intent labels corresponding to the descriptive text.

[0038] The equilibrium optimization module is configured to determine the influence degree of each training sample in the original sample set after changing the supervision label based on the influence function, and remove the training samples with relatively high influence degree to obtain a balanced sample set.

[0039] The augmentation and optimization module is configured to perform text augmentation processing on a portion of the training samples in the balanced sample set, thereby expanding the training samples through text augmentation to obtain an augmented sample set;

[0040] The clustering optimization module is configured to remove outlier training samples from the augmented sample set based on the clustering results of the deep semantic information of each training sample in the augmented sample set, thereby obtaining an optimized sample set.

[0041] According to another aspect of this application, a sample set optimization device is provided, including a central processing unit and a memory, wherein the central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the sample set optimization method described in this application.

[0042] According to another aspect of this application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the sample set optimization method in the form of computer-readable instructions, wherein the computer program, when invoked by a computer, executes the steps included in the method.

[0043] According to another aspect of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0044] Compared to existing technologies, this application expands the training samples based on a pre-labeled original sample set with a limited sample size. Before expansion, samples with high influence in the original sample set are removed based on measured influence, and the remaining training samples constitute a balanced sample set with relatively balanced influence. Text enhancement processing is performed on the balanced sample set to expand a large number of high-quality training samples, thus obtaining an augmented sample set. Then, the deep semantic information of the training samples in the augmented sample set is used for clustering, so that training samples with similarity are aggregated into the same cluster. Other training samples that are outside the various clusters are removed. The augmented sample set after removal constitutes the optimized sample set. The optimized sample set obtained in this way has a rich number of samples and excellent sample quality, which is suitable for training deep learning models corresponding to relevant downstream tasks, making the trained deep learning models easier to converge and achieve higher prediction accuracy. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart illustrating one embodiment of the sample set optimization method of this application;

[0047] Figure 2 This is a schematic diagram illustrating the process of constructing the original sample set in an embodiment of this application;

[0048] Figure 3 This is a schematic diagram of the process of optimizing the original sample set to obtain a balanced sample set in an embodiment of this application;

[0049] Figure 4 This is a schematic diagram of the process of obtaining an augmented sample set based on the balanced sample set in an embodiment of this application;

[0050] Figure 5 This is a schematic diagram illustrating the process of obtaining an optimized sample set based on an augmented sample set in an embodiment of this application;

[0051] Figure 6 This is a flowchart illustrating the application of a model trained on an optimized sample set in an embodiment of this application.

[0052] Figure 7 This is a schematic block diagram of the sample set optimization device of this application;

[0053] Figure 8 This is a schematic diagram of the structure of a sample set optimization device used in this application. Detailed Implementation

[0054] The models cited or potentially cited in this application, including traditional machine learning models or deep learning models, can be deployed on a remote server and invoked remotely on the client, or deployed on a client with the capability to invoke directly, unless explicitly specified in the text. In some embodiments, when running on the client, the corresponding intelligence can be obtained through transfer learning in order to reduce the requirements on the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0055] Please see Figure 1 According to a sample set optimization method provided in this application, in one embodiment, it includes the following steps:

[0056] Step S1100: Obtain the original sample set, which includes multiple training samples and their corresponding supervision labels. The training samples are descriptive text, and the supervision labels are intent labels corresponding to the descriptive text.

[0057] This application provides an original sample set for storing seed samples, which can be used as training samples for deep learning models. More training samples can be expanded based on these seed samples to augment the training samples in the original sample set, thereby enriching the training data required by the deep learning model.

[0058] The training samples, serving as seed samples in the original sample set, can be constructed using data with corresponding content to adapt to specific downstream training tasks. For example, in one embodiment, the training samples can be descriptive text corresponding to the name, features, functions, and uses of product items. The descriptive text can be manually edited text or text extracted from user access data from the e-commerce platform.

[0059] The supervision labels corresponding to the training samples in the original sample set are mainly used to represent the intent type pointed to by the semantics of the descriptive text, and can thus be represented as intent labels. Therefore, there is a semantic relationship between the training samples and their supervision labels. The intent type in the supervision labels usually corresponds to one or a class of objects that the descriptive text in the training samples intends to express. For example, the descriptive text in the training samples could be "shoes suitable for hiking," while the corresponding intent label in the supervision labels could represent "sports shoes"; the descriptive text in the training samples could be "want to buy a phone for Dad," while the corresponding intent label in the supervision labels could be "senior citizen mobile phone," and so on. It is evident that there is a semantic correspondence between the descriptive text and the intent labels.

[0060] It is easy to understand that one intent label corresponds to one intent type. Therefore, before setting the supervision labels for the training samples, an intent type system can be predefined. This system includes multiple intent types, each of which can be represented by its corresponding intent label. The intent label can be expressed as either character text or numerical text, and can be flexibly set by those skilled in the art. In one embodiment, the intent type system can be a pre-defined product classification system within an e-commerce platform or one of its independent websites. Therefore, the intent labels therein can be product categories within the product classification system.

[0061] In one embodiment, the intention labels corresponding to each training sample in the original sample set can be obtained based on the page. In another embodiment, the intention labels corresponding to each training sample can be obtained from the original basic dataset with existing mapping relationships. As long as the corresponding intention labels are used to annotate each training sample, the annotated training samples can be used as seed samples in this application.

[0062] Step S1200: Determine the influence degree of each training sample in the original sample set after changing the supervision label according to the influence function, remove the training samples with relatively high influence degree, and obtain a balanced sample set;

[0063] Based on the interpretability understanding of machine learning, influence functions can be used to determine the degree of influence of each training sample in a sample set on a trained model, thereby determining the corresponding influence degree of each training sample. The influence function is a widely used statistical learning concept that can be used to measure the degree of influence of training samples on model parameters; it is simply called influence degree, or the importance of the sample.

[0064] In one embodiment, the weights of each training sample in the original sample set can be appropriately weighted to adjust the weights of each training sample in the model training. For example, a training sample can be deleted, and then the classification model can be trained again using the original sample set. The training effect of the model can then be quantified by using an influence function, thereby determining the degree of influence of each training sample on the model training.

[0065] In another embodiment, a given classification model is first trained using each training sample in the original sample set and its corresponding supervision label. The model is trained until convergence. Then, the classification loss value obtained by predicting each training sample using the converged model can be used to determine the influence of each training sample. Using the converged model, each training sample in the original sample set is classified and predicted one by one to determine its corresponding classification result. Then, a different supervision label than the original supervision label of the predicted training sample is used to calculate the classification loss value. Based on the classification loss value, an influence function is further applied to calculate the influence of the corresponding training sample. This method achieves higher efficiency in calculating the influence of each training sample.

[0066] As can be seen, under the premise of applying the influence function, the influence degree corresponding to each training sample in the original sample set can be determined in various ways to characterize the degree of influence of the corresponding training sample on model training. It is easy to understand that training samples with excessively high influence will make the trained model more sensitive, easily causing an imbalance in sample influence, thus leading to less than ideal model training results. Therefore, in one embodiment, the original sample set can be cleaned by deleting some training samples with excessively high influence, and the remaining training samples can constitute a balanced sample set. In another embodiment, for some training samples with excessively low influence, it indicates that the model is not sensitive to them, and theoretically, they can also be cleaned as needed. This measure can be flexibly combined and used by those skilled in the art.

[0067] Step S1300: Perform text enhancement processing on a portion of the training samples in the balanced sample set to expand the training samples and obtain an augmented sample set.

[0068] The training samples in the balanced sample set are high-quality seed samples obtained through preliminary optimization based on influence. Text enhancement processing can be performed on these seed samples to derive more training samples, making the feature distribution of the training samples in the entire balanced sample set more balanced.

[0069] There are many ways to perform text enhancement processing, and it can be implemented according to any one or more of the following embodiments:

[0070] In one embodiment, new training samples can be derived by modifying the seed samples at the literal level. New training samples can be generated by randomly inserting, swapping, or replacing synonyms in the descriptive text of the seed samples within a certain range. The supervision label of the new training samples can be associated with the supervision label of their corresponding seed samples.

[0071] In another embodiment, new training samples can be derived by modifying the features of the seed samples based on the vector space. Specifically, the seed samples are first represented by features to obtain corresponding feature vectors. Based on these feature vectors, the features are modified, and then the modified feature vectors are mapped back to the text level to generate new training samples.

[0072] In another embodiment, new training samples can be generated based on the seed samples using a text generation method. Specifically, a pre-trained text generation tool is used to perform prediction tasks such as fill-in-the-blank and word embedding replacement based on the seed samples to generate corresponding new training samples.

[0073] In another embodiment, when there is an extreme imbalance of samples in the category corresponding to a particular intent label, that is, when the number of training samples is usually less than a certain amount, the seed samples in this category can be modified at the literal level. The local descriptive text of each seed sample in this category can be spliced ​​together to obtain new training samples. These new training samples are then used as seed samples. Furthermore, by combining the text generation method, more new training samples are generated on the basis of these seed samples, thereby expanding the training samples to make the training samples of the corresponding category more balanced.

[0074] By using or combining the above methods to achieve text enhancement, more new training samples are derived based on the training samples in the balanced sample set. These new training samples are then added to the balanced sample set to obtain an augmented sample set.

[0075] It is easy to understand that since training samples with excessive influence have been removed in advance, the remaining training samples are used as seed samples to expand new training samples. This ensures the quality of the seed samples, making the quality of the expanded new training samples more reliable, and also ensures that the training samples corresponding to each intent type are more balanced in feature distribution.

[0076] Step S1400: Based on the clustering results of the deep semantic information of each training sample in the augmented sample set, remove the outlier training samples in the augmented sample set to obtain an optimized sample set.

[0077] The training samples in the augmented sample set have the characteristics of balanced feature distribution and moderate influence, but there may also be some training samples whose semantics deviate from the context required by the downstream task. These training samples can be further optimized.

[0078] To this end, a pre-trained or specially trained feature extraction model can be used to represent the features of each training sample to obtain the deep semantic information corresponding to each training sample. Then, any vector compression method can be used to compress the deep semantic information of the training samples from high-dimensional vectors to low-dimensional vectors for clustering.

[0079] In one embodiment, when compressing the deep semantic information into vectors, a fully connected layer can be used. Typically, the fully connected layer maps the deep semantic information output by the feature extraction model from a high-dimensional space to a low-dimensional space to obtain the corresponding low-dimensional vector.

[0080] In another embodiment, an algorithm such as t-SNE can be used to perform vector compression on the deep semantic information. t-SNE (t-distributed stochastic neighbor embedding) is a machine learning algorithm for dimensionality reduction. As a non-linear dimensionality reduction algorithm, it is often used in the dimensionality reduction process of manifold learning and is compared with LLE. It is very suitable for reducing high-dimensional data to 2 or 3 dimensions, which facilitates visualization. After compressing the deep semantic information of each training sample into a low-dimensional vector using the t-SNE algorithm and visualizing it, in one embodiment, the clusters can be manually checked first, and training samples that deviate from the clusters can be manually removed.

[0081] After obtaining the low-dimensional vectors of each training sample, any feasible clustering algorithm can be used to cluster the training samples based on the low-dimensional vectors, thereby determining multiple clusters. Then, the outliers that are outside the clusters are removed from the augmented sample set, and the remaining training samples constitute the optimized sample set.

[0082] In one embodiment, step S1400 can be executed iteratively multiple times to remove outlier training samples from the augmented sample set multiple times, continuously improving the concentration of training samples in the optimized sample set, thereby maximizing the removal of training samples in the augmented sample set that belong to marginal data with long-tail effects, resulting in a higher quality optimized sample set.

[0083] After performing the above process on a demonstrative original sample set, an accuracy of 0.857 was achieved on the test dataset under the same model and test dataset, which is an 11% improvement in accuracy compared to optimizing the original dataset using traditional data augmentation methods.

[0084] As can be seen from the above embodiments, this application expands the training samples based on the pre-labeled original sample set with a limited number of samples. Before expansion, samples with high influence in the original sample set are cleaned up according to the measured influence. The remaining training samples constitute a balanced sample set with relatively balanced influence. Text enhancement processing is performed on the balanced sample set to expand a large number of high-quality training samples and obtain an augmented sample set. Then, the deep semantic information of the training samples in the augmented sample set is used for clustering, so that training samples with similarity are aggregated into the same cluster. Other training samples that are outside the various clusters are cleaned up. The cleaned augmented sample set constitutes the optimized sample set. The optimized sample set obtained in this way has a rich number of samples and excellent sample quality, which is suitable for training deep learning models corresponding to related downstream tasks, making the trained deep learning model more likely to converge and obtain higher prediction accuracy.

[0085] Based on any embodiment of this application, please refer to Figure 2 Obtain the original sample set, including:

[0086] Step S1110: Obtain the user search text corresponding to the product search event in the e-commerce platform;

[0087] In e-commerce platforms, user product searches are a frequent occurrence. Taking an online store on an independent e-commerce platform as an example, a user enters their search text on the product search page of the online store, expressing their desired product items in natural language, and then submits the corresponding search text, triggering a product search event. After receiving the user search text corresponding to the product search event, the independent website hosting the online store performs a search operation in its product database based on the user search text, obtaining a list of product results containing product information for multiple product items. This list of product results is then pushed to the relevant user to complete the response.

[0088] During this process, corresponding user access behavior naturally generates corresponding access behavior data. As each independent website operates, such access behavior data accumulates over time. Therefore, user search text can be extracted from the access behavior data generated by one or more independent websites to construct training samples for the original sample set.

[0089] Step S1120: Obtain the product category corresponding to the product item accessed by the user in the product results list obtained based on the user's search text;

[0090] For an independent website that obtains a list of product results based on the user's search text, after the list is pushed to the user, the user will typically visit the product details page of one or more of the product items in the list after the product information of each product item is displayed. This results in a conversion, that is, the push behavior of the corresponding product item is converted into an access behavior. The independent website can record such conversion events in the corresponding access behavior data. Thus, for a user's search text, the converted product item in the corresponding product results list can be determined from the access behavior data. When there are multiple converted product items, any one of them can be identified as the target product item.

[0091] After identifying the target product item, the product category to which the target product item belongs can be retrieved from its product information. This product category is then used as mapping data corresponding to the user's search text, serving as a supervisory label for the user's search text. It's easy to understand that user search text is descriptive text, and there is usually a semantic correspondence between the user search text and its mapped product category.

[0092] Step S1130: Use the user search text of each product search event as a training sample, and use the product category corresponding to the user search text as a supervision label, and store them in the original sample set.

[0093] As can be seen from the above introduction, for each product search event, the corresponding user search text and its corresponding product category can be obtained. The user search text can be used to construct training samples, and the product category mapped to the user search text can be defined as supervision labels. The training samples and supervision labels are mapped and stored in the original sample set, thus completing the preparation of the original sample set.

[0094] In one embodiment, when constructing user search text as training samples, the user search text can be preprocessed with text formatting, such as removing stop words and punctuation marks, to standardize it.

[0095] Since the supervisory labels are selected from product categories, the label space corresponding to all supervisory labels in the original sample set is also a classification space, which corresponds in fact to the product classification system. Therefore, the original sample set establishes an intent mapping relationship between the descriptive text in the training samples and the product categories semantically. Thus, the supervisory labels are actually intent labels, used to represent a corresponding intent type. The intent labels can be represented using either character-based text or numerical text, allowing for flexible representation.

[0096] As can be seen from the above embodiments, in the specific application scenario of an e-commerce platform, an original sample set can be automatically constructed based on the user search text in the access behavior data and the corresponding product category of the converted product items. Thus, the access behavior data has the function of truly reflecting the intent mapping result of the user search text. Therefore, it can save the cost of manual annotation, and the cost of obtaining the original sample set is very low and the efficiency is very high.

[0097] Based on any embodiment of this application, please refer to Figure 3 Based on the influence function, the influence degree of each training sample in the original sample set after changing the supervision label is determined. Then, the training samples with relatively high influence degrees are removed to obtain a balanced sample set, including:

[0098] Step S1210: Train the classification model to convergence using the training samples and their supervision labels in the original sample set;

[0099] A classification model can be constructed by combining any feature extraction model suitable for representing text encoded information with a classifier. Then, training samples from the original sample set are used to iteratively train the classification model until it converges. During training, a supervised training method is employed. For each training sample, the classification result predicted by the model maps the feature representation of the training sample to the corresponding classification probability set of each category in the classifier's classification space. The classification loss value of the classification result is calculated using the supervised labels. Based on this loss value, it is determined whether the classification model has reached convergence. If convergence is achieved, training terminates. If convergence has not been achieved, the next training sample from the original sample set is used to iteratively train the classification model, and gradient updates are performed to further approach convergence.

[0100] It is easy to understand that the classification model obtained therefrom can theoretically be used to predict the corresponding supervision label for the training sample, that is, to predict the intent label representing the corresponding intent type based on the descriptive text. For the e-commerce scenario exemplified in this application, it can be understood as predicting the corresponding product category based on the descriptive text.

[0101] Step S1220: Change the supervision label of each training sample in the original sample set, and input each training sample into the classification model one by one to predict the classification result.

[0102] According to the theory of influence functions, after the model is trained, it can be used to determine the influence of each training sample without relying on the model to be retrained. Accordingly, based on the converged classification model, each training sample in the original sample set is predicted one by one to obtain its corresponding classification result. In order to evaluate the influence of each training sample, for the training sample input into the classification model, its supervision label can be changed first, and its original supervision label can be modified to any other supervision label.

[0103] Step S1230: Determine the classification loss value of the corresponding training sample based on the changed supervision label, and apply the influence function to determine the influence degree of the corresponding training sample according to the classification loss value;

[0104] Furthermore, for the classification result, the classification loss value is calculated using the modified supervision label of the predicted training sample. Then, an influence function is applied to calculate the corresponding influence degree based on the classification loss value. The theory and specific formula of the influence function are known to those skilled in the art and will not be elaborated upon here.

[0105] After each training sample has its corresponding influence determined by the classification model, the mapping relationship between the training samples and the influence can be temporarily stored so that the training samples can be screened based on the influence.

[0106] Step S1240: Remove some training samples in the original sample set whose influence exceeds the set threshold, and retain the other training samples to form a balanced sample set.

[0107] After obtaining the corresponding influence value for each training sample in the original sample set, in one embodiment, the training samples can be sorted according to their influence values. Training samples with higher influence values ​​can be manually examined and removed from the original sample set for cleanup. In another embodiment, a set threshold can be provided. The influence value of each training sample is then compared with this threshold. If the influence value is higher than the threshold, the training sample is removed from the original sample set for cleanup; otherwise, it is retained. Since training samples with excessively high influence values ​​can easily cause the trained model to become oversensitive, removing these training samples from the original sample set ensures that the remaining training samples have a relatively balanced influence level, thus preventing the trained model from being excessively affected by individual training samples and avoiding overfitting or underfitting.

[0108] As can be seen from the above embodiments, after applying the influence function, the influence degree of each training sample in the original sample set can be determined by training a classification model once. Then, the seed samples in the original sample set are cleaned up according to the influence degree, so that the influence degree of the remaining seed samples on the trained model is more balanced, and the quality of the seed samples is ensured.

[0109] Based on any embodiment of this application, please refer to Figure 4 Text augmentation is performed on a subset of training samples from the balanced sample set to expand the training samples and obtain an augmented sample set, including:

[0110] Step S1310: Detect and determine the number of word segments in the training samples in the balanced sample set, and determine the training samples with a number of word segments greater than a preset threshold as first type training samples, and the rest as second type training samples.

[0111] The training samples in the balanced sample set can be used as seed samples to amplify more training samples. In this embodiment, different types of seed samples are first distinguished. To this end, each training sample in the balanced sample set can be segmented into words first. Then, based on the number of words segmented in each training sample, training samples with a number of words segmented greater than a preset threshold are classified as first-type training samples, and training samples with a number of words segmented not greater than the preset threshold are classified as second-type training samples.

[0112] It is easy to understand that the first type of training samples are mainly seed samples with a large number of words and sufficient semantics, while the second type of training samples have relatively fewer words and relatively insufficient semantics, so they can be treated differently.

[0113] Step S1320: Based on the first type of training samples, while keeping the original supervision label correspondence unchanged, new training samples are expanded by performing word segmentation granularity editing operations.

[0114] For the first type of training samples, literal-level text augmentation can be applied. This involves editing the segmented words individually, enhancing or deleting them to create new training samples. When editing a single training sample, the number of edited segments can be controlled to not exceed a certain proportion, for example, the total number of edited segments should not exceed 20%. Since the semantics of the new training samples expanded from the first type of training samples remain largely unchanged, the original supervision labels can be used to establish a mapping relationship with them and store them back in the balanced sample set.

[0115] Step S1330: Based on the second type of training samples, the local description texts of different training samples corresponding to the same original supervision label are randomly concatenated and expanded into new training samples corresponding to the original supervision label.

[0116] For the second type of training samples, they can be classified according to their original supervision labels. For training samples with the same original supervision labels, two or more training samples are extracted each time, and local description text is extracted from each training sample. Then, multiple such local description texts are randomly concatenated to obtain new training samples. These training samples are then stored back into the balanced sample set after establishing a mapping relationship with their original supervision labels.

[0117] In one embodiment, the decision to expand the training samples under a given original supervision label with new training samples can be determined based on the original supervision label. Specifically, a preset threshold can be set. When the total number of training samples associated with a certain original supervision label is lower than the preset threshold, new training samples are expanded for that original supervision label based on a second type of training samples. When the total number of training samples is not lower than the preset threshold, it indicates that the training samples corresponding to that original supervision label are sufficient, and there is no need to expand it with new training samples based on a second type of training samples. This can make the distribution of the total number of training samples corresponding to each supervision label more balanced.

[0118] Step S1340: Based on the expanded training samples, apply the text generation model to further expand more training samples, and add all the newly added training samples to the balanced sample set to form an augmented sample set.

[0119] Furthermore, based on the new training samples obtained through literal-level text augmentation, a text generation model can be used to generate more new training samples using the new training samples as input. Of course, the newly generated training samples are still mapped back to the balanced sample set with the supervision labels of the input training samples, so that the balanced sample set constitutes the augmented sample set.

[0120] In one embodiment, the text generation model can be implemented using a pre-trained model such as the BERT model. The BERT model performs tasks such as filling in blanks and word embedding replacements on the input training samples to generate new training samples.

[0121] Based on the above embodiments, it is easy to understand that in the process of expanding the training samples, this application distinguishes training samples with different total word segments and performs text enhancement processing for different situations. First, it expands the training samples to enrich the sample set by performing text enhancement at the literal level. Then, it further expands the training samples by using a text generation model to obtain an augmented sample set. In the process, it maintains the correct mapping relationship when determining the training samples and supervision labels. It effectively enriches the total number of training samples on the basis of the already preliminarily optimized balanced sample set, ensuring that the augmented sample set contains a large number of training samples and that these training samples can uniformly generalize various feature information.

[0122] Based on any embodiment of this application, please refer to Figure 5 Based on the clustering results of the deep semantic information of each training sample in the augmented sample set, outlier training samples in the augmented sample set are removed to obtain an optimized sample set, including:

[0123] Step S1410: Use the training samples in the augmented sample set to iteratively train the feature extraction model until convergence, so that the feature extraction model is suitable for feature representation of the training samples;

[0124] To further optimize the composition of the training samples in the augmented sample set, a feature extraction model is trained using contrastive learning with these training samples. The BERT model is recommended for this purpose, as it can undergo self-supervised training, typically training until convergence to acquire the ability to accurately represent the features of the input training samples. Of course, any other suitable basic model for representing textual information can be used instead; those skilled in the art can implement this flexibly.

[0125] Step S1420: Using the converged feature extraction model, extract the deep semantic information of each training sample in the augmented sample set one by one, and compress the deep semantic information into a low-dimensional vector of the same scale.

[0126] Once the feature extraction model is trained to convergence, it is used to represent the features of each training sample in the augmented sample set one by one, thereby obtaining the deep semantic information corresponding to each training sample. It is easy to understand that this deep semantic information is a high-dimensional dense vector.

[0127] In one embodiment, a fully connected layer can be added after the feature extraction model to map the deep semantic information of each training sample into a low-dimensional vector, or a pooling layer can be added after it to reduce the dimensionality of the deep semantic information into a low-dimensional vector. In another embodiment, the t-SNE algorithm can be used to map the deep semantic information into a low-dimensional vector. Regardless of the method used, the deep semantic information can be compressed to obtain its corresponding low-dimensional vector.

[0128] Step S1430: Use a clustering algorithm to cluster the low-dimensional vectors of each training sample to obtain multiple clusters, such that each cluster includes a portion of the training samples in the augmented sample set.

[0129] In one embodiment, the clustering algorithm used when clustering the training samples can be selected from a variety of algorithms, including but not limited to: K-Means clustering algorithm, mean-shift clustering algorithm, density-based clustering algorithm (DBSCAN), expectation-maximum clustering algorithm based on Gaussian mixture model, agglomerative hierarchical clustering algorithm, graph community detection clustering algorithm, etc.

[0130] It is easy to understand that after using any clustering algorithm to cluster the low-dimensional vectors of each training sample, multiple clusters can be obtained. Each cluster contains a portion of the training samples in the augmented sample set. However, there may be individual training samples that are outside of each cluster. These training samples are the outliers.

[0131] Step S1440: Take the training samples that do not fall into any cluster as outlier training samples, remove the outlier training samples from the augmented sample set, and obtain the optimized sample set.

[0132] Outlier training samples are usually marginal data corresponding to the long-tail effect. Therefore, these outlier training samples can be removed from the augmented sample set to achieve cleanup. After cleanup, the augmented sample set becomes the optimized sample set, which can be used to effectively train the model corresponding to the downstream task.

[0133] In one embodiment, the process from steps S1410 to S1440 can be executed iteratively. The optimized sample set obtained in each iteration is returned to step S1410 and used again as an augmented sample set for iteration. It is then used to train the feature extraction model again. The feature extraction model is used to extract deep semantic information and map it into a low-dimensional vector. Then, a clustering algorithm is used to eliminate the interference of outlier training samples, and an optimized sample set is obtained again. This process is repeated, and after a finite number of iterations, an optimized sample set that meets the expectations can be obtained.

[0134] As can be seen from the above embodiments, the feature extraction model trained using the augmented sample set is used to represent the training samples in the augmented sample set to obtain a low-dimensional vector of deep semantic information. Then, clustering is used to eliminate the interference of outlier training samples, thereby eliminating the marginal data corresponding to the long-tail effect and further improving the data quality of the final optimized sample set.

[0135] Based on any embodiment of this application, please refer to Figure 6 After the clustering results based on the deep semantic information of each training sample in the augmented sample set are obtained, and outlier training samples are removed from the augmented sample set, the optimized sample set is obtained, including:

[0136] Step S2100: Train the search redirection model to convergence using the training samples in the optimized sample set, so that the search redirection model is suitable for predicting the corresponding intent tag based on the input descriptive text.

[0137] Prepare a search redirection model. This model can use any suitable basic model for text feature representation as its feature extraction model, followed by a classification model. For example, the feature extraction model can use basic models such as DistilRoberta, DistilBert, Bert, or FasText. The classifier can be a multi-classifier, and its classification space can correspond to a product classification system, including multiple categories. Each category indicates a corresponding intent category to the user, i.e., a product category. For training samples that should be predicted as a certain intent category, the corresponding supervision label of the training sample is used for supervised training.

[0138] It is easy to understand that by iteratively training the search redirection model using training samples from the optimized sample set, it can be trained to a convergent state. Since the optimized sample set has been optimized using the technical solution of this application, it is easier to quickly train the search redirection model to a convergent state and ensure higher prediction accuracy. Therefore, this search redirection model can be used to predict the corresponding intent category for input descriptive text and represent it as a corresponding intent label. For example, when "hiking shoes" is input into the search redirection model, the corresponding intent label "sports shoes" can be predicted.

[0139] Step S2200: In response to the user's search request, input the user's search text carried in the request into the converged search redirection model to predict the intent tag corresponding to the user's search text;

[0140] The search redirection model trained to convergence can be configured as a redirection service to respond to user search requests from independent websites on e-commerce platforms. Based on the user search text carried in the user search request, it predicts the corresponding intent tag, thereby predicting the user's intent and redirecting the user search text to the intended target.

[0141] When a user initiates a search request on an e-commerce platform, the redirection service obtains the user's search text carried in the search request. Then, the encoded information of the user's search text is input into the search redirection model, which can then predict the corresponding intent tag.

[0142] Step S2300: Retrieve product items carrying the intent tags from the product database of the independent website, and construct a product information list to respond to the user's search request.

[0143] After obtaining the intent tag corresponding to the user's search text in the user's search request, a search can be performed on the product database of the corresponding independent website to retrieve all product items carrying the intent tag. Then, a portion of these product items are selected to construct a product information list. The product information list stores the product information of each selected product item, including but not limited to product title, product image, product price, and product link. This product information list is then pushed to the user to complete the response to their search request, providing results that better match the user's expressed intent.

[0144] As can be seen from the above embodiments, the optimized sample set obtained in this application is suitable for training a search redirection model to accurately predict the true intent of user search text, thereby enabling the redirection processing of user search text, making the search results for product items more accurate, and comprehensively improving the service experience of e-commerce platforms.

[0145] Please see Figure 7According to one aspect of this application, a sample set optimization apparatus includes a sample acquisition module 1100, a balance optimization module 1200, an augmentation optimization module 1300, and a clustering optimization module 1400, wherein: the sample acquisition module 1100 is configured to acquire an original sample set, including multiple training samples and their corresponding supervision labels, wherein the training samples are descriptive text and the supervision labels are intent labels corresponding to the descriptive text; the balance optimization module 1200 is configured to determine the influence degree of each training sample in the original sample set after changing the supervision label according to an influence function, and remove some training samples with relatively high influence degrees to obtain a balanced sample set; the augmentation optimization module 1300 is configured to perform text enhancement processing based on some training samples in the balanced sample set, and expand the training samples through text enhancement to obtain an augmented sample set; the clustering optimization module 1400 is configured to remove outlier training samples in the augmented sample set based on the clustering results of the deep semantic information of each training sample in the augmented sample set to obtain an optimized sample set.

[0146] Based on any embodiment of this application, the sample acquisition module 1100 includes: a material acquisition unit, configured to acquire user search text corresponding to product search events in an e-commerce platform; a tag acquisition unit, configured to acquire product categories corresponding to product items accessed by users in the product results list obtained based on the user search text; and a sample construction unit, configured to use the user search text of each product search event as a training sample and the product category corresponding to the user search text as a supervision tag, and store them in the original sample set.

[0147] Based on any embodiment of this application, the equilibrium optimization module 1200 includes: a classification model training unit, configured to train a classification model to convergence using training samples and their supervision labels in the original sample set; a classification model prediction unit, configured to change the supervision labels of each training sample in the original sample set, and input each training sample into the classification model one by one to predict the classification result; an influence calculation unit, configured to determine the classification loss value of the corresponding training sample based on the changed supervision label, and apply an influence function to determine the influence degree of the corresponding training sample according to the classification loss value; and a sample equilibrium cleaning unit, configured to remove some training samples in the original sample set whose influence degree is higher than a set threshold, and retain other training samples to form an equilibrium sample set.

[0148] Based on any embodiment of this application, the augmentation and optimization module 1300 includes: a type detection unit, configured to detect and determine the number of word segments in the training samples of the balanced sample set, and determine the word segments with a number greater than a preset threshold as first type training samples, and the rest as second type training samples; a first expansion unit, configured to expand new training samples based on the first type training samples, while keeping the original supervision label correspondence unchanged, by performing word segmentation granularity editing operations; a second expansion unit, configured to expand new training samples corresponding to the original supervision label by randomly concatenating the local description texts of different training samples corresponding to the same original supervision label based on the second type training samples; and an augmentation implementation unit, configured to further expand more training samples by applying a text generation model on the basis of the expanded training samples, and add all the newly added training samples to the balanced sample set to form an augmented sample set.

[0149] Based on any embodiment of this application, the clustering optimization module 1400 includes: an extraction model training unit, configured to iteratively train a feature extraction model to convergence using training samples in the augmented sample set, making the feature extraction model suitable for representing the training samples; a sample feature representation unit, configured to extract the deep semantic information of each training sample in the augmented sample set one by one using the converged feature extraction model, and compress the deep semantic information into a low-dimensional vector of the same scale; a sample clustering processing unit, configured to cluster the low-dimensional vectors of each training sample using a clustering algorithm, thereby obtaining multiple clusters, such that each cluster includes a portion of the training samples in the augmented sample set; and a sample outlier optimization unit, configured to treat training samples that do not fall into any cluster as outlier training samples, remove outlier training samples from the augmented sample set, and obtain an optimized sample set.

[0150] Based on any embodiment of this application, the clustering optimization module 1400 is run iteratively multiple times so that the training samples belonging to edge data with long-tail effects have been removed from the final optimized sample set.

[0151] Based on any embodiment of this application, following the clustering optimization module 1400, the sample set optimization device of this application further includes: a model training module, configured to train the search redirection model to convergence using training samples from the optimized sample set, so that the search redirection model is suitable for predicting the corresponding intent tag based on the input descriptive text; a model prediction module, configured to respond to a user search request, input the user search text carried by the request into the converged search redirection model, and predict the intent tag corresponding to the user search text; and a data retrieval module, configured to retrieve product items carrying the intent tag from the product database of the independent website based on the intent tag, and construct a product information list to respond to the user search request.

[0152] Another embodiment of this application also provides a sample set optimization device. For example... Figure 8 The diagram shows the internal structure of a sample set optimization device. This device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable, non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database stores information sequences, and when executed by the processor, these computer-readable instructions enable the processor to implement a sample set optimization method.

[0153] The processor of the sample set optimization device provides computational and control capabilities to support the operation of the entire device. The memory of the device can store computer-readable instructions, which, when executed by the processor, cause the processor to perform the sample set optimization method of this application. The network interface of the device is used for communication with a terminal.

[0154] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the sample set optimization device to which the present application is applied. The specific sample set optimization device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0155] In this embodiment, the processor is used to execute... Figure 7 The specific functions of each module are described, and the memory stores the program code and various data required to execute the above modules or sub-modules. The network interface is used to realize data transmission between user terminals or servers. In this embodiment, the non-volatile readable storage medium stores the program code and data required to execute all modules in the sample set optimization device of this application, and the server can call the server's program code and data to execute the functions of all modules.

[0156] This application also provides a non-volatile readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the sample set optimization method of any embodiment of this application.

[0157] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0158] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0159] In summary, as can be seen from this application, from a product perspective, the implementation of this application achieves better classification results by using a search redirection model to classify user search text into e-commerce product categories. Better classification results lead to higher ranking results, thereby increasing user click-through rate / conversion rate and bringing higher exposure benefits. From an algorithm model technology perspective, through comparative experiments, the accuracy of the model is improved under the same model and data source conditions. From a cost perspective, the manual annotation cost is significantly reduced in the data preparation stage of product intent classification.

Claims

1. A sample set optimization method, characterized in that, include: Obtaining an original sample set, including multiple training samples and their corresponding supervision labels, wherein the training samples are descriptive text and the supervision labels are intent labels corresponding to the descriptive text, includes: obtaining user search text corresponding to product search events in an e-commerce platform; obtaining product categories corresponding to product items accessed by users in the product results list obtained based on the user search text; storing the user search text of each product search event as a training sample and the product category corresponding to the user search text as a supervision label in the original sample set; The influence degree of each training sample in the original sample set after changing the supervision label is determined according to the influence function. Then, the training samples with relatively high influence degree are removed to obtain a balanced sample set. Text augmentation is performed on a portion of the training samples in the balanced sample set to expand the training samples and obtain an augmented sample set. Based on the clustering results of the deep semantic information of each training sample in the augmented sample set, outlier training samples in the augmented sample set are removed to obtain an optimized sample set.

2. The sample set optimization method according to claim 1, characterized in that, The influence of each training sample in the original sample set after changing the supervision label is determined based on the influence function. Then, the training samples with relatively high influence are removed to obtain a balanced sample set, including: The classification model is trained to convergence using the training samples and their supervision labels from the original sample set. The supervision labels of each training sample in the original sample set are changed, and each training sample is input into the classification model one by one to predict the classification result. Based on the modified supervision labels, the classification loss value of the corresponding training samples is determined, and the influence function is applied to determine the influence degree of the corresponding training samples according to the classification loss value. Remove training samples whose influence exceeds a set threshold from the original sample set, and retain the other training samples to form a balanced sample set.

3. The sample set optimization method according to claim 1, characterized in that, Text augmentation is performed on a subset of training samples from the balanced sample set to expand the training samples, resulting in an augmented sample set, including: The number of word segments in the training samples of the balanced sample set is detected and determined. Those with a number of word segments greater than a preset threshold are identified as first-type training samples, and the rest are identified as second-type training samples. Based on the first type of training samples, new training samples are expanded by performing word segmentation granularity editing operations while keeping the original supervision label correspondence unchanged. Based on the second type of training samples, the local description texts of different training samples corresponding to the same original supervision label are randomly concatenated and expanded into new training samples corresponding to the original supervision label. Based on the expanded training samples, a text generation model is applied to further expand the training samples. All the newly added training samples are added to the balanced sample set to form an augmented sample set.

4. The sample set optimization method according to claim 1, characterized in that, Based on the clustering results of the deep semantic information of each training sample in the augmented sample set, outlier training samples are removed from the augmented sample set to obtain an optimized sample set, including: The feature extraction model is iteratively trained until convergence using training samples from the augmented sample set, making the feature extraction model suitable for representing features from the training samples; The converged feature extraction model is used to extract the deep semantic information of each training sample in the augmented sample set one by one, and the deep semantic information is compressed into a low-dimensional vector of the same scale. Clustering algorithms are used to cluster the low-dimensional vectors of each training sample, resulting in multiple clusters, each of which includes a portion of the training samples in the augmented sample set. Training samples that do not fall into any cluster are taken as outliers. Outliers in the augmented sample set are removed to obtain an optimized sample set.

5. The sample set optimization method according to any one of claims 1 to 4, characterized in that, Based on the clustering results of the deep semantic information of each training sample in the augmented sample set, the step of removing outlier training samples from the augmented sample set and obtaining an optimized sample set is performed iteratively multiple times so that the final optimized sample set has removed training samples belonging to marginal data with long-tail effects.

6. The sample set optimization method according to any one of claims 1 to 4, characterized in that, After clustering the deep semantic information of each training sample in the augmented sample set, removing outlier training samples from the augmented sample set, and obtaining the optimized sample set, the process includes: The search redirection model is trained to convergence using training samples from the optimized sample set, making the search redirection model suitable for predicting the corresponding intent tag based on the input descriptive text. In response to a user's search request, the user's search text carried in the request is input into a converged search redirection model to predict the intent tag corresponding to the user's search text. Based on the intent tag, retrieve product items carrying the intent tag from the product database of the independent website, and construct a product information list to respond to the user's search request.

7. A sample set optimization device, characterized in that, include: The sample acquisition module is configured to acquire an original sample set, including multiple training samples and their corresponding supervision labels. The training samples are descriptive text, and the supervision labels are intent labels corresponding to the descriptive text. The module includes: acquiring user search text corresponding to product search events in an e-commerce platform; acquiring product categories corresponding to product items accessed by users in the product results list obtained based on the user search text; and storing the user search text of each product search event as a training sample and the product category corresponding to the user search text as a supervision label in the original sample set. The equilibrium optimization module is configured to determine the influence degree of each training sample in the original sample set after changing the supervision label based on the influence function, and remove the training samples with relatively high influence degree to obtain a balanced sample set. The augmentation and optimization module is configured to perform text augmentation processing on a portion of the training samples in the balanced sample set, thereby expanding the training samples through text augmentation to obtain an augmented sample set; The clustering optimization module is configured to remove outlier training samples from the augmented sample set based on the clustering results of the deep semantic information of each training sample in the augmented sample set, thereby obtaining an optimized sample set.

8. The sample set optimization apparatus according to claim 7, characterized in that, The device further includes: The model training module is configured to train the search redirection model to convergence using training samples from the optimization sample set, so that the search redirection model is suitable for predicting the corresponding intent label based on the input descriptive text. The model prediction module is configured to respond to user search requests by inputting the user search text carried in the request into a converged search redirection model and predicting the intent tag corresponding to the user search text. The data retrieval module is configured to retrieve product items carrying the intent tags from the product database of the independent website, and construct a product information list to respond to the user's search request.

9. A sample set optimization device, comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps included in the method as described in any one of claims 1 to 6.

10. A non-volatile readable storage medium, characterized in that, It stores a computer program in the form of computer-readable instructions, which, when invoked by a computer, performs the steps included in the method as described in any one of claims 1 to 6.

Images