Processing method and device of training data, equipment and readable medium

Abstract

The application provides a kind of training data processing method, device and equipment and readable medium, by from the multiple categories that classification model can classify, select multiple key optimization categories.Then at least according to first screening condition, training sample data is screened from candidate sample set.Wherein, the category of the candidate sample data to be screened and any one key optimization category match, candidate sample set includes multiple candidate sample data.Because the similarity degree between any two key optimization categories in multiple key optimization categories is less than similarity degree threshold, and the similarity degree between two key optimization categories is the similarity degree between sample data under two key optimization categories, therefore, according to the training sample data between different categories screened according to first screening condition, when using the training sample data to train the classification model to be optimized, the optimization effect of the classification model to be optimized can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, and readable medium for processing training data. Background Technology

[0002] Classification models are used to classify sample data. In existing technologies, the optimization process of classification models involves directly inputting a large amount of sample data into a feature classifier for model training, which then obtains and outputs the optimized classification model. The sample data input into the feature classifier carries the category information of the sample data.

[0003] However, the similarity between some sample data of different categories used in the optimization process is high. For example, the sample data "I want to recharge my phone credit" and "I want to recharge my data" are highly similar, but belong to different categories. As a result, similar sample data are easily confused during the optimization of the classification model, resulting in poor optimization effect of the classification model, that is, the improvement of classification accuracy is small. Summary of the Invention

[0004] In view of this, the training data processing method, apparatus, device, and readable medium provided in the embodiments of the present invention are used to train the classification model to be optimized by acquiring training sample data using multiple dissimilar categories, thereby improving the optimization effect of the classification model.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0006] The first aspect of this application discloses a method for processing training data, including:

[0007] From the multiple categories that can be classified by the classification model, multiple key optimization categories are selected; wherein, the similarity between any two key optimization categories is less than a similarity threshold; the similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories;

[0008] Training sample data is obtained from the candidate sample set based on at least a first screening condition; wherein, the first screening condition is that the category of the candidate sample data to be screened matches any one of the key optimization categories; the candidate sample set includes multiple candidate sample data.

[0009] The selected training sample data is added to the first training set to obtain the second training set; the second training set is used to train the optimized classification model.

[0010] Optionally, in the above-described method for processing training data, before selecting multiple key optimization categories from the multiple categories that can be classified by the classification model, the method further includes:

[0011] Obtain the first classification accuracy of the classification model to be optimized on the validation sample data of each category in the validation set; wherein, the classification model to be optimized is trained using the training sample data in the first training set;

[0012] Categories whose first classification accuracy is less than the first accuracy threshold are identified as categories to be optimized.

[0013] The classification model can classify multiple categories, which are the determined categories to be optimized.

[0014] Optionally, in the above-described method for processing training data, before obtaining training sample data from the candidate sample set based at least on the first screening condition, the method further includes:

[0015] For each of the aforementioned key optimization categories, features of each model sample data under that key optimization category are obtained; wherein, the model sample data is validation sample data or training sample data; the features of the model sample data are used to characterize the key information of the model sample data;

[0016] The step of selecting training sample data from the candidate sample set based at least on a first screening criterion includes:

[0017] Training sample data is obtained by filtering from the candidate sample set according to the first and second screening conditions; wherein, the second screening condition is the feature of the candidate sample data to be screened, which matches the feature of the model sample data under any of the key optimization categories.

[0018] Optionally, in the above training data processing method, the step of obtaining the features of each model sample data under each of the key optimization categories includes:

[0019] For each of the key optimization categories, each model sample data under the key optimization category is input into the feature extraction model, and the feature extraction model obtains and outputs the features of each model sample data; wherein, the feature extraction model is obtained by training the neural network model with multiple sample data.

[0020] Optionally, in the above-described method for processing training data, before obtaining training sample data from the candidate sample set based at least on the first screening condition, the method further includes:

[0021] Obtain the second classification accuracy of the classification model to be optimized for candidate sample data in each category of the candidate sample set;

[0022] Candidate sample data for categories whose second classification accuracy is less than the second accuracy threshold are determined as processed candidate sample data.

[0023] Wherein, obtaining training sample data from the candidate sample set based at least on the first screening condition includes:

[0024] Training sample data is obtained from the processed candidate sample data based at least on the first screening criteria.

[0025] Optionally, in the above training data processing method, the step of selecting multiple key optimization categories from the multiple categories that the classification model can classify includes:

[0026] From the multiple categories that can be classified by the classification model, several key optimization categories are selected based on the degree of similarity between each pair of categories.

[0027] Optionally, in the above training data processing method, the step of selecting multiple key optimization categories from the multiple categories that can be classified by the classification model based on the similarity between each pair of categories includes:

[0028] Select one category from the multiple categories that the classification model can classify, and determine it as the category to be optimized.

[0029] Among the remaining categories other than those already identified as key optimization categories, the category with the highest similarity to the target category and that meets the third screening condition is selected; wherein, the target category is the category most recently identified as a key optimization category; and the third screening condition is that the similarity between the category to be selected and the target category is less than a similarity threshold.

[0030] If a category with the highest similarity to the target category and that meets the third screening condition is selected, the selected category is determined as the key optimization category, and the selected category is used as the new target category. The process then returns to the step of selecting the category with the highest similarity to the target category and that meets the third screening condition from the remaining categories other than those already determined as key optimization categories, until no category with the highest similarity to the target category and that meets the third screening condition is selected.

[0031] Optionally, in the above-described method for processing training data, selecting the category with the highest similarity to the target category and satisfying the third screening condition from the remaining categories other than those already identified as key optimization categories includes:

[0032] For each validation sample data under the target category in the validation set, based on the validation sample data, two training sample data that match the validation sample data are obtained from all training sample data under the remaining categories except those that have been identified as the key optimization categories; wherein, the two training sample data that match the validation sample data are the two training sample data in the first training set that have the highest similarity to the validation sample data and belong to different categories.

[0033] For each validation sample data under the target category in the validation set, calculate the similarity difference between two training sample data that match the validation sample data. If the absolute value of the similarity difference is less than a first threshold, then increment the frequency count of similar pairs of the category combinations corresponding to the two training sample data that match the validation sample data by one.

[0034] For each category combination identified in the statistics, the ratio of the total frequency of similar pairs of the category combination to the total number of validation sample data under the target category in the validation set is determined as the similarity degree of the category combination; wherein, the similarity degree of the category combination is the similarity degree between the two categories included in the category combination;

[0035] Based on the similarity of all category combinations, the category with the highest similarity to the target category and that meets the third screening condition is selected from all the statistically analyzed category combinations.

[0036] Optionally, in the above method for processing training data, after adding the selected training sample data to the first training set to obtain the second training set, the method further includes:

[0037] The model is trained using training sample data from the second training set to obtain an optimized classification model.

[0038] Optionally, in the above training data processing method, after training the classification model to be optimized using the training sample data in the first training set to obtain the optimized classification model, the method further includes:

[0039] The optimized classification model is used as the new classification model to be optimized, and the process returns to the step of obtaining the first classification accuracy of the classification model to be optimized for each category of the validation sample data in the validation set.

[0040] Optionally, in the above training data processing method, before the step of using the optimized classification model as a new classification model to be optimized, and using the second training set as a new first training set, and returning to the step of obtaining the first classification accuracy of the classification model to be optimized for each category of the validation sample data in the validation set, the method further includes:

[0041] Update the candidate sample data in the candidate sample set.

[0042] A second aspect of this application discloses a training data processing apparatus, comprising:

[0043] The first selection unit is used to select multiple key optimization categories from multiple categories that can be classified by the classification model; wherein, the similarity between any two key optimization categories is less than a similarity threshold; the similarity between two key optimization categories is the similarity between sample data under the two key optimization categories;

[0044] The first screening unit is configured to screen training sample data from a candidate sample set based on at least a first screening condition; wherein the first screening condition is the category of the candidate sample data to be screened, which matches any one of the key optimization categories; the candidate sample set includes multiple candidate sample data.

[0045] An addition unit is used to add the selected training sample data to the first training set to obtain the second training set; wherein, the second training set is used to train the optimized classification model.

[0046] Optionally, the above-mentioned training data processing apparatus further includes:

[0047] The first acquisition unit is used to acquire the first classification accuracy of the classification model to be optimized for each category of the validation sample data in the validation set; wherein the classification model to be optimized is trained using the training sample data in the first training set.

[0048] The first determining unit is used to determine the categories whose first classification accuracy is less than the first accuracy threshold as categories to be optimized.

[0049] The classification model can classify multiple categories, which are the determined categories to be optimized.

[0050] Optionally, the above-mentioned training data processing apparatus further includes:

[0051] The second acquisition unit is used to acquire features of each model sample data under each of the key optimization categories; wherein, the model sample data is validation sample data or training sample data; the features of the model sample data are used to characterize the key information of the model sample data;

[0052] The first filtering unit includes:

[0053] The first screening subunit is used to screen training sample data from the candidate sample set according to the first screening condition and the second screening condition; wherein, the second screening condition is the feature of the candidate sample data to be screened, which matches the feature of the model sample data under any of the key optimization categories.

[0054] Optionally, in the above-mentioned training data processing apparatus, the second acquisition unit includes:

[0055] The output subunit is used to input each model sample data under each of the key optimization categories into the feature extraction model for each key optimization category, and the feature extraction model obtains and outputs the features of each model sample data; wherein, the feature extraction model is obtained by training the neural network model with multiple sample data.

[0056] Optionally, the above-mentioned training data processing apparatus further includes:

[0057] The third acquisition unit is used to acquire the second classification accuracy of the classification model to be optimized for candidate sample data under each category in the candidate sample set;

[0058] The second determining unit is used to determine the candidate sample data of the category whose second classification accuracy is less than the second accuracy threshold as the processed candidate sample data.

[0059] The first screening subunit includes:

[0060] The second screening subunit is used to screen training sample data from the processed candidate sample data based at least on the first screening condition.

[0061] Optionally, in the above-mentioned training data processing apparatus, the first selection unit includes:

[0062] The first selection subunit is used to select multiple key optimization categories from the multiple categories that can be classified by the classification model, based on the degree of similarity between each pair of categories.

[0063] Optionally, in the above-mentioned training data processing apparatus, the first selection subunit includes:

[0064] The first determining subunit is used to select one of the multiple categories that can be classified by the classification model and determine it as the key optimization category;

[0065] The second selection subunit is used to select the category with the highest similarity to the target category and that meets the third screening condition from the remaining categories other than those already identified as key optimization categories; wherein, the target category is the category most recently identified as a key optimization category; and the third screening condition is that the similarity between the category to be selected and the target category is less than a similarity threshold.

[0066] The second determining subunit is used to determine the selected category as the key optimization category if the category with the highest similarity to the target category and which meets the third screening condition is selected.

[0067] The first return subunit is used to return the selected category as the new target category to the second selection subunit until the second selection subunit fails to select a category that has the highest similarity to the target category and meets the third filtering condition.

[0068] Optionally, in the above-mentioned training data processing apparatus, the second selection subunit includes:

[0069] The matching subunit is used to, for each validation sample data under the target category in the validation set, obtain two training sample data that match the validation sample data from all training sample data under the remaining categories except those that have been identified as the key optimization categories; wherein, the two training sample data that match the validation sample data are the two training sample data in the first training set that have the highest similarity to the validation sample data and belong to different categories.

[0070] The calculation subunit is used to calculate the similarity difference between two training sample data that match the verification sample data for each verification sample data under the target category in the verification set. If the absolute value of the similarity difference is less than a first threshold, the frequency of similar pairs of the category combinations corresponding to the two training sample data that match the verification sample data is incremented by one.

[0071] The third determining subunit is used to determine the similarity degree of each category combination by the ratio of the total frequency of similar pairs of the category combination to the total number of verification sample data under the target category in the verification set; wherein, the similarity degree of the category combination is the similarity degree between the two categories included in the category combination;

[0072] The third selection subunit is used to select the category with the highest similarity to the target category and that meets the third screening condition from all the statistically obtained category combinations based on the similarity of all category combinations.

[0073] Optionally, the above-mentioned training data processing apparatus further includes:

[0074] The training unit is used to train the model using training sample data from the second training set to obtain an optimized classification model.

[0075] Optionally, the above-mentioned training data processing apparatus further includes:

[0076] The return unit is used to take the optimized classification model as the new classification model to be optimized, and the second training set as the new first training set, and return it to the first acquisition unit.

[0077] Optionally, the above-mentioned training data processing apparatus further includes:

[0078] The update unit is used to update the candidate sample data in the candidate sample set.

[0079] A third aspect of this application discloses a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the first aspects above.

[0080] The fourth aspect of this application discloses a training data processing apparatus, comprising:

[0081] One or more processors;

[0082] A storage device on which one or more programs are stored;

[0083] When the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method as described in any of the first aspects above.

[0084] Based on the training data processing method provided in the above embodiments of the present invention, multiple key optimization categories are selected from the multiple categories that can be classified by the classification model. Then, training sample data is obtained from the candidate sample set according to at least a first screening condition. The first screening condition is that the category of the candidate sample data to be screened matches any one of the key optimization categories, and the candidate sample set includes multiple candidate sample data. Since the similarity between any two key optimization categories is less than a similarity threshold, and the similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories, the training sample data of different categories obtained according to the first screening condition are dissimilar. Therefore, during training, it is less likely that the training sample data will be confused. Thus, using a second training set with the screened training sample data, the optimized classification model trained achieves better optimization results compared to existing optimized classification models trained directly using a large amount of sample data. Attached Figure Description

[0085] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0086] Figure 1 A flowchart illustrating the training data processing method disclosed in the embodiments of this application. Figure 1 ;

[0087] Figure 2 This is a flowchart illustrating a method for selecting key optimization categories disclosed in an embodiment of this application;

[0088] Figure 3 This is a flowchart illustrating another method for selecting key optimization categories disclosed in an embodiment of this application;

[0089] Figure 4 A flowchart illustrating the training data processing method disclosed in the embodiments of this application. Figure 2 ;

[0090] Figure 5 A flowchart illustrating the training data processing method disclosed in the embodiments of this application. Figure 3 ;

[0091] Figure 6 This is a schematic diagram of a process for constructing screening conditions disclosed in an embodiment of this application;

[0092] Figure 7 A flowchart illustrating the training data processing method disclosed in the embodiments of this application. Figure 4 ;

[0093] Figure 8 A flowchart illustrating the training data processing method disclosed in the embodiments of this application. Figure 5 ;

[0094] Figure 9 A flowchart illustrating the training data processing method disclosed in the embodiments of this application. Figure 6 ;

[0095] Figure 10 A flowchart illustrating the training data processing method disclosed in the embodiments of this application. Figure 7 ;

[0096] Figure 11 This is a schematic diagram of the structure of the training data processing device disclosed in the embodiments of this application. Detailed Implementation

[0097] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0098] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0099] Based on the aforementioned optimization issues of classification models, this application proposes a method for processing training data. Detailed descriptions of this method can be found in Embodiments 1, 2, 3, 4, 5, 6, and 7 below. Embodiments 2, 3, 4, 5, 6, and 7 can all be understood as alternative training data processing methods based on Embodiment 1. Furthermore, multiple embodiments can be combined to obtain new embodiments of training data processing methods.

[0100] To make the descriptions of the following embodiments clearer, the definitions of the terms appearing in the following embodiments are first introduced.

[0101] (1) Classification models are used to classify sample data. For example, classification models can be text classification models, image classification models, etc. Among them, text classification models are used to classify text sample data. Image classification models are used to classify image sample data.

[0102] (2) Sample data can be understood as the general term for data that can be used as input into a classification model for classification processing. Different classification models correspond to different types of sample data. For example, the sample data of a text classification model is text-type data (referred to as text sample data), while the sample data of an image classification model is image-type data (referred to as image sample data).

[0103] (3) Validation sample data can be understood as sample data used to validate the classification accuracy (i.e. model performance) of the classification model. Validation sample data is included in the validation set.

[0104] (4) Candidate sample data can be understood as the sample data that candidates add to the training set. Candidate sample data is included in the candidate sample set.

[0105] (5) Training sample data can be understood as sample data used to train the classification model. The training sample data is included in the training set. Specifically, the training set can refer to the first training set mentioned in the following embodiments, or it can refer to the second training set mentioned in the following embodiments. Both the first training set and the second training set include training sample data, but the specific training sample data included in the second training set is different from that in the first training set. Therefore, "first" and "second" are used to distinguish them.

[0106] Example 1

[0107] See Figure 1 To address the aforementioned technical problems, this application provides a method for processing training data, specifically including the following steps:

[0108] S101. Select multiple key optimization categories from the multiple categories that can be classified by the classification model. Among these, the similarity between any two key optimization categories is less than the similarity threshold. The similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories.

[0109] The multiple categories that the classification model can classify can be understood as multiple categories that the classification model can identify from the sample data. Optionally, the multiple categories that the classification model can classify can be all the categories of the sample data that the classification model can identify, or it can be a partial selection of the categories of all the sample data that the classification model can identify. The partial selection can be obtained by selecting from all the categories of the sample data that the classification model can identify. This embodiment of the application does not limit the selection method; it can be arbitrary or selected according to specific requirements.

[0110] The key optimization categories can be understood as those categories for which the classification model needs to improve classification performance. When the similarity between two categories is less than the similarity threshold, the sample data under these two categories are considered dissimilar. The similarity threshold can be set empirically, and the method of setting the similarity threshold is not limited in this embodiment. The lower the similarity between two categories, the lower the probability that the sample data under the two categories are similar. Sample data under a category can be understood as sample data belonging to that category.

[0111] In this embodiment, since the similarity between any two key optimization categories is less than the similarity threshold, any two key optimization categories are dissimilar to each other. That is, the similarity between sample data under any two key optimization categories is very low. When using sample data under multiple key optimization categories to train the classification model to be optimized, since there will be no category confusion, the situation of entanglement between training sample data of different categories due to excessive similarity can be avoided, thereby improving the optimization effect of the classification model.

[0112] Optionally, in one specific embodiment of this application, one implementation of step S101 includes:

[0113] From the multiple categories that can be classified by the classification model, several key optimization categories are selected based on the degree of similarity between each pair of categories.

[0114] Specifically, for each pair of categories that the classification model can classify, the similarity between those two categories is pre-calculated. This similarity can be understood as the overall similarity between sample data within those two categories. The higher the similarity between two categories, the higher the similarity between randomly selected sample data from those categories. Based on the similarity between each pair of categories, several key optimization categories can be selected that satisfy the condition that the similarity between any two categories is less than a similarity threshold.

[0115] There are many ways to calculate the similarity between two categories. For example, one can calculate the similarity between sample data from every two categories and then take the average of all calculated similarities between sample data from all two categories to obtain the similarity between the two categories. Alternatively, one can take the median of all calculated similarities between sample data from all two categories to obtain the similarity between the two categories. There are many ways to calculate the similarity between two categories, including but not limited to the methods proposed in the embodiments of this application.

[0116] Optionally, see Figure 2 In a specific embodiment of this application, selecting multiple key optimization categories from multiple categories that can be classified by the classification model based on the similarity between each pair of categories may include the following steps:

[0117] S201. Select one category from the multiple categories that can be classified by the classification model and determine it as the key optimization category.

[0118] The method of selecting one of the categories can be random selection or selection in a specific order; this application embodiment does not impose any restrictions on this.

[0119] S202. Among the remaining categories other than those already identified as key optimization categories, select the category with the highest similarity to the target category and that meets the third screening condition. The target category is the category that was recently identified as a key optimization category, and the third screening condition is that the similarity between the category to be screened and the target category is less than the similarity threshold.

[0120] The purpose of the third screening criterion is to filter out categories that are dissimilar to the key optimization category. If the similarity between the selected category and the target category is less than a similarity threshold, the category is considered dissimilar to the target category. Since there may be multiple categories dissimilar to the target category, the category with the highest similarity can be selected first, and then step S203 is executed to determine the selected category as the key optimization category. This selected category is then used as the new target category, and the process returns to step S202 to select the next key optimization category. When no category meets the third screening criterion, the process can end, i.e., the selection of key optimization categories stops.

[0121] It should be noted that, apart from the categories already identified as key optimization categories, categories with a similarity level greater than or equal to the key optimization category are not included. Categories with a similarity level greater than or equal to the key optimization category are considered similar to the key optimization category; therefore, before executing step S202, these similar categories will be preferentially eliminated, and then step S202 will be used to select the next key optimization category that meets the criteria.

[0122] Optionally, see Figure 3 In one specific embodiment of this application, one implementation of step S202 includes:

[0123] S301. For each validation sample data under the target category in the validation set, based on the validation sample data, obtain two training sample data that match the validation sample data from all training sample data under the other categories except those that have been identified as the key optimization categories. The two training sample data that match the validation sample data are the two training sample data in the first training set that have the highest similarity to the validation sample data and belong to different categories.

[0124] Specifically, the validation set includes multiple validation sample data points across multiple categories that the classification model can classify. Each validation sample data point under the target category is obtained from the validation set. Then, for each validation sample data point under the target category, the similarity between that validation sample data point and each training sample data point under the remaining categories in the first training set is calculated. From the similarity scores between the validation sample data point and each training sample data point in the first training set, the two training sample data points with the highest similarity scores and belonging to different categories are selected. These two selected training sample data points are the training sample data points that match the validation sample data point. Here, "remaining categories" refers to all categories among the multiple categories that the classification model can classify, excluding those already identified as the focus of optimization, as mentioned in step S101.

[0125] It should be noted that since the similarity between sample data in the same category is often the highest, in the two training sample data selected in step S301, one of the training sample data will be the target category, and the other training sample data will be a category different from the target category.

[0126] It should also be noted that when calculating the similarity between the verification sample data and each training sample data in the first training set, each training sample data in the first training set can be understood as the training sample data under multiple categories that the classification model can classify in step S101, that is, similarity matching calculation is performed on the training sample data under multiple categories among the candidate categories for key optimization.

[0127] For example, when the classification model is a text classification model, the validation set includes sample data of multiple validation sentence patterns under multiple knowledge points. Here, a knowledge point can be understood as a business category summarized after sorting through the collected sample sentence patterns. For instance, if the text classification model is applied to a telecommunications customer service Q&A scenario, the validation set includes sample data of sentence patterns under multiple knowledge points such as phone bill recharge, repair, and data traffic recharge.

[0128] During step S301, a certain knowledge point K-Zsd(i) is the target category, and knowledge point K-Zsd(i) contains several extended questions (i.e., verification sentence sample data), such as q1, q2, and q3. For each verification sentence sample data of knowledge point K-Zsd(i), two training sentence sample data that match the verification sentence sample data are obtained from the first training set. Specifically, taking q1 as an example, q1 is measured by calculating the cosine distance with each training sentence sample data in the first training set (i.e., a way to calculate similarity), resulting in the matched TOP1 sentence Q1 and TOP2 sentence Q2. The knowledge point to which TOP1 sentence Q1 belongs is Zsd_Q1, and the similarity between TOP1 sentence Q1 and q1 is Score_Q1. The knowledge point to which TOP2 sentence Q2 belongs is Zsd_Q2, and the similarity between TOP2 sentence Q2 and q1 is Score_Q2. Among them, Q1 and Q2 are two training sentence sample data with the highest similarity to q1, and Q1 and Q2 belong to different knowledge points.

[0129] S302. For each validation sample data under the target category in the validation set, calculate the similarity difference between two training sample data that match the validation sample data. If the absolute value of the similarity difference is less than the first threshold, increment the frequency of similar pairs of the category combinations corresponding to the two training sample data that match the validation sample data by one.

[0130] For each validation sample data in the target category of the validation set, the difference in similarity between two training samples that match the validation sample data is calculated. If the absolute value of the similarity difference is less than a first threshold, the two matched training samples are considered similar; if the absolute value of the similarity difference is not less than the first threshold, the two matched training samples are considered dissimilar. When the absolute value of the similarity difference is less than the first threshold, the matched training samples are considered similar, and the frequency of similarity pairs of the category combinations corresponding to the two training samples that match the validation sample data is incremented by one. The higher the similarity frequency of a category combination, the higher the similarity between the two categories included in that category combination.

[0131] Taking a text classification model as an example, taking q1 under the aforementioned knowledge point K-Zsd(i) as an example, after calculating Score_Q1 and Score_Q2 in step S301, during the execution of step S302, it is determined whether the absolute value of the similarity difference |Score_Q1-Score_Q2| ≤ the first threshold. If |Score_Q1-Score_Q2| ≤ the first threshold, the frequency of similar pairs of the category combination "Zsd_Q1-Zsd_Q2" (i.e., the combination of knowledge point Zsd_Q1 and knowledge point Zsd_Q2) is incremented by one.

[0132] After executing step S302, there may be one, multiple, or even none of the category combinations with similar pairs counted. If the frequency of similar pairs for all category combinations is 0, it means that the target category is not similar to any other category. Since the similarity between sample data within the same category is often the highest, the category combinations with similar pairs counted after executing step S302 all include the target category.

[0133] S303. For each category combination that has been statistically analyzed, the ratio of the total frequency of similar pairs of that category combination to the total number of validation sample data under the target category in the validation set is determined as the similarity of that category combination, where the similarity of the category combination is the similarity between the two categories included in the category combination.

[0134] For each category combination of similar pairs identified in step S302, the ratio of the total frequency of similar pairs in that category combination to the total number of validation samples belonging to the target category in the validation set is determined as the similarity of that category combination. The similarity of a category combination can be understood as the degree of similarity between the two categories included in the combination. The total number of validation samples under the target category in the validation set can be understood as the total number of all validation samples belonging to the target category in the validation set.

[0135] Continuing with the aforementioned text classification model as an example, during step S303, the similarity of the category combination “Zsd_Qn—Zsd_Qm” is calculated as: Num(“Zsd_Qn—Zsd_Qm”) / num(K-Zsd(i)). Here, Num(“Zsd_Qn—Zsd_Qm”) represents the frequency of similarity pairs between the category combinations of knowledge points Zsd_Qn and Zsd_Qm, while num(K-Zsd(i)) represents the total number of validation sentence samples in the target category K-Zsd(i) of the validation set.

[0136] As can be seen from step S303, the higher the total frequency of similar pairs of category combinations counted in steps S301 to S302, the higher the similarity of the category combinations. As described above, all counted category combinations include the target category; therefore, step S303 can be understood as calculating the similarity between the target category and all other categories except those identified as key optimization categories.

[0137] S304. Based on the similarity of all category combinations, select the category with the highest similarity to the target category and that meets the third screening condition from all statistically analyzed category combinations.

[0138] The category selected in step S304 can be understood as the category with the highest similarity among all categories that are not similar to the target category.

[0139] It should be noted that if no category with the highest similarity to the target category and meeting the third screening condition can be selected in step S304, the process ends, and the selection of the key optimization category ends. If a category with the highest similarity to the target category and meeting the third screening condition is selected, the process continues. Figure 2 Step S203 is shown.

[0140] S203. Determine the selected category as the key optimization category and use the selected category as the new target category, then return to step S202.

[0141] Since the selected category meets the third screening condition and is not similar to the target category, it can be identified as a key optimization category. Then, the newly identified key optimization category (i.e., the category selected in step S202) is used as the new target category, and the process returns to step S202 to continue selecting the next key optimization category until no category with the highest similarity to the target category and meeting the third screening condition is selected, at which point the process ends. Figure 2 The process shown outlines the selection of key optimization categories; proceed with the following steps. Figure 1 The next step, S102, is shown.

[0142] For example, Figure 2The process of selecting key optimization categories can be as follows: The classification model can classify multiple categories, including category A, category B, category C, category D, and category E. First, category A is selected as the key optimization knowledge point. Then, using category A as the target category, step S202 is executed. During the execution of step S202, it is found that category B has the highest similarity among the categories that meet the third screening condition. Therefore, step S203 is executed, and category B is determined as the key optimization category. At this point, categories A and B are determined as key optimization categories. Next, using category B as the new target category, step S202 is returned. During the execution of step S202, among the categories other than category A, category C has the highest similarity to category B and meets the third screening condition. Therefore, step S203 is executed, and category C is determined as the key optimization category. At this point, categories A, B, and C are determined as key optimization categories. Continuing with category C as the new target category, return to step S202. Select the category with the highest similarity among the remaining categories D and E, satisfying the third screening condition. If no category satisfying the third screening condition can be selected from categories D and E, the process ends. The final selected key optimization categories are categories A, B, and C.

[0143] S102. At least according to the first screening condition, training sample data is obtained from the candidate sample set, wherein the first screening condition is that the category of the candidate sample data to be screened matches any key optimization category, and the candidate sample set includes multiple candidate sample data.

[0144] Specifically, the process of executing step S102 can be as follows: for each key optimization category, select each candidate sample data belonging to the key optimization category from the candidate sample set, and then select all candidate sample data that meet the first selection condition, and determine the selected candidate sample data as training sample data.

[0145] During step S102, in addition to using the key optimization category for screening, other screening conditions can also be used to screen candidate sample data. Since the similarity between any two key optimization categories is less than the similarity threshold, and the similarity between two key optimization categories is the similarity between the sample data under those two categories, the training sample data of different categories selected according to the first screening condition will not be too similar and thus affect the training effect of the classification model to be optimized. Using this training sample data to train the classification model to be optimized can improve the classification accuracy of the model.

[0146] Optionally, the process of constructing the candidate sample set is as follows: obtain the current online feedback data, and add the feedback data as candidate sample data to the candidate sample set. Here, online feedback data can be understood as sample data collected online over a recent period. For example, if the classification model is a text classification model, and the application scenario of the text classification model is customer service, then user question phrases collected by the customer service platform can be added as candidate sample data to the candidate sample set. The method of constructing the candidate sample set is not limited in this embodiment.

[0147] Optionally, before executing step S102, the candidate sample data in the candidate sample set can be pre-processed so that each candidate sample data carries its category. For example, the format of the candidate sample data can be pre-converted to a format carrying category labels. For instance, the feature classifier can be used to convert the format of the candidate sample data into a format carrying category labels. Alternatively, the category of each candidate sample data can be manually labeled. After processing the candidate sample data, candidate sample data that meets the first screening condition can be selected from the candidate sample set and used as training sample data.

[0148] S103. Add the selected training sample data to the first training set to obtain the second training set, which is used to train the optimized classification model.

[0149] In this embodiment, the classification model to be optimized can be understood as the classification model before optimization. This classification model to be optimized can be obtained by training the model on a first training set before adding the training sample data from step S102. The first training set also includes multiple training sample data sets before adding the training sample data selected in step S102.

[0150] In this embodiment of the application, the first training set also includes multiple training sample data before the training sample data selected in step S102 is added.

[0151] Specifically, the process of pre-constructing the classification model to be optimized can be as follows: multiple training sample data from the first training set (before adding the training sample data obtained in step S102) are input into a feature classifier, which then obtains and outputs the classification model to be optimized. Here, the training sample data carries the category of that training sample data. That is, it can be understood as the training sample data carrying its own label.

[0152] Taking a text classification model as an example, the construction process of the text classification model to be optimized is as follows: First, the format of multiple training sample data in the first training set is converted into the format of classification feature text, which carries the category labels of the training sample data. The format of the classification feature text is one that can be recognized and processed by the model. Then, each classification feature text is input into the feature classifier, which labels each text, obtaining a pseudo-label for each. The model is trained using each classification feature text and its pseudo-label, and finally, the feature classifier obtains and outputs the text classification model to be optimized. This text classification model to be optimized can be understood as a primitive model with poor classification performance, which cannot meet the user's needs; therefore, optimization is required.

[0153] The training sample data selected in step S102 is added to the first training set, thus updating the training sample data included in the first training set and obtaining the second training set. When the classification model to be optimized is subsequently trained using the second training set, since the newly added training sample data for different categories in the second training set are not similar, excessive similarity will not affect the training effect of the classification model to be optimized. Therefore, using the training sample data of the second training set to train the classification model to be optimized can improve the classification effect of the model on each key optimization category, thereby optimizing the classification model.

[0154] In the training data processing method provided in this application embodiment, multiple key optimization categories are selected from the multiple categories that can be classified by the classification model. Then, training sample data is obtained from the candidate sample set according to at least a first screening condition. The first screening condition is that the category of the candidate sample data to be screened matches any one of the key optimization categories, and the candidate sample set includes multiple candidate sample data. Since the similarity between any two key optimization categories is less than a similarity threshold, and the similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories, the training sample data of different categories selected according to the first screening condition are dissimilar. Therefore, during training, it is less likely that the training sample data will be confused. Thus, using a second training set with the screened training sample data, the optimized classification model trained achieves better optimization results compared to existing optimized classification models trained directly using a large amount of sample data.

[0155] Example 2

[0156] See Figure 4 Based on the foregoing Figure 1 The method shown in this application embodiment also discloses a training data processing method, which can focus on optimizing categories with poor classification performance of the classification model, specifically including the following steps:

[0157] S401. Obtain the first classification accuracy of the classification model to be optimized on the validation sample data of each category in the validation set.

[0158] The validation set includes validation sample data for all categories that the classification model can classify. Each validation sample in the validation set carries its own category. The classification model to be optimized is trained using training sample data from the first training set.

[0159] Specifically, the process of executing step S401 can be as follows: Each validation sample data in the validation set is input into the classification model to be optimized. The classification model to be optimized obtains the predicted category of each validation sample data. Then, based on the predicted category of each validation sample data and the category of the validation sample data carried by each validation sample data, the first classification accuracy of the classification model to be optimized for the validation sample data under each category in the validation set can be calculated. Here, the predicted category of the validation sample data is the category identified by the classification model to be optimized for the validation sample data.

[0160] The process of calculating the first classification accuracy of the classification model to be optimized for each category in the validation set, based on the predicted category of each validation sample and the category of the validation sample data it carries, is as follows: When the predicted category of a validation sample matches the category it carries, the classification model to be optimized is accurate in classifying that validation sample. When the predicted category of a validation sample does not match the category it carries, the classification model to be optimized is inaccurate in classifying that validation sample. After obtaining the classification accuracy result for each validation sample, for each category, the proportion of accurately classified validation samples in that category out of the total number of validation samples is calculated to obtain the first classification accuracy of the classification model to be optimized for that category in the validation set.

[0161] The validation set can be pre-constructed. For example, multiple sample data can be collected from various channels such as the internet and operating platforms, and then a portion of the sample data can be selected as validation sample data and added to the validation set. The process of constructing the validation set is not limited in this embodiment.

[0162] The initial classification accuracy of the classification model to be optimized on the validation sample data of a certain category in the validation set can be used to illustrate the effectiveness of the model in processing sample data of that category. A higher initial classification accuracy on the validation sample data of a certain category indicates a better performance of the model, meaning it is more accurate in identifying the sample data of that category. Conversely, a lower initial classification accuracy on the validation sample data of that category indicates a worse performance of the model.

[0163] S402. Classes whose first classification accuracy is less than the first accuracy threshold are identified as categories to be optimized.

[0164] From the first classification accuracy of the classification model to be optimized obtained above on the validation sample data of each category in the validation set, select the categories whose first classification accuracy is less than the first accuracy threshold, and then use the selected categories as the categories to be optimized.

[0165] The first accuracy threshold can be set empirically, and the method of setting this first accuracy threshold is not limited in this embodiment. The category to be optimized can be understood as the category in which the model's classification performance is poor. When the first classification accuracy of the validation sample data under a certain category is less than the first accuracy threshold, it is considered that the classification model to be optimized has a poor classification performance for the sample data of that category and cannot accurately identify the sample data of that category.

[0166] S403. Select multiple key optimization categories from the multiple categories that can be classified by the classification model. The multiple categories that can be classified by the classification model are the determined categories to be optimized. The similarity between any two key optimization categories is less than the similarity threshold. The similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories.

[0167] As described above in step S402, the categories to be optimized can be understood as those with poor model classification performance. Therefore, in scenarios where the classification model to be optimized needs to be optimized, the categories that require priority optimization can be selected from the categories to be optimized. The execution process and principle of selecting multiple key optimization categories from the categories to be optimized in step S403 can be found in the description of step S101 above, and will not be repeated here.

[0168] The key optimization categories selected through steps S401 to S403 are not only dissimilar to each other, but also the classification model to be optimized performs poorly in these key optimization categories. Therefore, when training the classification model to be optimized using the training sample data of these key optimization categories, the training effect of the classification model to be optimized will not be affected by the excessive similarity between the training sample data. At the same time, it also focuses on improving the poor classification performance of the classification model to be optimized, thereby achieving the goal of significantly improving the optimization effect of the model.

[0169] S404. At least according to the first screening condition, select training sample data from the candidate sample set.

[0170] The execution process and principle of step S404 can be referred to the above. Figure 1 Step S102 shown will not be repeated here.

[0171] S405. Add the selected training sample data to the first training set to obtain the second training set.

[0172] The execution process and principle of step S405 can be referred to the above. Figure 1 Step S103 shown will not be repeated here.

[0173] Example 3

[0174] See Figure 5 ,based on Figure 1 The method shown in this application embodiment also discloses a training data processing method. In the process of selecting key optimization categories, in addition to using a second screening condition, a second screening condition is also used to screen out training sample data that can improve the model performance to a greater extent. The method specifically includes the following steps:

[0175] S501. Select multiple key optimization categories from the multiple categories that can be classified by the classification model. Among these, the similarity between any two key optimization categories is less than the similarity threshold. The similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories.

[0176] The execution process and principle of step S501 can be referred to the aforementioned step S101, and will not be repeated here.

[0177] S502. For each key optimization category, obtain the features of each model sample data under that key optimization category, where the model sample data is either validation sample data or training sample data, and the features of the model sample data are used to characterize the key information of the model sample data.

[0178] To more accurately select candidate sample data belonging to the key optimization category from the candidate sample set during the subsequent execution of step S503, step S502 also obtains the features of each model sample data under the key optimization category, and then uses the features of each model sample data under the key optimization category to construct a second screening condition. Since the features of the model sample data are used to characterize the key information of the model sample data, it can be considered that the features of the model sample data under the key optimization category are information unique to the sample data under the key optimization category. When the feature of a sample data matches the feature of any model sample data under any key optimization category, it can be considered that the sample data is very likely to belong to the key optimization category.

[0179] Wherein, the model sample data is either validation sample data or training sample data, and one implementation of step S502 may be: for each key optimization category, obtain the features of each validation sample data under the key optimization category in the validation set, and / or, obtain the features of each validation sample data under the key optimization category in the first training set.

[0180] For example, when the classification model is a text classification model, the features of the model sample data obtained in step S502 can be understood as semantic elements (tokens) of the model sample data. Optionally, the semantic elements of the model sample data may include at least one of the following: business token, action token, and attribute token. Attribute tokens can be understood as adjectives. For example, in the model sample data "I want to apply for an unsecured loan, what is the interest rate?", the semantic elements include query, interest rate, apply, unsecured loan, loan, and consultation. Another example is "I want to ask how to activate the service that doesn't require a password for deductions", the semantic elements include activation, consultation, password-free, deduction, and deduction.

[0181] Optionally, in one specific embodiment of this application, one implementation of step S502 includes: for each key optimization category, inputting each model sample data under that category into a feature extraction model, and having the feature extraction model obtain and output the features of each model sample data. The feature extraction model is obtained by training a neural network model using multiple sample data. For example, the feature extraction model is obtained by performing unsupervised training on the neural network model using multiple sample data. Exemplarily, the feature extraction model can be an existing BERT-BiLSTM-CRF model or a BERT-Att-BiLSTM model, etc.

[0182] S503. According to the first screening condition and the second screening condition, the training sample data is obtained by screening from the candidate sample set, wherein the second screening condition is that the features of the candidate sample data to be screened match the features of the model sample data under any key optimization category.

[0183] The specific meaning and principle of the first screening condition are as described above. Figure 1 The first screening condition in step S102 is the same as described above and will not be repeated here. The second screening condition is constructed based on the aforementioned step S502. Since the features of each model sample data under the key optimization category are obtained in step S502, the features of each model sample data under the key optimization category can be understood as features unique to the sample data of the key optimization category. Therefore, the obtained features can be used to screen the candidate sample data to obtain candidate sample data under the key optimization category from the candidate sample set.

[0184] For example, in step S502, the model sample data obtained under a key optimization category, such as "I want to apply for an unsecured loan, what is the interest rate?", contains semantic elements including query, interest rate, apply, unsecured loan, loan, and consultation. Therefore, when a candidate sample data is selected from the candidate sample set and contains the semantic elements of query, interest rate, apply, unsecured loan, loan, and consultation, it is considered that the semantic elements of the candidate sample data match the semantic features of the model sample data under the aforementioned key optimization category, i.e., the candidate sample data is considered to meet the second screening condition.

[0185] Optionally, one implementation of step S503 is as follows: Candidate sample data that simultaneously meets both the first and second screening conditions are selected from the candidate sample set, and the selected candidate sample data is used as training sample data. It should be noted that the training sample data selected in step S503 is a portion of the data used to train the classification model to be optimized. When training the classification model to be optimized subsequently, the training sample data selected in step S503, along with the training sample data included in the originally constructed first training set, are used to train the classification model to obtain the optimized classification model. That is, it can also be understood that before training the classification model to be optimized, the training sample data selected in step S503 is added to the first training set to obtain a second training set. Then, the optimized classification model is trained using the second training set. The second training set includes the training sample data from the first training set and the training sample data selected in step S503.

[0186] The order in which the two screening conditions are used is not limited in this embodiment. For example, candidate sample data that meets the first screening condition can be selected first, and then candidate sample data that meets the second screening condition can be selected from among them. Using a method that simultaneously meets the first and second screening conditions can accurately select candidate sample data belonging to the key optimization category. This prevents training sample data from being too similar between different categories, thus avoiding affecting the training effect of the classification model to be optimized. When using this training sample data to train the classification model to be optimized, the optimization effect of the classification model can be improved.

[0187] Optionally, another implementation of step S503 is as follows: Candidate sample data that satisfies at least one of the first and second screening conditions are selected from the candidate sample set, and the selected candidate sample data is used as training sample data. As described above, both the first and second screening conditions are used to select candidate sample data belonging to the key optimization category. Therefore, even if only one screening condition is met, it can still be concluded that the category of the candidate sample data is the key optimization category.

[0188] The following example uses a text classification model to illustrate the process by which the text classification model obtains the first and second selection criteria, based on the training data processing method proposed in the aforementioned embodiments of this application:

[0189] See Figure 6Multiple sample sentences are divided into a validation set and a first training set at a ratio of 1:9. A classification model to be optimized (which can be understood as a raw classification model) is pre-trained using the training sample sentences from the first training set. Then, validation sample sentences from each category in the validation set are input into the classification model to be optimized, and the processing effect of the model on each knowledge point is obtained (specifically, reflected by the first classification accuracy of the model on the validation sample sentences for each knowledge point in the validation set). For each knowledge point, it is determined whether the first classification accuracy is less than a first accuracy threshold. If it is less than the first accuracy threshold, the knowledge point is considered to have a lower than expected processing effect and is therefore identified as a knowledge point to be optimized. Then, it is determined whether the frequency of similar sentences for each knowledge point is less than p%. If the frequency of similar sentences for a knowledge point is less than p%, it is identified as a key optimization knowledge point, thus becoming the key optimization knowledge point as screening condition I (equivalent to the aforementioned first screening condition). Then, each sentence token under the key optimization knowledge point is obtained as screening condition II (equivalent to the aforementioned second screening condition). When multiple sample sentences that satisfy both screening condition I and screening condition II are selected, there will be no confusion between the sample sentences from different categories. As a result, when the selected sample sentences are used to train the classification model, the optimization effect is improved.

[0190] in, Figure 6 The principles and detailed processes involved in the illustrated procedure can be found in the foregoing. Figure 4 Steps S401 to S403, and Figure 5 The relevant content of steps S501 to S503 shown will not be repeated here.

[0191] S504. Add the selected training sample data to the first training set to obtain the second training set, which is used to train the optimized classification model.

[0192] The execution process and principle of step S504 can be found in the aforementioned text. Figure 1 Step S103 shown will not be repeated here.

[0193] Example 4

[0194] See Figure 7 ,based on Figure 1 The training data processing method shown in this application is disclosed in another embodiment. This method preprocesses the candidate sample data in the candidate sample set before filtering, so that the training sample data obtained later can better optimize the classification model when used to train the model. Specifically, Figure 7 The method shown includes the following steps:

[0195] S701. Select multiple key optimization categories from the multiple categories that can be classified by the classification model. Among these, the similarity between any two key optimization categories is less than the similarity threshold. The similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories.

[0196] The execution process and principle of step S701 can be found in [reference needed]. Figure 1 Step S101 shown will not be repeated here.

[0197] S702. Obtain the second classification accuracy of the classification model to be optimized for candidate sample data under each category in the candidate sample set.

[0198] The execution process and principle of step S702 can be found by referring to Figure 4 The only difference in step S401 is that step S401 obtains the first classification accuracy of each category in the validation set, while step S702 obtains the second classification accuracy of each category in the candidate sample set. This will not be elaborated further here.

[0199] S703. Candidate sample data of categories whose second classification accuracy is less than the second accuracy threshold are identified as processed candidate sample data.

[0200] The second accuracy threshold can be set arbitrarily based on experience, and the method of setting the second accuracy threshold is not limited in this embodiment.

[0201] Specifically, for each category in the candidate sample set, if the second classification accuracy of the candidate sample data under that category is greater than or equal to the second accuracy threshold, then that category is considered a non-key optimization category. Here, a non-key optimization category can be understood as a category for which the classification model to be optimized does not require a significant improvement in classification performance.

[0202] Therefore, in step S703, candidate sample data of categories whose second classification accuracy is less than the second accuracy threshold are preferentially selected and determined as processed candidate sample data. Then, in step S704, further screening is carried out from the processed candidate sample data to obtain training sample data that needs to be added to the first training set. The portion of the candidate sample set that is not determined as processed candidate sample data does not participate in the screening of training sample data in the subsequent step S704.

[0203] By prioritizing the exclusion of candidate sample data for categories whose second classification accuracy is greater than or equal to the second accuracy threshold, the training sample data selected in the final step S704 will not contain categories for which classification performance does not need to be improved. Therefore, when the training sample data obtained in step S704 is used to train the classification model to be optimized, it can optimize the classification performance of the classification model to be optimized to a greater extent.

[0204] It should be noted that the execution order of steps S702 to S703 and step S701 is not limited in this embodiment of the application. Steps S701 and S702 to S703 only need to be executed before step S704.

[0205] S704. At least according to the first screening condition, select training sample data from the processed candidate sample data.

[0206] The execution process and principle of step S704 can be referred to the above. Figure 1 The difference between step S102 shown is that step S102 filters from all candidate sample data in the candidate sample set, while step S704 filters only from the processed candidate sample data determined by step S703. This will not be elaborated further here.

[0207] Optionally, in some embodiments, training sample data can be obtained from the processed candidate sample data based on a first screening condition and a second screening condition.

[0208] S705. Add the selected training sample data to the first training set to obtain the second training set, which is used to train the optimized classification model.

[0209] The execution process and principle of step S705 can be referred to the above. Figure 1 Step S103 shown will not be repeated here.

[0210] Example 5

[0211] See Figure 8 ,based on Figure 4 The present application also proposes another method for processing training data, which is used to optimize a classification model to be optimized, and specifically includes the following steps:

[0212] S801. Obtain the first classification accuracy of the classification model to be optimized on the validation sample data of each category in the validation set.

[0213] The execution process and principle of step S801 can be referred to the above. Figure 4 Step S401 shown will not be repeated here.

[0214] S802. Classes with a first classification accuracy less than a first accuracy threshold are identified as categories to be optimized.

[0215] The execution process and principle of step S802 can be referred to the above. Figure 4 Step S402 shown will not be repeated here.

[0216] S803. Select multiple key optimization categories from the multiple categories that can be classified by the classification model. The multiple categories that can be classified by the classification model are the determined categories to be optimized. The similarity between any two of the key optimization categories is less than the similarity threshold. The similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories.

[0217] The execution process and principle of step S803 can be referred to the above. Figure 4 Step S403 shown will not be repeated here.

[0218] S804. At least according to the first screening condition, select training sample data from the candidate sample set.

[0219] The execution process and principle of step S804 can be referred to the above. Figure 4 Step S404, as shown, will not be repeated here.

[0220] S805. Add the selected training sample data to the first training set to obtain the second training set.

[0221] The execution process and principle of step S805 can be referred to the above. Figure 4 Step S405, as shown, will not be repeated here.

[0222] S806. Use the training sample data in the second training set to train the model and obtain the optimized classification model.

[0223] The second training set mentioned in step S806 includes at least the training sample data obtained in step S804 and the training sample data in the first training set.

[0224] Optionally, the process of performing step S806 may be: inputting multiple training sample data from the second training set into a feature classifier for model training, and obtaining and outputting an optimized classification model from the feature classifier.

[0225] Optionally, the process of performing step S806 can be: using multiple training sample data from the second training set to input into the classification model to be optimized for model training, and obtaining the optimized classification model.

[0226] It should be noted that step S806 can also be performed in... Figure 1 The procedure is performed after step S103 shown.

[0227] Because the training sample data used in training the classification model to be optimized consists of samples from multiple dissimilar key optimization categories, the training performance of the classification model will not be affected by excessive similarity between training sample data from different categories. The final optimized classification model, compared to the original model, shows a significant improvement in classification performance for the key optimization categories.

[0228] Example 6

[0229] See Figure 9 ,based on Figure 8 The present application also proposes another method for processing training data, which is used to perform multiple optimizations of the classification model to be optimized, specifically including the following steps:

[0230] S901. Obtain the first classification accuracy of the classification model to be optimized on the validation sample data of each category in the validation set.

[0231] The execution process and principle of step S901 can be referred to the above. Figure 8 Step S801 shown will not be repeated here.

[0232] S902. Categories whose first classification accuracy is less than the first accuracy threshold are identified as categories to be optimized.

[0233] The execution process and principle of step S902 can be referred to the above. Figure 8 Step S802 shown will not be repeated here.

[0234] S903. Select multiple key optimization categories from the multiple categories that can be classified by the classification model. The multiple categories that can be classified by the classification model are the determined categories to be optimized. The similarity between any two key optimization categories is less than the similarity threshold. The similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories.

[0235] The execution process and principle of step S903 can be referred to the above. Figure 8 Step S803 shown will not be repeated here.

[0236] S904. At least according to the first screening condition, training sample data are obtained from the candidate sample set.

[0237] The execution process and principle of step S904 can be referred to the above. Figure 8 Step S804, as shown, will not be repeated here.

[0238] S905. Add the selected training sample data to the first training set to obtain the second training set.

[0239] The execution process and principle of step S905 can be referred to the above. Figure 8 Step S805, as shown, will not be repeated here.

[0240] S906. Use the training sample data in the second training set to train the model and obtain the optimized classification model.

[0241] The execution process and principle of step S906 can be referred to the above. Figure 8 Step S806, as shown, will not be repeated here.

[0242] S907. Use the optimized classification model as the new classification model to be optimized, and use the second training set as the new first training set, then return to step S901.

[0243] After returning to step S901, the improved classification model is used as the new model to be optimized. Because the new model's classification performance is better, the initial classification accuracy of the validation sample data for each category may change when step S901 is executed again. Consequently, the category to be optimized determined when step S902 is executed again will also change. Repeating step S907 multiple times will improve the initial classification accuracy of the validation sample data for each category. Execution can continue until no category with an initial classification accuracy lower than the initial accuracy threshold can be selected, at which point execution stops. Figure 9 The process. Among them, stopping execution. Figure 9 There are many ways to perform the process, including but not limited to the content proposed in the embodiments of this application.

[0244] Example 7

[0245] See Figure 10 ,based on Figure 9 The present application also proposes another method for processing training data, which is used to continuously update the training model during the process of repeatedly optimizing the classification model to be optimized, and specifically includes the following steps:

[0246] S1001. Obtain the first classification accuracy of the classification model to be optimized on the validation sample data of each category in the validation set.

[0247] The execution process and principle of step S1001 can be referred to the above. Figure 9 Step S901 shown will not be repeated here.

[0248] S1002. Classes with a first classification accuracy less than a first accuracy threshold are identified as categories to be optimized.

[0249] The execution process and principle of step S1002 can be referred to the above. Figure 9 Step S902 shown will not be repeated here.

[0250] S1003. Select multiple key optimization categories from the multiple categories that can be classified by the classification model. The multiple categories that can be classified by the classification model are the determined categories to be optimized. The similarity between any two key optimization categories is less than the similarity threshold. The similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories.

[0251] The execution process and principle of step S1003 can be referred to the above. Figure 9 Step S903 shown will not be repeated here.

[0252] S1004. At least according to the first screening condition, select training sample data from the candidate sample set.

[0253] The execution process and principle of step S1004 can be referred to the above. Figure 9 Step S904, as shown, will not be repeated here.

[0254] S1005. Add the selected training sample data to the first training set to obtain the second training set.

[0255] The execution process and principle of step S1005 can be referred to the above. Figure 9 Step S905, as shown, will not be repeated here.

[0256] S1006. Use the training sample data in the second training set to train the model and obtain the optimized classification model.

[0257] The execution process and principle of step S1006 can be referred to the above. Figure 9 Step S906, as shown, will not be repeated here.

[0258] S1007. Use the optimized classification model as the new classification model to be optimized, and use the second training set as the new first training set.

[0259] The execution process and principle of step S1007 can be found by referring to Figure 9The relevant part of step S907 is shown.

[0260] S1008. Update the candidate sample data in the candidate sample set and return to step S1001.

[0261] There are many ways to update the candidate sample data in the candidate sample set. For example, you can continuously collect the latest sample data generated online over a certain period and then add it to the candidate sample set as candidate sample data to update the candidate sample set. Another example is to use the latest sample data collected within a specific period as candidate sample data and use this candidate sample data to construct a new candidate sample set.

[0262] Because the candidate sample data in the candidate sample set is constantly changing, the training sample data obtained after returning to step S1001 will also change. Continuously filtering the training sample data using the updated candidate sample data can improve the optimization effect of the classification model, thereby continuously improving the classification accuracy of the model.

[0263] See Figure 11 Based on the training data processing method proposed in the above embodiments of this application, this application also discloses a training data processing device, including: a first selection unit 1101, a first filtering unit 1102, and an adding unit 1103.

[0264] The first selection unit 1101 is used to select multiple key optimization categories from the multiple categories that can be classified by the classification model. The similarity between any two key optimization categories is less than a similarity threshold. The similarity between two key optimization categories is the similarity between the sample data under those two key optimization categories.

[0265] Optionally, in a specific embodiment of this application, the first selection subunit 1101 includes: a first determination subunit, a second selection subunit, a second determination subunit, and a first return subunit.

[0266] The first determining sub-unit is used to select one of the multiple categories that can be classified by the classification model and determine it as the key optimization category.

[0267] The second selection sub-unit is used to select the category with the highest similarity to the target category and that meets the third screening criterion from the remaining categories excluding those already identified as key optimization categories. The target category is the category most recently identified as a key optimization category. The third screening criterion is that the similarity between the selected category and the target category is less than a similarity threshold.

[0268] The second determining sub-unit is used to determine the selected category as the key optimization category if the category with the highest similarity to the target category and meets the third screening condition is selected.

[0269] The first return sub-unit is used to return the selected category as the new target category to the second selection sub-unit, until the second selection sub-unit fails to select a category with the highest similarity to the target category and that meets the third filtering condition.

[0270] Optionally, in a specific embodiment of this application, the second selection subunit includes: a matching subunit, a calculation subunit, a third determination subunit, and a third selection subunit.

[0271] The matching subunit is used to, for each validation sample data in the target category of the validation set, obtain two training sample data that match the validation sample data from all training sample data in all categories other than those already identified as the focus of optimization. These two matching training sample data are the two training sample data in the first training set that have the highest similarity to the validation sample data and belong to different categories.

[0272] The calculation subunit is used to calculate the similarity difference between two training samples that match the validation sample data for each validation sample data under the target category in the validation set. If the absolute value of the similarity difference is less than the first threshold, the frequency of similar pairs of the category combinations corresponding to the two training samples that match the validation sample data is incremented by one.

[0273] The third determining subunit is used to determine the similarity of each category combination by the ratio of the total frequency of similar pairs in the category combination to the total number of validation sample data under the target category in the validation set. The similarity of a category combination refers to the degree of similarity between the two categories included in the combination.

[0274] The third selection subunit is used to select the category with the highest similarity to the target category and that meets the third screening condition from all statistically analyzed category combinations based on the similarity of all category combinations.

[0275] The first screening unit 1102 is used to screen training sample data from the candidate sample set at least according to a first screening condition. The first screening condition is the category of the candidate sample data to be screened, which matches any one of the key optimization categories. The candidate sample set includes multiple candidate sample data.

[0276] Add unit 1103 to add the selected training sample data to the first training set to obtain the second training set. The second training set is used to train the optimized classification model.

[0277] Optionally, in a specific embodiment of this application, the training data processing device further includes: a first acquisition unit and a first determination unit.

[0278] The first acquisition unit is used to acquire the first classification accuracy of the classification model to be optimized on the validation sample data of each category in the validation set. The classification model to be optimized is trained using the training sample data in the first training set.

[0279] The first determining unit is used to determine categories whose first classification accuracy is less than a first accuracy threshold as categories to be optimized. The multiple categories that the classification model can classify are the determined categories to be optimized.

[0280] Optionally, in one specific embodiment of this application, the training data processing apparatus further includes:

[0281] The second acquisition unit is used to acquire the features of each model sample data under each key optimization category. The model sample data can be either validation sample data or training sample data. The features of the model sample data are used to characterize the key information of the model sample data. The first filtering unit includes a first filtering subunit, used to filter training sample data from the candidate sample set according to a first filtering condition and a second filtering condition. The second filtering condition is that the features of the candidate sample data to be filtered match the features of the model sample data under any key optimization category.

[0282] Optionally, in a specific embodiment of this application, the second acquisition unit includes:

[0283] The output subunit is used to input each model sample data under each key optimization category into the feature extraction model, which then obtains and outputs the features of each model sample data. The feature extraction model is trained on the neural network model using multiple sample data sets.

[0284] Optionally, in a specific embodiment of this application, it further includes: a third acquisition unit and a second determination unit.

[0285] The third acquisition unit is used to acquire the second classification accuracy of the classification model to be optimized for candidate sample data of each category in the candidate sample set.

[0286] The second determining unit is used to determine candidate sample data under categories whose second classification accuracy is less than a second accuracy threshold as processed candidate sample data. The first filtering subunit 1102 includes a second filtering subunit, used to filter training sample data from the processed candidate sample data based at least on a first filtering condition.

[0287] Optionally, in a specific embodiment of this application, it further includes: a training unit, used to train the model using training sample data in the second training set to obtain an optimized classification model.

[0288] Optionally, in a specific embodiment of this application, it further includes: a return unit, used to return the optimized classification model as a new classification model to be optimized and the second training set as a new first training set to the first acquisition unit.

[0289] Optionally, in a specific embodiment of this application, it further includes: an update unit, used to update the candidate sample data in the candidate sample set.

[0290] The execution principles of each unit and subunit in the training data processing apparatus proposed in this application embodiment can be found in the relevant parts of the training data processing method proposed in the above-mentioned application embodiment, and will not be repeated here.

[0291] In the training data processing apparatus proposed in this application embodiment, a first selection unit 1101 selects multiple key optimization categories from multiple categories that can be classified by the classification model. Then, a first filtering unit 1102 filters training sample data from a candidate sample set at least according to a first filtering condition. The first filtering condition is that the category of the candidate sample data to be filtered matches any one of the key optimization categories, and the candidate sample set includes multiple candidate sample data. Since the similarity between any two key optimization categories is less than a similarity threshold, and the similarity between two key optimization categories is the similarity between the sample data under the two key optimization categories, the training sample data of different categories filtered according to the first filtering condition are dissimilar. Therefore, during training, it is less likely that the training sample data will be confused. Thus, using a second training set with the filtered training sample data, the optimized classification model trained achieves better optimization results compared to existing optimized classification models trained directly using a large amount of sample data.

[0292] This application also discloses a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements any of the methods described above.

[0293] This application also discloses a training data processing device, comprising: one or more processors; a storage device having one or more programs stored thereon; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement any of the methods described above.

[0294] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0295] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0296] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing training data, characterized in that, include: From the multiple categories that can be classified by the classification model, multiple key optimization categories are selected; wherein, the similarity between any two key optimization categories is less than a similarity threshold; the similarity between two key optimization categories is the similarity between sample data under the two key optimization categories; wherein, when the classification model is a text classification model, the sample data is text sample data; when the classification model is an image classification model, the sample data is image sample data; Training sample data is obtained from the candidate sample set based on at least a first screening condition; wherein, the first screening condition is that the category of the candidate sample data to be screened matches any one of the key optimization categories; the candidate sample set includes multiple candidate sample data. The selected training sample data is added to the first training set to obtain the second training set; the second training set is used to train the optimized classification model.

2. The method according to claim 1, characterized in that, Before selecting multiple key optimization categories from the multiple categories that can be classified by the classification model, the process also includes: Obtain the first classification accuracy of the classification model to be optimized on the validation sample data of each category in the validation set; wherein, the classification model to be optimized is trained using the training sample data in the first training set; Categories whose first classification accuracy is less than the first accuracy threshold are identified as categories to be optimized. The classification model can classify multiple categories, which are the determined categories to be optimized.

3. The method according to claim 1, characterized in that, Before obtaining training sample data from the candidate sample set based at least on the first screening condition, the method further includes: For each of the aforementioned key optimization categories, each model sample data under that category is input into a feature extraction model, which then obtains and outputs features for each model sample data. The feature extraction model is obtained by training a neural network model using multiple sample data, which are either validation or training sample data. The features of the model sample data are used to characterize the key information of the model sample data. The step of selecting training sample data from the candidate sample set based at least on a first screening criterion includes: Training sample data is obtained by filtering from the candidate sample set according to the first and second screening conditions; wherein, the second screening condition is the feature of the candidate sample data to be screened, which matches the feature of the model sample data under any of the key optimization categories.

4. The method according to claim 1, characterized in that, Before obtaining training sample data from the candidate sample set based at least on the first screening condition, the method further includes: Obtain the second classification accuracy of the classification model to be optimized for candidate sample data in each category of the candidate sample set; Candidate sample data for categories whose second classification accuracy is less than the second accuracy threshold are determined as processed candidate sample data. Wherein, obtaining training sample data from the candidate sample set based at least on the first screening condition includes: Training sample data is obtained from the processed candidate sample data based at least on the first screening criteria.

5. The method according to claim 1, characterized in that, From the multiple categories that can be classified by the classification model, several key optimization categories are selected, including: Select one category from the multiple categories that the classification model can classify, and determine it as the category to be optimized. Among the remaining categories other than those already identified as key optimization categories, the category with the highest similarity to the target category and that meets the third screening condition is selected; wherein, the target category is the category most recently identified as a key optimization category; and the third screening condition is that the similarity between the category to be selected and the target category is less than a similarity threshold. If a category with the highest similarity to the target category and that meets the third screening condition is selected, the selected category is determined as the key optimization category, and the selected category is used as the new target category. The process then returns to the step of selecting the category with the highest similarity to the target category and that meets the third screening condition from the remaining categories other than those already determined as key optimization categories, until no category with the highest similarity to the target category and that meets the third screening condition is selected.

6. The method according to claim 5, characterized in that, The selection of categories from those already identified as key optimization categories, which have the highest similarity to the target category and meet the third screening criterion, includes: For each validation sample data under the target category in the validation set, based on the validation sample data, two training sample data that match the validation sample data are obtained from all training sample data under the remaining categories except those that have been identified as the key optimization categories; wherein, the two training sample data that match the validation sample data are the two training sample data in the first training set that have the highest similarity to the validation sample data and belong to different categories. For each validation sample data under the target category in the validation set, calculate the similarity difference between two training sample data that match the validation sample data. If the absolute value of the similarity difference is less than a first threshold, then increment the frequency count of similar pairs of the category combinations corresponding to the two training sample data that match the validation sample data by one. For each category combination identified in the statistics, the ratio of the total frequency of similar pairs of the category combination to the total number of validation sample data under the target category in the validation set is determined as the similarity degree of the category combination; wherein, the similarity degree of the category combination is the similarity degree between the two categories included in the category combination; Based on the similarity of all category combinations, the category with the highest similarity to the target category and that meets the third screening condition is selected from all the statistically analyzed category combinations.

7. The method according to claim 2, characterized in that, After adding the selected training sample data to the first training set to obtain the second training set, the process further includes: The model is trained using training sample data from the second training set to obtain an optimized classification model. Update the candidate sample data in the candidate sample set; The optimized classification model is used as the new classification model to be optimized, and the second training set is used as the new first training set. The process returns to the step of obtaining the first classification accuracy of the classification model to be optimized on the validation sample data of each category in the validation set.

8. A training data processing apparatus, characterized in that, include: The first selection unit is used to select multiple key optimization categories from multiple categories that can be classified by the classification model; wherein the similarity between any two key optimization categories is less than a similarity threshold; the similarity between two key optimization categories is the similarity between sample data under the two key optimization categories; wherein, when the classification model is a text classification model, the sample data is text sample data; when the classification model is an image classification model, the sample data is image sample data; The first screening unit is configured to screen training sample data from a candidate sample set based on at least a first screening condition; wherein the first screening condition is the category of the candidate sample data to be screened, which matches any one of the key optimization categories; the candidate sample set includes multiple candidate sample data. An addition unit is used to add the selected training sample data to the first training set to obtain the second training set; wherein, the second training set is used to train the optimized classification model.

9. A computer-readable medium, characterized in that, It stores a computer program thereon, wherein the program, when executed by a processor, implements the method as described in any one of claims 1 to 7.

10. A training data processing device, characterized in that, include: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method as described in any one of claims 1 to 7.

Images