Classification model training method, device and electronic equipment

Abstract

The present specification discloses a training method, device and electronic device for a classification model, and relates to the field of artificial intelligence technology, wherein the method comprises: obtaining an original sample set; looping and performing the following iterative operations until a preset termination condition is met: oversampling the original sample set and the reference sample set to obtain a training sample set; performing the following operations on each sample pair in the training sample set: inputting the feature data of the current sample pair into the classification model to obtain a prediction result; calculating the confidence of the classification model according to the category label of the current sample pair and the probability that the current sample pair is divided into each category; when the confidence is within a predetermined range, updating the parameters of the classification model through the current sample pair; when the confidence is within a predetermined range and the category label of the current sample pair belongs to a predetermined category, placing the current sample pair as a reference sample pair into the reference sample set. This scheme can improve the prediction accuracy and generalization of the classification model for small category samples.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method, device and electronic device for a classification model. Background Art

[0002] In recent years, the problem of unbalanced dataset classification has become one of the key and difficult problems in the field of intelligent machine learning. The so-called unbalanced dataset refers to a dataset in which the sample sizes of each category are extremely unbalanced. Taking the binary classification problem as an example, assuming that the number of samples in the positive class is much larger than the number of samples in the negative class, the dataset in which the ratio of samples of different categories is close to 100:1 is usually called an unbalanced dataset. These categories with a small sample size are also called small categories or small sample classes. Samples of small categories are often the focus of classification models, that is, the purpose of classification models is to find these small sample class data from massive data in practical applications.

[0003] When traditional classification models classify data, they usually assume that the categories of data samples obtained through a certain channel have been calibrated and the number of data samples in each category is equivalent. In this way, it is only necessary to adopt corresponding effective classification models, such as neural networks, support vector machines, etc., to fully learn the data samples to obtain relatively good classification accuracy.

[0004] However, this classification model cannot fully learn from a small number of samples, resulting in poor generalization performance when faced with imbalanced data classification. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a training method, device and electronic device for a classification model to solve the problem that the generalization performance of existing classification models is low when facing unbalanced data classification.

[0006] To solve the above technical problems, an embodiment of the present specification provides a training method for a classification model, comprising: obtaining an original sample set, wherein the original sample set includes multiple sample pairs, each sample pair includes at least: feature data and a category label; looping through the following iterative operations until a preset termination condition is met: oversampling the original sample set and the reference sample set to obtain a training sample set; performing the following operations on each sample pair in the training sample set: inputting the feature data of the current sample pair into the classification model to obtain a prediction result of the current sample pair, the prediction result including the probability of being divided into various categories; calculating the confidence of the classification model according to the category label of the current sample pair and the probability of the current sample pair being divided into various categories; when the confidence is within a predetermined range, updating the parameters of the classification model through the current sample pair; when the confidence is within the predetermined range and the category label of the current sample pair belongs to a predetermined category, placing the current sample pair as a reference sample pair into the reference sample set; the predetermined category is a category whose proportion in the original sample set is less than a preset threshold.

[0007] The embodiment of the present specification also provides a training device for a classification model, comprising: an acquisition module, used to acquire an original sample set, wherein the original sample set includes a plurality of sample pairs, each sample pair includes at least: feature data and a category label; an iteration module, used to cyclically perform an iteration operation until a preset termination condition is met; the iteration module includes: an oversampling submodule, used to oversample the original sample set and the reference sample set to obtain a training sample set; an execution submodule, used to perform an operation on each sample pair in the training sample set; the execution submodule includes: a prediction submodule, used to input the feature data of the current sample pair into the classification model to obtain the current sample pair a prediction result, the prediction result including the probability of being divided into each category; a calculation submodule, used to calculate the confidence of the classification model according to the category label of the current sample pair and the probability of the current sample pair being divided into each category; an update submodule, used to update the parameters of the classification model through the current sample pair when the confidence is within a predetermined range; a reference submodule, used to put the current sample pair into a reference sample set as a reference sample pair when the confidence is within the predetermined range and the category label of the current sample pair belongs to a predetermined category; the predetermined category is a category in the original sample set whose proportion is less than a preset threshold.

[0008] A training method, device and electronic device for a classification model provided in an embodiment of the present specification perform multiple iterative operations. In each iterative operation, the original sample set and the reference sample set are oversampled to obtain a training sample set, and the samples in the training sample set are input into the classification model in sequence, which can avoid the problem of the prediction result being biased towards a category with a larger number of sample pairs caused by inputting small category sample pairs as a small part of the training sample set into the classification model; after updating the parameters of the classification model with sample pairs in one iteration, if the sample pair is a small category sample pair, it is put into the reference sample set so that a new sample pair can be obtained based on the sample pair by oversampling in the next iteration, thereby further improving the prediction accuracy and generalization of the classification model for small category samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some implementation methods recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0010] Figure 1 A flow chart of a classification model training method according to an embodiment of this specification is shown;

[0011] Figure 2 A flowchart of an embodiment of oversampling the original sample set and the reference sample set to obtain a training sample set is shown;

[0012] Figure 3 A flowchart of another classification model training method according to an embodiment of this specification is shown;

[0013] Figure 4 A principle block diagram of a training device for a classification model according to an embodiment of this specification is shown;

[0014] Figure 5 A principle block diagram of another classification model training device according to an embodiment of this specification is shown;

[0015] Figure 6 A principle block diagram of an embodiment of a computing submodule is shown;

[0016] Figure 7 A principle block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0018] In some scenario examples, there may be control-type transaction records among the massive financial transaction records, and the field values ​​of such transaction records are often different from those of normal transaction records. In order to find control-type transaction records from the massive transaction records, these massive transaction records can be input into the classification model one by one, and the classification model will determine whether each transaction record belongs to the control category.

[0019] In some scenario examples, there may be access and modification records of network intrusion in the massive network supervision data, which is often different from the network supervision data of normal network communication. Network intrusion behavior can be tracked based on the massive network supervision data. To this end, the massive network supervision records can be input into the classification model one by one, and the classification model will determine whether each record data corresponds to network intrusion behavior.

[0020] The embodiments of this specification provide a classification model training method to train the above classification model. Figure 1 As shown, the training method includes the following steps:

[0021] S101: Acquire an original sample set, wherein the original sample set includes a plurality of sample pairs, and each sample pair includes at least feature data and a category label.

[0022] The original sample set is a set of sample pairs prepared for training the classification model. This set can be data recorded in actual production and life scenarios. For example, transaction records and category labels generated when transactions are conducted in financial institutions such as banks. Each transaction record corresponds to a transaction. The transaction record has multiple fields, and each field represents information about one aspect of the transaction.

[0023] Feature data is data recorded in actual production and life scenarios and used to represent the characteristics of things. For example, in a transaction record sample of a financial institution, each field of the transaction is a feature data, and the transaction fields can be, for example, user name, bank account number, bank card region, counterparty account number, transaction type, transaction amount, etc.

[0024] Category tags are usually given to data based on events that have occurred in actual production and life scenarios. For example, if a transaction record of a financial institution is related to a regulatory case that has occurred, a special tag will be given to the transaction record based on this fact. Since category tags are given based on facts, the category to which the sample identified by the category tag belongs is credible and unquestionable.

[0025] S102: Oversampling the original sample set and the reference sample set to obtain a training sample set.

[0026] The training sample set is obtained based on oversampling of the original sample set and the reference sample set, that is, samples are extracted from a mixed set of the original sample set and the reference sample set to generate new samples.

[0027] S103: Select a sample pair from the training sample set as the current sample pair, and remove the selected sample pair from the training sample set.

[0028] S104: Inputting the feature data of the current sample pair into the classification model to obtain the prediction result of the current sample pair, where the prediction result includes the probability of being classified into each category.

[0029] It can be seen from step S104 that the number of sample pairs used in training the classification model is one, which is different from the method of using all sample pairs to train the classification model at the same time. The single sample pair training method can fully consider the impact of the sample pairs of the small sample class as an individual on the classification model, and can avoid the problem of the prediction results being biased towards the class with more sample pairs due to the input of the sample pairs of the small class as a small part of the training sample set into the classification model.

[0030] When the classification model is applied in actual production and life scenarios, the prediction result of its output is only one category, rather than the probability of the output being divided into each category as described in the embodiments of this specification. This is because after the classification model is trained, when it is put into actual production and life scenarios, a result processing unit will be added after the output of the classification model, and its function is to filter out the category corresponding to the largest probability value and use this category as the output result.

[0031] In order to obtain a more accurate classification model, in the training method provided in the embodiments of this specification, the output result of the classification model is the prediction probability corresponding to each category.

[0032] The classification model described in this specification may be a neural network model, a support vector machine model, etc. This specification does not limit the type of classification model.

[0033] The parameters of the classification model described in step S104 have been preliminarily determined, and may be parameters randomly initialized before this step, or may be determined by updating the classification model parameters using the previous sample pair.

[0034] S105: Calculate the confidence of the classification model according to the category label of the current sample pair and the probability of the current sample pair being classified into each category.

[0035] Confidence represents the degree to which the category corresponding to the maximum probability value is distinguished from other categories.

[0036] For example, for sample pair A, the probabilities of being classified into the first and second categories are 0.55 and 0.45 respectively, and the difference between the two is 0.1; for sample pair B, the probabilities of being classified into the first and second categories are 0.2 and 0.8 respectively, and the difference between the two is 0.6. It can be seen that in the actual application scenario, that is, when a result processing unit is added after the output of the classification model to filter out the category corresponding to the largest probability value as the output result, the classification model can more certainly classify sample pair B into the second category, while for sample pair A, it may be classified into the first category this time, but when a certain sample pair is used to update the parameters of the classification model, that is, after fine-tuning the parameters of the classification model, the classification model is likely to classify sample pair A into the second category.

[0037] It can be seen from this that when the confidence of the current sample pair is small, there is a high possibility of prediction error when predicting the sample pair after the parameters of the classification model are updated.

[0038] The confidence level in step S105 may be the difference between the two probability values ​​with the largest probability values ​​in the prediction result, or may be obtained by multiplying the difference by a coefficient or a variable.

[0039] In some embodiments, the confidence of the classification model can be calculated by the following formula: σ = y c -max i≠c {y i}, where σ is the confidence, c is the category label of the current sample pair, and y c is the probability that the current sample pair is classified into category c, y i is the probability that the current sample pair is classified into categories other than category c.

[0040] For example, the prediction result of the current sample pair is: the probability of being classified into the first category is 0.6, the probability of being classified into the second category is 0.1, and the probability of being classified into the third category is 0.3. If the category of the current sample pair is marked as the first category, the confidence is σ=0.6-0.3=0.3; if the category of the current sample pair is marked as the third category, the confidence is σ=0.3-0.6=-0.3.

[0041] From this, it can be seen that in actual production life, when a result processing unit is added after the output of the classification model to screen out the category corresponding to the largest probability value as the output result, when σ is greater than 0, it means that the classification model prediction in actual production life is accurate, and when σ is less than 0, it means that the classification model prediction in actual production life is inaccurate.

[0042] In some embodiments, Figure 3 As shown in steps S308 and S309, the confidence of the classification model can be calculated by the following steps:

[0043] Calculate σ according to the following formula: σ = y c -max i≠c {y i}, where c is the category label of the current sample pair, y c is the probability that the current sample pair is classified into category c, y i is the probability that the current sample pair is classified into categories other than category c;

[0044] Calculate Δ according to the following formula: in, n c is the number of c-type sample pairs in the original sample set, n is the total number of sample pairs in the original sample set, Z=min i {r i}, n i is the number of sample pairs of categories other than category c in the original sample set, and i represents categories other than category c;

[0045] Calculate p according to the following formula: Among them, p is the confidence level.

[0046] It can be seen from the calculation formula of Δ and p that: when the classification model prediction is inaccurate, the value of p is 1. In this case, the current sample pair should be used to update the parameters of the classification model; when the classification model prediction is inaccurate, the value of p is negatively correlated with Δ, that is, the value of p is negatively correlated with n. c There is a negative correlation, that is, n c The smaller it is, the larger the value of p is, which can increase the probability that the small category sample pair is selected to update the classification model parameters; in the case of inaccurate classification model prediction, since Z = min i {r i}, that is, when the number of sample pairs of categories other than category c is smaller, the value of Z is smaller, and the value of p is smaller, that is, for sample pairs of large categories (categories with a larger proportion in the original data set), the value of p is smaller, and the probability of the sample pair being selected for updating the classification model is smaller. Through these settings, measuring the accuracy of prediction based on the value of p can increase the probability of updating the classification model parameters for sample pairs of small categories and reduce the probability of updating the classification model parameters for sample pairs of large categories, thereby improving the prediction accuracy of the classification model for sample pairs of small categories and reducing the amount of calculation. Usually, the prediction accuracy requirement for sample pairs of large categories is not very high.

[0047] When the value of p is used to measure the accuracy of the prediction, the predetermined range in step S105 may be a random number less than 0-1. Figure 3 As shown in steps S310 and S311.

[0048] S106: Determine whether the confidence level is within a predetermined range. If yes, execute steps S107 and S108; otherwise, jump to S110 to continue execution.

[0049] S107: Update the parameters of the classification model through the current sample pair.

[0050] When the confidence level is low, in order to improve the prediction accuracy of the current sample pair, the parameters of the classification model may be updated using the current sample pair.

[0051] However, after updating the parameters of the classification model, although the classification accuracy of the current sample pair can be improved, the classification accuracy of other sample pairs, especially sample pairs similar to the current sample pair, may be reduced. For this reason, the current sample pair can be put into the reference sample set so that the training sample set can be expanded based on the current sample pair in the next iteration.

[0052] S108: Determine whether the category label of the current sample pair belongs to a predetermined category. The predetermined category is a category whose proportion in the original sample set is less than a preset threshold. If yes, execute steps S109 and S110; otherwise, jump to S110 to continue execution.

[0053] S109: Put the current sample pair as a reference sample pair into the reference sample set.

[0054] The predetermined category is the small category. Since the classification problem of the unbalanced data set pays more attention to whether the prediction results of the small category samples are accurate, in this embodiment, only the sample pairs of the small category are put into the reference sample set.

[0055] S110: Determine whether the training sample set is empty. If so, execute step S111; otherwise, jump to step S103 to continue execution.

[0056] S111: Determine whether the preset termination condition is met. If yes, the training of the classification model is completed; otherwise, jump to step S102 to continue execution.

[0057] In some embodiments, the preset termination condition may be that the number of loop executions (ie, the number of iterations) reaches a preset number of iterations.

[0058] In some embodiments, the preset termination condition may be that no sample pairs are put into the reference sample set in this round of cycle (or this iteration), that is, the prediction confidence of the classification model for small category sample pairs meets the requirements, thereby further ensuring the prediction accuracy of the classification model for small category samples.

[0059] The above steps S102 to S109 constitute an iterative operation. The "iteration" here refers to the activity of repeating the feedback process, the purpose of which is to make the category corresponding to the maximum value of the probability predicted by the classification model consistent with the category label of the sample pair. Each repetition of the process is called an "iteration", and the result of each iteration will be used for the next iteration.

[0060] The above steps S103 and S108 provide a method for executing steps S104 to S107 for each sample pair in the training sample set without repeatedly executing the same sample pair.

[0061] Of course, in some embodiments, other methods can be used to ensure that the samples in the training sample set execute steps S104 to S107 without causing the same sample pair to execute repeatedly. For example, the sample pairs in the training sample set are numbered in sequence, and the classification model is input one by one according to the number of the sample pairs. This specification does not list other methods one by one.

[0062] A training method, device and electronic device for a classification model provided in an embodiment of the present specification perform multiple iterative operations. In each iterative operation, the original sample set and the reference sample set are oversampled to obtain a training sample set, and the samples in the training sample set are input into the classification model in sequence, which can avoid the problem of the prediction result being biased towards a category with a larger number of sample pairs caused by inputting small category sample pairs as a small part of the training sample set into the classification model; after updating the parameters of the classification model with sample pairs in one iteration, if the sample pair is a small category sample pair, it is put into the reference sample set so that a new sample pair can be obtained based on the sample pair by oversampling in the next iteration, thereby further improving the prediction accuracy and generalization of the classification model for small category samples.

[0063] In the above step S102, "oversampling the original sample set and the reference sample set to obtain a training sample set" can be performed by mixing elements in the original sample set and the reference sample set to obtain a second sample set, and randomly oversampling the second sample set to obtain a training sample set; or randomly oversampling the second sample set to obtain a training sample set. Figure 3 The training sample set is obtained by the SMOTE oversampling method shown; or other oversampling methods. The present application does not limit the specific method of oversampling. The category to which the virtual sample generated by oversampling the sample of a small category belongs should be the same as the category to which the small category belongs.

[0064] In the next iteration, since the training sample set is obtained by oversampling based on the sample pairs given in this iteration, the training sample set in the next iteration has sample pairs similar to the sample pairs given in this iteration, so the prediction accuracy of the similar sample pairs can be verified in the next iteration.

[0065] Based on this setting, the training method provided in the embodiments of this specification can further improve the accuracy and generalization ability of predicting small category samples.

[0066] like Figure 2 As shown, the method of "oversampling the original sample set and the reference sample set to obtain a training sample set" in step S102 may include the following steps:

[0067] S1021: Filter out all sample pairs whose category labels belong to the predetermined category in the original sample set, and mix them with the sample pairs in the reference sample set to obtain a first sample set.

[0068] The predetermined category is a small category. Since a small category can be one category or multiple categories, step S210 refers to screening out all sample pairs of each small category. That is, when there are two small categories, all sample pairs of the first small category and all sample pairs of the second small category are screened out.

[0069] S1022: Oversample each type of sample in the first sample set to generate n i ·ρ i sample pairs, where n i is the number of sample pairs of class i in the first sample set, ρ i is the ratio of the number of class i sample pairs obtained by oversampling to the number of class i sample pairs in the first sample set.

[0070] ρ i , i.e. the sampling ratio, is usually a given value.

[0071] In some embodiments, at each iteration, n is the number of samples sampled from the category with the largest number of sample pairs in the original sample set. That is to say, in each iteration, the number of sample pairs of the small category is expanded to be equal to the number of sample pairs sampled from the large category. Among them, the large category refers to the category with the largest number of sample pairs in the original sample set.

[0072] In some embodiments, Figure 3 As shown in step S317, only in the first iteration operation, n is the number of sample pairs sampled from the category with the largest number of sample pairs in the original sample set; in the second and subsequent iterations, according to ρ i Negatively correlated with the number of iterations performed Adjust ρ i That is, as the number of iterations increases (1st, 2nd, 3rd, 4th, etc.), the sampling ratio ρ i It becomes smaller and smaller to prevent the classification model from overfitting.

[0073] The specific adjustment method can be, for example, Among them, ρ i,ep is the sampling ratio of this iteration (i.e., the epth iteration), ρ i,ep-1 is the sampling ratio of the previous iteration, and ep is the number of current iterations. Of course, an adjustment method can also be used, as long as the sampling ratio is reduced as the number of iterations increases.

[0074] S1023: Mix various sample pairs generated by oversampling with sample pairs in the first sample set to obtain a training sample set.

[0075] It can be seen from this step that in this embodiment, the training sample set includes all samples of each small category in the original sample set, as well as virtual samples generated by oversampling. Of course, the training sample set should also include samples of other categories other than the small category, which are usually extracted from the original sample set.

[0076] The present specification also provides a classification model training device that can be used to perform Figure 1 The training method of the classification model shown in Figure 4 As shown, the device includes an acquisition module 10 and an iteration module 20. The iteration module 20 includes an oversampling submodule 21 and an execution submodule 22. The execution submodule 22 further includes a prediction submodule 221, a calculation submodule 222, an update submodule 223 and a reference submodule 224.

[0077] The acquisition module 10 is used to acquire an original sample set, wherein the original sample set includes a plurality of sample pairs, each sample pair includes at least: feature data and a category label. The iteration module 20 is used to perform iteration operations cyclically until a preset termination condition is met.

[0078] The oversampling submodule 21 is used to oversample the original sample set and the reference sample set to obtain a training sample set. The execution submodule 22 is used to perform operations on each sample pair in the training sample set.

[0079] The prediction submodule 221 is used to input the feature data of the current sample pair into the classification model to obtain the prediction result of the current sample pair, and the prediction result includes the probability of being divided into each category. The calculation submodule 222 is used to calculate the confidence of the classification model according to the category label of the current sample pair and the probability of the current sample pair being divided into each category. The update submodule 223 is used to update the parameters of the classification model through the current sample pair when the confidence is within a predetermined range. The reference submodule 224 is used to put the current sample pair as a reference sample pair into the reference sample set when the confidence is within the predetermined range and the category label of the current sample pair belongs to a predetermined category; the predetermined category is a category whose proportion in the original sample set is less than a preset threshold.

[0080] In some embodiments, Figure 5 As shown, the oversampling submodule 21 includes a screening submodule 211 , a sampling submodule 212 and a mixing submodule 213 .

[0081] The screening submodule 211 is used to screen out all sample pairs whose category labels belong to the predetermined category in the original sample set, and mix them with the sample pairs in the reference sample set to obtain the first sample set.

[0082] The sampling submodule 212 is used to oversample each type of sample in the first sample set to generate n i ·ρ i sample pairs, where n i is the number of sample pairs of class i samples in the first sample set, ρ i is the ratio of the number of class i sample pairs obtained by oversampling to the number of class i sample pairs in the first sample set.

[0083] The mixing submodule 213 is used to mix various sample pairs generated by oversampling with sample pairs in the first sample set to obtain a training sample set.

[0084] In some embodiments, Figure 5 As shown, the iteration module 20 also includes a determination submodule 23 for determining n is the number of sample pairs sampled from the category with the largest number of sample pairs in the original sample set.

[0085] In some embodiments, Figure 5As shown, the iteration module 20 also includes an adjustment submodule 24 for adjusting the value of p in the second and subsequent iterations. i Negatively correlated with the number of iterations performed Adjust ρ i .

[0086] In some embodiments, the calculation submodule 222 calculates the confidence of the classification model by the following formula: σ=y c -max i≠c {y i}, where σ is the confidence, c is the category label of the current sample pair, and y c is the probability that the current sample pair is classified into category c, y i is the probability that the current sample pair is classified into categories other than category c.

[0087] In some embodiments, Figure 6 As shown, the calculation submodule 222 includes a first calculation submodule 2221 , a second calculation submodule 2222 and a third calculation submodule 2223 .

[0088] The first calculation submodule 2221 is used to calculate σ according to the following formula: σ=y c -max i≠c {y i}, where c is the category label of the current sample pair, y c is the probability that the current sample pair is classified into category c, y i is the probability that the current sample pair is classified into categories other than category c.

[0089] The second calculation submodule 2222 is used to calculate Δ according to the following formula: in, n c is the number of c-type sample pairs in the original sample set, n is the total number of sample pairs in the original sample set, Z=min i {r i}, n i is the number of sample pairs of categories other than category c in the original sample set, and i represents categories other than category c.

[0090] The third calculation submodule 2223 is used to calculate p according to the following formula: Among them, p is the confidence level.

[0091] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, the electronic device may include a processor 71 and a memory 72, wherein the processor 71 and the memory 72 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0092] The processor 71 may be a central processing unit (CPU). The processor 71 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0093] The memory 72 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the training method of the classification model in the embodiment of the present invention (for example, Figure 4 The processor 71 executes various functional applications and data classification of the processor by running the non-transitory software programs, instructions and modules stored in the memory 72, that is, the training method of the classification model in the above method embodiment is implemented.

[0094] The memory 72 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created by the processor 71, etc. In addition, the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 72 may optionally include a memory remotely arranged relative to the processor 71, and these remote memories may be connected to the processor 71 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0095] The one or more modules are stored in the memory 72 and when executed by the processor 71, perform the following steps: Figures 1 to 3 The training method of the classification model in the illustrated embodiment.

[0096] For details of the above electronic equipment, please refer to Figures 1 to 3 The corresponding descriptions and effects in the embodiments can be understood and will not be repeated here.

[0097] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0098] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip2. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog2. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0099] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0100] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions.

[0101] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0102] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute certain parts of the methods of each implementation method of the present application.

[0103] The present application can be used in many general or special computer system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0104] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0105] Although the present application has been described through embodiments, those skilled in the art will appreciate that there are many modifications and variations to the present application without departing from the spirit of the present application, and it is intended that the appended claims include these modifications and variations without departing from the spirit of the present application.

Claims

1. A classification model training method, characterized in that: include: Acquire an original sample set, wherein the original sample set includes a plurality of sample pairs, and each sample pair includes at least: feature data and a category label; The loop performs the following iterative operations until the preset termination condition is met: Oversampling the original sample set and the reference sample set to obtain a training sample set; Perform the following operations on each sample pair in the training sample set: Inputting feature data of the current sample pair into a classification model to obtain a prediction result of the current sample pair, wherein the prediction result includes a probability of being classified into each category; Calculating the confidence of the classification model according to the category label of the current sample pair and the probability that the current sample pair is classified into each category; When the confidence level is within a predetermined range, updating the parameters of the classification model using the current sample pair; When the confidence level is within the predetermined range and the category label of the current sample pair belongs to a predetermined category, the current sample pair is placed in a reference sample set as a reference sample pair; the predetermined category is a category whose proportion in the original sample set is less than a preset threshold; The oversampling of the original sample set and the reference sample set to obtain a training sample set includes: Screen out all sample pairs whose category labels belong to the predetermined category in the original sample set, and mix them with the sample pairs in the reference sample set to obtain a first sample set; For each class of samples in the first sample set, oversampling generates n i ·ρ i sample pairs, where n i is the number of sample pairs of class i samples in the first sample set, ρ i is the ratio of the number of class i sample pairs obtained by oversampling to the number of class i sample pairs in the first sample set; Various sample pairs generated by oversampling are mixed with sample pairs in the first sample set to obtain a training sample set.

2. The method according to claim 1, characterized in that The preset termination condition includes at least one of the following: When the number of loop executions reaches the preset number of iterations, no sample pairs are put into the reference sample set.

3. The method according to claim 1, characterized in that Also includes: Sure n is the number of sample pairs sampled from the category with the largest number of sample pairs in the original sample set.

4. The method according to claim 1, characterized in that: Also includes: In the second and subsequent iterations, according to ρ i Negatively correlated with the number of iterations performed Adjust ρ i .

5. The method according to claim 1, characterized in that The confidence of the classification model is calculated by the following formula: σ=y c -max i≠c {y i }, where σ is the confidence, c is the category label of the current sample pair, and y c is the probability that the current sample pair is classified into category c, y i is the probability that the current sample pair is classified into categories other than category c.

6. The method according to claim 1, characterized in that The confidence of the classification model is calculated by the following steps: Calculate σ according to the following formula: σ = y c -max i≠c {y i }, where c is the category label of the current sample pair, y c is the probability that the current sample pair is classified into category c, y i is the probability that the current sample pair is classified into categories other than category c; Calculate Δ according to the following formula: in, n c is the number of c-type sample pairs in the original sample set, n is the total number of sample pairs in the original sample set, Z=min i {r i }, n i is the number of sample pairs of categories other than category c in the original sample set, and i represents categories other than category c; Calculate p according to the following formula: Among them, p is the confidence level.

7. A training device for a classification model, characterized in that: include: An acquisition module, used for acquiring an original sample set, wherein the original sample set includes a plurality of sample pairs, each sample pair includes at least: feature data and a category label; An iteration module is used to cyclically execute the iteration operation until a preset termination condition is met; the iteration module includes: An oversampling submodule, used for oversampling the original sample set and the reference sample set to obtain a training sample set; An execution submodule is used to perform operations on each sample pair in the training sample set; the execution submodule includes: A prediction submodule, used for inputting feature data of the current sample pair into a classification model to obtain a prediction result of the current sample pair, wherein the prediction result includes a probability of being classified into each category; A calculation submodule, used for calculating the confidence of the classification model according to the category label of the current sample pair and the probability of the current sample pair being classified into each category; An updating submodule, configured to update the parameters of the classification model using the current sample pair when the confidence level is within a predetermined range; A reference submodule, configured to, when the confidence level is within the predetermined range and the category label of the current sample pair belongs to a predetermined category, place the current sample pair as a reference sample pair into a reference sample set; the predetermined category is a category whose proportion in the original sample set is less than a preset threshold; The oversampling submodule comprises: A screening submodule, configured to screen out all sample pairs in the original sample set whose category labels belong to a predetermined category, and mix them with the sample pairs in the reference sample set to obtain a first sample set; The sampling submodule is used to oversample each type of sample in the first sample set to generate n i ·ρ i sample pairs, where n i is the number of sample pairs of class i samples in the first sample set, ρ i is the ratio of the number of class i sample pairs obtained by oversampling to the number of class i sample pairs in the first sample set; The mixing submodule is used to mix various sample pairs generated by oversampling with the sample pairs in the first sample set to obtain a training sample set.

8. The device according to claim 7, characterized in that The calculation submodule includes: The first calculation submodule is used to calculate σ according to the following formula: σ = y c -max i≠c {y i }, where c is the category label of the current sample pair, y c is the probability that the current sample pair is classified into category c, y i is the probability that the current sample pair is classified into categories other than category c; The second calculation submodule is used to calculate Δ according to the following formula: in, n c is the number of c-type sample pairs in the original sample set, n is the total number of sample pairs in the original sample set, Z=min i {r i }, n i is the number of sample pairs of categories other than category c in the original sample set, and i represents categories other than category c; The third calculation submodule is used to calculate p according to the following formula:

9. An electronic device, characterized in that: include: A memory and a processor, wherein the processor and the memory are communicatively connected to each other, the memory stores computer instructions, and the processor implements the steps of the method according to any one of claims 1 to 6 by executing the computer instructions.

10. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, and when the computer program instructions are executed, the steps of the method according to any one of claims 1 to 6 are implemented.

Images