Sample weight self-adaption method and system for unbalanced data of positive and negative samples and storage medium

Abstract

The invention discloses a sample weight self-adaption method and system for unbalanced data of positive and negative samples and a storage medium, and the method comprises the steps: transmitting samples to a classification network model through a data set, and enabling the transmitted samples to at least comprise a target sample, a positive sample and a negative sample; modifying a classification loss function of the classification network model; the performance of the classification network model is monitored according to evaluation indexes of the classification network model on different types of samples, the evaluation indexes are accuracy rates or / and recall rates, and the accuracy rates at least comprise the accuracy rates of positive samples and negative samples; the sample weight is dynamically adjusted along with classification network model training. According to the method, the sample weight can be self-adapted along with the change of the training gradient even on a data set with large difference between positive and negative samples, and training is assisted.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and specifically relates to a sample weight adaptive method, system and storage medium for positive and negative sample imbalanced data. Background Art

[0002] In machine learning and deep learning, dealing with imbalanced data is a common challenge, especially when the model is required to compare a pair of data samples and learn to map them to a common representation space. The ratio of positive and negative samples is particularly important to the model. When training on an imbalanced dataset, the model may focus too much on the features of the majority class samples, thereby performing well on the training data but having poor generalization ability on unseen data.

[0003] Currently, the commonly used solutions are usually to increase or decrease data, such as oversampling, undersampling, ensemble methods and generating new samples (Synthetic Minority Over-sampling Technique, SMOTE), but the actual application cost is relatively high. Summary of the invention

[0004] Therefore, in order to solve the above-mentioned shortcomings, the present invention provides a sample weight adaptation method, system and storage medium for positive and negative sample imbalance data; the present invention can achieve adaptive sample weights as the training gradient changes even on a data set with a large difference between positive and negative samples, thereby assisting training.

[0005] In a first aspect, the present invention provides a sample weight adaptive method for positive and negative sample imbalance data, comprising:

[0006] Samples are passed into the classification network model through the data set, and the passed samples include at least target samples, positive samples and negative samples;

[0007] Modify the classification loss function of the classification network model;

[0008] Monitoring the performance of the classification network model according to evaluation indicators of the classification network model on samples of different categories, wherein the evaluation indicators are accuracy and / or recall, wherein the accuracy includes at least the accuracy of positive samples and negative samples;

[0009] Dynamically adjust sample weights as the classification network model is trained.

[0010] Optionally, the method to dynamically adjust sample weights is:

[0011] Define an adjustment factor f(acc) about the performance index to the accuracy, apply the adjustment factor to the sample weight, and obtain the dynamically adjusted sample weight w t+1 ;

[0012] Where: w t =f(acc)*w t+1 ;

[0013] w t is the sample weight of the previous step, w t+1 is the sample weight after dynamic adjustment.

[0014] Optionally, the adjustment factor for the positive sample weight in the sample weight is The adjustment factor for the negative sample weight is

[0015] Among them, e is the exponent, acc is the accuracy of the positive sample obtained during verification, and acc 负 is the accuracy of negative samples during verification; k is a constant.

[0016] Optionally, the initial value of k is set to a constant greater than 1 according to the difference between the samples.

[0017] Optionally, it approaches 1 in the subsequent process of positive and negative samples gradually approaching each other (it can also be set to 1 when the subsequent difference is not large, but the k value must not be less than 1).

[0018] The principle of the above sample weight adaptive method for positive and negative sample imbalanced data is:

[0019] Use evaluation indicators (accuracy and / or recall, preferably accuracy) to evaluate each round of network training and monitor model performance. The accuracy of positive and negative samples should be calculated separately, that is, The validation set should ensure that the features and rules that need to be learned are similar to those in the training set so that the validation results are representative and the sample weights are dynamically adjusted based on the data obtained from monitoring model performance.

[0020] In a second aspect, the present invention provides a sample weight adaptive system for positive and negative sample imbalanced data, comprising:

[0021] The sample input module is used to input samples to the classification network model through the data set. The input samples include at least target samples, positive samples and negative samples.

[0022] A modification module is used to modify the classification loss function of the classification network model;

[0023] A monitoring module, used to monitor the performance of the classification network model according to evaluation indicators of the classification network model on samples of different categories, wherein the evaluation indicators at least include the accuracy of positive samples and negative samples; and

[0024] The dynamic adjustment module is used to dynamically adjust the sample weights as the classification network model is trained.

[0025] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the sample weight adaptation method for positive and negative sample imbalanced data.

[0026] The present invention has the following advantages:

[0027] The present invention considers modifying the loss function to weight samples, and provides a sample weight adaptation method, system and storage medium for positive and negative sample imbalance data as the model is trained and the indicator weights are adapted. The present invention can achieve adaptive sample weights as the training gradient changes even on data sets with large differences between positive and negative samples, thereby assisting training. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the sample weight adaptation method for positive and negative sample imbalanced data;

[0029] Figure 2 It is a systematic block diagram of the sample weight adaptive system for positive and negative sample imbalance data;

[0030] In the figure: 100, sample input module; 200, modification module; 300, monitoring module; 400, dynamic adjustment module. DETAILED DESCRIPTION

[0031] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0032] In this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises", or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or apparatus.

[0033] As described in the background technology, in machine learning and deep learning, dealing with imbalanced positive and negative samples (Imbalanced Data) is a common challenge, especially when the model is required to compare a pair of data samples and learn to map them to a common representation space. The ratio of positive and negative samples is particularly important to the model. When training on an imbalanced data set, the model may pay too much attention to the characteristics of the majority class samples, so that it performs well on the training data, but has poor generalization ability on unseen data. The commonly used solution is usually to increase or decrease data, such as oversampling, undersampling, ensemble methods, and generate new samples (Synthetic Minority Over-sampling Technique, SMOTE), but the actual application cost is high.

[0034] Based on the above reasons, this embodiment provides a sample weight adaptive method for positive and negative sample imbalance data, such as Figure 1 As shown, the method comprises the following steps:

[0035] S100, passing samples to the classification network model through the data set, where the passed samples include at least target samples, positive samples, and negative samples.

[0036] In this step, when creating the data set, it is necessary to count the number of positive and negative samples in the label to facilitate the subsequent calculation of the accuracy and other performance of the monitoring network model.

[0037] S200, modifying the classification loss function (such as ternary loss, etc.) of the classification network model.

[0038] This step uses the loss function to influence the model's attention and weight distribution to different samples by means of sample weighting, thereby adjusting the learning behavior of the network model during the training process, making it pay more attention to important samples and improving the performance and generalization ability of the network model.

[0039] S300, monitoring the performance of the classification network model according to the evaluation index of the classification network model on samples of different categories to determine whether the sample weighting has achieved the expected effect.

[0040] In this step, the evaluation index is accuracy and / or recall, where accuracy at least includes positive sample accuracy acc 正 And the accuracy of negative samples acc 负 ,in TP is a true positive sample, FP is a false positive sample, TN is a true negative sample, and FN is a false negative sample.

[0041] S400, dynamically adjust the weights of positive and negative samples as the classification network model is trained.

[0042] For example, the initial weight value of the positive and negative samples is defined as 1, and as the model is trained, the sample weight is dynamically adjusted. An adjustment factor f(acc) related to the accuracy of the performance indicator is defined, and the adjustment factor is applied to the sample weight to obtain the dynamically adjusted sample weight w t+1 ; Among them: w t =f(acc)*w t+1 ;w t is the sample weight of the previous step, w t+1 is the sample weight after dynamic adjustment.

[0043] Through the above steps S100 to S400, it is possible to achieve adaptive sample weights as the training gradient changes, even on a data set with a large difference between positive and negative samples, to assist training.

[0044] In one embodiment, the adjustment factor of the positive sample weight is The adjustment factor for the negative sample weight is Among them, e is the exponent, acc is the accuracy of the positive sample obtained during verification, and acc 负 is the accuracy of negative samples during verification; the k value can be set to a number greater than 1 at the beginning according to the difference in samples, and approach 1 in the subsequent gradual approach of positive and negative samples; it can also be set to 1 when the difference between positive and negative samples is not large, but the k value must not be less than 1. The adjustment factor in the network model can make the adjustment of weights change with the gradient of the model, which is more flexible in application and has a greater response.

[0045] The above adjustment factor is an exponential function of the accuracy. The exponential function selected here can make the model pay more attention to difficult samples in the initial stage, and gradually reduce the degree of attention to difficult samples as the training progresses, so as to achieve the purpose of balancing samples. The part of the adjustment factor formula that most affects its calculation result is the absolute value of acc positive - acc negative. As the weight factor is continuously adjusted during the training process, the accuracy of the model is also optimized. Under the action of the adjustment factor formula, the overall accuracy is improved while the difference in the accuracy of positive and negative samples is reduced (the difference will be very small in the final stage of training, which is not a big difference), thus avoiding the situation where one of the positive or negative samples performs well while the other performs poorly.

[0046] In another embodiment, if Figure 2 As shown, a sample weight adaptive system for positive and negative sample imbalanced data includes:

[0047] The sample input module 100 is used to input samples to the classification network model through the data set, and the input samples include at least target samples, positive samples and negative samples;

[0048] A modification module 200, used to modify the classification loss function of the classification network model;

[0049] A monitoring module 300, configured to monitor the performance of the classification network model according to evaluation indicators of the classification network model on samples of different categories, wherein the evaluation indicators at least include the accuracy of positive samples and negative samples; and

[0050] The dynamic adjustment module 400 is used to dynamically adjust the sample weights as the classification network model is trained.

[0051] The sample weight adaptation system for positive and negative sample imbalanced data is used to implement a sample weight adaptation method for positive and negative sample imbalanced data, which can achieve adaptive sample weights as training gradients change even on data sets with large differences between positive and negative samples, thereby assisting training.

[0052] Based on the above embodiments, the third aspect of the present invention further provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is used to enable the computer to execute the sample weight adaptation method for positive and negative sample imbalance data disclosed in the embodiments of the present invention.

[0053] Based on the above embodiments, the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the sample weight adaptation method for positive and negative sample imbalanced data disclosed in the embodiments of the present invention.

[0054] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0055] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0056] In the several embodiments provided in the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0057] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0058] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0059] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present application.

[0060] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A sample weight adaptive method for positive and negative sample imbalance data, characterized in that: It includes passing samples into the classification network model through the data set, and the passed samples at least include target samples, positive samples and negative samples; Modify the classification loss function of the classification network model; Monitoring the performance of the classification network model according to evaluation indicators of the classification network model on samples of different categories, wherein the evaluation indicators are accuracy and / or recall, wherein the accuracy includes at least the accuracy of positive samples and negative samples; Dynamically adjust sample weights as the classification network model is trained.

2. According to claim 1, a sample weight adaptive method for positive and negative sample imbalance data is characterized in that: The method to dynamically adjust sample weights is: Define an adjustment factor f(acc) about the performance index to the accuracy, apply the adjustment factor to the sample weight, and obtain the dynamically adjusted sample weight w t+1 ; Where: w t =f(acc)*w t+1 ; w t is the sample weight of the previous step, w t+1 is the sample weight after dynamic adjustment.

3. According to claim 2, a sample weight adaptive method for positive and negative sample imbalance data is characterized in that: The adjustment factor of the positive sample weight in the sample weight is The adjustment factor for the negative sample weight is Among them, e is the exponent, acc is the accuracy of the positive sample obtained during verification, and acc 负 is the accuracy of negative samples during verification; k is a constant.

4. According to claim 3, a sample weight adaptive method for positive and negative sample imbalance data is characterized in that: The initial value of k is set to a constant greater than 1 according to the difference of the samples.

5. According to claim 3, a sample weight adaptive method for positive and negative sample imbalanced data is characterized in that: The value of k is 1.

6. A sample weight adaptive system for positive and negative sample imbalance data, characterized in that: include: The sample input module is used to input samples to the classification network model through the data set. The input samples include at least target samples, positive samples and negative samples. A modification module is used to modify the classification loss function of the classification network model; A monitoring module, used to monitor the performance of the classification network model according to evaluation indicators of the classification network model on samples of different categories, wherein the evaluation indicators at least include the accuracy of positive samples and negative samples; as well as The dynamic adjustment module is used to dynamically adjust the sample weights as the classification network model is trained.

7. A storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the sample weight adaptation method for positive and negative sample imbalanced data as described in any one of claims 1 to 5.

Images