Momentum-based data evaluation method, system, and storage medium

Abstract

The application discloses a momentum-based data evaluation method and system and a storage medium. Streaming data commonly used in machine learning is input; the contribution of data samples to a classification model is determined according to the change in momentum, wherein the momentum adopts a heavy ball momentum algorithm, and the change value of the momentum is calculated according to two adjacent iterations; the original data samples are dynamically adjusted, the data samples are evaluated according to the change value of the momentum, a threshold value is set, the current iteration input data samples and previous data samples are weighted to form a new data sample set; the model determines whether the classification is correct according to the label and outputs the classification result. The application is more suitable for an algorithm framework in a deep learning background, can effectively evaluate the redundancy and imbalance of data, can effectively evaluate text, video, voice and other training sample data under the condition of limited resources, and the performance of the momentum algorithm can still surpass SGD under the condition that the data set is unbalanced.

Description

TECHNICAL FIELD

[0001] The present application relates to a data evaluation method and system, more particularly, to a momentum-based data evaluation method, evaluation system and storage medium. BACKGROUND

[0002] Data, model and training optimization algorithm are the three elements of machine learning. At present, the evaluation research in the field of machine learning mainly focuses on the model and training optimization algorithm level, but few scholars study the evaluation of data. The evaluation index of the model includes accuracy, F1 value, etc.; the evaluation index of the algorithm includes convergence speed, iteration number, CPU time and cycle number, etc. There is still little research on data evaluation.

[0003] The standard data quality evaluation system mainly refers to the following indicators: integrity, accuracy, effectiveness, timeliness and consistency. Due to the development of machine learning, higher requirements are put forward for data evaluation, especially the redundancy and imbalance of data lack effective evaluation methods. From the perspective of machine learning, the existing data evaluation technology and system are mainly based on gradient method. With the development of deep learning technology, adaptive momentum optimization algorithm has become the mainstream optimization method. The gradient-based evaluation method cannot meet the effective evaluation of large-scale training sample data at present, mainly because the momentum change can more accurately measure the contribution of training sample data to the objective function than the gradient change, so the momentum can be regarded as a key factor in data quality evaluation based on deep learning technology.

[0004] At present, deep reinforcement learning has achieved success in Go, Texas Hold'em, StarCraft, and King Glory. The main reason is that there are many data samples in the game, and data can be obtained without cost. In real life, the data in some fields is not easy to obtain, especially in the military field, the data has the characteristics of high value and small quantity, and it is impossible to obtain data without cost, so how to learn efficiently under the condition of resource limitation is an important problem to be solved. In addition, the data in real application. Therefore, it is particularly important to effectively evaluate the sample data during the training process.

[0005] In addition, the evaluation research in the field of machine learning mainly focuses on the level of model and training optimization algorithm, but few scholars study the evaluation problem of training data samples. At present, deep learning technology develops rapidly and achieves great success in the fields of computer vision and natural language processing. The redundancy and imbalance of data become important factors that affect the model and algorithm. In terms of algorithm, the adaptive step algorithm and the momentum method have achieved success, and the adaptive step algorithm that comprehensively uses the two strategies has become the mainstream method. In terms of machine learning, especially deep learning, the traditional gradient-based method cannot well evaluate the redundancy and imbalance of data, and the momentum-based method is used instead. The change of momentum has not been noticed in the field of data evaluation, which can better evaluate the quality and importance of data samples. The rapid development of deep learning technology has higher requirements for training sample data, and the standard data quality system evaluation lacks the indicators of redundancy and imbalance.

[0006] In reality, many data in machine learning tasks arrive in the form of a stream. On the one hand, it is impossible to obtain all training samples at once for massive high-dimensional training data, and there may not be enough memory space to store them. On the other hand, there are problems of redundancy and imbalance in specific applications that contain fewer high-quality data samples. Online learning refers to a class of learning algorithms that learn to optimize the prediction model in the sequentially arriving data stream instances. This dynamic learning makes online learning more scalable and more memory efficient, and the online learning model has been widely used in machine learning. The general process of the online learning classification model is shown in (Fig. 1): ① input data samples (text, image, voice, etc. streaming data); ② training through optimization algorithm and classification model; ③ the model judges whether the classification is correct according to the label and outputs the classification result. Figure 1

[0007] Inspired by the principle of momentum in physics, the contribution of data samples to the objective function can be judged according to the change of momentum in machine learning. In physics, momentum is a physical quantity that describes the inertia of a particle, and the momentum method has become the mainstream optimization algorithm in the field of deep learning in computer science. SUMMARY

[0008] 1. The purpose of the present application

[0009] The present application aims to solve the problem of limited data resources and proposes a data evaluation method and system based on momentum from the perspective of machine learning, thereby effectively evaluating the redundancy and imbalance of data samples.

[0010] 2. The technical solution of the present application

[0011] The present application proposes a data evaluation method based on momentum, comprising:​

[0012] Inputting streaming data common in machine learning;

[0013] According to the change of momentum to determine the contribution of data samples to the classification model, wherein the momentum adopts the heavy ball momentum algorithm, and the change value of the momentum is calculated according to adjacent two-step iterations;

[0014] The original data samples are dynamically adjusted, the data samples are evaluated according to the change value of the momentum, a threshold is set, the data samples input in the current iteration are weighted with the previous data samples to form a new data sample set;

[0015] The model determines whether the classification is correct according to the label and outputs the classification result.

[0016] Preferably, the input streaming data includes one or more of text, image, and voice.

[0017] Preferably, the contribution of the data samples to the classification model is determined according to the change of the momentum, which is trained by the adaptive momentum optimization algorithm and the online learning model.

[0018] Preferably, the heavy ball momentum algorithm is used for iteration to display the accumulated gradient information.

[0019] Further,

[0020] Step 1, input data samples (text, image, voice, and other streaming data);

[0021] Step 2, training by the adaptive momentum optimization algorithm and the online learning model, iteration number t from 1 to T;

[0022] Using the heavy ball momentum algorithm: w t+1 =w t -αg t +β(w t -w t-1 ), wherein α is the learning rate, β represents the momentum coefficient, w is the optimization variable, and w t refers to the optimization variable of the t-th iteration of the optimization algorithm. It can be seen that M t+1 =β(w t -w t-1 ) is the momentum term of the current t+1 step, which displays the accumulated gradient information of the previous t+1 steps.

[0023] Further, the change value of the momentum is calculated according to adjacent two-step iterations, specifically:

[0024] Step 3, algorithm training process, the change value of the momentum is calculated according to adjacent two-step iterations,

[0025] M t+1M t = beta(w t -w t-1 ) - beta(w t-1 -w t-2 ).

[0026] Further,

[0027] According to the change value of the momentum, a threshold value epsilon is set to reflect the importance of the data sample, M t+1 M t >= epsilon is an important data sample, M t+1 M t < epsilon indicates that the importance of the sample is low. The data sample input in the tth step iteration is weighted with the data sample in the previous t-1 step to form a new data sample set.

[0028] The application provides a momentum-based data evaluation system, comprising a memory and a processor, the memory stores a computer program, and the processor implements the method steps when executing the computer program.

[0029] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method steps.

[0030] 3、The beneficial effects of the application

[0031] (1) The application provides a novel momentum-based data evaluation method, which is more suitable for algorithm framework in deep learning background, can effectively evaluate the redundancy and imbalance of data, and fills the lack of two standard data quality system evaluation indexes.

[0032] (2) The momentum-based data evaluation system provided by the application can effectively evaluate the training sample data such as text, video and voice under the condition of limited resources, and the performance of the momentum algorithm can still surpass SGD under the condition of unbalanced data set.

[0033] In summary, the application can meet the requirements of current machine learning, especially deep learning, for data quality, and make the standard evaluation system index more perfect. DETAILED DESCRIPTION

[0034] Figure 1 It is a general process of online learning classification model;

[0035] Figure 2 It is a flowchart of the momentum-based data evaluation system;

[0036] Figure 3 It is a comparison chart of text classification data set training loss and test accuracy. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0038] The embodiments of the present application will be described in further detail below with reference to the drawings.

[0039] The momentum optimization algorithm refers to a method of adding momentum operation to the original gradient descent method. The momentum represents the difference between the current iteration and the previous iteration, which embodies the full use of the "inertia" of the previous iteration. According to the change of the momentum, the importance of the data sample to the model classification can be effectively observed. The adaptive optimization algorithm is formed by combining the momentum method with the adaptive step strategy. For the task based on deep learning, the momentum optimization method of the adaptive strategy all shows good performance.

[0040] The present application mainly proposes a new evaluation method and system for application under resource conditions. Specifically, for the streaming data (including text, video, audio, etc.) commonly used in machine learning, the contribution of the data sample to the classification model is judged according to the change of the momentum, and then the original data sample is dynamically adjusted.

[0041] It should be noted that the redundancy and uniqueness of the data do not mean the same thing. The redundant data do not necessarily refer to repeated data samples or attributes, but the data does not work for the performance of the classification model.

[0042] Table 1 Data evaluation system index and meaning

[0043]

[0044]

[0045] Embodiment 1

[0046] The present application adds a data evaluation module in the general process of machine learning training, that is, the training data sample is evaluated in real time in the process of model training, so as to further dynamically adjust the data sample. The general process of the data evaluation system based on momentum is shown as follows: Figure 2

[0047] Step 1, input data sample (text, image, voice, etc. Streaming data);

[0048] ​Step 2, training by adaptive momentum optimization algorithm and online learning model, iteration number t from 1 to T;

[0049] Using heavy ball momentum algorithm: w t+1 = w t - a g t + b (w t - w t-1 ), it can be seen that M t+1 = b (w t - w t-1 ) is the momentum term of the current t+1 step, which shows the information of the accumulated gradient of the previous t+1 steps.

[0050] Step 3, algorithm training process, according to the adjacent two steps iteration to calculate the change value of momentum,

[0051] M t+1 - M t = b (w t - w t-1 ) - b (w t-1 - w t-2 );

[0052] Step 4, evaluate the data samples according to the change of momentum, set a threshold value e, which reflects the importance of data samples, M t+1 - M t > e is an important data sample, M t+1 - M t < e indicates that the importance of the sample is low. The data samples input in the t step iteration are weighted with the data samples in the previous t-1 step to form a new data sample set;

[0053] Step 5, the model judges whether the classification is correct according to the label and outputs the classification result.

[0054] It can be found that the data system does not change the general process of machine learning training, only steps 3 and 4 are added. Specifically, the core of the data evaluation method is mainly based on the change of momentum to judge the size of the data contribution to the objective function, where the large momentum change indicates that the data sample has a large effect on the model classification, and the small momentum change indicates that the data sample has a small effect on the classification. At this time, after judging the change of momentum, the data samples are weighted to form new data samples, so as to dynamically adjust or generate new training data samples.

[0055] (1) Experimental scheme for verifying that momentum can evaluate data redundancy

[0056] The data adopts a benchmark data set THUCNews about text, which is generated by filtering historical data of Sina News RSS channel from 2005 to 2011. The text length is between 20 and 30. There are 10 categories, 20,000 pieces of text in each category. The categories are: finance, real estate, stock, education, technology, society, politics, sports, game and entertainment. There are 180,000 pieces of training set, 10,000 pieces of verification set and 10,000 pieces of test set. The random optimization method can simulate the online learning mode of the given data set, that is, a small batch of data is extracted from the given data for training. The comparison method is the basic stochastic gradient descent (SGD) method and the momentum-based stochastic gradient descent (Momentum) method. The experiment can verify that the momentum-based method has an advantage over the gradient-based method on the text classification data set, which can show that the momentum can accelerate the convergence and improve the classification performance of the model.

[0057] (2) The experimental scheme for verifying the momentum can evaluate the data imbalance

[0058] Waimai_10k is a Chinese corpus for sentiment classification, which collects user reviews collected by a certain takeout platform, 4000 positive reviews and about 8000 negative reviews. The data field includes: Label: 1 represents positive comments, and 0 represents negative comments. Review: The number of positive and negative samples is not balanced, which belongs to an unbalanced data set. The random optimization method: SGD and Momentum method are still used for comparison to verify that momentum can be more effective for unbalanced data.

[0059] (3) Simulation experiment results

[0060] The experimental results of the two data sets are shown in Figure 3 . The dark curve represents the momentum algorithm, and the light curve is the classic SGD algorithm. It can be seen that for large-scale text classification data sets, the momentum algorithm can converge faster and have higher test accuracy. It shows that momentum can effectively overcome the influence of data redundancy. The Waimai_10k data set is an unbalanced data set, and the performance of the momentum algorithm can still surpass SGD, which shows that momentum can also overcome the influence of data imbalance.

[0061] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the scope of the disclosed technology can be easily thought of by those skilled in the art, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A momentum-based data evaluation method, characterized in that: Input streaming data, a common type of data used in machine learning; The contribution of data samples to the classification model is determined by the change in momentum, where the momentum is calculated using the heavy ball momentum algorithm based on the change in momentum between two adjacent iterations. The original data samples are dynamically adjusted. The data samples are evaluated based on the momentum change value. A threshold is set, and the data samples input in the current iteration are weighted with the previous data samples to form a new data sample set. The contribution of the data to the objective function is judged based on the momentum change. A large momentum change indicates that the data sample has a large role in the model classification, while a small momentum change indicates that the data sample has a small role in classification. After the change, the data samples are weighted to form new data samples, thereby dynamically adjusting or generating new training data samples. The model determines whether the classification is correct based on the labels and outputs the classification result; The method further includes: Step 1: Input data sample; Step 2: Train the model using an adaptive momentum optimization algorithm and an online learning model, iterating a certain number of times. t From 1 to T ; Using the heavy ball momentum algorithm: ,in α It's the learning rate. β Represents the momentum coefficient. w It is an optimization variable. w t The optimization algorithm iteration number is... t The optimization variables of the step, i.e. , For the present t The momentum term of +1 step shows the previous... t +1 step accumulated gradient information; Step 3: Algorithm training process, calculating the change in momentum based on two adjacent iterations: ; The data samples are evaluated based on the change in momentum, and a threshold ε is set to reflect the importance of the data samples. These are important data samples. This indicates that the data sample has low importance; for the first t The input data samples in each iteration are the same as those in the previous iterations. t The data samples from step -1 are weighted to form a new data sample set.

2. The momentum-based data evaluation method according to claim 1, characterized in that: The streaming data includes one or more of the following: text, images, and audio.

3. A momentum-based data evaluation system, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that; When the processor executes the computer program, it implements the method as described in any one of claims 1-2.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method as described in any one of claims 1-2.

Images