A method, system and device for generating a target variable prediction model

By generating independent variable random matrix and using adversarial network models to train neural networks to establish causal relationships, the problem of low robustness of machine learning models based on correlation training in the existing technology is solved, and the data distribution changes are achieved. Stable and accurate prediction of effects.

CN112101516BActive Publication Date: 2024-11-26PENG CHENG LAB +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010750071.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-07-30
Publication Date
2024-11-26
Estimated Expiration
2040-07-30

AI Technical Summary

Technical Problem

The machine learning models in the prior art are based on correlation training, resulting in low robustness and inability to meet the accurate prediction requirements of the target dependent variables, especially when the data distribution changes, the prediction effect is unstable.

Method used

By obtaining the original data set, a random matrix of independent variables is generated, and a weight vector is obtained using an adversarial network model, which is used to train neural network models and establish causal relationships to improve the robustness of the predictive model.

Benefits of technology

Even when the data distribution changes, the generated target variable prediction model can maintain good prediction capabilities, improving the stability and accuracy of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112101516B_ABST
    Figure CN112101516B_ABST
Patent Text Reader

Abstract

The present invention proposes a method, system and device for generating a target variable prediction model, which obtains an original data set; inputs the independent variable data contained in the original data set and the independent variable random matrix randomly generated by the independent variable data into a preset adversarial network model, and uses the preset adversarial network model to obtain a weight vector; uses the original data set and the weight vector to train the preset neural network model to be trained and adjust the model parameters until a trained target variable prediction model is obtained. The embodiment of the present invention uses a preset adversarial network model to generate a weight vector according to the original data set, and uses the weight vector to train the loss function of the preset network model to obtain a target variable prediction model, and the target variable prediction model can still maintain good prediction capabilities even when the data distribution of the training data set has changed significantly.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a method, system and device for generating a target variable prediction model. Background Art

[0002] Machine learning methods are currently widely used in many fields such as image recognition, speech recognition, natural language processing, and bioinformatics. However, there is a problem with the machine learning methods in the prior art. Traditional machine learning models are almost all established through correlation. Although the correlation describes the characteristics of events changing simultaneously, this characteristic does not represent causality, which leads to the output of machine learning being likely to be local and unstable. Therefore, the correlation in the training data set may not exist or change in the test set and the actual use of the model, resulting in the robustness of the machine learning model trained using the training data set not meeting the requirements.

[0003] Therefore, the prior art needs to be further improved. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method, system and device for generating a target variable prediction model, aiming to solve the defect in the prior art that the target variable prediction model trained based on correlation has low robustness and cannot meet the demand for accurate prediction of the target dependent variable.

[0005] In a first aspect, this embodiment discloses a method for generating a target variable prediction model, which includes:

[0006] Acquire an original data set; wherein the original data set contains a plurality of independent variable data and a dependent variable, a preset number of independent variable data are formed into an independent variable data matrix in the form of columns, and each row of data in the independent variable data matrix is ​​used as a sample data;

[0007] Obtaining an independent variable random matrix according to the independent variable data matrix;

[0008] Input the independent variable data matrix and the independent variable random matrix into a preset adversarial network model, and use the preset adversarial network model to obtain a weight vector corresponding to the independent variable data matrix; wherein the weight vector is a weight vector composed of weights corresponding to each sample data;

[0009] The original data set and the weight vector are used to train the preset neural network model to be trained and adjust the model parameters until a trained target variable prediction model is obtained.

[0010] Optionally, the step of obtaining an independent variable random matrix according to the independent variable data matrix includes:

[0011] The order of each column of the independent variable data matrix is ​​randomly adjusted to generate a random matrix of independent variables.

[0012] Optionally, the preset adversarial network model includes a generation module and a differentiation module;

[0013] The step of inputting the independent variable data matrix and the independent variable random matrix into a preset adversarial network model and obtaining a weight vector corresponding to the independent variable data matrix using the preset adversarial network model comprises:

[0014] Vertically concatenate the independent variable data matrix and its corresponding independent variable random matrix to obtain a concatenated training data matrix;

[0015] The training data matrix is ​​used to perform multiple iterations of training on the distinguishing module and the generating module to obtain the weight vector.

[0016] Optionally, the step of performing multiple iterations of training on the distinguishing module and the generating module using the training data matrix to obtain the weight vector includes:

[0017] Inputting the training data matrix into the differentiation module to obtain the classification result output by the differentiation module;

[0018] Using the classification result and the preset initial weight vector to weight the loss function of the differentiation module, an initial total loss value of the preset adversarial network model is calculated, and model parameters of the preset adversarial network model are adjusted according to the initial total loss value;

[0019] Input the preset initial weight vector and the loss value corresponding to each sample data into the generation module to obtain the updated weight vector corresponding to the sample data output by the generation module;

[0020] Inputting the training data matrix into the differentiation module to obtain an updated classification result output by the differentiation module;

[0021] Calculating the loss value of the differentiation module according to the classification result and the updated weight vector, adjusting the updated weight vector according to the loss value, and calculating the updated total loss value of the preset adversarial network model;

[0022] Repeat the steps of inputting the loss value of each sample data and the updated weight vector into the generation module to obtain the iterative update weight vector output by the generation module corresponding to the independent variable data matrix; until the updated total loss value of the preset adversarial network model meets the preset requirements, and obtain the weight vector corresponding to the independent variable data matrix.

[0023] Optionally, the loss function of the differentiation module is a weighted cross entropy loss function.

[0024] Optionally, the step of using the original data set and the weight vector to train the preset neural network model to be trained and adjusting model parameters until a trained target variable prediction model is obtained includes:

[0025] Inputting the original data set into a preset neural network model to obtain a target feature result output by the preset neural network model;

[0026] Weighting the loss function of the preset neural network model according to the weight vector, and calculating the loss value of the preset neural network model according to the target feature result and the weighted loss function;

[0027] The model parameters of the preset neural network model are adjusted according to the loss value of the preset neural network model, and the step of inputting the original data set into the preset neural network model is repeated until the target network model is obtained through training.

[0028] Optionally, the independent variable data contained in the original data set is price index data synthesized from futures daily frequency price and volume data, and the original data set also contains dependent variable data, the dependent variable data is daily yield, and the target network model is a yield prediction model for predicting futures yields.

[0029] In a second aspect, this embodiment discloses a system for generating a target variable prediction model, which includes:

[0030] The data set acquisition module is used to acquire an original data set; wherein the original data set contains a plurality of independent variable data and a dependent variable, and a preset number of independent variable data are combined into an independent variable data matrix in the form of columns, and each row of data in the independent variable data matrix is ​​used as a sample data;

[0031] A random processing module, used for obtaining an independent variable random matrix according to the independent variable data matrix;

[0032] A weight updating module, used to input the independent variable data matrix and the independent variable random matrix into a preset adversarial network model, and use the preset adversarial network model to obtain a weight vector corresponding to the independent variable data matrix; wherein the weight vector is a weight vector composed of weights corresponding to each sample data;

[0033] The model training module is used to train the preset neural network model to be trained and adjust the model parameters using the original data set and the weight vector until a trained target variable prediction model is obtained.

[0034] In the third aspect, this embodiment also discloses a device for generating a target variable prediction model, which includes a processor and a storage medium communicatively connected to the processor, wherein the storage medium is suitable for storing multiple instructions; the processor is suitable for calling the instructions in the storage medium to execute the steps of the method for generating the target variable prediction model.

[0035] In a fourth aspect, this embodiment also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the method for generating the target variable prediction model.

[0036] Beneficial effects: The present invention proposes a method, system and device for generating a target variable prediction model, obtaining an original data set; obtaining an independent variable random matrix according to the independent variable data matrix corresponding to each independent variable data; inputting the independent variable data matrix and the independent variable random matrix into a preset adversarial network model, and using the preset adversarial network model to obtain a weight vector corresponding to each independent variable data matrix; using the original data set and the weight vector to train a preset neural network model to be trained and adjust model parameters until a trained target variable prediction model is obtained. The embodiment of the present invention generates a weight vector according to the original data set using a preset adversarial network model, and uses the weight vector to train the loss function of the preset network model to obtain a target variable prediction model, and the target variable prediction model can still maintain good prediction capabilities even when the data distribution of the training data set has changed significantly. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flowchart of the steps of the method for generating the target variable prediction model of the present invention;

[0038] Figure 2 is a schematic diagram of the structure of the differentiation module of the present invention;

[0039] Figure 3 It is a structural principle block diagram of the generation system of the target variable prediction model of the present invention;

[0040] Figure 4 It is a structural schematic diagram of a device for generating a target variable prediction model in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0043] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as herein.

[0044] The inventors found that in the prior art, when training the target variable prediction model, the model is established and trained based on the corresponding relationship between the known variables and the variables to be predicted. However, the correlation between the known variables and the variables to be predicted only represents the local characteristics between the known quantities and the variables. Therefore, the method of model training based on correlation is not suitable for the training of the target variable prediction model with unstable correlation between the known quantities and the variables. If the target variable prediction model with weak correlation between the known quantities and the variables is trained based on the correlation relationship, the network model obtained by the training may have the problem of low robustness, and it is impossible to obtain a more accurate and stable prediction result.

[0045] Since the relationship between known variables and predicted variables is essentially a causal relationship, and the causal relationship can better reflect the global and stable relationship between variables, and will not change due to changes in data in the data set, the invention uses the relationship between known variables and predicted variables to train the preset neural network model, thereby obtaining a neural network model with higher robustness.

[0046] The method for generating the target variable prediction model provided in this embodiment is not only applicable to the training process of the corresponding relationship in which the correlation between the known variables and the predicted variables is strong, but is more applicable to the training process of the corresponding relationship in which the correlation between the known variables and the predicted variables is weak, but the causal correlation between the known variables and the predicted variables is strong. For example, in the field of quantitative investment, when the switching of stock market styles causes a large change in the distribution of data, because the correlation between the known price and volume data and the yield to be predicted changes greatly, the machine learning model established based on the correlation between the two is likely to be unable to make an accurate prediction of the yield, but if the fund company relies on causal relationship training of the neural network model and makes investment decisions based on the trained target variable prediction model, better prediction results will be achieved.

[0047] The method for generating the target variable prediction model provided by this embodiment is applicable to many fields such as image recognition, speech recognition, natural language processing, and bioinformatics. Since the method for generating the target variable prediction model provided by this embodiment is based on causal relationships, even if the distribution of the known training sample data has changed significantly, a relatively accurate prediction result can be made. Therefore, it is particularly suitable for fields where the correspondence between the known training sample data and the result data to be predicted changes significantly, such as: prediction of changes in password characters identified by image recognition, or speech information recognition in the field of speech recognition, etc. Because the information to be predicted has a large degree of randomness, the target variable prediction model generation method provided by this embodiment is applied to perform corresponding model training, and a better training effect can be obtained.

[0048] The method of the present invention is explained in more detail below in conjunction with specific application examples of the embodiments of the present invention.

[0049] The present invention provides a method for generating a target variable prediction model, such as Figure 1 As shown, including:

[0050] Step S1, obtaining an original data set; wherein the original data set contains a plurality of independent variable data and a dependent variable, a preset number of independent variable data form an independent variable data matrix in the form of columns, and each row of the independent variable data matrix is ​​used as a sample data.

[0051] The original data set obtained in this step can obtain original data sets in different fields according to different models to be generated. The original data set contains multiple independent variable data and one dependent variable. Specifically, a preset number of independent variable data in a column form an independent variable data matrix, and the independent variable data matrix is ​​used as the input of the preset neural network model. Each row of the independent variable data matrix corresponds to a value of a dependent variable. The independent variable data matrix and the dependent variable are both in vector form.

[0052] For example, if the generated model is used to predict the rate of return, the original data set obtained in this step is the daily price and volume data of the main continuous contracts of various varieties in the Chinese commodity futures market (including daily opening price, closing price, highest price and lowest price, and total trading volume of the day). Use price and volume data to synthesize a batch of technical indicators, such as MACD, CCI, RSI, BOP, ATR as the independent variable data matrix X, and the dependent variable Y is the daily rate of return. The goal of the prediction is to predict the rate of return of futures varieties in the future.

[0053] Specifically, during the training process, it is necessary to divide the original data set into training sets and test sets. In the actual use stage, training sets and validation sets are also divided, and predictions are made based on real-time data.

[0054] Step S2: obtaining an independent variable random matrix according to the independent variable data matrix.

[0055] After obtaining the data of each independent variable in the original data set in the above steps, the order of each column in the independent variable data matrix is ​​randomly adjusted (for example: randomly moving the order of the dependent variable in each column) to generate an adjusted independent variable random matrix corresponding to each independent variable data.

[0056] Perform a shuffle operation on each column of the independent variable data matrix, that is, randomly adjust the order of samples for any independent variable. After performing this operation on all columns, a new independent variable data matrix S is obtained, which is also the independent variable random matrix.

[0057] Step S3: input the independent variable data matrix and the independent variable random matrix into a preset adversarial network model, and use the preset adversarial network model to obtain a weight vector corresponding to the independent variable data matrix; wherein the weight vector is a weight vector composed of weights corresponding to each sample data.

[0058] In this step, a preset adversarial network model is used to generate a weight vector based on an independent variable data matrix and an independent variable random matrix. Specifically, the independent variable data matrix and the independent variable random matrix are input into the preset adversarial network model, and the weight of the loss function of the preset adversarial network model is adjusted according to the output result of the preset adversarial network model, and the adjusted loss function is used to calculate the loss value, and the model parameters of the adversarial network model are adjusted according to the calculated loss value, and the step of inputting the independent variable data matrix and the independent variable random matrix into the preset adversarial network model is repeated until the independent variable data matrix obtained based on the preset adversarial network model meets the preset requirements, and the weight vector corresponding to the weight of the loss function of the final adjustment adversarial network model is used as the weight vector corresponding to the independent variable data matrix to be obtained in this step, and a corresponding weight vector is generated based on each independent variable data.

[0059] Specifically, the preset adversarial network model includes a generation module and a differentiation module;

[0060] The step of inputting the independent variable data matrix and the independent variable random matrix into a preset adversarial network model and obtaining a weight vector corresponding to each independent variable data matrix by using the preset adversarial network model comprises:

[0061] Vertically concatenate the independent variable data matrix and its corresponding independent variable random matrix to obtain a concatenated training data matrix;

[0062] The training data matrix is ​​used to perform multiple iterations of training on the distinguishing module and the generating module to obtain the weight vector.

[0063] In one embodiment, the loss function of the differentiation module is a weighted cross entropy loss function.

[0064] Furthermore, the step of performing multiple iterations of training on the distinguishing module and the generating module using the training data matrix to obtain the weight vector includes:

[0065] Step S31, inputting the training data matrix into the differentiation module to obtain the classification result output by the differentiation module.

[0066] The discrimination module is a discriminator, which is used to identify whether the data input therein is generated by the generator or the original data. Therefore, in this step, the training data matrix is ​​first input into the discrimination module, and the training data matrix input by the discrimination module is distinguished to identify whether it is the data in the target data set. If it is the data in the target data set, 1 is output and it is judged to be real data. Otherwise, 0 is input and it is judged to be data generated by the generation module. Therefore, the classification result is the judgment result of whether the input training data matrix is ​​the data in the target data set.

[0067] Step S32: Utilize the classification result and the preset initial weight vector to weight the loss function of the differentiation module, and calculate the initial total loss value of the preset adversarial network model.

[0068] In this step, the loss function of the differentiation module is first weighted according to the preset initial weight vector, and then the initial total loss value obtained by the first training of the preset adversarial network model is calculated according to the classification result obtained in the above step S31 and the weighted differentiation module, and the model parameters of the preset adversarial network model are adjusted according to the initial total loss value. The initial total loss value is the sum of the loss values ​​corresponding to the training samples corresponding to each row data in the training data matrix weighted by the preset initial weight vector, that is: first calculate the loss value corresponding to the training sample corresponding to each row data in the training data matrix, and then calculate the weighted value of each loss value and the preset initial weight vector respectively, to obtain the weighted loss value of the loss value corresponding to each training sample, and finally add the sum of the weighted loss values ​​corresponding to each training sample to obtain the initial total loss value.

[0069] Step S33: input the preset initial weight vector and the loss value corresponding to each sample data into the generation module to obtain the updated weight vector corresponding to each sample data output by the generation module.

[0070] The preset initial weight vector and the loss value corresponding to each sample data are input into the generation module of the adversarial network model to obtain the updated weight vector output by the generation module.

[0071] Step S34: input the training data matrix into the differentiation module to obtain an updated classification result output by the differentiation module.

[0072] The training data matrix is ​​input into the differentiation module again to obtain the updated classification result output by the differentiation module. This classification result is a new classification result obtained after adjusting the model parameters.

[0073] Step S35, calculating the loss value of the differentiation module according to the classification result and the updated weight vector, adjusting the updated weight vector according to the loss value, and calculating the updated total loss value of the preset adversarial network model.

[0074] The loss value of the differentiation module is calculated according to the updated classification result and the updated weight vector, and the updated weight vector is adjusted according to the loss value, and the updated total loss value of the preset adversarial network model is calculated. The calculation method of the updated total loss value here is the same as that of the above step S31, and the updated total loss value is the sum of the loss values ​​calculated after the updated loss values ​​corresponding to each sample data are weighted by the updated weight vector.

[0075] Step S36, repeatedly inputting the loss value of each sample data and the updated weight vector into the generation module, and obtaining the step of iteratively updating the weight vector output by the generation module corresponding to the independent variable data matrix; until the updated total loss value of the preset adversarial network model meets the preset requirements, and obtaining the weight vector corresponding to the independent variable data matrix.

[0076] Repeat the steps of inputting the loss value of each sample data and the updated weight vector into the generation module to obtain the step of iteratively updating the weight vector, and the step of calculating the total loss value of the model according to the updated weight vector, until the updated total loss value meets the preset requirements, and obtain the final updated weight vector which is the weight vector corresponding to the independent variable data matrix.

[0077] Step S4: using the original data set and the weight vector to train the preset neural network model to be trained and adjust the model parameters until a trained target variable prediction model is obtained.

[0078] After the weight vector is generated by the adversarial network model in the above step S3, the weight vector is used to weight the sample data in the objective function of the network model to be generated. Specifically, the loss function is weighted, so as to achieve the goal of training a preset model with higher robustness by learning the weights according to the distribution of the data itself.

[0079] Furthermore, the step of using the original data set and the weight vector to train the preset neural network model to be trained and adjusting model parameters until a trained target variable prediction model is obtained includes:

[0080] Inputting the original data set into a preset neural network model to obtain a target feature result output by the preset neural network model;

[0081] Weighting the loss function of the preset neural network model according to the weight vector, and calculating the loss value of the preset neural network model according to the target feature result and the weighted loss function;

[0082] The model parameters of the preset neural network model are adjusted according to the loss value of the preset neural network model, and the step of inputting the original data set into the preset neural network model is repeated until the target network model is obtained through training.

[0083] In one embodiment, taking the futures field as an example, the independent variable data contained in the original data set is price index data synthesized from futures daily frequency price and volume data, and the original data set also contains dependent variable data, the dependent variable data is daily yield, and the target network model is a yield prediction model for predicting futures yields.

[0084] The method of the present invention is further explained below by taking a specific application embodiment as an example.

[0085] 1. First, obtain the data set and clarify the learning task. That is, obtain the independent variable data matrix X and the dependent variable Y (in vector form, each row of the independent variable data matrix corresponds to a value of the dependent variable);

[0086] 2. Using the independent variable matrix, shuffle each column of the independent variable matrix, that is, randomly adjust the order of samples for any independent variable, and after doing this for all columns, obtain a new matrix S; wherein each row of the independent variable data matrix is ​​taken as a sample data; the new matrix S is the independent variable random matrix;

[0087] 3. Using the adversarial learning method, generate sample weight vectors so that the overall distribution of the independent variable matrix X after being weighted is close to the new matrix S;

[0088] About weight vector: By assigning weights to each sample, the overall distribution of the sample set can be changed. Assuming there are N samples, then assign weight w_i to the i-th sample, then all the weights together are a vector of length N.

[0089] 4. Using the weight vector obtained in 3, according to the learning task defined in 1, weight the samples in the objective function of the machine learning algorithm, so that the pseudo-correlation between different independent variables due to selective bias and confounding variables is reduced, so that a robust machine learning model can be trained.

[0090] In the framework of causal analysis, due to the existence of sample selection bias and confounding variables, there will be a certain deviation between the observed data distribution and the global data distribution. In the observed data, there will be unstable correlations between the independent variables, which is inconsistent with the global data distribution characteristics. When the prediction model established using these observed data is used, the prediction effect will be very unstable when facing a data set with a changing distribution. This embodiment hopes to solve the above problems in some scenarios.

[0091] In step 2, the shuffle operation is actually used to obtain an ideal independent variable matrix. Regardless of the distribution of the original independent variable matrix and the correlation between the independent variables, the shuffle operation is performed on each column (that is, each independent variable). The final matrix S is the ideal matrix. The marginal distribution of each independent variable in S has not changed, while the correlation between the independent variables has completely disappeared.

[0092] In step 3, the data matrix S obtained in the previous step is used to generate the weight vector w that adjusts the distribution of the X independent variable matrix. In this step, the adversarial learning method will be applied. In brief, adversarial learning is given a target data set and two models are constantly competing. One is the generation module, and the goal is to generate data similar to the target data set; the other is the distinction module, which distinguishes whether the input data comes from the target data set or is generated by the generation module. When the final model converges, the generation module can generate data that is very close to the target data set.

[0093] The method provided in this embodiment uses adversarial learning to generate a weight vector w, so that the independent variable data matrix X is adjusted by the weight vector w. w The distribution of is very close to the distribution of the ideal independent variable data matrix S, thus achieving the purpose of blocking the correlation between independent variables. If the correlation between independent variables is blocked, the influence of selective bias and confounding variables will be weakened, and the weighted matrix X w The trained model will be more stable.

[0094] The specific method of calculating the weight vector w using the adversarial learning method is as follows. Assume that there are n samples and p independent variables, that is, the matrix X is n*p dimensional. First, mark all real samples, that is, the independent variable matrix X is 0, and all samples from the ideal data matrix S are 1. The distinguishing module is d(x), and the weight vector is w=(w1,w2,…w n ), the initial values ​​will be set to equal weights. Figure 2 As shown, the discrimination module d(x) uses a two-layer neural network with 10 neurons each, and its structure includes: an input layer 100, a first hidden layer 200, a second hidden layer 300 and an output layer 400. The input of d(x) is a data matrix formed by vertically splicing S and X, and the label used for supervision is a vector composed of 0 and 1. Samples from S are marked as 1, and samples from X are marked as 0.

[0095] In one embodiment, the total loss function uses cross entropy and is weighted by w, and its functional formula is:

[0096]

[0097] The generation module here is an exponential gradient descent algorithm, which will adjust the weight w during the iteration process. The specific iterative algorithm is Here, t means the algorithm has been performed to the tth iteration.

[0098] The loss function of the distinguishing module is defined as l(t), where l can generally use the commonly used loss function of the binary classifier, such as cross entropy.

[0099] The total loss function during the optimization process is

[0100]

[0101] where s i and x i are the i-th samples respectively.

[0102] The optimization goal is

[0103] in The purpose of setting Δ is to normalize w. The specific iterative algorithm is shown in the following table. The distinguishing module d(x) and the generating module for updating w are iterated alternately until the algorithm converges and the required weight vector w is obtained.

[0104]

[0105] In step 4, during the model training process, the weight vector obtained in the previous step is used to weight the independent variable matrix. The specific operation is weighting on the loss function. For example, the original loss function of the regression model is The weighted loss function is

[0106] Compared with conventional machine learning methods, the method provided in this embodiment starts from the analytical framework of causal inference, rather than modeling based solely on correlation. From the experimental results, the model trained by the method provided in this embodiment maintains better stability than conventional machine learning methods when the distribution of the data set changes.

[0107] Compared with other algorithms for enhancing model robustness, the method for generating the target variable prediction model provided in this embodiment does not require any prior information about the distribution or statistical characteristics of the test data set, while other methods generally require such information or make corresponding settings in the assumptions. In practice, the specific distribution and statistical characteristics of the prediction data are generally unknown, so the method of this embodiment has a wider and more practical use value.

[0108] Compared with other machine learning methods that use propensity scores or reweighting, the method provided in this embodiment does not need to specify the specific form or calculation method of sample weights, but directly uses adversarial learning to learn sample weights based on the distribution of the data itself; while general methods that use propensity scores and reweighting need to specify the specific form of weights or scores. The disadvantage of specifying a specific form is that these specific forms correspond to applicable statistical features, such as linear relationships, etc. Once the actual data does not meet these features, the effectiveness of these methods will decrease. The method provided in this embodiment learns sample weights based on the data itself and has an adaptive characteristic. Therefore, the method provided in this embodiment has a wider applicability to data with different statistical characteristics.

[0109] This embodiment also provides a system for generating a target variable prediction model, such as Figure 3 As shown, including:

[0110] The data set acquisition module 310 is used to acquire the original data set; wherein the original data set contains multiple independent variable data and one dependent variable, and a preset number of independent variable data form an independent variable data matrix in the form of columns, and each row of the independent variable data matrix is ​​used as a sample data; its function is as described in step S1.

[0111] The random processing module 320 is used to obtain an independent variable random matrix according to the independent variable data matrix; its function is as described in step S2;

[0112] The weight updating module 330 is used to input the independent variable data matrix and the independent variable random matrix into a preset adversarial network model, and obtain a weight vector corresponding to the independent variable data matrix using the preset adversarial network model; wherein the weight vector is a weight vector composed of weights corresponding to each sample data, and its function is as described in step S3;

[0113] The model training module 340 is used to train the preset neural network model to be trained and adjust the model parameters using the original data set and the weight vector until a trained target variable prediction model is obtained. Its function is as described in step S4.

[0114] Based on the above-mentioned method for generating a target variable prediction model, this embodiment also provides a device for generating a target variable prediction model, which includes a processor and a storage medium communicatively connected to the processor, wherein the storage medium is suitable for storing multiple instructions; the processor is suitable for calling the instructions in the storage medium to execute the steps of implementing the method for generating a target variable prediction model.

[0115] Specifically, Figure 4 As shown, the electronic device includes at least one processor (processor) 20 and a memory (memory) 22, and may also include a display screen 21, a communication interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0116] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0117] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 30 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0118] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0119] In addition, this embodiment also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the method for generating the target variable prediction model.

[0120] The specific process of loading and executing the plurality of instructions in the storage medium and the processor in the terminal has been described in detail in the method above, and will not be described here one by one.

[0121] The present invention proposes a method, system and device for generating a target variable prediction model, obtaining an original data set; obtaining an independent variable random matrix according to the independent variable data matrix corresponding to each independent variable data; inputting the independent variable data matrix and the independent variable random matrix into a preset adversarial network model, using the preset adversarial network model, obtaining a weight vector corresponding to each independent variable data matrix; using the original data set and the weight vector to train a preset neural network model to be trained and adjusting model parameters, until a trained target variable prediction model is obtained. The embodiment of the present invention uses a preset adversarial network model to generate a weight vector according to the original data set, and uses the weight vector to train the loss function of the preset network model to obtain a target variable prediction model, and the target variable prediction model can still maintain good prediction capabilities even when the data distribution of the training data set has changed significantly.

[0122] It is understandable that those skilled in the art can make equivalent substitutions or changes based on the technical solution and inventive concept of the present invention, and all these changes or substitutions should fall within the protection scope of the claims attached to the present invention.

Claims

1. A method for generating a target variable prediction model, characterized in that: include: Acquire an original data set; wherein the original data set contains a plurality of independent variable data and a dependent variable, a preset number of independent variable data are formed into an independent variable data matrix in the form of columns, and each row of data in the independent variable data matrix is ​​used as a sample data; Obtaining an independent variable random matrix according to the independent variable data matrix; The step of obtaining an independent variable random matrix according to the independent variable data matrix comprises: Randomly adjust the order of each column of the independent variable data matrix to generate a random matrix of independent variables; Input the independent variable data matrix and the independent variable random matrix into a preset adversarial network model, and use the preset adversarial network model to obtain a weight vector corresponding to the independent variable data matrix; wherein the weight vector is a weight vector composed of weights corresponding to each sample data; Using the original data set and the weight vector to train the preset neural network model to be trained and adjust the model parameters until a trained target variable prediction model is obtained; The preset adversarial network model includes a generation module and a differentiation module; The step of inputting the independent variable data matrix and the independent variable random matrix into a preset adversarial network model and obtaining a weight vector corresponding to the independent variable data matrix using the preset adversarial network model comprises: Vertically concatenate the independent variable data matrix and its corresponding independent variable random matrix to obtain a concatenated training data matrix; Performing multiple iterations of training on the distinguishing module and the generating module using the training data matrix to obtain the weight vector; The calculation method of the weight vector includes: assuming that there are n samples and p independent variables, the training data matrix X is n*p dimensions, the independent variable matrix X is marked as 0, and all samples from the ideal data matrix S are 1; the input of the distinguishing module is a data matrix formed by vertically splicing S and X, the total loss function uses cross entropy and is weighted by the weight vector, and the distinguishing module and the generating module are alternately iterated until the algorithm converges to obtain the required weight vector; The generation method is applied to image recognition or speech recognition, and the target variable prediction model is used for prediction of password character changes obtained through image recognition or speech information recognition in speech recognition.

2. The method for generating a target variable prediction model according to claim 1, characterized in that: The step of performing multiple iterations of training on the distinguishing module and the generating module using the training data matrix to obtain the weight vector comprises: Inputting the training data matrix into the differentiation module to obtain the classification result output by the differentiation module; Using the classification result and the preset initial weight vector to weight the loss function of the differentiation module, an initial total loss value of the preset adversarial network model is calculated, and model parameters of the preset adversarial network model are adjusted according to the initial total loss value; Input the preset initial weight vector and the loss value corresponding to each sample data into the generation module to obtain the updated weight vector corresponding to the sample data output by the generation module; Inputting the training data matrix into the differentiation module to obtain an updated classification result output by the differentiation module; Calculating the loss value of the differentiation module according to the updated classification result and the updated weight vector, adjusting the updated weight vector according to the loss value, and calculating the updated total loss value of the preset adversarial network model; Repeat the steps of inputting the loss value of each sample data and the updated weight vector into the generation module to obtain the iterative update weight vector output by the generation module corresponding to the independent variable data matrix; until the updated total loss value of the preset adversarial network model meets the preset requirements, and obtain the weight vector corresponding to the independent variable data matrix.

3. The method for generating a target variable prediction model according to claim 2, characterized in that: The loss function of the differentiation module is a weighted cross entropy loss function.

4. The method for generating a target variable prediction model according to claim 2, characterized in that: The step of training the preset neural network model to be trained using the original data set and the weight vector and adjusting the model parameters until a trained target variable prediction model is obtained comprises: Inputting the original data set into a preset neural network model to obtain a target feature result output by the preset neural network model; Weighting the loss function of the preset neural network model according to the weight vector, and calculating the loss value of the preset neural network model according to the target feature result and the weighted loss function; The model parameters of the preset neural network model are adjusted according to the loss value of the preset neural network model, and the step of inputting the original data set into the preset neural network model is repeated until the target variable prediction model is obtained through training.

5. A system for generating a target variable prediction model, characterized in that: include: The data set acquisition module is used to acquire an original data set; wherein the original data set contains a plurality of independent variable data and a dependent variable, and a preset number of independent variable data are combined into an independent variable data matrix in the form of columns, and each row of data in the independent variable data matrix is ​​used as a sample data; A random processing module, used for obtaining an independent variable random matrix according to the independent variable data matrix; wherein the order of each column of the independent variable data matrix is ​​randomly adjusted to generate an independent variable random matrix; A weight updating module, used to input the independent variable data matrix and the independent variable random matrix into a preset adversarial network model, and use the preset adversarial network model to obtain a weight vector corresponding to the independent variable data matrix; wherein the weight vector is a weight vector composed of weights corresponding to each sample data; A model training module, used to train the preset neural network model to be trained and adjust the model parameters using the original data set and the weight vector until a trained target variable prediction model is obtained; The preset adversarial network model includes a generation module and a differentiation module; The step of inputting the independent variable data matrix and the independent variable random matrix into a preset adversarial network model and obtaining a weight vector corresponding to the independent variable data matrix using the preset adversarial network model comprises: Vertically concatenate the independent variable data matrix and its corresponding independent variable random matrix to obtain a concatenated training data matrix; Performing multiple iterations of training on the distinguishing module and the generating module using the training data matrix to obtain the weight vector; The calculation method of the weight vector includes: assuming that there are n samples and p independent variables, the training data matrix X is n*p dimensions, the independent variable matrix X is marked as 0, and all samples from the ideal data matrix S are 1; the input of the distinguishing module is a data matrix formed by vertically splicing S and X, the total loss function uses cross entropy and is weighted by the weight vector, and the distinguishing module and the generating module are alternately iterated until the algorithm converges to obtain the required weight vector; The generation method is applied to image recognition or speech recognition, and the target variable prediction model is used for prediction of password character changes obtained through image recognition or speech information recognition in speech recognition.

6. A device for generating a target variable prediction model, characterized in that: It includes a processor and a storage medium communicatively connected to the processor, wherein the storage medium is suitable for storing multiple instructions; the processor is suitable for calling the instructions in the storage medium to execute the steps of the method for generating a target variable prediction model as described in any one of claims 1 to 4 above.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the method for generating a target variable prediction model as described in any one of claims 1 to 4.