Dual learning method for heterogeneous data sample selection deviation correction

Abstract

The invention discloses a dual learning method, device and equipment for heterogeneous data sample selection deviation correction. The method comprises the following steps: acquiring description text data, item feature data and user feature data; according to the text data, the item feature data and the user feature data, determining a prediction matrix and a first user item real label matrix; inputting a first user item real label matrix into an estimation model of a pre-constructed double-robust model to obtain a first pseudo-label matrix, and filling a corresponding part of the first pseudo-label matrix with elements in the prediction matrix to obtain a second pseudo-label matrix, and inputting the second pseudo-label matrix and the first user project real label matrix into the prediction model of the double-robust model to obtain the feedback data of the project, the prediction matrix containing heterogeneous information is obtained through heterogeneous feature fusion, and the prediction matrix is introduced into the double-robust model, so that the rationality of pseudo-label data generation is improved.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a dual learning method, device and equipment for correcting selection bias of heterogeneous data samples. Background Art

[0002] Today, data-driven decisions have a profound impact on all aspects of society, from recommendation systems to healthcare services and even criminal sentencing. The credibility of an AI system's decisions is closely related to the authenticity and reliability of the data, because the phenomena reflected by the system represent the state of the data to a certain extent. Data preprocessing is the first step in decision analysis. Although many systems now have high decision accuracy, data often has missing and uneven distribution, which skews the system's decision results. Specifically, the non-random missing of data causes the collected data set to fail to fully represent the target group being investigated, that is, the data has sample selection bias. Correcting sample selection bias plays a vital role in ensuring the accuracy and effectiveness of data-driven systems.

[0003] Collecting large, complex data sets has become a common phenomenon in all fields, and the trend is gradually shifting from observation-based, intuitive decisions to data-based decisions. Data-based decisions have produced many excellent research results, but there are also cases of failure. This may be because the researchers' default assumptions do not take into account the sample selection bias of the data, or the large data sets mask many problems. If enough data can be extracted from "unpopular areas", the data will be highly representative, and the data collection is usually independent of the research process. Therefore, it is quite difficult to ensure that the data is unbiased. Sample selection bias usually stems from the process of selecting the data. Researchers have limited or no control over sample selection. They only select the samples to be observed, thereby truncating the entire sample, resulting in some samples having a higher proportion than others, resulting in sample selection bias. In addition, bias may also occur when some data cannot be collected due to technical or audience willingness limitations.

[0004] The existence of sample selection bias can cause many problems, such as inaccurate recommendations from recommendation systems and skewed statistical data from questionnaires. Alleviating the adverse effects of sample selection bias can help data-oriented research obtain more accurate results. Nowadays, research on bias correction mainly addresses two problems, namely data imbalance (missing subsets) and missing data points. Data imbalance is mainly reflected in the lack of data of a certain feature. The current solution mostly uses inverse propensity weighting to improve the representativeness of small-proportion subsets. For example, dynamic weighting methods based on visual cohort comparisons; Heckman models that integrate non-independent selection mechanisms into statistical models to correct biases through treatment effect modules and sample selection modules; non-parametric empirical Bayes methods based on data stratification, etc. For the problem of non-random missing of some data points, data interpolation is one of the most commonly used methods. The doubly robust (DR) estimator combines the inverse propensity score (IPS) estimator and the error-in-difference compensation (EIB) estimator in a doubly robust way, which not only alleviates the data imbalance but also realizes data interpolation, and has good performance.

[0005] Although the research on sample selection bias correction has made significant progress, the survey found that the existing methods are still difficult to solve the following problems: First, with the in-depth study of heterogeneous data and its widespread existence in various fields, how to make full use of the information of heterogeneous data to assist in correcting sample selection bias has become a new research direction; Second, the data inserted in the existing data interpolation method is random, and the bias caused by non-random missing data is not fully considered. Third, some correction models using the inverse tendency weighting method often ignore the problem of high variance. Summary of the invention

[0006] The main purpose of this application is to provide a dual learning method, device and equipment for correcting the selection bias of heterogeneous data samples, aiming to obtain accurate project feedback data using the corrected sample data.

[0007] To achieve the above-mentioned objectives, the present application provides a dual learning method for correcting selection bias of heterogeneous data samples, including: obtaining descriptive text data, project feature data and user feature data; determining a prediction matrix and a first user-project true label matrix based on the text data, the project feature data and the user feature data; inputting the first user-project true label matrix into the estimation model of a pre-built dual robust model to obtain a first pseudo label matrix, and using the elements in the prediction matrix to fill the corresponding parts of the first pseudo label matrix to obtain a second pseudo label matrix, and inputting the second pseudo label matrix and the first user-project true label matrix into the prediction model of the dual robust model to obtain feedback data of the project, wherein the dual robust model is trained based on a preset first objective function and a preset second function.

[0008] Optionally, determining a prediction matrix based on the text data, the project feature data and the user feature data includes: obtaining representative context feature data of the description text based on the description text data, and obtaining project potential feature data and user potential feature data based on the project feature data and the user feature data, respectively; combining the representative context feature data and the project potential feature data to obtain merged project feature data; and using a multi-layer perception module to capture the nonlinear relationship between the user potential feature data and the merged project feature data to obtain the prediction matrix.

[0009] Optionally, the representative context feature data of the description text is obtained based on the description text data, including: using the first embedding layer to perform word embedding representation processing on the description text data to obtain a word vector matrix; using the convolution layer to slice the word vector matrix to obtain corresponding context features, wherein the convolution window used in each slice is obtained by weighting based on each shared weight; using the pooling layer to select the largest context feature data from each of the context features to obtain the representative context feature data.

[0010] Optionally, the combining of the representative context feature data and the project latent feature data to obtain merged project feature data includes: processing the project feature data using the second embedding layer to obtain project latent feature data; and processing the representative context feature data and the project feature data based on a pre-constructed fusion layer to obtain the merged project feature data.

[0011] Optionally, the use of a multi-layer perception module to capture the nonlinear relationship between the user potential feature data and the merged project feature data to obtain a prediction matrix includes: using the third embedding layer to process the user feature data to obtain the user potential feature data; using stacked multiple MLP layers to sequentially process the input user potential feature data and the merged project feature data to obtain the prediction matrix.

[0012] Optionally, the training process of the dual robust model includes: randomly initializing the parameters of the prediction model and the estimation model; obtaining a set of user-item pairs, and obtaining user and item interaction data based on the set of user-item pairs, wherein the set of user-item pairs is determined based on user feature data and item feature data; obtaining a second user-item true label matrix and a second prediction matrix; using the estimation model to process the second user-item true label matrix to obtain a third pseudo label matrix, and using the elements in the second prediction matrix to fill the corresponding parts of the third pseudo label matrix to obtain a fourth pseudo label matrix; obtaining an estimation error based on the second prediction matrix and the fourth pseudo label matrix; replacing the square loss term in the loss function of the estimation model based on the estimation error to obtain a new loss function of the estimation model; inputting the fourth pseudo label matrix and the second in the prediction model. A user-item true label matrix is ​​obtained to obtain a third prediction matrix, wherein the prediction model and the estimation model have the same structure; the cross entropy between the third prediction matrix and the second user-item true label matrix is ​​calculated to obtain a prediction error; a blank event correction error is obtained based on the estimation error and the prediction error; a loss function of the prediction model is obtained based on the blank event correction error, the prediction error and the item interaction data; the estimation model and the prediction model are iteratively trained based on the new loss function of the estimation model and the loss function of the prediction model, and the parameters of the prediction model are copied to the estimation model in each round of iterative training, and in the next round of iteration, the second user-item true label matrix is ​​continuously input into the estimation model, and the iterative training is performed until both the estimation model and the prediction model converge or a preset number of iteration termination times is reached.

[0013] Optionally, the new loss function of the estimation model is:

[0014]

[0015] Among them, L e (θ) represents the new loss function of the estimation model, represents the fourth pseudo label matrix, represents the second prediction matrix, O represents the user and item interaction data, O = {(u,i)|(u,i)∈D,o u,i =1},o u,i =1 means that there is contact between user u and item i, λ represents the parameter that controls the strength of L2 regularization to avoid overfitting, and θ represents all parameters of the estimation model. represents the prediction tendency, CE() represents the cross entropy, and F represents the regularization operation.

[0016] Optionally, the loss function of the prediction model is:

[0017]

[0018] in, represents the loss function of the prediction model, represents all parameters of the prediction model, o u,i =1 means that there is contact between user u and item i, o u,i = 0 means there is no contact between user u and item i, μ represents the parameter for controlling the L2 regularization strength to avoid overfitting, D represents the set of user-item pairs, Indicates the prediction tendency, e u,i represents the estimation error, represents the prediction error, and F represents the regularization operation.

[0019] To achieve the above-mentioned purpose, the present application also provides a dual learning device for correcting sample selection bias for heterogeneous data, including: an acquisition module, used to acquire descriptive text data, project feature data and user feature data; an input data determination module, used to determine a prediction matrix and a first user-project true label matrix based on the text data, the project feature data and the user feature data; a feedback data determination module, used to input the first user-project true label matrix into the estimation model of a pre-built dual robust model to obtain a first pseudo label matrix, and use the elements in the prediction matrix to fill the corresponding parts of the first pseudo label matrix to obtain a second pseudo label matrix, and input the second pseudo label matrix and the first user-project true label matrix into the prediction model of the dual robust model to obtain feedback data of the project, wherein the dual robust model is trained based on a preset first objective function and a preset second function.

[0020] To achieve the above-mentioned purpose, the present application also provides an electronic device, which includes: at least one processor, a memory and an input-output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the dual learning method for correcting sample selection bias for heterogeneous data provided in any of the aforementioned embodiments.

[0021] The embodiment of the present application proposes a dual learning method, device and equipment for correcting sample selection bias of heterogeneous data, which obtains descriptive text data, project feature data and user feature data; determines a prediction matrix and a first user-project true label matrix according to the text data, the project feature data and the user feature data; inputs the first user-project true label matrix into the estimation model of the pre-built dual robust model to obtain a first pseudo label matrix, and fills the corresponding part of the first pseudo label matrix with the elements in the prediction matrix to obtain a second pseudo label matrix, and inputs the second pseudo label matrix and the first user-project true label matrix into the prediction model of the dual robust model to obtain feedback data of the project, wherein the dual robust model is trained based on a preset first objective function and a preset second function. The present invention obtains a prediction matrix containing heterogeneous information through heterogeneous feature fusion, and introduces it into the dual robust model, thereby improving the rationality of pseudo label data generation and achieving the effect of correcting sample selection bias. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart diagram of an embodiment of a dual learning method for correcting sample selection bias for heterogeneous data provided by the present application;

[0023] Figure 2 A network structure diagram used in the process of obtaining a prediction matrix provided in an embodiment of the dual learning method for correcting sample selection bias for heterogeneous data of the present application;

[0024] Figure 3 A schematic diagram of an embodiment of a dual learning method for correcting sample selection bias for heterogeneous data provided by the present application;

[0025] Figure 4 A schematic diagram of a matrix filling process provided in an embodiment of a dual learning method for correcting sample selection bias for heterogeneous data of the present application;

[0026] Figure 5 This is a schematic diagram of the structure of a dual learning device for correcting sample selection bias for heterogeneous data in this application.

[0027] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0028] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] Regarding the dataset used in this application, the dataset used in this application is the ML-1M dataset, which is a classic recommendation system research dataset, including about 1 million user rating records for movies, ranging from 1-5, and 6040 users, each of whom rated at least 20 movies. The specific content is divided into the following three categories, namely user information: user ID, gender MF, age group (such as 18-24, 25-34, etc.) and occupation (21 categories, such as lawyers, scientists, farmers, etc.);

[0030] Rating information: user D, project ID, rating value and timestamp;

[0031] Movie information; item ID, item name, item description text, and item category (such as action, comedy, romance, etc.).

[0032] The present application provides a dual learning method for data interpolation that rationally utilizes heterogeneous data information to solve the problems existing in the prior art. This method can be widely used in different fields, for example, it can be applied to medical health, recommendation systems, urban construction and other fields. This method alleviates the problem of data imbalance by interpolating data and inverse tendency weighting, and can calibrate sample selection bias for a variety of application fields to solve the problem of biased sample data selection in the prior art. Compared with existing research work, this application has the following advantages: 1) Constructing a sample selection bias identification index to quantify the degree of data imbalance; 2) Using prediction data containing heterogeneous information to replace the pseudo-label data generated by the estimation model to achieve the purpose of improving prediction accuracy and shortening training time; 3) The bias correction process uses a dynamic parameter correction method, that is, a dual learning method, which provides a dynamically changing estimation error for the prediction model through pseudo-label technology.

[0033] The technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0034] Reference Figure 1 The first embodiment of the present application provides a dual learning method for correcting sample selection bias for heterogeneous data. The execution subject of the method may be a processor in a host or a server. The method may include:

[0035] S10, obtaining description text data, project feature data and user feature data;

[0036] S20, determining a prediction matrix and a first user-item true label matrix according to the text data, the item feature data and the user feature data;

[0037] In this step, the purpose of this application is to learn the context features of the project and the interaction features between the user and the project. The network structure diagram used to learn the context features of the project and the interaction features between the user and the project is shown in the attached figure. Figure 2 Attached Figure 2 Starting from the bottom, enter the text data including the project description C j , project features User characteristics Where U={u1,u2,...,u m} represents m users, I={i1,i2,...,i n} represents n items;

[0038] In an embodiment of the present application, step S20 may include the following execution process:

[0039] S201, obtaining representative context feature data of the description text based on the description text data, and obtaining project potential feature data and user potential feature data based on the project feature data and the user feature data respectively;

[0040] Specifically, the processor may obtain representative context feature data of the description text based on the description text data, including the following execution process:

[0041] S2011, use the first embedding layer to perform word embedding representation processing on the description text data to obtain a word vector matrix:

[0042] refer to Figure 2 The first branch on the left side of the lower half is used to learn text features. In the process of learning text features, the processor uses the embedding layer processed by the first convolutional neural network to perform word embedding representation processing on the description text data and convert the text information into a word vector matrix. For example, for any comment text C j By connecting the word vectors in the text, the processor can generate a word vector matrix and preserve the word order, which allows the convolutional layer to extract more accurate context features. The word vector matrix can be expressed as: T = [..., l i-1 ,l i ,l i+1 ,...],l i Represents C j The word vector at the i-th position in .

[0043] S2012, using a convolutional layer to slice the word vector matrix to obtain corresponding context features, wherein the convolution window used in each slice is obtained by weighting based on each shared weight;

[0044] This step can be used to extract contextual features of the comment text. Specifically, the i-th contextual feature is the sth shared weight Extracted by convolution window ω, The extracted formula is:

[0045]

[0046] Among them, T (:,(i+ω-1)) is the slice of the matrix T and the convolution window ω, f(·) represents the nonlinear activation function, which can be the ReLU function. It is worth noting that a shared weight can only capture one context feature, and the number of context features is set to n c , for example, a review text document is represented as n c A context feature vector of variable length.

[0047] S2013. Use the pooling layer to select the largest context feature data from each context feature to obtain representative context feature data.

[0048] The pooling layer selects representative context features c from the output of the previous layer. j , wherein the selection method can be to extract the maximum context feature to represent the representative upstream feature data.

[0049] S202, combining representative context feature data and project potential feature data to obtain merged project feature data;

[0050] In an embodiment of the present application, step S202 may also include the following execution process:

[0051] S2021, using the second embedding layer to process the project feature data to obtain project potential feature data;

[0052] S2022. Process the representative context feature data and the project feature data based on the pre-built fusion layer to obtain merged project feature data.

[0053] Specifically, the processor can integrate two heterogeneous information features (context feature data and item feature data) through a fusion layer. The merged item features are connected to a single dense neuron layer, which calculates the weighted sum of all dimensions and obtains the merged item feature data q through tanh nonlinear activation. i =f(W i f (q i ·c i )), where W i f is the weight matrix, and f(·) represents the activation function of the fusion layer.

[0054] S203, using a multi-layer perception module to capture the nonlinear relationship between the user's potential feature data and the merged project feature data to obtain a prediction matrix.

[0055] In an embodiment of the present application, step S203 may also include the following execution process:

[0056] S2031, using the third embedding layer to process the user feature data to obtain user potential feature data;

[0057] refer to Figure 2 The second and third branches on the left side of the lower half use neural networks to map user feature data and project feature data into low-dimensional dense vectors. This process can be expressed as:

[0058]

[0059] in, and are the rating latent factor matrix of the user and the rating latent factor matrix of the item, respectively, u and q i Represent the latent features of the user and the latent features of the project respectively. The user's rating latent factor matrix and the project's rating latent factor matrix are obtained through the embedding layers of the second convolutional neural network and the third convolutional neural network respectively.

[0060] S2032. Use stacked multiple MLP layers to sequentially process the input user potential feature data and merged item feature data to obtain a prediction matrix.

[0061] In this step, the processor can obtain a prediction matrix containing heterogeneous information by learning the nonlinear relationship between features. Specifically, the processor can use the stacked MLP to learn the nonlinear interaction between features, thereby obtaining the interaction process between users and items. The stacked MLP can be used to simulate the user potential feature data p u and the item latent characteristic q i To realize the interaction between multiple users and items, the final prediction function of the stacked MLP is as follows:

[0062]

[0063] in, is the prediction matrix, corresponding to the scoring application scenario represents the predicted score, z l is the output of the l-1th layer, is the weight matrix of the prediction layer, and l is the number of layers of the MLP.

[0064] S30, constructing an identification index for sample selection bias. In this step, the processor reflects the overall instability of the data by calculating the entropy value of the propensity score corresponding to different features, where the propensity score represents the degree of inclination of the feature data, and the entropy value can reflect the overall instability of the feature data. The calculation formula is as follows:

[0065]

[0066] in is the propensity score of the jth feature;

[0067] S40. Input the first user-project true label matrix into the estimation model of the pre-built dual robust model to obtain the first pseudo label matrix, and use the elements in the prediction matrix to fill the corresponding parts of the first pseudo label matrix to obtain the second pseudo label matrix, and input the second pseudo label matrix and the first user-project true label matrix into the prediction model of the dual robust model to obtain feedback data of the project, wherein the dual robust model is trained based on a preset first objective function and a preset second function.

[0068] Specifically, the training process of the dual robust model includes:

[0069] Randomly initialize the parameters of the prediction model and the estimation model;

[0070] Acquire a user-item pair set, and obtain user-item interaction data based on the user-item pair set, wherein the user-item pair set is determined based on user feature data and item feature data;

[0071] Obtain the second user-item true label matrix, and obtain the second prediction matrix according to the prediction matrix;

[0072] Processing the second user-item true label matrix using the estimation model to obtain a third pseudo label matrix, and filling corresponding parts of the third pseudo label matrix using elements in the second prediction matrix to obtain a fourth pseudo label matrix;

[0073] Obtaining an estimated error based on the second prediction matrix and the fourth pseudo label matrix;

[0074] Replacing the squared loss term in the loss function of the estimation model based on the estimation error to obtain a new loss function of the estimation model;

[0075] Inputting the fourth pseudo label matrix and the second user-item real label matrix into the prediction model to obtain a third prediction matrix, wherein the prediction model and the estimation model have the same structure;

[0076] Calculate the cross entropy between the third prediction matrix and the second user-item true label matrix to obtain the prediction error;

[0077] The blank event correction error is obtained based on the estimation error and the prediction error;

[0078] Based on the blank event correction error, prediction error and item interaction data, the loss function of the prediction model is obtained;

[0079] The estimation model and the prediction model are iteratively trained based on the new loss function of the estimation model and the loss function of the prediction model respectively, and the parameters of the prediction model are copied to the estimation model in each round of iterative training. In the next round of iteration, the second user project true label matrix continues to be input into the estimation model, and the iterative training is carried out until both the estimation model and the prediction model converge or the preset number of iteration termination times is reached.

[0080] The above steps are described in detail below. In the above steps, this application proposes a dual learning and correction method based on a dual robust (DR) model. Figure 3 , the dual learning method consists of two models: the prediction model

[0081] and estimation model Among them, x u,i Represents user and project features. The present invention combines the two models by parameter transfer and iteration. The workflow of the dual learning method is as follows: Figure 1 As shown. The input includes the set of all user-item pairs D = U × I, the second user-item true label matrix And the obtained second prediction matrix containing heterogeneous information

[0082] It should be noted that, in order to distinguish from the actual data process, the data used in the training process of this application are project interaction data, the second user-project true label matrix, and the second prediction matrix obtained based on the second user-project true label matrix.

[0083] Before training the dual robust (DR) model, the processor needs to randomly train the initialization parameters to estimate all the parameters of the model θ and predict all the parameters of the model After starting training, the processor first copies the parameters of the prediction model to update the parameters of the estimation model. The prediction model and the estimation model have the same structure and different parameters. The dual robust (DR) model in the present application has the function of dual learning. Since the dual learning method only enhances the anti-tendency weight of the estimation learning, it is still affected by high variance. Therefore, in each round of dynamic learning of the estimation model, the estimation model parameters are updated using the prediction model parameters, so that the estimation model can be regularly corrected while retaining the information brought by the enhanced anti-tendency weights. The processor can then use the second user-item true label matrix R to train the estimation model to obtain a third pseudo-label matrix and use the elements of the second prediction matrix to replace the corresponding parts of the third pseudo-label matrix, such as Figure 3 As shown in the table above, this method not only realizes data interpolation, but also provides dynamically changing estimation errors for the prediction model.

[0084] Continue to refer Figure 3 Specifically, the processor first uses the estimation model to process the second user item true label to obtain a third pseudo label matrix refer to Figure 4 , Figure 4 This is the process of using the second prediction matrix to fill the third pseudo label matrix in this application. The corresponding part uses the second prediction matrix The fourth pseudo label matrix is ​​replaced by the elements of to improve the accuracy and scientificity of pseudo label data generation. Then, the estimated error is obtained based on the second prediction matrix and the fourth pseudo label matrix. The expression of the estimated error is: at this time, is the second prediction matrix, is the fourth pseudo label matrix.

[0085] That is, by calculating the second prediction matrix of the prediction model and the fourth pseudo-label matrix The cross entropy between them is used to estimate the estimation error

[0086] Furthermore, the squared loss term in the loss function of the estimation model is replaced based on the estimation error expression to obtain a new loss function of the estimation model. The new loss function of the estimation model is:

[0087]

[0088] Among them, L e (θ) represents the new loss function of the estimation model, represents the fourth pseudo label matrix, represents the second prediction matrix, O represents the user and item interaction data, O = {(u,i)|(u,i)∈D,ou,i =1},o u,i =1 means that there is contact between user u and item i, λ represents the parameter that controls the strength of L2 regularization to avoid overfitting, and θ represents all parameters of the estimation model. represents the prediction tendency, CE() represents the cross entropy, and F represents the regularization operation. So far, the processor has built the loss function of the estimation model.

[0089] Continue to refer Figure 3 Next, the processor can construct a loss function of the prediction model. Specifically, the processor first inputs the fourth pseudo label matrix and the second user-item real label matrix into the prediction model to obtain a third prediction matrix. Then, the processor calculates the cross entropy between the third prediction matrix and the second user-item real label matrix to obtain a prediction error.

[0090] That is, using the estimated error Calculate prediction errors for all events and correct the error bias for blank events in, represents the third prediction matrix, r u,o represents the second user-item true label matrix, e u,i Indicates prediction error; tendency With error deviation δ u,i Inversely proportional to eliminate the influence of sample selection bias. Further, the processor obtains the blank event correction error based on the estimation error and the prediction error, and obtains the loss function of the prediction model based on the blank event correction error, the prediction error and the item interaction data; the loss function of the prediction model is:

[0091]

[0092] in, represents the loss function of the prediction model, represents all parameters of the prediction model, o u,i =1 means that there is contact between user u and item i, o u,i = 0 means there is no contact between user u and item i, μ represents the parameter for controlling the L2 regularization strength to avoid overfitting, D represents the set of user-item pairs, Indicates the prediction tendency, e u,i represents the prediction error, represents the estimation error, and F represents the regularization operation. At this point, the processor has constructed the loss function of the estimation model.

[0093] Finally, the processor iteratively trains the estimation model and the prediction model based on the new loss function of the estimation model and the loss function of the prediction model, and copies the parameters of the prediction model to the estimation model in each round of iterative training, and continues to input the second user project real label matrix into the estimation model in the next round of iteration, and iterates the training until both the estimation model and the prediction model converge or reach a preset number of iteration terminations. It is worth noting that in each round of iteration, the second prediction matrix remains unchanged until the loss function of the prediction model or the estimation model converges to the optimization stagnation point.

[0094] By comparing the bias identification entropy of the final pseudo-label matrix (the prediction data generated by the estimation model and the prediction data containing heterogeneous information) with the bias identification entropy of the initial data, it is found that the overall imbalance of the prediction score after correction is reduced by an average of 49.17%. The example is finally applied in the field of recommendation system score prediction. The comparison of the recommendation index Recall@K before and after correction is an average increase of 16.68%, and NDCG@K is an average increase of 12.35%.

[0095] Reference Figure 5 On the basis of the above embodiments, the present application may also include a dual learning device for correcting sample selection bias for heterogeneous data to solve the same technical problem. The dual learning device 100 may include an acquisition module 1001, an input data determination module 1002 and a feedback data determination module 1003, wherein the acquisition module 1001 is used to acquire descriptive text data, project feature data and user feature data; the input data determination module 1002 is used to determine the prediction matrix and the first user-project true label matrix according to the text data, project feature data and user feature data; the feedback data determination module 1003 is used to input the first user-project true label matrix into the estimation model of the pre-built dual robust model to obtain the first pseudo label matrix, and use the elements in the prediction matrix to fill the corresponding part of the first pseudo label matrix to obtain the second pseudo label matrix, and input the second pseudo label matrix and the first user-project true label matrix into the prediction model of the dual robust model to obtain the feedback data of the project, wherein the dual robust model is trained based on the preset first objective function and the preset second function.

[0096] Based on the above embodiments, the present application may also include an electronic device, characterized in that the electronic device includes: at least one processor, a memory and an input-output unit; wherein the memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute any one of the method embodiments of the dual learning method for correcting sample selection bias for heterogeneous data.

[0097] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A dual learning method for correcting selection bias of heterogeneous data samples, characterized in that: include: Obtaining description text data, item feature data, and user feature data; Determine a prediction matrix and a first user-item true label matrix according to the text data, the item feature data and the user feature data; The first user-project true label matrix is ​​input into the estimation model of the pre-built dual robust model to obtain a first pseudo label matrix, the corresponding parts of the first pseudo label matrix are filled with elements in the prediction matrix to obtain a second pseudo label matrix, and the second pseudo label matrix and the first user-project true label matrix are input into the prediction model of the dual robust model to obtain feedback data of the project, wherein the dual robust model is trained based on a preset first objective function and a preset second function.

2. The dual learning method for correcting sample selection bias for heterogeneous data as claimed in claim 1, characterized in that: The step of determining a prediction matrix according to the text data, the item feature data and the user feature data comprises: Obtain representative context feature data based on the descriptive text data, and obtain project potential feature data and user potential feature data based on the project feature data and the user feature data, respectively; Combining the representative context feature data and the project potential feature data to obtain combined project feature data; A multi-layer perception module is used to capture the nonlinear relationship between the user's potential feature data and the merged item feature data to obtain the prediction matrix.

3. The dual learning method for correcting sample selection bias for heterogeneous data as claimed in claim 2, characterized in that: The first convolutional neural network includes at least a first embedding layer, a convolutional layer, and a pooling layer, and the representative context feature data is obtained based on the description text data, including: Using the first embedding layer to perform word embedding representation processing on the description text data to obtain a word vector matrix; The word vector matrix is ​​sliced ​​by using the convolution layer to obtain corresponding context features, wherein the convolution window used in each slice is obtained by weighting based on each shared weight; The pooling layer is used to select the maximum context feature data from each of the context features to obtain the representative context feature data.

4. The dual learning method for correcting sample selection bias for heterogeneous data as claimed in claim 3, characterized in that: The second convolutional neural network includes at least a second embedding layer, and the combining the representative context feature data and the project potential feature data to obtain the merged project feature data includes: Processing the project feature data using the second embedding layer to obtain project potential feature data; The representative context feature data and the item feature data are processed based on a pre-constructed fusion layer to obtain the merged item feature data.

5. The dual learning method for correcting sample selection bias for heterogeneous data as claimed in claim 4, characterized in that: The third convolutional neural network includes at least a third embedding layer, wherein the multi-layer perception module is used to capture the nonlinear relationship between the user potential feature data and the merged project feature data to obtain a prediction matrix, including: Processing the user feature data using the third embedding layer to obtain user potential feature data; The input user potential feature data and the merged item feature data are processed in sequence using a plurality of stacked MLP layers to obtain the prediction matrix.

6. The dual learning method for correcting sample selection bias for heterogeneous data as claimed in claim 1, characterized in that: The training process of the dual robust model includes: Randomly initializing parameters of the prediction model and the estimation model; Acquire a user-item pair set, and obtain user-item interaction data based on the user-item pair set, wherein the user-item pair set is determined based on the user feature data and the item feature data; Obtaining a second user-item true label matrix, and obtaining a second prediction matrix according to the prediction matrix; Processing the second user-item real label matrix using the estimation model to obtain a third pseudo label matrix, and filling corresponding parts of the third pseudo label matrix with elements in the second prediction matrix to obtain a fourth pseudo label matrix; Obtaining an estimated error based on the second prediction matrix and the fourth pseudo label matrix; Replacing the squared loss term in the loss function of the estimation model based on the estimation error to obtain a new loss function of the estimation model; Inputting the fourth pseudo label matrix and the second user-item real label matrix into the prediction model to obtain a third prediction matrix, wherein the prediction model has the same structure as the estimation model; Calculating the cross entropy between the third prediction matrix and the second user-item true label matrix to obtain a prediction error; Obtaining a blank event correction error based on the estimation error and the prediction error; Obtaining a loss function of the prediction model based on the blank event correction error, the prediction error and the item interaction data; The estimation model and the prediction model are iteratively trained based on the new loss function of the estimation model and the loss function of the prediction model respectively, and the parameters of the prediction model are copied to the estimation model in each round of iterative training, and in the next round of iteration, the second user-item true label matrix continues to be input into the estimation model, and the iterative training is carried out until both the estimation model and the prediction model converge or a preset number of iteration termination times is reached.

7. The dual learning method for correcting sample selection bias for heterogeneous data according to claim 6, characterized in that: The new loss function of the estimation model is: Among them, L e (θ) represents the new loss function of the estimation model, represents the fourth pseudo label matrix, represents the second prediction matrix, O represents the user and item interaction data, O = {(u,i)|(u,i)∈D,o u,i =1},o u,i =1 means that there is contact between user u and item i, D represents the set of user-item pairs, λ represents the parameter that controls the strength of L2 regularization to avoid overfitting, and θ represents all parameters of the estimation model. represents the prediction tendency, CE() represents the cross entropy, and F represents the regularization operation.

8. The dual learning method for correcting sample selection bias for heterogeneous data as claimed in claim 6, characterized in that: The loss function of the prediction model is: in, represents the loss function of the prediction model, represents all parameters of the prediction model, o u,i =1 means that there is contact between user u and item i, o u,i = 0 means there is no contact between user u and item i, μ represents the parameter for controlling the L2 regularization strength to avoid overfitting, D represents the set of user-item pairs, Indicates the prediction tendency, e u,i represents the prediction error, represents the estimation error, and F represents the regularization operation.

9. A dual learning device for correcting sample selection bias for heterogeneous data, characterized in that: include: An acquisition module, used to acquire description text data, project feature data and user feature data; An input data determination module, used to determine a prediction matrix and a first user-item true label matrix according to the text data, the item feature data and the user feature data; A feedback data determination module is used to input the first user-project true label matrix into the estimation model of the pre-built dual robust model to obtain a first pseudo label matrix, fill the corresponding part of the first pseudo label matrix with the elements in the prediction matrix to obtain a second pseudo label matrix, and input the second pseudo label matrix and the first user-project true label matrix into the prediction model of the dual robust model to obtain feedback data of the project, wherein the dual robust model is trained based on a preset first objective function and a preset second function.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor, memory, and input-output unit; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the dual learning method for correcting sample selection bias for heterogeneous data according to any one of claims 1 to 8.

Images