Predictive method and system based on non-complete data under missing distribution shift

Abstract

The application provides a kind of missing distribution offset based on the prediction method and system of incomplete data, comprising: with incomplete feature as input, construct the prediction function network based on multilayer perceptron, predict complete label;Construct the prediction function parameter network based on multilayer perceptron, learn the relevance between prediction function parameter and missing condition;Using random Fourier feature, the relevance between incomplete feature and missing condition is measured;According to random Fourier feature, the weight of incomplete sample in data set is redistributed;By minimizing the prediction reconstruction error, the prediction function parameter network and the prediction function network are jointly trained, and the overall predictability of the combined two networks is optimized.The application uses a prediction framework based on decorrelation, which can effectively learn the underlying relevance between complete labels and incomplete features, thereby achieving good generalization performance when the missing distribution is offset.

Description

Technical Field

[0001] This invention relates to the fields of weakly supervised learning and distributional out-generalization, and more specifically, to a prediction method and system based on incomplete data under missing distribution offset. Background Technology

[0002] In many applications, incomplete data is prevalent due to sensor malfunctions, incomplete sensing range, and other reasons. In recent years, much work has focused on prediction problems based on incomplete data, i.e., predicting complete labels based on incomplete features and missing data. These works only consider the case where the training and test distributions are consistent. However, in real-world scenarios, the training and test distributions may differ. For example, the training data might contain 20% missing data, but as sensor deployment increases and sensor quality improves, the test data might contain only 10% missing data in subsequent testing scenarios. Furthermore, during the training phase, the missing data in the test data is often unknown.

[0003] Patent document CN104899457A (application number: CN201510329302.8) discloses a satellite data prediction method based on an improved unbiased GM(1,1) model, which addresses the technical problem of poor prediction accuracy in existing satellite data prediction methods. The technical solution involves first preprocessing the raw satellite telemetry data, including filling in missing bits and removing outliers; secondly, normalizing and smoothing the preprocessed data; then, establishing an unbiased GM(1,1) model, using the data processed in the above steps as input for prediction, and updating the prediction by incorporating the new prediction results into the unbiased GM(1,1) model; finally, performing anti-smoothing and anti-normalization processing on the obtained prediction data to obtain the final prediction result.

[0004] First, many works can be used for prediction based on incomplete data. The NeuMiss model mainly targets linear regression problems with Gaussian distribution characteristics; many data completion works can also solve the prediction problem by treating the predicted labels as missing variables. However, these works only consider the case where the training distribution and the test distribution are consistent.

[0005] Secondly, many works focus on the out-of-distribution generalization problem in prediction, especially when the test distribution is unknown during training. However, they only consider scenarios with complete data, which makes their methods inapplicable to scenarios with incomplete data. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a prediction method and system based on incomplete data under missing distribution shift.

[0007] The prediction method based on incomplete data under missing distribution shift provided by the present invention includes:

[0008] Step A: Construct a prediction function network based on a multilayer perceptron, using incomplete features from the sensor's sensing range as input, learn the correlation between complete labels and incomplete features, and predict complete labels;

[0009] Step B: Construct a prediction function parameter network based on a multilayer perceptron to learn the correlation between prediction function parameters and missing cases;

[0010] Step C: Utilize random Fourier features to measure the correlation between incomplete features and missing features;

[0011] Step D: Based on the random Fourier features, reassign weights to incomplete samples in the dataset by minimizing the correlation between incomplete features and missing cases;

[0012] Step E: Based on the reassigned weights, by minimizing the prediction reconstruction error, jointly train the prediction function parameter network and the prediction function network, and optimize the overall predictability by combining the two networks.

[0013] Step F: Predict complete labels based on incomplete features and missing features.

[0014] Preferably, step A includes:

[0015] Step A1: Define the symbolic representations of incomplete features and complete labels, using uppercase and lowercase letters to represent random variables and their values, respectively, denoted by x∈R. n and y∈R d Features and labels are represented by m∈0,1 d The missing feature is indicated by x, where 1 represents observable and 0 represents missing. m This represents the observable portion of a feature;

[0016] Step A2: Use the incomplete feature x⊙m as the input to the prediction function network, which means that x and m are multiplied in the corresponding dimension;

[0017] Step A3: Construct a prediction function network based on a multilayer perceptron. The number of neurons in the input layer is consistent with the dimension of the features, the number of neurons in the output layer is consistent with the dimension of the labels, the number of hidden layers and the number of neurons in each layer are hyperparameters, and the inter-layer activation function is the ReLU function.

[0018] If we define a single fully connected layer with n-dimensional input and d-dimensional output, then this layer can be characterized by the following formula:

[0019] y = ReLU(Wx)

[0020] Where W∈R d×n The weight parameter matrix of the fully connected layer is used to describe the parameters of the prediction function. The weight parameter matrix of each layer in the multilayer perceptron is used to describe the relationship between complete labels and incomplete features. The final output of the multilayer perceptron is the predicted complete label.

[0021] Preferably, step B includes:

[0022] Step B1: Use the missing case m as input to the prediction function parameter network;

[0023] Step B2: Construct a prediction function parameter network based on a multilayer perceptron. The number of neurons in the input layer is consistent with the dimension of the missing case, and the number of neurons in the output layer is consistent with the dimension of the prediction function network parameters. The number of hidden layers and the number of neurons in each layer are hyperparameters. The inter-layer activation function is the ReLU function. The weight parameter matrix of each layer in this multilayer perceptron together constitutes the parameters of the prediction function, which characterizes the correlation between the prediction function parameters and the missing case. The final output of the multilayer perceptron is the parameters of the prediction function.

[0024] Step B3: Use the output of the prediction function parameter network as the parameter of the prediction function, apply it to the incomplete input features, and finally obtain the predicted complete label.

[0025] Preferably, step C includes:

[0026] Step C1: Define the function space of the random Fourier features as follows:

[0027]

[0028] Where h is a function in the function space, defined by the form after the colon; z is the independent variable of the function; ω is the angular frequency of the cosine trigonometric function, obtained by sampling from the standard Gaussian distribution; β is the phase of the cosine trigonometric function, obtained by uniform sampling from [0, 2π).

[0029] Step C2: Define the correlation measure between variables in the case of a complete sample. Let there be variables A and B, and N samples. Define the cross-covariance matrix of A and B as follows:

[0030]

[0031] u(A (i) )=(u1(A (i) ), ..., u q (A (i) )), u j ∈H

[0032] v(B (i) )=(v1(A(i) ), ..., v q (A (i) )), v j ∈H

[0033] Among them, A (i) Let A be a sample of the i-th variable A; A (j) Let be the sample of the j-th variable A; i be the label of the nth A or B sample; j be the label of the nth A or B sample; u() is the high-dimensional representation of variable A; v() is the high-dimensional representation of variable B; u(A (i) ) and v(B (i) ) is a high-dimensional representation of variables A and B, and the Frobenius norm squared measure of the correlation between A and B is used to measure the correlation between A and B.

[0034] Step C3: Define the correlation between incomplete features and missing features in the case of weighted incomplete samples, and assign a weight w to each sample. i Then the cross-covariance matrix for the k-th and l-th features is:

[0035]

[0036] Where, N kl N represents the number of samples where both the k-th and l-th dimension features are observable simultaneously. k and N l Let be the number of observable samples for the k-th and l-th dimension features, respectively;

[0037] The cross-covariance matrix for the k-th feature and the l-th missing feature is as follows:

[0038]

[0039] The cross-covariance matrices for the k-th missing dimension and the l-th missing dimension are as follows:

[0040]

[0041] in, For the i-th missing case in the l-th dimension, the total correlation between incomplete features and missing cases is the sum of the above three items.

[0042] Preferably, step D includes:

[0043] By minimizing the correlation between incomplete features and missing cases, and solving for the weight of each incomplete sample in the dataset, the optimization problem to be solved is:

[0044]

[0045] Where is the standard deviation of w; min indicates that the optimization objective is to minimize the first term CrossCovariance. w The overall correlation between incomplete features and missing features; the second term This is to prevent assigning extremely large weights to a small number of low-relevance samples; γ is a parameter that balances the two; the optimized w is the weight of each incomplete sample.

[0046] Step E includes:

[0047] By minimizing the weighted prediction reconstruction error, and jointly training the prediction function parameter network and the prediction function network, the optimization problem requiring a solution is:

[0048]

[0049] Where, min θ This indicates that the optimization objective is to minimize based on the parameter θ; This indicates that the sample (X, M, Y) follows the expected distribution of the training set. denoted as the prediction function network; w() represents the sample weights, g() represents the prediction function network, φ() represents the prediction function parameter network, and θ represents the parameters of the prediction function parameter network.

[0050] The prediction system based on incomplete data under missing distribution shift provided by the present invention includes:

[0051] Module A: Construct a prediction function network based on a multilayer perceptron, using incomplete features from the sensor's sensing range as input, learning the correlation between complete labels and incomplete features, and predicting complete labels;

[0052] Module B: Construct a prediction function parameter network based on a multilayer perceptron to learn the correlation between prediction function parameters and missing cases;

[0053] Module C: Utilizes random Fourier features to measure the correlation between incomplete features and missing features;

[0054] Module D: Based on the random Fourier features, the weights of incomplete samples in the dataset are reassigned by minimizing the correlation between incomplete features and missing cases;

[0055] Module E: Based on the redistributed weights, the prediction function parameter network and the prediction function network are jointly trained by minimizing the prediction reconstruction error, and the overall predictability is optimized by combining the two networks.

[0056] Step F: Predict complete labels based on incomplete features and missing features.

[0057] Preferably, module A includes:

[0058] Module A1: Defines the symbolic representation of incomplete features and complete labels, using uppercase and lowercase letters to represent random variables and their values, respectively, denoted by x∈R. n and y∈R d Features and labels are represented by m∈0,1 d The missing feature is indicated by x, where 1 represents observable and 0 represents missing. m This represents the observable portion of a feature;

[0059] Module A2: The incomplete feature x⊙m is used as the input to the prediction function network, which means that x and m are multiplied in the corresponding dimension;

[0060] Module A3: Construct a prediction function network based on a multilayer perceptron. The number of neurons in the input layer is consistent with the dimension of the features, the number of neurons in the output layer is consistent with the dimension of the labels, the number of hidden layers and the number of neurons in each layer are hyperparameters, and the inter-layer activation function is the ReLU function.

[0061] If we define a single fully connected layer with n-dimensional input and d-dimensional output, then this layer can be characterized by the following formula:

[0062] y = ReLU(Wx)

[0063] Where W∈R d×n The weight parameter matrix of the fully connected layer is used to describe the parameters of the prediction function. The weight parameter matrix of each layer in the multilayer perceptron is used to describe the relationship between complete labels and incomplete features. The final output of the multilayer perceptron is the predicted complete label.

[0064] Preferably, module B includes:

[0065] Module B1: Takes the missing case m as input to the prediction function parameter network;

[0066] Module B2: Construct a prediction function parameter network based on a multilayer perceptron. The number of neurons in the input layer is consistent with the dimension of the missing case, and the number of neurons in the output layer is consistent with the dimension of the prediction function network parameters. The number of hidden layers and the number of neurons in each layer are hyperparameters. The inter-layer activation function is the ReLU function. The weight parameter matrix of each layer in this multilayer perceptron together constitutes the parameters of the prediction function, which describes the correlation between the prediction function parameters and the missing case. The final output of the multilayer perceptron is the parameters of the prediction function.

[0067] Module B3: The output of the prediction function parameter network is used as the parameter of the prediction function, which is applied to the incomplete input features to finally obtain the predicted complete label.

[0068] Preferably, module C includes:

[0069] Module C1: Defines the function space of random Fourier features as follows:

[0070]

[0071] Where h is a function in the function space, defined by the form after the colon; z is the independent variable of the function; ω is the angular frequency of the cosine trigonometric function, obtained by sampling from the standard Gaussian distribution; β is the phase of the cosine trigonometric function, obtained by uniform sampling from [0, 2π).

[0072] Module C2: Defines the measure of the association between variables in the case of a complete sample. Given variables A and B, and N samples. Define the cross-covariance matrix of A and B as follows:

[0073]

[0074] u(A (i) )=(u1(A (i) ), ..., u q (A (i) )), u j ∈H

[0075] v(B (i) )=(v1(A (i) ), ..., v q (A (i) )), v j ∈H

[0076] Among them, A (i) Let A be a sample of the i-th variable A; A (j) Let be the sample of the j-th variable A; i be the label of the nth A or B sample; j be the label of the nth A or B sample; u() is the high-dimensional representation of variable A; v() is the high-dimensional representation of variable B; u(A (i) ) and v(B (i) ) is a high-dimensional representation of variables A and B, and the Frobenius norm squared measure of the correlation between A and B is used to measure the correlation between A and B.

[0077] Module C3: Defines the correlation between incomplete features and missing features in the case of weighted incomplete samples, and assigns a weight w to each sample. i Then the cross-covariance matrix for the k-th and l-th features is:

[0078]

[0079] Where, N kl N represents the number of samples where both the k-th and l-th dimension features are observable simultaneously. kand N l Let be the number of observable samples for the k-th and l-th dimension features, respectively;

[0080] The cross-covariance matrix for the k-th feature and the l-th missing feature is as follows:

[0081]

[0082] The cross-covariance matrices for the k-th missing dimension and the l-th missing dimension are as follows:

[0083]

[0084] in, For the i-th missing case in the l-th dimension, the total correlation between incomplete features and missing cases is the sum of the above three items.

[0085] Preferably, module D includes:

[0086] By minimizing the correlation between incomplete features and missing cases, and solving for the weight of each incomplete sample in the dataset, the optimization problem to be solved is:

[0087]

[0088] Where is the standard deviation of w; min indicates that the optimization objective is to minimize the first term CrossCovariance. w The overall correlation between incomplete features and missing features; the second term This is to prevent assigning extremely large weights to a small number of low-relevance samples; γ is a parameter that balances the two; the optimized w is the weight of each incomplete sample.

[0089] The module E includes:

[0090] By minimizing the weighted prediction reconstruction error, and jointly training the prediction function parameter network and the prediction function network, the optimization problem requiring a solution is:

[0091]

[0092] Where, min θ This indicates that the optimization objective is to minimize based on the parameter θ; This indicates that the sample (X, M, Y) follows the expected distribution of the training set. denoted as the prediction function network; w() represents the sample weights, g() represents the prediction function network, φ() represents the prediction function parameter network, and θ represents the parameters of the prediction function parameter network.

[0093] Compared with the prior art, the present invention has the following beneficial effects:

[0094] (1) The present invention adopts a prediction framework based on deassociation, which can effectively learn the underlying correlation between complete labels and incomplete features, thereby achieving better generalization performance when the missing distribution shifts.

[0095] (2) This invention can be directly applied to datasets with incomplete features;

[0096] (3) The present invention is reasonably constructed and easy to use, and can overcome the defects of the prior art. Attached Figure Description

[0097] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0098] Figure 1 This is a flowchart of the prediction method of the present invention;

[0099] Figure 2 This is an example diagram of a prediction framework based on incomplete data in an embodiment of the present invention. Detailed Implementation

[0100] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0101] Example 1:

[0102] like Figure 1 and Figure 2 As shown, this invention provides a prediction method based on incomplete data under missing distribution offset.

[0103] Specifically, this embodiment includes the following steps:

[0104] Step A: Using incomplete features as input, construct a prediction function network based on a multilayer perceptron. Through the parameters of the prediction function, learn the correlation between complete labels and incomplete features, and predict complete labels.

[0105] Step B: Construct a prediction function parameter network based on a multilayer perceptron to learn the correlation between prediction function parameters and missing cases;

[0106] Step C: Utilize random Fourier features to measure the correlation between incomplete features and missing features;

[0107] Step D: Based on the random Fourier features, reassign weights to incomplete samples in the dataset by minimizing the correlation between incomplete features and missing cases;

[0108] Step E: Based on the reassigned weights, by minimizing the prediction reconstruction error, jointly train the prediction function parameter network and the prediction function network, and optimize the overall predictability by combining the two networks.

[0109] Step F: Predict complete labels based on incomplete features and missing features.

[0110] Step A includes the following:

[0111] Step A1: Define the symbolic representations of incomplete features and complete labels. Specifically, use uppercase and lowercase letters to represent random variables and their values, respectively. Let x∈R n and y∈R d Features and labels are represented by m∈0,1 d This indicates the missing feature status, with 1 representing observable and 0 representing missing. (x) m If the observable portion of a feature is represented, then incomplete features can be represented using (x). m , m) represents;

[0112] Step A2: Use the incomplete feature x⊙m as the input to the prediction function network, which means that x and m are multiplied in the corresponding dimensions, that is, the original value in the observable dimension and 0 in the missing dimension;

[0113] Step A3: Construct a prediction function network based on a multilayer perceptron. The number of neurons in the input layer matches the dimension of the features, and the number of neurons in the output layer matches the dimension of the labels. The number of hidden layers and the number of neurons in each layer are hyperparameters, and the inter-layer activation function is the ReLU function. Mathematically, taking a single fully connected layer as an example, with an input of n dimensions and an output of d dimensions, this layer can be characterized by the following formula:

[0114] y = ReLU(Wx)

[0115] Where W∈R d×n This represents the weight parameter matrix of the fully connected layer. The weight parameter matrices of each layer in a multilayer perceptron collectively form the parameters of the prediction function, characterizing the correlation between complete labels and incomplete features. The final output of the multilayer perceptron is the predicted complete label.

[0116] Step B includes the following:

[0117] Step B1: Use the missing case m as input to the prediction function parameter network;

[0118] Step B2: Construct a prediction function parameter network based on a multilayer perceptron. The number of neurons in the input layer matches the dimension of the missing case, and the number of neurons in the output layer matches the dimension of the prediction function network parameters. The number of hidden layers and the number of neurons in each layer are hyperparameters, and the inter-layer activation function is the ReLU function. The weight parameter matrix of each layer in this multilayer perceptron collectively constitutes the parameters of the prediction function, characterizing the correlation between the prediction function parameters and the missing case. The final output of the multilayer perceptron is the prediction function parameter.

[0119] Step B3: Use the output of the prediction function parameter network as the parameter of the prediction function, apply it to the incomplete input features, and finally obtain the predicted complete label.

[0120] Step C includes the following steps:

[0121] Step C1: Define the function space of the random Fourier features as follows:

[0122]

[0123] Where ω is sampled from a standard Gaussian distribution, and β is uniformly sampled from [0, 2π);

[0124] Step C2: Define the measure of association between variables in the case of a complete sample. Assume there are variables A and B, and N samples. Define the cross-covariance matrix of A and B as follows:

[0125]

[0126] in

[0127] u(A (i) )=(u1(A (i) ), ..., u q (A (i) )), u j ∈H

[0128] v(B (i) )=(v1(A (i) ), ..., v q (A (i) )), v j ∈H

[0129] u(A (i) ) and v(B (i) This can be viewed as a high-dimensional representation of variables A and B. The Frobenius norm squared measure of the cross-covariance matrix of A and B is used to measure the correlation between A and B.

[0130] Step C3: Define the correlation between incomplete features and missing features in the case of weighted incomplete samples. Assign a weight w to each sample. i Since incomplete features are only partially observable, the correlation is calculated based only on the observable parts. Therefore, the cross-covariance matrix for the k-th and l-th dimension features is:

[0131]

[0132] Where, N kl N represents the number of samples where both the k-th and l-th dimension features are observable simultaneously. k and N l Let be the number of observable samples for the k-th and l-th features respectively. Without loss of generality, the corresponding observable samples can be arranged at the beginning of the dataset during calculation. Similarly, the cross-covariance matrix for the k-th and l-th missing features is:

[0133]

[0134] The cross-covariance matrix for the k-th missing dimension and the l-th missing dimension is as follows:

[0135]

[0136] The overall correlation between incomplete features and missing features is the sum of the above three items.

[0137] Step D includes the following steps:

[0138] Step D1: Solve for the weight of each incomplete sample in the dataset by minimizing the correlation between incomplete features and missing features. Specifically, the optimization problem requiring a solution is:

[0139]

[0140] The first term represents the overall correlation between incomplete features and missing features, the second term aims to prevent assigning excessively large weights to a small subset of low-correlation samples, and γ is a parameter that balances these two factors. The optimized w is the weight for each incomplete sample.

[0141] Step E includes the following steps:

[0142] Step E1: Jointly train the prediction function parameter network and the prediction function network by minimizing the weighted prediction reconstruction error. Specifically, the optimization problem for the required solution is:

[0143]

[0144] Where w() represents the sample weights, g() represents the prediction function network, φ() represents the prediction function parameter network, and θ represents the parameters of the prediction function parameter network. The prediction function corresponding to parameter θ obtained by solving this optimization problem achieves good generalization performance on prediction tasks based on incomplete data under missing distribution shifts.

[0145] This invention primarily addresses the practical problem of prediction based on incomplete data under missing distribution shifts, and proposes a framework for prediction based on incomplete data from the perspective of out-of-distribution generalization. On one hand, it learns the correlations between complete labels, incomplete features, prediction function parameters, and missing cases through a prediction function network and a prediction function parameter network. On the other hand, it promotes the prediction framework to learn the underlying correlations of the data through a decorrelation method, thereby overcoming the negative impact of distribution shifts. Experiments on artificially generated Gaussian and Gaussian mixture datasets and real-world housing price datasets fully demonstrate the effectiveness of this embodiment. From the perspective of root mean square error (RMSE), the performance of this invention significantly outperforms the best current prediction methods based on incomplete data and out-of-distribution generalization methods under complete data.

[0146] Example 2:

[0147] The present invention also provides a prediction system based on incomplete data under missing distribution shift. The prediction system based on incomplete data under missing distribution shift can be implemented by executing the process steps of the prediction method based on incomplete data under missing distribution shift. That is, those skilled in the art can understand the prediction method based on incomplete data under missing distribution shift as a preferred embodiment of the prediction system based on incomplete data under missing distribution shift.

[0148] The prediction system based on incomplete data under missing distribution offset provided by the present invention includes: Module A: Constructing a prediction function network based on a multilayer perceptron, using incomplete features within the sensor's sensing range as input, learning the correlation between complete labels and incomplete features, and predicting complete labels; Module B: Constructing a prediction function parameter network based on a multilayer perceptron, learning the correlation between prediction function parameters and missing cases; Module C: Using stochastic Fourier features to measure the correlation between incomplete features and missing cases; Module D: Based on stochastic Fourier features, redistributing weights to incomplete samples in the dataset by minimizing the correlation between incomplete features and missing cases; Module E: Based on the redistributed weights, jointly training the prediction function parameter network and the prediction function network by minimizing the prediction reconstruction error, and optimizing the overall predictive performance of the combined networks; Step F: Predicting complete labels based on incomplete features and missing cases.

[0149] Module A includes: Module A1: Defines the symbolic representation of incomplete features and complete labels, using uppercase and lowercase letters to represent random variables and their values, respectively, and using x∈R n and y∈R d Features and labels are represented by m∈0,1 d The missing feature is indicated by x, where 1 represents observable and 0 represents missing. m Module A1: Represents the observable part of the features; Module A2: Takes the incomplete features x⊙m as the input to the prediction function network, representing the multiplication of x and m in the corresponding dimensions; Module A3: Constructs a prediction function network based on a multilayer perceptron, where the number of neurons in the input layer is consistent with the dimension of the features, the number of neurons in the output layer is consistent with the dimension of the labels, the number of hidden layers and the number of neurons in each layer are hyperparameters, and the inter-layer activation function is the ReLU function; Assuming it is a single fully connected layer with n-dimensional input and d-dimensional output, this layer can be characterized by the following formula: y=ReLU(Wx), where W∈R d×n The weight parameter matrix of the fully connected layer is used to describe the parameters of the prediction function. The weight parameter matrix of each layer in the multilayer perceptron is used to describe the relationship between complete labels and incomplete features. The final output of the multilayer perceptron is the predicted complete label.

[0150] Module B includes: Module B1: using the missing case m as the input to the prediction function parameter network; Module B2: constructing a prediction function parameter network based on a multilayer perceptron, where the number of neurons in the input layer is consistent with the dimension of the missing case, the number of neurons in the output layer is consistent with the dimension of the prediction function network parameters, the number of hidden layers and the number of neurons in each layer are hyperparameters, the inter-layer activation function is the ReLU function, and the weight parameter matrix of each layer in this multilayer perceptron together constitutes the parameters of the prediction function, characterizing the correlation between the prediction function parameters and the missing case, and the final output of the multilayer perceptron is the prediction function parameters; Module B3: using the output of the prediction function parameter network as the parameters of the prediction function, applying it to the incomplete input features, and finally obtaining the predicted complete label.

[0151] The module C includes: Module C1: defining the function space of random Fourier features as: Where h is a function in the function space, defined by the form after the colon; z is the independent variable of the function; ω is the angular frequency of the cosine trigonometric function, obtained by sampling from the standard Gaussian distribution; β is the phase of the cosine trigonometric function, obtained by uniform sampling from [0, 2π); Module C2: Defines the correlation measure between variables in the case of complete samples. Let there be variables A and B, and N samples. Define the cross-covariance matrix of A and B as follows:

[0152] u(A(i) )=(u1(A (i) ), ..., u q (A (i) )), u j ∈H, v(B (i) )=(v1(A (i) ), ..., v q (A (i) )), v j ∈H, where A (i) Let A be the sample of the i-th variable A; A(j) be the sample of the j-th variable A; i is the label of the nth A or B sample; j is the label of the nth A or B sample; u() is the high-dimensional representation of variable A; v() is the high-dimensional representation of variable B; u(A (i) ) and v(B (i) Module C1: Provides high-dimensional representations of variables A and B, using the Frobenius norm squared of the cross-covariance matrices of A and B to measure the correlation between A and B; Module C2: Defines the correlation between incomplete features and missing features in the case of weighted incomplete samples, assigning a weight w to each sample. i Then the cross-covariance matrix for the k-th and l-th features is: Where, N kl N represents the number of samples where both the k-th and l-th dimension features are observable simultaneously. k and N l Let be the number of observable samples for the k-th and l-th features, respectively; the cross-covariance matrices for the missing k-th and l-th features are: The cross-covariance matrices for the k-th missing dimension and the l-th missing dimension are as follows: in, For the i-th missing case in the l-th dimension, the total correlation between incomplete features and missing cases is the sum of the above three items.

[0153] Module D includes: solving for the weight of each incomplete sample in the dataset by minimizing the correlation between incomplete features and missing cases, and the optimization problem to be solved is: Where is the standard deviation of w; min indicates that the optimization objective is to minimize the first term CrossCovariance. w The overall correlation between incomplete features and missing features; the second term This is to prevent assigning extremely large weights to a small number of low-relevance samples; γ is a parameter that balances the two; the optimized w is the weight of each incomplete sample.

[0154] The module E includes: jointly training a prediction function parameter network and a prediction function network by minimizing the weighted prediction reconstruction error, and the optimization problem requiring a solution is: Where, min θ This indicates that the optimization objective is to minimize based on the parameter θ; This indicates that the sample (X, M, Y) follows the expected distribution of the training set. denoted as the prediction function network; w() represents the sample weights, g() represents the prediction function network, φ() represents the prediction function parameter network, and θ represents the parameters of the prediction function parameter network.

[0155] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0156] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A prediction method based on incomplete data under missing distribution shift, characterized in that, include: Step A: Construct a prediction function network based on a multilayer perceptron, using incomplete features from the sensor's sensing range as input, learn the correlation between complete labels and incomplete features, and predict complete labels; Step B: Construct a prediction function parameter network based on a multilayer perceptron to learn the correlation between prediction function parameters and missing values; Step B includes: Step B1: Missing cases As input to the prediction function parameter network; Step B3: Use the output of the prediction function parameter network as the parameter of the prediction function, apply it to the incomplete input features, and finally obtain the predicted complete label; Step C: Utilize random Fourier features to measure the correlation between incomplete features and missing features; Step D: Based on the random Fourier features, reassign weights to incomplete samples in the dataset by minimizing the correlation between incomplete features and missing cases; Step E: Based on the reassigned weights, by minimizing the prediction reconstruction error, jointly train the prediction function parameter network and the prediction function network, and optimize the overall predictability by combining the two networks. Step F: Predict complete labels based on incomplete features and missing features; The incomplete characteristics of the sensor's sensing range are used as input, where the incomplete characteristics of the sensor's sensing range are caused by sensor malfunctions and incomplete sensing range.

2. The prediction method based on incomplete data under missing distribution shift according to claim 1, characterized in that, Step A includes: Step A1: Define the symbolic representations for incomplete features and complete labels, using uppercase and lowercase letters to represent random variables and their values, respectively. and To represent features and labels, use This indicates the missing feature status, where 1 represents observable and 0 represents missing. This represents the observable portion of a feature; Step A2: Remove incomplete features As input to the prediction function network, it represents and Multiply along the corresponding dimensions; Step A3: Construct a prediction function network based on a multilayer perceptron. The number of neurons in the input layer is consistent with the dimension of the features, the number of neurons in the output layer is consistent with the dimension of the labels, the number of hidden layers and the number of neurons in each layer are hyperparameters, and the inter-layer activation function is the ReLU function. Set as a single fully connected layer, with the input as... Dimension, output is If the dimension is defined, then the layer can be characterized by the following formula: in, The weight parameter matrix of the fully connected layer is used to describe the parameters of the prediction function. The weight parameter matrix of each layer in the multilayer perceptron is used to describe the relationship between complete labels and incomplete features. The final output of the multilayer perceptron is the predicted complete label.

3. The prediction method based on incomplete data under missing distribution shift according to claim 1, characterized in that, Step B further includes: Step B2: Construct a prediction function parameter network based on a multilayer perceptron. The number of neurons in the input layer is consistent with the dimension of the missing case, and the number of neurons in the output layer is consistent with the dimension of the prediction function network parameters. The number of hidden layers and the number of neurons in each layer are hyperparameters. The inter-layer activation function is the ReLU function. The weight parameter matrix of each layer in this multilayer perceptron together constitutes the parameters of the prediction function, which characterizes the correlation between the prediction function parameters and the missing case. The final output of the multilayer perceptron is the parameters of the prediction function.

4. The prediction method based on incomplete data under missing distribution shift according to claim 1, characterized in that, Step C includes: Step C1: Define the function space of the random Fourier features as follows: in, For functions in the function space, defined by the form following the colon; is the independent variable of the function; for The angular frequencies of trigonometric functions are obtained by sampling from a standard Gaussian distribution; for The phase of a trigonometric function is determined by... Obtained by uniform sampling in the middle; Step C2: Define a measure of the association between variables in the case of a complete sample. Let there be variables. and And there are Sample , define about and The cross covariance matrix is: in, For the first Variables ; For the first Variables ; , Variables , High-dimensional representation, using and Frobenius norm squared measure of the cross covariance matrix and The correlation between them; Step C3: Define the correlation between incomplete features and missing features in the case of weighted incomplete samples, and assign a weight to each sample. Then regarding the first Dimensional features and the first The cross-covariance matrix of the dimensional features is: in, For the first Dimensional features and the first The number of samples that can be observed simultaneously for a given feature. and For the first Dimensional features and the first The number of observable samples for each of the dimensional features; Regarding the first Dimensional features and the first The cross-covariance matrix for the missing dimension case is: Regarding the first Dimensional missing cases and the first The cross-covariance matrix for the missing dimension case is: in, For the first Vidi For each missing sample, the total correlation between incomplete features and missing cases is the sum of the above three items.

5. The prediction method based on incomplete data under missing distribution shift according to claim 1, characterized in that, Step D includes: By minimizing the correlation between incomplete features and missing cases, and solving for the weight of each incomplete sample in the dataset, the optimization problem to be solved is: in, for Standard deviation; The optimization objective is to minimize the first term. The overall correlation between incomplete features and missing features; the second term This is to prevent assigning extremely large weights to a small number of low-relevance samples; To balance the parameters of both, the optimized result is... Weights for each incomplete sample; Step E includes: By minimizing the weighted prediction reconstruction error, and jointly training the prediction function parameter network and the prediction function network, the optimization problem requiring a solution is: in, The optimization objective is based on parameters. Minimize; Indicates sample It follows the expected distribution of the training set. For prediction function networks; For sample weights, For predicting function parameter networks, These are the parameters of the prediction function parameter network.

6. A prediction system based on incomplete data under missing distribution shift, characterized in that, include: Module A: Construct a prediction function network based on a multilayer perceptron, using incomplete features from the sensor's sensing range as input, learning the correlation between complete labels and incomplete features, and predicting complete labels; Module B: Constructs a prediction function parameter network based on a multilayer perceptron to learn the correlation between prediction function parameters and missing values; Module B includes: Module B1: Missing Cases As input to the prediction function parameter network; Module B3: The output of the prediction function parameter network is used as the parameter of the prediction function, which is applied to the incomplete input features to finally obtain the predicted complete label; Module C: Utilizes random Fourier features to measure the correlation between incomplete features and missing features; Module D: Based on the random Fourier features, the weights of incomplete samples in the dataset are reassigned by minimizing the correlation between incomplete features and missing cases; Module E: Based on the redistributed weights, the prediction function parameter network and the prediction function network are jointly trained by minimizing the prediction reconstruction error, and the overall predictability is optimized by combining the two networks. Step F: Predict complete labels based on incomplete features and missing features; The incomplete characteristics of the sensor's sensing range are used as input, where the incomplete characteristics of the sensor's sensing range are caused by sensor malfunctions and incomplete sensing range.

7. The prediction system based on incomplete data under missing distribution shift according to claim 6, characterized in that, Module A includes: Module A1: Defines the symbolic representation of incomplete features and complete labels, using uppercase and lowercase letters to represent random variables and their values, respectively. and To represent features and labels, use This indicates the missing feature status, where 1 represents observable and 0 represents missing. This represents the observable portion of a feature; Module A2: Incomplete features As input to the prediction function network, it represents and Multiply along the corresponding dimensions; Module A3: Construct a prediction function network based on a multilayer perceptron. The number of neurons in the input layer is consistent with the dimension of the features, the number of neurons in the output layer is consistent with the dimension of the labels, the number of hidden layers and the number of neurons in each layer are hyperparameters, and the inter-layer activation function is the ReLU function. Set as a single fully connected layer, with the input as... Dimension, output is If the dimension is defined, then the layer can be characterized by the following formula: in, The weight parameter matrix of the fully connected layer is used to describe the parameters of the prediction function. The weight parameter matrix of each layer in the multilayer perceptron is used to describe the relationship between complete labels and incomplete features. The final output of the multilayer perceptron is the predicted complete label.

8. The prediction system based on incomplete data under missing distribution shift according to claim 6, characterized in that, Module B also includes: Module B2: Construct a prediction function parameter network based on a multilayer perceptron. The number of neurons in the input layer is consistent with the dimension of the missing case, and the number of neurons in the output layer is consistent with the dimension of the prediction function network parameters. The number of hidden layers and the number of neurons in each layer are hyperparameters, and the inter-layer activation function is the ReLU function. The weight parameter matrix of each layer in this multilayer perceptron together constitutes the parameters of the prediction function, which characterizes the correlation between the prediction function parameters and the missing case. The final output of the multilayer perceptron is the parameters of the prediction function.

9. The prediction system based on incomplete data under missing distribution shift according to claim 6, characterized in that, Module C includes: Module C1: Defines the function space of random Fourier features as follows: in, For functions in the function space, defined by the form following the colon; is the independent variable of the function; for The angular frequencies of trigonometric functions are obtained by sampling from a standard Gaussian distribution; for The phase of a trigonometric function is determined by... Obtained by uniform sampling in the middle; Module C2: Defines the measure of the association between variables in the case of a complete sample, given variables. and And there are Sample , define about and The cross covariance matrix is: in, For the first Variables ; For the first Variables ; , Variables , High-dimensional representation, using and Frobenius norm squared measure of the cross covariance matrix and The correlation between them; Module C3: Defines the correlation between incomplete features and missing features in the case of weighted incomplete samples, and assigns a weight to each sample. Then regarding the first Dimensional features and the first The cross-covariance matrix of the dimensional features is: in, For the first Dimensional features and the first The number of samples that can be observed simultaneously for a given feature. and For the first Dimensional features and the first The number of observable samples for each of the dimensional features; Regarding the first Dimensional features and the first The cross-covariance matrix for the missing dimension case is: Regarding the first Dimensional missing cases and the first The cross-covariance matrix for the missing dimension case is: in, For the first Vidi For each missing sample, the total correlation between incomplete features and missing cases is the sum of the above three items.

10. The prediction system based on incomplete data under missing distribution shift according to claim 6, characterized in that, Module D includes: By minimizing the correlation between incomplete features and missing cases, and solving for the weight of each incomplete sample in the dataset, the optimization problem to be solved is: in, for Standard deviation; The optimization objective is to minimize the first term. The overall correlation between incomplete features and missing features; the second term This is to prevent assigning extremely large weights to a small number of low-relevance samples; To balance the parameters of both, the optimized result is... Weights for each incomplete sample; The module E includes: By minimizing the weighted prediction reconstruction error, and jointly training the prediction function parameter network and the prediction function network, the optimization problem requiring a solution is: in, The optimization objective is based on parameters. Minimize; Indicates sample It follows the expected distribution of the training set. For prediction function networks; For sample weights, For predicting function parameter networks, These are the parameters of the prediction function parameter network.

Images