A method and apparatus for filling missing data

Abstract

The embodiment of the present application provides a kind of missing data filling method and device, to realize the accurate filling of missing value, this filling method includes: pre-filling to missing data, obtain pre-filling missing data, according to pre-filling missing data, determine and pre-filling missing data corresponding interpolation data and interpolation training data, generate adversarial network model, according to preset length, interpolation training data is divided, and interpolation training vector is obtained, according to interpolation training vector, adversarial network model is trained, interpolation data is input into the adversarial network model after training, and preliminary filling interpolation data is obtained, determine and preliminary filling interpolation data corresponding weight value, according to preliminary filling interpolation data and the weight value corresponding to preliminary filling interpolation data, determine and missing data corresponding filling value.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to a method and apparatus for filling in missing data. Background Technology

[0002] The use of electronic devices generates massive amounts of data, causing the data volume to grow exponentially. However, this rapid increase in data volume is often accompanied by the problem of missing data. Since missing data may contain important information, improper handling can negatively impact statistical analysis and applications. Therefore, how to reasonably and effectively solve the problem of missing values, improve the quality of raw data, and ultimately enhance the results of data analysis has become a focus of attention. Summary of the Invention

[0003] In view of the above problems, the purpose of this invention is to provide a method and apparatus for filling missing data, thereby achieving accurate filling of missing data. The specific solution is as follows:

[0004] In a first aspect, embodiments of this application provide a method for filling in missing data, characterized in that the filling method includes:

[0005] The missing data is pre-filled to obtain pre-filled missing data;

[0006] Based on the pre-filled missing data, interpolation data and interpolation training data corresponding to the pre-filled missing data are determined; the interpolation data is the data in the row and column where the pre-filled missing data is located in the data matrix; the interpolation data includes horizontal interpolation data and vertical interpolation data; the interpolation training data is the complete row and column data parallel to the interpolation data in the data matrix where the pre-filled missing data is located; the interpolation training data includes horizontal interpolation training data and vertical interpolation training data;

[0007] Generative adversarial network models;

[0008] The interpolation training data is divided according to a preset length to obtain interpolation training vectors;

[0009] The adversarial network model is trained based on the interpolated training vectors;

[0010] The interpolated data is input into the trained adversarial network model to obtain preliminary filled interpolated data;

[0011] Determine the weight values ​​corresponding to the initial fill interpolation data;

[0012] Based on the preliminary fill interpolation data and the weight values ​​corresponding to the preliminary fill interpolation data, the fill value corresponding to the missing data is determined.

[0013] Optionally, the generative adversarial network model includes:

[0014] Generate a horizontal adversarial network model and a vertical adversarial network model. Optionally, the step of dividing the interpolation training data according to a preset length to obtain interpolation training vectors includes:

[0015] The interpolation training vectors include horizontal interpolation training vectors and vertical interpolation training vectors;

[0016] Taking the leftmost data in the horizontal interpolation training data as the starting point, the vector is divided to the right according to the preset length to obtain the horizontal interpolation training vector;

[0017] Using the topmost data in the vertical interpolation training data as the starting point, the vector is divided downwards according to the preset length to obtain the vertical interpolation training vector.

[0018] Optionally, determining the weight value corresponding to the initial interpolation data includes:

[0019] Calculate the total distance between the horizontal interpolated data and the missing data, and the total distance between the vertical interpolated data and the missing data;

[0020] The total distance between the horizontal interpolated data and the missing data and the total distance between the vertical interpolated data and the missing data are normalized to obtain the weight value corresponding to the initial filled interpolated data.

[0021] The weight values ​​corresponding to the preliminary filling interpolation data include the horizontal preliminary filling result weight value and the vertical preliminary filling result weight value.

[0022] Optionally, calculating the total distance between the horizontal interpolated data and the missing data, and the total distance between the vertical interpolated data and the missing data, includes:

[0023] Calculate the horizontal coordinate distance between each data point in the horizontal interpolation data and the missing data;

[0024] The lateral inverse distance weight of each data point in the lateral interpolation data is determined based on the lateral coordinate distance.

[0025] Based on the horizontal inverse distance weight, determine the total distance between the horizontal interpolated data and the missing data;

[0026] Calculate the vertical coordinate distance between each data point in the vertical interpolation data and the missing data;

[0027] The longitudinal inverse distance weight of each data point in the longitudinal interpolation data is determined based on the longitudinal coordinate distance.

[0028] The total distance between the horizontal interpolated data and the missing data is determined based on the vertical inverse distance weight.

[0029] Optionally, determining the fill value corresponding to the missing data based on the preliminary fill interpolation data and the weight value corresponding to the preliminary fill interpolation data includes:

[0030] Based on the weight values ​​corresponding to the initial fill difference data, the initial fill interpolation data is weighted and summed to obtain the fill value corresponding to the missing data.

[0031] Secondly, embodiments of this application provide a data filling device for missing data, the device comprising:

[0032] Filler cells are used to pre-fill missing data;

[0033] The data acquisition unit is used to determine interpolation data and interpolation training data corresponding to the pre-filled missing data based on the pre-filled missing data; the interpolation data is the data in the row and column where the pre-filled missing data is located in the data matrix; the interpolation data includes horizontal interpolation data and vertical interpolation data; the interpolation training data is the complete row data and complete column data parallel to the interpolation data in the data matrix where the pre-filled missing data is located; the interpolation training data includes horizontal interpolation training data and vertical interpolation training data;

[0034] Generative units are used to generate adversarial network models;

[0035] A partitioning unit is used to partition the interpolation training data according to a preset length to obtain interpolation training vectors;

[0036] Training unit, used to train the adversarial network model based on the interpolation training vector;

[0037] The input unit is used to input the interpolated data into the trained adversarial network model to obtain preliminary filled interpolated data;

[0038] A calculation unit is used to determine the weight value corresponding to the preliminary fill interpolation data; and to determine the fill value corresponding to the missing data based on the preliminary fill interpolation data and the weight value corresponding to the preliminary fill interpolation data.

[0039] Optionally, the generation unit is specifically used for:

[0040] Generate multiple sets of horizontal vectors and multiple sets of vertical vectors;

[0041] A lateral adversarial network model is generated based on the multiple sets of lateral vectors;

[0042] A longitudinal adversarial network model is generated based on the multiple sets of longitudinal vectors.

[0043] Optionally, the partitioning unit is specifically used for:

[0044] The interpolation training vectors include horizontal interpolation training vectors and vertical interpolation training vectors;

[0045] Taking the leftmost data in the horizontal interpolation training data as the starting point, the vector is divided to the right according to the preset length to obtain the horizontal interpolation training vector;

[0046] Using the topmost data in the vertical interpolation training data as the starting point, the vector is divided downwards according to the preset length to obtain the vertical interpolation training vector.

[0047] Optionally, the computing unit is specifically used for:

[0048] Calculate the total distance between the horizontal interpolated data and the missing data, and the total distance between the vertical interpolated data and the missing data;

[0049] The total distance between the horizontal interpolated data and the missing data and the total distance between the vertical interpolated data and the missing data are normalized to obtain the weight value corresponding to the initial filled interpolated data.

[0050] The weight values ​​corresponding to the preliminary filling interpolation data include the horizontal preliminary filling result weight value and the vertical preliminary filling result weight value.

[0051] Thirdly, embodiments of this application provide a computer-readable storage medium including computer operation instructions, which, when executed on a computer, cause the computer to perform any of the aforementioned methods for filling in missing data.

[0052] Compared with the prior art, this application has the following advantages:

[0053] By pre-filling missing data, an adversarial network model is built by making full use of existing data. The corresponding adversarial network model is trained by horizontal and vertical interpolation training vectors. The interpolated data is input into the trained adversarial network model to obtain relatively accurate preliminary interpolated data. The weight values ​​corresponding to the preliminary interpolated data are determined. Based on the preliminary interpolated data and the corresponding weight values, the filling values ​​corresponding to the missing data are determined. Thus, by pre-filling the missing data, the accuracy of the preliminary interpolated data is improved, and more accurate filling values ​​are obtained. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0055] Figure 1 A flowchart illustrating a method for filling in missing data provided in an embodiment of this application;

[0056] Figure 2 This is a schematic diagram of a missing data filling device provided in an embodiment of this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] There are two main methods for handling missing values ​​in existing technologies: one is to directly delete the data item containing the missing value, and the other is to impute the missing value. Direct deletion, while simple, leads to data waste and the loss of a large amount of important information, and therefore is not commonly used in practice. Existing methods for imputing missing values ​​include zero imputation, mean imputation, regression imputation, cluster imputation, and KNN imputation. These methods are greatly affected by the missing value rate, cannot fully reflect the relationships between data points, have poor variability in the imputation results, and have low imputation efficiency and accuracy.

[0060] To address the above problems, embodiments of this application provide a method for filling in missing data, such as... Figure 1 As shown, the method includes:

[0061] S101: Pre-fill missing data.

[0062] Missing data is pre-filled with a value of 0. For example, when the missing data is 'a', the data matrix [x1 a x3] is pre-filled to obtain the data matrix [x1 0 x3].

[0063] S102: Based on the pre-filled missing data, determine the interpolation data and interpolation training data corresponding to the pre-filled missing data; the interpolation data is the data in the row and column where the pre-filled missing data is located in the data matrix; the interpolation data includes horizontal interpolation data and vertical interpolation data; the interpolation training data is the complete row data and complete column data parallel to the interpolation data in the data matrix where the pre-filled missing data is located; the interpolation training data includes horizontal interpolation training data and vertical interpolation training data.

[0064] Determine the interpolated data and interpolation training data corresponding to the pre-impaired missing data. The interpolated data includes horizontal interpolated data and vertical interpolated data. The horizontal interpolated data is the data in the row containing the pre-impaired missing data in the data matrix, and the vertical interpolated data is the data in the column containing the pre-impaired missing data in the data matrix.

[0065] The interpolation training data includes horizontal and vertical interpolation training data. Horizontal interpolation training data consists of complete rows of data parallel to the horizontal interpolation data in the data matrix containing the pre-filled missing data, while vertical interpolation training data consists of complete columns of data parallel to the vertical interpolation data in the data matrix containing the pre-filled missing data. Thus, by extracting horizontal and vertical interpolation data from the data matrix centered on the missing values, full utilization of existing data resources is achieved.

[0066] For example, when the data matrix has three rows and three columns, if the first row of the data matrix is ​​[x11 x12 x13], the second row of the data matrix is ​​[x21 a x23], and the third row of the data matrix is ​​[x31 x32 x33], and 'a' represents missing data, then the horizontal interpolation data is [x21 a x23], the vertical interpolation data is [x12 a x32], the horizontal interpolation training data is [x11 x12 x13] and [x31 x32 x33], and the vertical interpolation training data is [x11 x21 x31] and [x13 x23 x33].

[0067] S103: Generative Adversarial Network Model.

[0068] Build an adversarial network model.

[0069] In one alternative embodiment, the generative adversarial network model includes:

[0070] Generate lateral adversarial network (LAN) and longitudinal adversarial network (LAN) models. Each LAN and LAN model consists of a generator and a discriminator. The generator uses mean squared error as the loss function, and the discriminator uses cross-entropy loss. The generator is an autoencoder with a self-attention mechanism.

[0071] S104: Divide the interpolation training data according to the preset length to obtain the interpolation training vector.

[0072] The horizontal interpolation training data and the vertical interpolation training data are divided according to a preset length to obtain the interpolation training vectors corresponding to the horizontal interpolation training data and the vertical interpolation training data.

[0073] In one optional embodiment, the interpolation training data is divided according to a preset length to obtain interpolation training vectors, including:

[0074] Interpolation training vectors include horizontal interpolation training vectors and vertical interpolation training vectors;

[0075] Starting from the leftmost data in the horizontal interpolation training data, the vector is divided to the right according to a preset length to obtain the horizontal interpolation training vector.

[0076] Starting with the topmost data point in the vertical interpolation training data, vectors are partitioned downwards according to a preset length to obtain the vertical interpolation training vectors. In the adversarial network model, the autoencoder divides the input data into several groups of vectors of the preset length. When partitioning the horizontal vectors, the leftmost data point in the horizontal interpolation training data is used as the starting point, and partitioning proceeds to the right according to the preset length. Except for the last group of horizontal vectors, adjacent horizontal vectors do not overlap. If the length of the last group of horizontal vectors cannot reach the preset length, it is padded by borrowing data from the left to ensure that the padded last group of horizontal vectors meets the preset length. This introduces a self-attention mechanism to improve the accuracy of the initial padding results of the autoencoder.

[0077] For example, if the data in the horizontal interpolation training data are [x1 x2 x3 x4 x5 x6 x7] from left to right, and the preset length is 2, then x1 is used as the starting point to divide to the right. The first group of horizontal vectors is [x1 x2], the second group of horizontal vectors is [x3 x4], and the third group of horizontal vectors is [x5 x6]. At this time, the remaining data in the row where the horizontal data is located cannot reach the preset length, so one data needs to be borrowed from the left. Therefore, the fourth group of horizontal vectors is [x6 x7].

[0078] When dividing the vertical vectors, the topmost data in the vertical interpolation training data is used as the starting point for the division, and the vectors are divided downwards according to the preset length. Except for the last group of vertical vectors, there is no overlap between adjacent vertical vectors. If the length of the last group of vertical vectors cannot reach the preset length, the last group of vertical vectors is made to borrow data upwards to fill in the gaps, so that the last group of vertical vectors after filling in the gaps meets the preset length.

[0079] It should be noted that the preset length can be determined based on actual needs.

[0080] S105: Train the adversarial network model based on the interpolated training vectors.

[0081] The missing data is filled in by the generator in the adversarial network model. The horizontal adversarial network model is trained by using horizontal interpolation training vectors, and the vertical adversarial network model is trained by using vertical interpolation training vectors, resulting in the trained horizontal adversarial network model and the trained vertical adversarial network model.

[0082] During training, the discriminator determines whether the horizontal and vertical data generated by the generator are real horizontal and vertical data. The generator and discriminator are updated synchronously until the discriminator can no longer determine whether the horizontal and vertical data generated by the generator are real horizontal and vertical data. At this point, the corresponding generator is taken as the horizontal and vertical adversarial network model.

[0083] S106: Input the interpolated data into the trained adversarial network model to obtain preliminary filled interpolated data.

[0084] The preliminary filling interpolation data includes the horizontal preliminary filling results and the vertical preliminary filling results.

[0085] The horizontal interpolation data is input into the horizontal adversarial network model to obtain the preliminary horizontal filling result f1, and the vertical interpolation data is input into the vertical adversarial network model to obtain the preliminary vertical filling result f2.

[0086] S107: Determine the weight values ​​corresponding to the initial interpolation data.

[0087] Determine the weight values ​​for the horizontal preliminary filling results and the vertical preliminary filling results respectively.

[0088] In one optional embodiment, determining the weight values ​​corresponding to the initial fill interpolation data includes:

[0089] Calculate the total distance between the horizontally interpolated data and the missing data, and the total distance between the vertically interpolated data and the missing data;

[0090] The total distance between the horizontal interpolated data and the missing data, and the total distance between the vertical interpolated data and the missing data are normalized to obtain the weight values ​​corresponding to the preliminary filling interpolated data. The weight values ​​corresponding to the preliminary filling interpolated data include the horizontal preliminary filling result weight value and the vertical preliminary filling result weight value.

[0091] According to the formula The total distances between the horizontally interpolated data and the missing data, as well as the total distances between the vertically interpolated data and the missing data, are normalized, with r ranging from 1 to 2. When calculating the weight values ​​for the initial horizontal imputation results, r = 1, according to the formula... Calculate the weight values ​​of the preliminary horizontal imputation results. Where d1 is the total distance between the horizontal interpolated data and the missing data, d2 is the total distance between the vertical interpolated data and the missing data, and ω1 is the weight value of the preliminary horizontal imputation results. When calculating the weight values ​​of the preliminary vertical imputation results, when r = 2, according to the formula... Calculate the weight value of the preliminary vertical filling result. Where ω2 is the weight value of the preliminary vertical filling result.

[0092] In one optional embodiment, calculating the total distance between the horizontally interpolated data and the missing data, and the total distance between the vertically interpolated data and the missing data, includes:

[0093] Calculate the horizontal coordinate distance between each data point in the horizontally interpolated data and the missing data;

[0094] The lateral inverse distance weights of each data point in the lateral interpolation data are determined based on the lateral coordinate distance.

[0095] The total distance between the interpolated data and the missing data is determined based on the horizontal inverse distance weight.

[0096] Calculate the vertical coordinate distance between each data point in the vertical interpolation data and the missing data;

[0097] The vertical inverse distance weights of each data point in the vertical interpolation data are determined based on the vertical coordinate distance.

[0098] The total distance between the horizontal interpolated data and the missing data is determined based on the vertical inverse distance weight.

[0099] According to the formula Calculate the lateral inverse distance weights for each data point in the lateral interpolation data, where λ i h represents the inverse lateral distance weights for each data point in the lateral interpolation data. i represents the coordinate distance between the coordinates of each data matrix in the horizontal interpolation data and the coordinates of the missing data matrix, where i represents the i-th horizontal interpolation data and n represents the total number of horizontal interpolation data.

[0100] According to the formula d1=∑ tx row，i *λ i Determine the total distance between the interpolated data and the missing data, where d1 is the total distance between the interpolated data and the missing data, and x... row，i Let t be the i-th lateral interpolation data, and t be the total number of lateral inverse distance weights.

[0101] According to the formula Calculate the longitudinal inverse distance weights for each data point in the longitudinal interpolation data, where λ j h represents the inverse longitudinal distance weights for each data point in the longitudinal interpolation data. j represents the coordinate distance between the coordinates of each data matrix in the vertical interpolation data and the coordinates of the missing data matrix, where j represents the j-th vertical interpolation data and m represents the total number of vertical interpolation data.

[0102] According to the formula d2=∑ e x row，j *λ j Determine the total distance between the interpolated data and the missing data, where d2 is the total distance between the interpolated data and the missing data, and x... row，j Let j be the j-th horizontal interpolation data, and e be the total number of vertical inverse distance weights.

[0103] By dividing the weight calculation into two steps, firstly, the distance between the interpolated data and the missing values ​​is considered, and each interpolated data is assigned an inverse distance weight (the farther the interpolated data is from the missing value, the smaller the weight). Secondly, the amount of interpolated data in both the horizontal and vertical directions is considered, and the total distance between the horizontal and vertical interpolated data and the missing value data is normalized to obtain the weights of the preliminary horizontal and vertical filling results.

[0104] S108: Determine the filling value corresponding to the missing data based on the preliminary filling interpolation data and the weight value corresponding to the preliminary filling interpolation data.

[0105] The imputation values ​​for missing data are calculated based on the initial imputation interpolation and the weight values ​​corresponding to the initial imputation interpolation data.

[0106] In one optional embodiment, determining the imputation value corresponding to the missing data based on the preliminary imputation interpolation data and the weight value corresponding to the preliminary imputation interpolation data includes:

[0107] Based on the weight values ​​corresponding to the initial filled interpolation data, the initial filled interpolation data is weighted and summed to obtain the filled values ​​corresponding to the missing data.

[0108] The preliminary imputation interpolation data are weighted and summed according to the formula F = ω1*f1 + ω2*f2 to determine the imputation value corresponding to the missing data, where F is the imputation value corresponding to the missing data. By weighting and summing the weighted values ​​of the preliminary imputation results in both the horizontal and vertical directions, overfitting is avoided and the variability of the imputation results is improved.

[0109] To make it easier to understand, the following example is provided:

[0110] Taking a 14*12 matrix as an example, if xk and k are taken as missing data, the matrix is:

[0111]

[0112] A: x k，k Pre-fill with zeros to obtain the complete data matrix.

[0113] B: {x k，1 , ..., x k，k-1 ,0,x k，k+1 , ..., x k，12} serves as the horizontal interpolation data corresponding to the missing data. The horizontal interpolation training data consists of other complete rows of data, with {x} as the interpolation data. 1，k , ..., x k-1，k ,0,x k+1，k , ..., x 14，k The vertical interpolation data corresponding to the missing data is the other complete column data.

[0114] C: Construct a horizontal adversarial network model and a vertical adversarial network model, with a preset length of 3. The generator's encoder divides each set of horizontal interpolation training data into 4 vectors of length 3. Taking the first row as an example, the 4 vectors are (x 1，1 x 1，2 x 1，3 ), (x 1，4 x 1，5 x 1，6 ), (x 1，7 x 1，8 x 1，9 ), (x 1，10 x 1，11 x 1，12 Each set of vertical interpolation training data is divided into 5 vectors of length 3. Taking the first column as an example, the 5 vectors are (x... 1，1 x 2，1 x 3，1 ), (x 4，1 x 5，1 x 6，1 ), (x 7，1 x 8，1 x9，1 ), (x 10，1 x 11，1 x 12，1 ), (x 12，1 x 13，1 x 14，1 ).

[0115] D: Train the horizontal adversarial network model using horizontal interpolation training vectors, and train the vertical adversarial network model using vertical interpolation training vectors. During training, the generator and discriminator are updated synchronously and iteratively until the discriminator can no longer distinguish whether the horizontal / vertical data generated by the generator is real horizontal / vertical data. At this point, the generator can be used as an adversarial network model.

[0116] E: Input the horizontal interpolation data into the trained horizontal adversarial network model to obtain the preliminary horizontal filling result f1, and input the vertical interpolation data into the trained vertical adversarial network model to obtain the preliminary vertical filling result f2.

[0117] F: According to the formula Calculate the lateral inverse distance weights of each data point in the lateral interpolation data using the formula d1=∑ t x row，i *λ i Determine the total distance between the interpolated data and the missing data. According to the formula... Calculate the longitudinal inverse distance weights of each data point in the longitudinal interpolation data using the formula d2=∑ e x row，j *λ j Determine the total distance between the interpolated data and the missing data according to the formula. The total distance between the horizontal interpolated data and the missing data, as well as the total distance between the vertical interpolated data and the missing data, are normalized to obtain the weight values ​​of the preliminary horizontal filling result and the preliminary vertical filling result.

[0118] G: The preliminary interpolated data is weighted and summed according to the formula F=ω1*f1+ω2*f2 to obtain the filling value corresponding to the missing data.

[0119] By pre-filling missing data, an adversarial network model is built by making full use of existing data. The corresponding adversarial network model is trained by horizontal and vertical interpolation training vectors. The interpolated data is input into the trained adversarial network model to obtain relatively accurate preliminary interpolated data. The weight values ​​corresponding to the preliminary interpolated data are determined. Based on the preliminary interpolated data and the corresponding weight values, the filling values ​​corresponding to the missing data are determined. Thus, by pre-filling the missing data, the accuracy of the preliminary interpolated data is improved, and more accurate filling values ​​are obtained.

[0120] like Figure 2 As shown, this application embodiment provides a data filling device for missing data, the device comprising:

[0121] Filling unit 201 is used to pre-fill missing data;

[0122] The data acquisition unit 202 is used to determine the interpolation data and interpolation training data corresponding to the pre-filled missing data based on the pre-filled missing data. The interpolation data is the data in the row and column where the pre-filled missing data is located in the data matrix. The interpolation data includes horizontal interpolation data and vertical interpolation data. The interpolation training data is the complete row and column data parallel to the interpolation data in the data matrix where the pre-filled missing data is located. The interpolation training data includes horizontal interpolation training data and vertical interpolation training data.

[0123] Generation unit 203 is used to generate adversarial network models;

[0124] The partitioning unit 204 is used to partition the interpolation training data according to a preset length to obtain the interpolation training vector;

[0125] Training unit 205 is used to train the adversarial network model based on the interpolated training vectors;

[0126] Input unit 206 is used to input interpolated data into the trained adversarial network model to obtain preliminary filled interpolated data;

[0127] The calculation unit 207 is used to determine the weight value corresponding to the preliminary filling interpolation data; and to determine the filling value corresponding to the missing data based on the preliminary filling interpolation data and the weight value corresponding to the preliminary filling interpolation data.

[0128] Optionally, generation unit 203 is specifically used for:

[0129] Generate multiple sets of horizontal vectors and multiple sets of vertical vectors;

[0130] Generate a lateral adversarial network model based on multiple sets of lateral vectors;

[0131] A longitudinal adversarial network model is generated based on multiple sets of longitudinal vectors.

[0132] Optionally, partitioning unit 204 is specifically used for:

[0133] In the data matrix containing pre-filled missing data, starting from the leftmost data in the row containing the horizontal data, the vector is divided to the right according to the preset length to obtain multiple sets of horizontal vectors.

[0134] In the data matrix containing the pre-filled missing data, the top of the column containing the vertical data is used as the starting point, and the vectors are divided downwards according to the preset length to obtain multiple sets of vertical vectors.

[0135] Optionally, the computing unit 207 is specifically used for:

[0136] Calculate the total distance between the horizontally interpolated data and the missing data, and the total distance between the vertically interpolated data and the missing data;

[0137] The total distance between the horizontal interpolated data and the missing data and the total distance between the vertical interpolated data and the missing data are normalized to obtain the weight values ​​corresponding to the initial interpolated data.

[0138] The weight values ​​corresponding to the initial filling interpolation data include the weight values ​​of the horizontal initial filling results and the weight values ​​of the vertical initial filling results.

[0139] Optionally, the computing unit 207 is specifically used for:

[0140] Calculate the horizontal coordinate distance between each data point in the horizontal interpolation data and the missing data;

[0141] The lateral inverse distance weight of each data point in the lateral interpolation data is determined based on the lateral coordinate distance.

[0142] Based on the horizontal inverse distance weight, determine the total distance between the horizontal interpolated data and the missing data;

[0143] Calculate the vertical coordinate distance between each data point in the vertical interpolation data and the missing data;

[0144] The longitudinal inverse distance weight of each data point in the longitudinal interpolation data is determined based on the longitudinal coordinate distance.

[0145] The total distance between the horizontal interpolated data and the missing data is determined based on the vertical inverse distance weight.

[0146] Optionally, the computing unit 207 is specifically used for:

[0147] Based on the weight values ​​corresponding to the initial fill difference data, the initial fill interpolation data is weighted and summed to obtain the fill value corresponding to the missing data.

[0148] By pre-filling missing data, an adversarial network model is built by making full use of existing data. The corresponding adversarial network model is trained by horizontal and vertical interpolation training vectors. The interpolated data is input into the trained adversarial network model to obtain relatively accurate preliminary interpolated data. The weight values ​​corresponding to the preliminary interpolated data are determined. Based on the preliminary interpolated data and the corresponding weight values, the filling values ​​corresponding to the missing data are determined. Thus, by pre-filling the missing data, the accuracy of the preliminary interpolated data is improved, and more accurate filling values ​​are obtained.

[0149] This application provides a computer-readable storage medium including computer operation instructions. When the computer operation instructions are executed on a computer, the computer performs any of the above-described methods for filling in missing data.

[0150] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0151] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for filling missing data, characterized in that, The filling method includes: The missing data is pre-filled to obtain pre-filled missing data; Based on the pre-filled missing data, interpolation data and interpolation training data corresponding to the pre-filled missing data are determined; the interpolation data is the data in the row and column where the pre-filled missing data is located in the data matrix; the interpolation data includes horizontal interpolation data and vertical interpolation data; the interpolation training data is the complete row and column data parallel to the interpolation data in the data matrix where the pre-filled missing data is located; the interpolation training data includes horizontal interpolation training data and vertical interpolation training data; A generative adversarial network (GAN) model is provided, comprising a generator and a discriminator; wherein the generator is an autoencoder incorporating a self-attention mechanism; wherein the GAN model comprises: training a horizontal adversarial network model using horizontal interpolation training vectors, and training a vertical adversarial network model using vertical interpolation training vectors, to obtain a trained horizontal adversarial network model and a trained vertical adversarial network model, wherein the adversarial network model comprises a trained horizontal adversarial network model and a trained vertical adversarial network model; The interpolation training data is divided according to a preset length to obtain interpolation training vectors; The adversarial network model is trained by synchronously iteratively updating the generator and the discriminator based on the interpolated training vector until the discriminator can no longer distinguish between the interpolated training vector generated by the generator and the real interpolated training vector. The interpolated data is input into the trained adversarial network model to obtain preliminary filled interpolated data; Determine the weight value corresponding to the preliminary interpolated data; wherein, determining the weight value corresponding to the preliminary interpolated data includes: calculating the total distance between the horizontal interpolated data and the missing data, and the total distance between the vertical interpolated data and the missing data; The total distance between the horizontal interpolated data and the missing data and the total distance between the vertical interpolated data and the missing data are normalized to obtain the weight value corresponding to the initial filled interpolated data. The weight values ​​corresponding to the preliminary filling interpolation data include the horizontal preliminary filling result weight value and the vertical preliminary filling result weight value; Based on the preliminary fill interpolation data and the weight values ​​corresponding to the preliminary fill interpolation data, the fill value corresponding to the missing data is determined.

2. The filling method according to claim 1, characterized in that, The step of dividing the interpolation training data according to a preset length to obtain interpolation training vectors includes: The interpolation training vectors include horizontal interpolation training vectors and vertical interpolation training vectors; Taking the leftmost data in the horizontal interpolation training data as the starting point, the vector is divided to the right according to the preset length to obtain the horizontal interpolation training vector; Using the topmost data in the vertical interpolation training data as the starting point, the vector is divided downwards according to the preset length to obtain the vertical interpolation training vector.

3. The filling method according to claim 1, characterized in that, The calculation of the total distance between the horizontal interpolated data and the missing data, and the total distance between the vertical interpolated data and the missing data, includes: Calculate the horizontal coordinate distance between each data point in the horizontal interpolation data and the missing data; The lateral inverse distance weight of each data point in the lateral interpolation data is determined based on the lateral coordinate distance. Based on the horizontal inverse distance weight, determine the total distance between the horizontal interpolated data and the missing data; Calculate the vertical coordinate distance between each data point in the vertical interpolation data and the missing data; The longitudinal inverse distance weight of each data point in the longitudinal interpolation data is determined based on the longitudinal coordinate distance. The total distance between the horizontal interpolated data and the missing data is determined based on the vertical inverse distance weight.

4. The filling method according to claim 3, characterized in that, The step of determining the imputation value corresponding to the missing data based on the preliminary imputation interpolation data and the weight value corresponding to the preliminary imputation interpolation data includes: Based on the weight values ​​corresponding to the initial fill interpolation data, the initial fill interpolation data is weighted and summed to obtain the fill value corresponding to the missing data.

5. A device for filling missing data, characterized in that, The device includes: Filler cells are used to pre-fill missing data; The data acquisition unit is used to determine interpolation data and interpolation training data corresponding to the pre-filled missing data based on the pre-filled missing data; the interpolation data is the data in the row and column where the pre-filled missing data is located in the data matrix; the interpolation data includes horizontal interpolation data and vertical interpolation data; the interpolation training data is the complete row data and complete column data parallel to the interpolation data in the data matrix where the pre-filled missing data is located; the interpolation training data includes horizontal interpolation training data and vertical interpolation training data; A generation unit is used to generate an adversarial network model, the adversarial network model including a generator and a discriminator; wherein, the generator is an autoencoder with a self-attention mechanism; wherein, the generative adversarial network model includes: training a horizontal adversarial network model using horizontal interpolation training vectors, and training a vertical adversarial network model using vertical interpolation training vectors, to obtain a trained horizontal adversarial network model and a trained vertical adversarial network model, the adversarial network model including the trained horizontal adversarial network model and the trained vertical adversarial network model; A partitioning unit is used to partition the interpolation training data according to a preset length to obtain interpolation training vectors; The training unit is used to train the adversarial network model based on the interpolated training vector through synchronous iterative updates of the generator and the discriminator, until the discriminator can no longer distinguish between the interpolated training vector generated by the generator and the real interpolated training vector. The input unit is used to input the interpolated data into the trained adversarial network model to obtain preliminary filled interpolated data; A calculation unit is used to determine the weight value corresponding to the preliminary fill interpolation data; and to determine the fill value corresponding to the missing data based on the preliminary fill interpolation data and the weight value corresponding to the preliminary fill interpolation data. Specifically, the calculation unit is used to: calculate the total distance between the horizontal interpolated data and the missing data, and the total distance between the vertical interpolated data and the missing data; The total distance between the horizontal interpolated data and the missing data and the total distance between the vertical interpolated data and the missing data are normalized to obtain the weight value corresponding to the initial filled interpolated data. The weight values ​​corresponding to the preliminary filling interpolation data include the horizontal preliminary filling result weight value and the vertical preliminary filling result weight value.

6. The filling device according to claim 5, characterized in that, The partitioning unit is specifically used for: The interpolation training vectors include horizontal interpolation training vectors and vertical interpolation training vectors; Taking the leftmost data in the horizontal interpolation training data as the starting point, the vector is divided to the right according to the preset length to obtain the horizontal interpolation training vector; Using the topmost data in the vertical interpolation training data as the starting point, the vector is divided downwards according to the preset length to obtain the vertical interpolation training vector.

Images