Data repair method and device, computer device and storage medium

Abstract

The application relates to a data repairing method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining sample data to be repaired, internal feature items and a feature similarity matrix, determining a data repairing item according to the missing number and missing position of missing attribute data in the sample data to be repaired, fusing the sample data to be repaired, the internal feature items and the data repairing item to obtain a target difference item, determining a constraint condition based on the data repairing matrix, the internal feature items and the feature similarity matrix, fusing the target difference item and the constraint condition to obtain a target iteration item, finally, iteratively determining a target data repairing matrix for the target iteration item, and obtaining target repaired sample data according to the target data repairing matrix, a data repairing index matrix and the sample data to be repaired. The method can effectively improve the accuracy of data repairing.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data recovery method, apparatus, computer device, and storage medium. Background Technology

[0002] With the development of computer technology, the technology of accurately and effectively repairing and supplementing missing data when faced with data gaps in multi-perspective data samples is an important technical component in the field of text data mining.

[0003] In traditional techniques, the restoration of multi-view data samples is achieved by using deep learning networks to roughly estimate the missing data of the multi-view data samples, resulting in low data restoration accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide a data repair method, apparatus, computer equipment, or computer-readable storage medium to address the aforementioned technical problems and improve the accuracy of data repair.

[0005] A data repair method includes:

[0006] Obtain the sample data to be repaired, internal feature terms, and feature similarity matrix. The internal feature terms are used to characterize the internal distribution characteristics of the missing attribute data in the sample data to be repaired, and the feature similarity matrix is ​​used to characterize the degree of similarity between the column vectors in the corresponding internal feature terms.

[0007] The data repair items are determined based on the number and location of missing attribute data in the sample data to be repaired. The data repair items include a data repair matrix and a data repair index matrix. The data repair index matrix is ​​used to represent the correspondence between the elements of the data repair matrix and the missing locations.

[0008] The target difference term is obtained by fusing the sample data to be repaired, internal feature terms, and data repair terms. The target difference term is used to characterize the degree of difference between the internal features and internal feature terms of the repaired complete sample data.

[0009] Constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix.

[0010] The target iteration term is obtained by fusing the target difference term and constraint conditions.

[0011] The target data repair matrix is ​​determined by iterating over the target iteration term, and the target repair sample data is obtained based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired.

[0012] In one embodiment, determining data repair items based on the number and location of missing attribute data in the sample data to be repaired includes:

[0013] The dimension of the data repair matrix is ​​determined by the number of missing attribute data in the sample data to be repaired, and random values ​​are assigned to each element in the data repair matrix to obtain the data repair matrix.

[0014] The value of each element in the data repair index matrix is ​​determined based on the location of the missing attribute data in the sample data to be repaired.

[0015] Data repair items are obtained by fusing the data repair index matrix and the data repair matrix.

[0016] In one embodiment, the target difference item is obtained by fusing the sample data to be repaired, internal feature items, and data repair items, including:

[0017] The sample data to be repaired is merged with the data repair items to obtain the repaired sample;

[0018] The target difference term is obtained by calculating the difference between the repaired sample and the internal feature term.

[0019] In one embodiment, constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix, including:

[0020] Obtain the dimensional similarity matrix, which is determined based on the similarity of the row vector data of the sample data to be repaired;

[0021] The data repair difference term is obtained by calculating the difference between the row vectors of the data repair matrix;

[0022] The first constraint is obtained by fusing the data repair difference term with the dimensional similarity matrix.

[0023] In one embodiment, constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix, including:

[0024] The internal feature difference term is calculated based on the difference between the column vectors of the internal feature terms;

[0025] The second constraint is obtained by fusing the internal feature difference term with the feature similarity matrix.

[0026] In one embodiment, constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix, including:

[0027] The feature similarity difference value is obtained by calculating the difference between each row vector of the feature similarity matrix;

[0028] The third constraint is obtained by fusing the feature similarity difference values.

[0029] In one embodiment, the target data repair matrix is ​​determined by iteratively processing the target iteration term, and the target repair sample data is obtained based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired, including:

[0030] Calculate the iteration value of the target iteration term under the preset conditions, which are that each row vector of the feature similarity matrix is ​​a unit vector and all elements are greater than or equal to 0;

[0031] When the iteration value is less than the preset threshold, the iteration stops and the target data repair matrix is ​​obtained;

[0032] The target data repair items are obtained based on the target data repair matrix and the data repair index matrix;

[0033] The target repair sample data is obtained by fusing the target data repair items with the sample data to be repaired.

[0034] A data repair device, comprising:

[0035] The data acquisition module is used to acquire the sample data to be repaired, internal feature terms, and feature similarity matrix. The internal feature terms are used to characterize the internal distribution characteristics of the missing attribute data in the sample data to be repaired, and the feature similarity matrix is ​​used to characterize the similarity between the column vectors in the corresponding internal feature terms. The data repair terms are determined based on the number and location of the missing attribute data in the sample data to be repaired. The data repair terms include a data repair matrix and a data repair index matrix. The data repair index matrix is ​​used to characterize the correspondence between the elements of the data repair matrix and the missing locations.

[0036] The target difference item determination module is used to obtain the target difference item based on the fusion of the sample data to be repaired, internal feature items, and data repair items. The target difference item is used to characterize the degree of difference between the internal features and internal feature items of the repaired complete sample data.

[0037] The constraint construction module is used to determine constraints based on the data repair matrix, internal feature terms, and feature similarity matrix.

[0038] The target iteration term generation module is used to obtain the target iteration term based on the fusion of target difference terms and constraint conditions.

[0039] The target repair sample data determination module is used to iterate the target iteration term to determine the target data repair matrix, and obtain the target repair sample data based on the target data repair matrix, the data repair index matrix and the sample data to be repaired.

[0040] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0041] Obtain the sample data to be repaired, internal feature terms, and feature similarity matrix. The internal feature terms are used to characterize the internal distribution characteristics of the missing attribute data in the sample data to be repaired, and the feature similarity matrix is ​​used to characterize the degree of similarity between the column vectors in the corresponding internal feature terms.

[0042] The data repair items are determined based on the number and location of missing attribute data in the sample data to be repaired. The data repair items include a data repair matrix and a data repair index matrix. The data repair index matrix is ​​used to represent the correspondence between the elements of the data repair matrix and the missing locations.

[0043] The target difference term is obtained by fusing the sample data to be repaired, internal feature terms, and data repair terms. The target difference term is used to characterize the degree of difference between the internal features and internal feature terms of the repaired complete sample data.

[0044] Constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix.

[0045] The target iteration term is obtained by fusing the target difference term and constraint conditions.

[0046] The target data repair matrix is ​​determined by iterating over the target iteration term, and the target repair sample data is obtained based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired.

[0047] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0048] Obtain the sample data to be repaired, internal feature terms, and feature similarity matrix. The internal feature terms are used to characterize the internal distribution characteristics of the missing attribute data in the sample data to be repaired, and the feature similarity matrix is ​​used to characterize the degree of similarity between the column vectors in the corresponding internal feature terms.

[0049] The data repair items are determined based on the number and location of missing attribute data in the sample data to be repaired. The data repair items include a data repair matrix and a data repair index matrix. The data repair index matrix is ​​used to represent the correspondence between the elements of the data repair matrix and the missing locations.

[0050] The target difference term is obtained by fusing the sample data to be repaired, internal feature terms, and data repair terms. The target difference term is used to characterize the degree of difference between the internal features and internal feature terms of the repaired complete sample data.

[0051] Constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix.

[0052] The target iteration term is obtained by fusing the target difference term and constraint conditions.

[0053] The target data repair matrix is ​​determined by iterating over the target iteration term, and the target repair sample data is obtained based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired.

[0054] The aforementioned data repair method, apparatus, computer equipment, and storage medium acquire the sample data to be repaired, internal feature terms, and feature similarity matrix. They determine data repair terms based on the number and location of missing attribute data in the sample data, then fuse the sample data, internal feature terms, and data repair terms to obtain target difference terms. Constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix. Target iteration terms are obtained by fusing the target difference terms and constraints. Finally, the target iteration terms are iterated to determine the target data repair matrix, and the target repaired sample data is obtained based on the target data repair matrix, data repair index matrix, and the sample data to be repaired. In this way, by determining data repair terms based on the number and location of missing attribute data in the sample data, determining constraints based on the data repair matrix, internal feature terms, and feature similarity matrix, and then iterating the target data repair matrix based on the constraints, the target data is repaired. This fully exploits the unique feature distribution within the sample data, considers the consistency of the internal feature distribution and global consistency, ensuring that the final repaired data meets the overall data characteristics of the sample data, effectively improving the accuracy of data repair. Attached Figure Description

[0055] Figure 1 This is a diagram illustrating the application environment of a data repair method in one embodiment;

[0056] Figure 2 This is a flowchart illustrating a data repair method in one embodiment;

[0057] Figure 3 This is a flowchart illustrating a data repair item generation method in one embodiment;

[0058] Figure 4 This is a flowchart illustrating a method for generating target difference items in one embodiment;

[0059] Figure 5 This is a flowchart illustrating the process of generating the first constraint in one embodiment;

[0060] Figure 6 This is a flowchart illustrating the process of generating the second constraint in one embodiment;

[0061] Figure 7 This is a flowchart illustrating the process of generating a third constraint in one embodiment;

[0062] Figure 8 This is a schematic diagram of the process for generating target repair sample data in one embodiment;

[0063] Figure 9 This is a structural block diagram of a data repair device in one embodiment;

[0064] Figure 10 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0066] The data repair method provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, computer device 102 acquires the sample data to be repaired, internal feature terms, and a feature similarity matrix. It then determines data repair terms based on the number and location of missing attribute data in the sample data. A target difference term is obtained by fusing the sample data to be repaired, internal feature terms, and data repair terms. This target difference term characterizes the degree of difference between the internal features and internal feature terms of the repaired complete sample data. Constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix. A target iteration term is then obtained by fusing the target difference term and constraints. The target iteration term is iteratively applied to determine the target data repair matrix. Finally, the target repaired sample data is obtained based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired. Specifically, computer device 102 may include, but is not limited to, various personal computers, laptops, servers, smartphones, tablets, smart cameras, and portable wearable devices.

[0067] In one embodiment, such as Figure 2 As shown, a data repair method is provided, which can be applied to... Figure 1 Taking computer device 102 as an example, the following steps are included:

[0068] Step S202: Obtain the sample data to be repaired, internal feature terms, and feature similarity matrix. The internal feature terms are used to characterize the internal distribution characteristics of the missing attribute data in the sample data to be repaired, and the feature similarity matrix is ​​used to characterize the similarity between the column vectors in the corresponding internal feature terms.

[0069] The sample data to be repaired consists of data samples with missing attributes. It contains diverse features and is multi-perspective data, lacking sample data from certain perspectives. The internal feature terms and elements in the feature similarity matrix are randomly assigned initial values. Multi-perspective data is essentially a collection of multi-source features of an object. For example, an image has features such as color, texture, and shape, while a piece of text has English, French, and Chinese versions. Compared to single-perspective data, multi-perspective data possesses richer semantic information.

[0070] Step S204: Determine data repair items based on the number and location of missing attribute data in the sample data to be repaired. Data repair items include a data repair matrix and a data repair index matrix. The data repair index matrix is ​​used to represent the correspondence between the elements of the data repair matrix and the missing locations.

[0071] Among them, missing attribute data refers to missing data existing in a certain perspective in the sample data to be repaired, missing number refers to the number of missing data in the perspective of the sample data to be repaired, missing location refers to the position of the missing data in the corresponding perspective of the perspective of the sample data to be repaired, and the elements in the data repair matrix serve as the initial values ​​for repairing the missing data in the sample data to be repaired. The elements in the data repair matrix are random values.

[0072] Specifically, the computer equipment determines the dimension and number of matrix elements in the data repair matrix based on the number of missing attribute data in the sample data to be repaired. All matrix elements are assigned random values, and the positions of the elements can be random or arranged according to certain rules, thus obtaining the data repair matrix. Then, the data repair index matrix is ​​determined based on the missing positions of the missing attribute data in the sample data to be repaired. Specifically, the dimension of the data repair index matrix and the size of each element are determined based on the missing positions of the missing data in the sample data to be repaired and the positions of each element in the data repair matrix. This allows each element in the data repair matrix to determine the position of its corresponding missing data in the sample data to be repaired based on the data repair index matrix. Finally, the data repair items are obtained based on the generated data repair index matrix and data repair matrix.

[0073] Step S206: Based on the fusion of the sample data to be repaired, internal feature items, and data repair items, a target difference item is obtained. The target difference item is used to characterize the degree of difference between the internal features and internal feature items of the repaired complete sample data.

[0074] Specifically, the computer equipment fuses the data repair items with the sample data to be repaired to obtain initial repair sample data. Then, the initial repair sample data and the internal feature items are used to calculate the difference to generate a target difference item. The target difference item represents the degree of difference between the internal distribution characteristics of the initial repair sample data and the internal feature items. The larger the result of the target difference item, the larger the gap between the internal distribution characteristics of the initial repair sample data repaired based on the current data repair items and the actual missing data.

[0075] Step S208: Determine the constraints based on the data repair matrix, internal feature terms, and feature similarity matrix.

[0076] The constraint condition is used to combine the iteration of the target difference item obtained in the previous steps to perform the iteration constraint condition, so that if the target difference item does not meet the iteration stopping condition, the iteration is performed under the premise that each iteration parameter meets the constraint condition, and the iteration range is specified for the iteration.

[0077] Specifically, the computer device obtains the first constraint condition for the iteration of the repair matrix based on the distribution characteristics between the row vectors of the data repair matrix, then obtains the second constraint condition for the iteration of the internal feature based on the distribution characteristics between the column vectors of the internal feature terms, and finally generates the third constraint condition for the iteration of the feature similarity matrix through the distribution characteristics between the feature similarity matrices of different perspectives.

[0078] Step S210: Obtain the target iteration term based on the fusion of the target difference term and the constraint conditions.

[0079] Specifically, the computer device can construct the target iteration term by fusing the target difference term with the constraints corresponding to the data repair matrix, internal feature term, and feature similarity matrix respectively, or it can construct the target iteration term by fusing the target difference term with some constraints.

[0080] Step S212: Iterate the target iteration term to determine the target data repair matrix, and obtain the target repair sample data based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired.

[0081] Specifically, the computer device iterates over the target iteration item, calculating the iteration value for each iteration. When the iteration value is less than a preset threshold, the iteration stops, and the data repair matrix at the current iteration number is taken as the target data repair matrix. Then, the target data repair item is generated based on the target data repair matrix and the data repair index matrix. Finally, the target data item is used to repair the sample data to be repaired to obtain the target repaired sample data.

[0082] In this embodiment, by acquiring the sample data to be repaired, internal feature terms, and a feature similarity matrix, data repair terms are determined based on the number and location of missing attribute data in the sample data to be repaired. Then, a target difference term is obtained by fusing the sample data to be repaired, internal feature terms, and data repair terms. Constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix. A target iteration term is obtained by fusing the target difference term and constraints. Finally, the target iteration term is iterated to determine the target data repair matrix, and the target repaired sample data is obtained based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired. In this way, by determining data repair terms based on the number and location of missing attribute data in the sample data to be repaired, and determining constraints based on the data repair matrix, internal feature terms, and feature similarity matrix, and then iterating the target data repair matrix based on the constraints, the target data repair is completed. This fully explores the unique feature distribution within the sample data, considers the consistency of the feature distribution within the sample data as well as the global consistency, and ensures that the final repaired data meets the data characteristics of the overall sample data, effectively improving the accuracy of data repair.

[0083] In one embodiment, such as Figure 3 As shown, the data repair items are determined based on the number and location of missing attribute data in the sample data to be repaired, including:

[0084] Step S302: Determine the dimension of the data repair matrix based on the number of missing attribute data in the sample data to be repaired, and assign random values ​​to each element in the data repair matrix to obtain the data repair matrix.

[0085] Specifically, the computing device obtains the number of missing attribute data in the sample data to be repaired, constructs a data repair matrix with the same number of elements as the number of missing data, and each matrix element serves as the repair data for the corresponding missing data, participating in the iterative process of subsequent steps. Random values ​​are assigned to each element in all data repair matrices to obtain the data repair matrix.

[0086] Step S304: Determine the value of each element in the data repair index matrix based on the missing location of the missing attribute data in the sample data to be repaired.

[0087] Specifically, the computer equipment defines the data repair index matrix W according to the following formula 1. (v) :

[0088]

[0089] in, Indicates the data repair index matrix W (v) The element in the i-th row and j-th column, X (v)For the multi-perspective text dataset {X (1) ,X (2) ,…,X (m) The sample data to be repaired from the vth perspective in}.

[0090] Step S306: Obtain the data repair item based on the fusion of the data repair index matrix and the data repair matrix.

[0091] Specifically, the computer device performs a weighted multiplication of the data repair index matrix and the data repair matrix or directly calculates the product to obtain the data repair item, wherein the position of the element in the data repair matrix in the data repair item corresponds to the position of the missing data in the matrix to be repaired.

[0092] In this embodiment, the computer device determines the dimension of the data repair matrix based on the number of missing attribute data in the sample data to be repaired, and assigns random values ​​to each element in the data repair matrix to obtain the data repair matrix. Then, it determines the value of each element in the data repair index matrix based on the missing position of the missing attribute data in the sample data to be repaired. Finally, it obtains data repair items based on the fusion of the data repair index matrix and the data repair matrix. The data repair matrix and the data repair index matrix are constructed using the number and missing position information of the missing attribute data. Then, the data repair items corresponding to the missing data are generated by multiplying the data repair matrix and the data repair index matrix. This realizes a one-to-one mapping relationship between the repaired data and the missing data, and improves the efficiency of constructing data repair items.

[0093] In one embodiment, such as Figure 4 As shown, the target difference item is obtained by fusing the sample data to be repaired, internal feature items, and data repair items, including:

[0094] Step S402: The sample data to be repaired is merged with the data repair items to obtain the repaired sample.

[0095] Specifically, the computer equipment merges the sample data to be repaired with the data repair items by directly adding or weighted summing to obtain the repaired sample. The repaired sample is obtained by filling the missing data positions in the corresponding sample data to be repaired with the elements in the data repair items, thereby completing the sample repair.

[0096] Step S404: Calculate the target difference term based on the difference between the repaired sample and the internal feature term.

[0097] The repair sample is the sample data that is repaired using the data repair items determined in the above steps, and the elements in the data repair items are randomly assigned elements.

[0098] Specifically, the computer equipment calculates the target difference item according to the following formula 2:

[0099]

[0100] Among them, E (v) For data repair matrix, U (v) X is an internal characteristic term. (v) For the multi-perspective text dataset {X (1) ,X (2) ,…,X (m) The sample data to be repaired from the v-th viewpoint in}, W (v) Repair the index matrix for the data, and E (v) W (v) For data repair, X (v) +E (v) W (v) To repair the sample.

[0101] In this embodiment, a repair sample is obtained by fusing the sample data to be repaired with the data repair items. Then, the target difference item is obtained by calculating the difference between the repair sample and the internal feature items. The correlation between each data repair item and the internal feature items is established by the difference between the repair data and the internal feature items, which effectively improves the efficiency of data difference feedback during the iteration process.

[0102] In one embodiment, such as Figure 5 As shown, the constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix, including:

[0103] Step S502: Obtain the dimensional similarity matrix, which is determined based on the similarity of the row vector data of the sample data to be repaired.

[0104] Specifically, the computer device measures the similarity between different feature dimensions of the sample data to be repaired by constructing a dimensional similarity matrix. This similarity is determined based on the k-nearest neighbor rule between the row vectors of the sample data to be repaired. When the sample data to be repaired X... (v) The i-th row vector With the vector in the j-th row The k-nearest neighbor similarity matrix G (v) The element in the i-th row and j-th column Otherwise, it is 0.

[0105] Step S504: Calculate the data repair difference term based on the difference of the row vectors of the data repair matrix.

[0106] Specifically, the computer equipment constructs the data repair discrepancy item according to the following formula 3:

[0107]

[0108] Where, dv This represents the dimension of the feature matrix for the v-th viewpoint. For data repair matrix E (v) The i-th row vector, For data repair matrix E (v) The j-th row vector.

[0109] Step S506: The first constraint condition is obtained by fusing the data repair difference item and the dimensional similarity matrix.

[0110] Specifically, the computer device performs a weighted fusion or direct multiplication of the data repair difference items obtained in the aforementioned steps with the corresponding dimensional similarity matrix to obtain the first constraint condition. The first constraint condition is constructed according to the following formula 4:

[0111]

[0112] Where Tr(·) represents the matrix trace operation, G represents the dimensional similarity matrix (v) The Laplace matrix, Where D (v) It is a diagonal matrix, and its diagonal values ​​are calculated as follows: d v This represents the dimension of the feature matrix for the v-th viewpoint.

[0113] In this embodiment, a dimensional similarity matrix is ​​obtained, which is determined based on the similarity of the row vector data of the sample data to be repaired. A data repair difference term is calculated based on the difference of the row vectors in the data repair matrix. Finally, the first constraint condition is obtained by fusing the data repair difference term with the dimensional similarity matrix. The first constraint condition is obtained by fusing the distribution characteristics of each row vector in the data repair matrix and the magnitude of the difference between the row vectors, and participates in the subsequent iteration process. This makes the iterated data repair matrix more consistent with the internal distribution characteristics of the original sample data, which can effectively improve the accuracy of the iteration.

[0114] In one embodiment, such as Figure 6 As shown, the constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix, including:

[0115] Step S602: Calculate the internal feature difference term based on the difference of the column vectors of the internal feature terms.

[0116] Among them, the internal feature difference term is used to characterize the distribution characteristics among the column vectors in the internal feature term, such as the difference characteristics among the column vectors.

[0117] Specifically, the computer device calculates the difference between the column vectors of the internal feature terms to obtain the internal feature difference terms.

[0118] For example, the internal characteristic difference term ΔU of a computer device can be calculated as shown in Formula 5 below:

[0119]

[0120] in Represents the internal characteristic term U (v) The i-th column vector, Represents the internal characteristic term U (v) The j-th column vector.

[0121] Step S604: The internal feature difference term is fused with the feature similarity matrix to obtain the second constraint condition.

[0122] Specifically, the computer equipment can be fused to obtain the second constraint condition in the manner shown in Formula 6 below:

[0123]

[0124] in, The feature similarity matrix S (v) The Laplace matrix, D (v) It is a degree matrix, and its calculation method is as follows:

[0125] In this embodiment, the feature distribution between the current column vectors is obtained by calculating the difference between each column vector in the internal feature terms. Then, the second constraint is obtained by fusing the internal feature difference terms with the feature similarity matrix. In this way, the difference calculation of each column vector in the internal feature terms is used to participate in the construction of the second constraint and in the subsequent iteration process, so that the iterated data repair matrix is ​​more consistent with the internal distribution characteristics of the original sample data, which can effectively improve the accuracy of the iteration.

[0126] In one embodiment, such as Figure 7 As shown, the constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix, including:

[0127] Step S702: Calculate the difference based on the row vectors of the feature similarity matrix to obtain the feature similarity difference value.

[0128] Step S704: The feature similarity difference values ​​are fused to obtain the third constraint.

[0129] Specifically, the feature similarity matrix {S} of the computer device corresponding to multiple perspectives with missing data. (1) ,…,S(m) To leverage the complementarity between the feature similarity matrices, enhance their consistency, and avoid losing excessive semantic information, the computer device calculates the difference values ​​between different feature similarity matrices to construct the third constraint. The feature similarity difference values ​​are calculated according to Formula 7 below:

[0130]

[0131] Among them, S (v) Let S represent the feature similarity matrix corresponding to the v-th viewpoint. (w) Let w represent the feature similarity matrix corresponding to the w-th viewpoint.

[0132] In this embodiment, feature similarity difference values ​​are obtained by calculating the differences between each row vector of the feature similarity matrix. Then, the feature similarity difference values ​​are fused to obtain the third constraint. Cross-view graph matching is used to constrain the feature similarity matrix. By calculating the difference values ​​between each feature similarity matrix, the distribution characteristics between feature similarity matrices corresponding to different views are mined, and then the third constraint is constructed to participate in the subsequent iteration process. This makes the iterated data repair matrix more consistent with the internal distribution characteristics of the original sample data, which can effectively improve the accuracy of the iteration.

[0133] In one embodiment, such as Figure 8 As shown, the target data repair matrix is ​​determined by iteratively applying the target iteration term. Based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired, the target repair sample data is obtained, including:

[0134] Step S802: Calculate the iteration value of the target iteration term under the preset condition that each row vector of the feature similarity matrix is ​​a unit vector and all elements are greater than or equal to 0.

[0135] Specifically, the computer device fuses the target difference terms and constraints determined in the aforementioned steps to obtain the target iteration term, which can be constructed in the manner shown in Formula 8 below:

[0136]

[0137] Where E (v) For data repair matrix, U (v) S is an internal characteristic term. (v) Let X be the feature similarity matrix. (v) For the sample data to be repaired, W (v) Repair the index matrix for the data. This is the first constraint condition. This is the second constraint. This is the third constraint. For the target difference term, λ1 and λ2 are positive penalty parameters. Represents the feature similarity matrix S (v) It is a unit vector, and all its elements are greater than or equal to 0.

[0138] Step S804: When the iteration value is less than the preset threshold, stop the iteration and obtain the target data repair matrix.

[0139] Specifically, the computer device can use the alternating multiplier method, specifically solving Equation 8 above according to Equations 9 to 14 to obtain the iterative result:

[0140] Update data repair matrix E (v) Due to matrix E (v) The characteristic matrix X corresponding to all values ​​in (v) The values ​​in are all 0, therefore, regarding E (v) The subproblems are:

[0141]

[0142] For E in Formula 9 (v) Taking the derivative and setting the partial derivative to 0, we get E (v) Update formula:

[0143]

[0144] Update U (v) In fixed formula 8, U (v) Irrelevant variables, about U (v) The subproblems are as follows:

[0145]

[0146] For U in the above formula (v) Taking the derivative and setting the partial derivative to 0, we can obtain U. (v) Update formula:

[0147]

[0148] Update S (v) In fixed formula 8, S (v) Irrelevant variables, regarding S (v) The subproblems are as follows:

[0149]

[0150] make Transform into the following form

[0151]

[0152] Where m represents the number of feature similarity matrices corresponding to each of the m viewpoints.

[0153] When the iteration value calculated by the target iteration term shown in Formula 8 is less than the preset threshold, it indicates that the iteration stopping condition is met, and the iteration is stopped. The data repair matrix under the current iteration number is used as the target data repair matrix.

[0154] Step S806: Obtain the target data repair item based on the target data repair matrix and the data repair index matrix.

[0155] Specifically, the computer device multiplies the target data repair matrix with the data repair index matrix to obtain the target data repair item. Alternatively, they can be fused together in other ways, such as by adding or weighted summation. No specific restrictions are placed on the fusion method here.

[0156] For example, computer equipment can obtain the target data repair item H according to the following formula 15. (v) :

[0157] H (v) =E (v) W (v) , Formula 15

[0158] Among them, E (v) W represents the data restoration matrix corresponding to the v-th viewpoint. (v) This represents the data repair index matrix corresponding to the v-th viewpoint.

[0159] Step S808: Target repair sample data is obtained by fusing the target data repair item with the sample data to be repaired.

[0160] Specifically, the computer equipment can repair the sample data to be repaired according to the target data repair item in the following formula 16:

[0161] X (v) +E (v) W (v) -U (v) , Formula 16

[0162] Where X (v) E represents the sample data to be repaired from the v-th viewpoint. (v) W represents the data restoration matrix corresponding to the v-th viewpoint. (v) U represents the data repair index matrix corresponding to the v-th viewpoint. (v) Let be the internal feature term corresponding to the v-th viewpoint.

[0163] In this embodiment, the iteration value of the target iteration item under the preset conditions is first calculated. When the iteration value is less than the preset threshold, the iteration stops and the target data repair matrix is ​​obtained. The target data repair item is obtained based on the target data repair matrix and the data repair index matrix. The target repair sample data is obtained by fusing the target data repair item with the sample data to be repaired. By judging whether the iteration value meets the iteration stopping condition, the target data repair matrix that meets the internal consistency and global consistency of the multi-view sample data is iterated, so that the accuracy of repairing the sample data to be repaired based on the target data repair matrix is ​​high.

[0164] This application also provides an application scenario in which the above-described data repair method is applied. Specifically, the data repair method is applied in this scenario as follows:

[0165] Computer devices acquire a complete multi-view text dataset as {X} (1) ,X (2) ,…,X (m)},in d v Let X represent the dimension of the feature matrix for the v-th view, n represent the number of samples, and m represent the number of views. For the feature matrix X of the v-th view... (v) The number of missing data points is represented by n. v The elements are filled with zeros. We introduce a matrix. Modeling n v For each missing data point, an index matrix is ​​introduced. Describe the observation data X (v) With missing data E (v) The relationship between W (v) Specifically defined as:

[0166]

[0167] The computer equipment uses the following optimization objectives to recover the missing data for each viewpoint:

[0168]

[0169] Where λ1 is a positive penalty parameter, E (v) W (v) This corresponds to the missing data for each perspective, X. (v) +E (v) W (v) This represents the complete data after completion, U (v) It is a complete representation that satisfies the consistency of feature distribution within the view. Represents matrix E (v) The i-th row, This represents the feature similarity matrix of the v-th viewpoint. This measures the similarity between different feature dimensions. Specifically, G (v) Constructed based on k-nearest neighbors, if and If they are k-nearest neighbors, then Otherwise, it is 0. According to graph embedding theory, the following equation holds:

[0170]

[0171] Where Tr(.) represents the matrix trace operation, Representation matrix G (v) The Laplace matrix, specifically, D (v) It is a diagonal matrix, and its diagonal values ​​are calculated as follows:

[0172] Computer equipment then according to U (v) The constraints for calculating the differences between the column vectors are as follows:

[0173]

[0174] in, Let matrix S (v) The Laplace matrix, D (v) It is a degree matrix, and its calculation method is as follows: If we combine representation completion learning and graph learning into a unified process, then we have the following objective function:

[0175]

[0176] Here, 1 represents a column vector whose elements are all 1s.

[0177] For multiple similarity matrices {S (1) ,…,S (m) To leverage their complementarity, enhance their consistency, and avoid losing excessive semantic information, a cross-view graph matching mechanism is used. The specific optimization objective is as follows:

[0178]

[0179] Combining formulas (3-5) and (3-6), we propose the following final optimization objective:

[0180]

[0181]

[0182] After the model is iteratively updated, a set of similarity matrices {S} can be obtained.(1) ,…,S (m) We used an averaging method. The final similarity matrix is ​​obtained and used as input to the spectral clustering algorithm to obtain data labels, thereby completing a multi-perspective text representation clustering method for data repair and missing data scenarios.

[0183] In this embodiment, by acquiring the sample data to be repaired, internal feature terms, and a feature similarity matrix, data repair terms are determined based on the number and location of missing attribute data in the sample data to be repaired. Then, a target difference term is obtained by fusing the sample data to be repaired, internal feature terms, and data repair terms. Constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix. A target iteration term is obtained by fusing the target difference term and constraints. Finally, the target iteration term is iterated to determine the target data repair matrix, and the target repaired sample data is obtained based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired. In this way, by determining data repair terms based on the number and location of missing attribute data in the sample data to be repaired, and determining constraints based on the data repair matrix, internal feature terms, and feature similarity matrix, and then iterating the target data repair matrix based on the constraints, the target data repair is completed. This fully explores the unique feature distribution within the sample data, considers the consistency of the feature distribution within the sample data as well as the global consistency, and ensures that the final repaired data meets the data characteristics of the overall sample data, effectively improving the accuracy of data repair.

[0184] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0185] In one embodiment, such as Figure 9 As shown, a data repair device is provided. This device can be a software module, a hardware module, or a combination of both integrated into a computer device. Specifically, the device includes: a data acquisition module 902, a target difference term determination module 904, a constraint condition construction module 906, a target iteration term generation module 908, and a target repair sample data determination module 910, wherein:

[0186] The data acquisition module 902 is used to acquire the sample data to be repaired, internal feature terms, and feature similarity matrix. The internal feature terms are used to characterize the internal distribution characteristics of the missing attribute data in the sample data to be repaired, and the feature similarity matrix is ​​used to characterize the similarity between the column vectors in the corresponding internal feature terms. The data repair terms are determined according to the number and location of the missing attribute data in the sample data to be repaired. The data repair terms include a data repair matrix and a data repair index matrix. The data repair index matrix is ​​used to characterize the correspondence between the elements of the data repair matrix and the missing locations.

[0187] The target difference item determination module 904 is used to obtain the target difference item based on the fusion of the sample data to be repaired, internal feature items, and data repair items. The target difference item is used to characterize the degree of difference between the internal features and internal feature items of the repaired complete sample data.

[0188] The constraint construction module 906 is used to determine constraints based on the data repair matrix, internal feature terms, and feature similarity matrix.

[0189] The target iteration term generation module 908 is used to obtain the target iteration term based on the fusion of the target difference term and the constraint conditions.

[0190] The target repair sample data determination module 910 is used to iteratively determine the target data repair matrix by the target iteration term, and obtain the target repair sample data based on the target data repair matrix, the data repair index matrix and the sample data to be repaired.

[0191] The aforementioned data repair device acquires the sample data to be repaired, internal feature terms, and a feature similarity matrix. It determines data repair terms based on the number and location of missing attribute data in the sample data, then fuses these terms to obtain a target difference term. Constraints are determined based on the data repair matrix, internal feature terms, and feature similarity matrix. Finally, a target iteration term is obtained by fusing the target difference term and constraints. The target iteration term is then iterated to determine the target data repair matrix, and the target repaired sample data is obtained based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired. This process, by determining data repair terms based on the number and location of missing attribute data in the sample data, determining constraints based on the data repair matrix, internal feature terms, and feature similarity matrix, and then iterating the target data repair matrix based on the constraints, completes the repair of the target data. It fully leverages the unique feature distribution within the sample data, considering both the consistency of the internal feature distribution and the global consistency, ensuring that the final repaired data meets the overall data characteristics of the sample data, thus effectively improving the accuracy of data repair.

[0192] For specific limitations regarding the data recovery device, please refer to the limitations of the data recovery method above, which will not be repeated here. Each module in the aforementioned data recovery device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0193] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a data recovery method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0194] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0195] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0196] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0197] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0198] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0199] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0200] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A data repair method, characterized in that, The method includes: Obtain the sample data to be repaired, internal feature terms, and feature similarity matrix. The internal feature terms are used to characterize the internal distribution characteristics of the missing attribute data in the sample data to be repaired, and the feature similarity matrix is ​​used to characterize the degree of similarity between the column vectors in the corresponding internal feature terms. Data repair items are determined based on the number and location of missing attribute data in the sample data to be repaired. The data repair items include a data repair matrix and a data repair index matrix. The data repair index matrix is ​​used to characterize the correspondence between the elements of the data repair matrix and the missing locations. Based on the fusion of the sample data to be repaired, the internal feature items, and the data repair items, a target difference item is obtained. The target difference item is used to characterize the degree of difference between the internal features of the repaired complete sample data and the internal feature items. Constraints are determined based on the data repair matrix, the internal feature terms, and the feature similarity matrix. The target iteration term is obtained by fusing the target difference term and the constraint conditions. The process involves iteratively determining the target data repair matrix by iterating over the target iteration term, and obtaining target repair sample data based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired. This includes: calculating the iteration value of the target iteration term under a preset condition, where each row vector of the feature similarity matrix is ​​a unit vector and all elements are greater than or equal to 0; stopping the iteration when the iteration value is less than a preset threshold to obtain the target data repair matrix; obtaining the target data repair term based on the target data repair matrix and the data repair index matrix; and fusing the target data repair term with the sample data to be repaired to obtain the target repair sample data.

2. The method according to claim 1, characterized in that, The step of determining data repair items based on the number and location of missing attribute data in the sample data to be repaired includes: The dimension of the data repair matrix is ​​determined based on the number of missing attribute data in the sample data to be repaired, and random values ​​are assigned to each element in the data repair matrix to obtain the data repair matrix. The value of each element in the data repair index matrix is ​​determined based on the missing location of the missing attribute data in the sample data to be repaired. Data repair items are obtained by fusing the data repair index matrix and the data repair matrix.

3. The method according to claim 1, characterized in that, The process of fusing the sample data to be repaired, the internal feature terms, and the data repair terms to obtain the target difference term includes: The sample data to be repaired is fused with the data repair items to obtain a repaired sample; The target difference term is obtained by calculating the difference between the repaired sample and the internal feature term.

4. The method according to claim 1, characterized in that, The determination of constraints based on the data repair matrix, the internal feature terms, and the feature similarity matrix includes: Obtain a dimensional similarity matrix, which is determined based on the similarity of the row vector data of the sample data to be repaired; The data repair difference term is calculated based on the difference in the row vectors of the data repair matrix; The first constraint condition is obtained by fusing the data repair difference item with the dimensional similarity matrix.

5. The method according to claim 1, characterized in that, The determination of constraints based on the data repair matrix, the internal feature terms, and the feature similarity matrix includes: The internal feature difference term is calculated based on the difference in the column vectors of the internal feature term; The second constraint is obtained by fusing the internal feature difference term with the feature similarity matrix.

6. The method according to claim 1, characterized in that, The determination of constraints based on the data repair matrix, the internal feature terms, and the feature similarity matrix includes: The feature similarity difference value is obtained by calculating the difference between each row vector of the feature similarity matrix; The third constraint is obtained by fusing the feature similarity difference values.

7. The method according to claim 1, characterized in that, The sample data to be repaired is a data sample with missing attribute data, containing diverse features and being multi-perspective data. The missing attribute data refers to the missing data in the sample data to be repaired that exists from different perspectives.

8. A data repair device, characterized in that, The device includes: The data acquisition module is used to acquire the sample data to be repaired, internal feature terms, and a feature similarity matrix. The internal feature terms are used to characterize the internal distribution characteristics of the missing attribute data in the sample data to be repaired, and the feature similarity matrix is ​​used to characterize the similarity between the column vectors in the corresponding internal feature terms. Data repair items are determined based on the number and location of the missing attribute data in the sample data to be repaired. The data repair items include a data repair matrix and a data repair index matrix. The data repair index matrix is ​​used to characterize the correspondence between the elements of the data repair matrix and the missing locations. The target difference item determination module is used to obtain a target difference item based on the fusion of the sample data to be repaired, the internal feature item, and the data repair item. The target difference item is used to characterize the degree of difference between the internal features of the repaired complete sample data and the internal feature item. A constraint construction module is used to determine constraint conditions based on the data repair matrix, the internal feature terms, and the feature similarity matrix; The target iteration term generation module is used to obtain the target iteration term based on the target difference term and the constraint conditions. The target repair sample data determination module is used to iteratively determine the target data repair matrix by the target iteration item, and obtain the target repair sample data based on the target data repair matrix, the data repair index matrix, and the sample data to be repaired. The module includes: calculating the iteration value of the target iteration item under a preset condition, where the preset condition is that each row vector of the feature similarity matrix is ​​a unit vector and all elements are greater than or equal to 0; stopping the iteration when the iteration value is less than a preset threshold to obtain the target data repair matrix; obtaining the target data repair item based on the target data repair matrix and the data repair index matrix; and fusing the target data repair item with the sample data to be repaired to obtain the target repair sample data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images