Data completion method and device, equipment, storage medium and product

Abstract

The application discloses a data completion method and device, equipment, storage medium and product, and relates to the technical field of big data intelligent analysis. The method creates a bipartite graph according to object identifiers, attribute identifiers and non-missing object attribute values in a current data table; the object identifiers and the attribute identifiers correspond to nodes in the bipartite graph, and the non-missing object attribute values correspond to the weight values of connecting edges of the bipartite graph; missing object attribute values of the current data table are determined, at least one effective path corresponding to the missing object attribute values in the bipartite graph is determined, the effective path is a path meeting a set weight value condition, and the path includes at least three connecting edges connecting the object identifiers and the attribute identifiers corresponding to the missing object attribute values; and the completion attribute values of the missing object attribute values are determined according to the set weight values of the at least one effective path. The application solves the problem that the existing data completion method has poor flexibility, and improves the flexibility of data completion.

Description

Technical Field

[0001] This invention relates to the field of big data intelligent analysis technology, and in particular to a data completion method, apparatus, device, storage medium and product. Background Technology

[0002] For large-scale missing data, data completion is currently mainly achieved through trained data completion models. These models primarily include feature-based statistical models and deep learning models. The former tends to make strong assumptions about the data distribution and lacks flexibility in handling mixed data types, such as mixed data including continuous and discrete variables. The latter cannot flexibly determine missing values ​​based on other observations, and during model training, it is usually initialized with specific default values, which can lead to biased assumptions about missing values. Furthermore, both of these data completion models require retraining when encountering new data samples, resulting in poor flexibility.

[0003] In summary, existing data completion methods suffer from a lack of flexibility. Summary of the Invention

[0004] This invention provides a data completion method, apparatus, device, storage medium, and product to address the problem of poor flexibility in existing data completion methods.

[0005] According to one aspect of the present invention, a data completion method is provided, comprising:

[0006] A bipartite graph is created based on the object identifiers, attribute identifiers, and attribute values ​​of non-missing objects in the current data table; wherein the object identifiers and attribute identifiers correspond to nodes in the bipartite graph, and the attribute values ​​of non-missing objects correspond to the weights of the connecting edges in the bipartite graph;

[0007] Determine the missing object attribute value of the current data table, and determine at least one valid path corresponding to the missing object attribute value in the bipartite graph. The valid path is a path that meets the set weight conditions, and the path includes at least three connecting edges that connect the object identifier and attribute identifier corresponding to the missing object attribute value.

[0008] The completion attribute value of the missing object attribute value is determined based on the weight set for the at least one valid path.

[0009] According to another aspect of the present invention, a data completion device is provided, comprising:

[0010] The bipartite graph module is used to create a bipartite graph based on the object identifier, attribute identifier, and non-missing object attribute values ​​in the current data table; wherein, the object identifier and the attribute identifier correspond to the nodes in the bipartite graph, and the non-missing object attribute values ​​correspond to the weights of the connecting edges in the bipartite graph;

[0011] The effective path module is used to determine the missing object attribute value of the current data table and to determine at least one effective path corresponding to the missing object attribute value in the bipartite graph. The effective path is a path that meets the set weight conditions and includes at least three connecting edges that connect the object identifier and the attribute identifier corresponding to the missing object attribute value.

[0012] The completion module is used to determine the completion attribute value of the missing object attribute value based on the set weight of the at least one valid path.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data completion method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data completion method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the data completion method described in any embodiment.

[0019] Compared to existing technologies, the technical solution of this invention improves the flexibility of determining effective paths by determining the valid paths based on bipartite graphs to identify missing object attribute values. Since the weights of the connecting edges of the valid paths correspond to the non-missing object attribute values, the completion attribute values ​​corresponding to the missing object attribute values ​​are determined by setting the weights of all valid paths corresponding to the missing object attribute values. This achieves the technical effect of determining the completion attribute values ​​of missing attribute values ​​based on the association between the missing object attribute values ​​and the non-missing object attribute values, thereby improving the flexibility, accuracy, and universality of data completion.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of data completion provided according to an embodiment of the present invention;

[0023] Figure 2 This is a bipartite graph corresponding to data table 1 provided in the embodiments of the present invention;

[0024] Figure 3 This is another data completion flowchart provided according to an embodiment of the present invention;

[0025] Figure 4 This is another data completion flowchart provided according to an embodiment of the present invention;

[0026] Figure 5 This is another data completion flowchart provided according to an embodiment of the present invention;

[0027] Figure 6 This is a structural block diagram of the data completion device provided according to an embodiment of the present invention;

[0028] Figure 7 This is a structural block diagram of another data completion device provided according to an embodiment of the present invention;

[0029] Figure 8 This is a schematic diagram of the structure of an electronic device that implements an embodiment of the present invention. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0031] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0033] Example

[0034] Figure 1 This is a flowchart of a data completion method provided in an embodiment of the present invention. This embodiment is applicable to cases where data completion of missing object attribute values ​​in a data table is achieved by determining the valid path of the missing object attribute value based on the bipartite graph corresponding to the data table. This method can be executed by a data completion device, which can be implemented in hardware and / or software, and can be configured in software or hardware. Figure 1 As shown, the method includes:

[0035] S110. Create a bipartite graph based on the object identifier, attribute identifier, and non-missing object attribute values ​​in the current data table; where the object identifier and attribute identifier correspond to the nodes in the bipartite graph, and the non-missing object attribute values ​​correspond to the weights of the connecting edges in the bipartite graph.

[0036] Among them, the object identifier is the identifier of the analysis object, such as customer identifier, employee identifier, project identifier, etc.

[0037] Among them, the attribute identifier is the statistical item identifier of the object, such as phone bill identifier, electricity bill identifier, water bill identifier, food consumption identifier, etc.

[0038] Among them, the non-missing object attribute values ​​are the object attribute values ​​that are explicitly recorded in the current data table.

[0039] A bipartite graph, also known as a two-part graph, is a graph in which the vertex set V can be divided into two disjoint subsets. Each edge in the graph is attached to two vertices belonging to either of these two disjoint subsets, and the vertices within the two subsets are not adjacent. In this embodiment, the two disjoint subsets are the subset of object nodes corresponding to all object identifiers and the subset of attribute nodes corresponding to all attribute identifiers in the current data table.

[0040] In this embodiment, each connecting edge of the bipartite graph has a weight, which is equal to the attribute value of the corresponding non-missing object.

[0041] It is understandable that when there are missing object attribute values ​​in the current data table, in the bipartite graph, there are no connecting edges between the object identifier and the attribute identifier corresponding to the missing object attribute value. A missing object attribute value refers to an object attribute value that is not recorded in the current data table. Let data table 1 be the current data table, where S1, S2, and S3 are object identifiers, and F1, F2, F3, and F4 are attribute identifiers. The value corresponding to the intersection of these identifiers is the object attribute value. The bipartite graph corresponding to table 1 is as follows: Figure 2 As shown.

[0042] Table 1

[0043] S1 0.4 0.5 NA 0.2 S2 NA NA 0.7 0.2 S3 0.4 NA NA 0.5

[0044] Comparison of data in Table 1 and Figure 2 The bipartite graph in the image can be observed to... Figure 2 The connecting edges in the table correspond one-to-one with the attribute values ​​of the non-missing objects in Table 1, and Figure 2 The weight of the connecting edge in Table 1 is equal to the attribute value of its corresponding non-missing object; while the attribute values ​​of the missing objects in Table 1 are... Figure 2 There are no corresponding connecting edges in the bipartite graph.

[0045] S120. Determine the missing object attribute values ​​in the current data table, and determine at least one valid path corresponding to the missing object attribute values ​​in the bipartite graph. The valid path is a path that meets the set weight conditions, and the path includes at least three connecting edges that connect the object identifier and attribute identifier corresponding to the missing object attribute values.

[0046] The weighting condition is set as follows: the difference between the weights of two adjacent connecting edges of the node corresponding to each attribute identifier is within a set threshold range. In one embodiment, this set threshold range can be selected as 15% or 20%.

[0047] The following steps are used to determine the path corresponding to the missing object attribute value:

[0048] Step a1: In the bipartite graph, determine the object identifier corresponding to the missing object attribute value and the object node and attribute node corresponding to the attribute identifier, respectively, and take the object node as the starting object node and the attribute node as the ending attribute node.

[0049] Step a2: Search the bipartite graph for all paths that connect the starting object node and the ending attribute node, and each path must contain at least three connecting edges.

[0050] Since the path is a non-linear connection between the starting object node and the ending attribute node, it needs to pass through another object node, so the path includes at least three connecting edges.

[0051] The missing object attribute values ​​determined through the above steps do not contain any closed loops on any path. Thus, for any path, the ending attribute node can be uniquely reached along its starting object node.

[0052] For example, consider setting a threshold range of less than or equal to 20%. Figure 2 As shown, for the missing object attribute value (S3, F3), there are two corresponding paths: path A (S3-F4-S2-F3) and path B (S3-F1-S1-F4-S2-F3). Both path A and path B include the attribute node F4' corresponding to the attribute identifier F4. The weights of the first connecting edge (S3-F4) and the second connecting edge (F4-S2) of this attribute node are 0.5 and 0.2, respectively, with an error between them. The error is outside the set threshold range, therefore path A is not a valid path. Path B includes attribute node F1' corresponding to attribute identifier F1 and attribute node F4' corresponding to attribute identifier F4. The weights of the first connecting edge (S3-F1) and the second connecting edge (F1-S1) of attribute node F1 are 0.4 and 0.4 respectively, and the error between them is... The error is within the set threshold range; the weights of the first connecting edge (S1-F4) and the second connecting edge (F4-S2) of attribute node F4' are 0.2 and 0.2 respectively, and the error between them is... The error is within the set threshold range, therefore path B is a valid path. In this embodiment, the error is the percentage of the ratio of the difference between the larger and smaller weights to the smaller weight. It is understood that this error can also be defined in other forms, such as the percentage of the ratio of the difference between the weights of the first and second connecting edges to the weight of the first connecting edge, or the percentage of the ratio of the difference between the weights of the first and second connecting edges to the weight of the second connecting edge. In actual use, it can be set according to the specific situation. The first and second connecting edges are located on opposite sides of the attribute node, and the ending node of the first connecting edge is an attribute node, and the starting node of the second connecting edge is an attribute node.

[0053] S130. Determine the complete attribute value for the missing object attribute value based on the set weight of the at least one valid path.

[0054] Since the weight of the connecting edge of the valid path is the corresponding non-missing object attribute value, the completion value of the missing object attribute value is determined based on the weight of the valid path corresponding to the missing object attribute value. In other words, the completion value of the missing object attribute value is determined based on the set non-missing object attribute value associated with the valid path. This improves the flexibility and universality of completing the missing object attribute value.

[0055] Compared to existing technologies, the technical solution of this invention improves the flexibility of determining effective paths by determining the valid paths based on bipartite graphs to identify missing object attribute values. Since the weights of the connecting edges of the valid paths correspond to the non-missing object attribute values, the completion attribute values ​​corresponding to the missing object attribute values ​​are determined by setting the weights of all valid paths corresponding to the missing object attribute values. This achieves the technical effect of determining the completion attribute values ​​of missing attribute values ​​based on the association between the missing object attribute values ​​and the non-missing object attribute values, thereby improving the flexibility, accuracy, and universality of data completion.

[0056] Figure 3 This is a flowchart illustrating a data completion method provided in yet another embodiment of the present invention. For example... Figure 3 As shown, compared to the previous embodiment, the data completion method in this embodiment adds the steps of "updating the current data table, updating the bipartite graph according to the updated current data table; returning to determine the missing object attribute values ​​of the current data table, and determining at least one valid path corresponding to the missing object attribute values ​​in the bipartite graph, until there are no missing object attribute values ​​in the current data table or there is no valid path for the missing object attribute values ​​in the current data table". The method includes the following steps:

[0057] S210. Create a bipartite graph based on the object identifier, attribute identifier, and attribute values ​​of non-missing objects in the current data table; where the object identifier and attribute identifier correspond to the nodes in the bipartite graph, and the attribute values ​​of non-missing objects correspond to the weights of the connecting edges in the bipartite graph.

[0058] S220. Determine the missing object attribute values ​​in the current data table, and determine at least one valid path corresponding to the missing object attribute values ​​in the bipartite graph. The valid path is a path that meets the set weight conditions, and the path includes at least three connecting edges that connect the object identifier and attribute identifier corresponding to the missing object attribute values.

[0059] S230. Determine the complete attribute value for missing object attribute values ​​based on the set weight of at least one valid path.

[0060] S240. Update the current data table and update the bipartite graph based on the updated current data table; return the steps of determining the missing object attribute values ​​of the current data table and determining at least one valid path corresponding to the missing object attribute values ​​in the bipartite graph, until there are no missing object attribute values ​​in the current data table or there is no valid path for the missing object attribute values ​​in the current data table.

[0061] This step aims to achieve incremental data completion of the data table, and its incrementality can be demonstrated through the following steps:

[0062] Step b1: Update the current data table, and update the bipartite graph based on the updated current data table.

[0063] Step b2: Determine if there are any missing object attribute values ​​in the updated current data table. If not, proceed to step b3; otherwise, proceed to step b4.

[0064] Step b3: Use the updated current data table as the current data table after data completion.

[0065] Step b4: Determine whether each missing object attribute value corresponds to a valid path in the updated bipartite graph.

[0066] Step b5: If not, end; if yes, determine the completion attribute value for each missing object attribute value based on the set weight of all valid paths corresponding to the missing object attribute value, and return to step b1.

[0067] It is understandable that for the current data table, there may be one or more missing object attribute values ​​that do not have a valid path in the bipartite graph corresponding to the current data table. After performing one or more rounds of data completion on the current data table to obtain the updated current data table, some or all of the missing object attribute values ​​will have a valid path in the bipartite graph corresponding to the updated current data table. At this time, the completed attribute values ​​of the missing object attribute values ​​can be determined based on the valid paths corresponding to the missing object attribute values.

[0068] Compared to existing technologies, the data completion method provided in this invention improves the likelihood and accuracy of completing missing object attribute values ​​in the current data table through a progressive data completion approach.

[0069] Figure 4 This is a flowchart illustrating a data completion method provided in another embodiment of the present invention. This embodiment further optimizes the "determining the missing object attribute values ​​of the current data table" in the aforementioned embodiments. Accordingly, this embodiment includes:

[0070] S310. Create a bipartite graph based on the object identifier, attribute identifier, and attribute values ​​of non-missing objects in the current data table; wherein the object identifier and attribute identifier correspond to the nodes in the bipartite graph, and the attribute values ​​of non-missing objects correspond to the weights of the connecting edges in the bipartite graph.

[0071] S3201. Determine all missing object attribute values ​​in the current data table, and determine the number of valid paths for each missing object attribute value based on the bipartite graph, as well as the confidence level of the valid paths corresponding to the number of valid paths.

[0072] In the bipartite graph, identify all valid paths corresponding to the attribute values ​​of each missing object, count the number of valid paths corresponding to each attribute value of the missing object, and use the number of valid paths as the confidence score of the valid paths, thereby obtaining the confidence score of the valid paths corresponding to the attribute values ​​of each missing object.

[0073] In determining the completion attribute value for a missing object attribute, the more valid paths there are, the more weights can be set for the valid paths that can be referenced, and the more non-missing object attribute values ​​can be referenced, thus increasing the accuracy of the determined completion attribute value. Therefore, the confidence of the valid paths corresponding to the missing object attribute value in this step reflects the number of non-missing object attribute values ​​that can be referenced during the completion process, as well as the accuracy of the corresponding completion attribute value.

[0074] S3202. Sort all missing object attribute values ​​in descending order of confidence level, and take the missing object attribute values ​​corresponding to the first set number of confidence levels in the descending sort result as at least one missing object attribute value of the current data table.

[0075] Using the missing object attribute values ​​corresponding to the first set number of confidence levels in the descending sort results as at least one missing object attribute value for the current data table can, on the one hand, ensure the accuracy of the completed attribute values ​​determined in the current round; on the other hand, it can allow other missing object attribute values ​​in the descending sort results to enter the data completion process in subsequent rounds. Moreover, after the data completion process of the current round is completed, the number of valid paths corresponding to these other missing object attribute values ​​may increase, and the increase in valid paths will improve the accuracy of the completed attribute values ​​for the corresponding missing object attribute values, thereby improving the overall accuracy of the data completion for the current data table.

[0076] S3203. Determine the missing object attribute values ​​in the current data table, and determine at least one valid path corresponding to the missing object attribute values ​​in the bipartite graph. The valid path is a path that meets the set weight conditions, and the path includes at least three connecting edges that connect the object identifier and attribute identifier corresponding to the missing object attribute values.

[0077] S330. Determine the complete attribute value for the missing object attribute value based on the set weight of the at least one valid path.

[0078] Compared to existing technologies, the technical solution of this embodiment selects the missing object attribute values ​​that enter the current round of data completion based on the confidence of the effective path, thereby improving the accuracy of the current round of data completion and increasing the number of effective paths for some or all missing object attribute values ​​that have not entered the current round of data completion. This improves the accuracy of data completion for the missing object attribute values ​​and the overall accuracy of data completion for the current data table.

[0079] Figure 5 This is a flowchart of a data completion method provided in another embodiment of the present invention. This embodiment further optimizes the "determining the completion attribute value of the missing object attribute value based on the weight and relevance of the associated connection edges of the at least one valid path" in the previous embodiment. Accordingly, this embodiment includes:

[0080] S410. Create a bipartite graph based on the object identifier, attribute identifier, and attribute values ​​of non-missing objects in the current data table; where the object identifier and attribute identifier correspond to the nodes in the bipartite graph, and the attribute values ​​of non-missing objects correspond to the weights of the connecting edges in the bipartite graph.

[0081] S420. Determine the missing object attribute values ​​in the current data table, and determine at least one valid path corresponding to the missing object attribute values ​​in the bipartite graph. The valid path is a path that meets the set weight conditions, and the path includes at least three connecting edges that connect the object identifier and attribute identifier corresponding to the missing object attribute values.

[0082] S4301. The last connected edge of the valid path is taken as the associated connected edge of the valid path, and a node of the last connected edge corresponds to the attribute identifier of the missing object attribute value.

[0083] An associated connection edge is a connection edge whose weight contributes to the determination of the attribute value of a missing object.

[0084] Since the path includes three connecting edges, the valid path selected from the path also includes at least three connecting edges. In this embodiment, the connecting edge corresponding to the ending attribute node is called the final connecting edge, and this final connecting edge is used as the associated connecting edge of the valid path.

[0085] S4302. Determine the relevance of the associated connection edges of at least one valid path corresponding to the missing object attribute value, wherein the relevance is used to represent the association strength between the corresponding associated connection edge and the missing object attribute value.

[0086] In this embodiment, if the effective path includes 3 connecting edges (or 1 attribute node), then its associated connecting edge is determined to be a direct connecting edge to the missing object attribute value, and this connecting edge has the highest correlation with the corresponding missing object attribute value. If the effective path includes more than 3 connecting edges (or more than 2 attribute nodes), then its associated connecting edge is determined to be an indirect connecting edge to the missing object attribute value, and the more connecting edges or attribute nodes the effective path has, the lower the correlation between the associated connecting edge and the corresponding missing object attribute value.

[0087] In one embodiment, the depth of at least one valid path corresponding to the missing object attribute value is determined; the relevance of at least one associated connection edge is determined based on the depth of the at least one valid path. Specifically, the relevance of the associated connection edge can be determined through the following steps:

[0088] Step c1: Use the reciprocal of the depth of each valid path as the initial relevance of the corresponding associated edge.

[0089] Step c2: Normalize the initial relevance of the at least one associated connection edge to obtain the relevance of the at least one associated connection edge.

[0090] Using the reciprocal of the depth of the effective path as the initial relevance of the corresponding associated edges allows associated edges of effective paths with fewer edges to have a larger initial relevance, while associated edges of effective paths with more edges have a smaller initial relevance. Normalizing the initial relevance of at least one associated edge ensures that the sum of the relevances corresponding to at least one associated edge is 1.

[0091] S440. Determine the complete attribute value of the missing object attribute value based on the weight and relevance of the associated connection edge corresponding to the missing object attribute value.

[0092] The relevance of the associated connection edges is used as the weight of the associated connection edges; the weights of the corresponding at least one associated connection edge are weighted and summed according to the weight of the corresponding at least one associated connection edge to obtain the corresponding completion attribute value.

[0093] Understandably, since the weights of the connecting edges in a bipartite graph are the attribute values ​​of their corresponding non-missing objects, the corresponding completion attribute value is obtained by weighting and summing the weights of at least one associated connecting edge based on the weights of the corresponding at least one non-missing object attribute value. This achieves the technical effect of determining the completion attribute value of a missing object attribute value based on at least one associated non-missing object attribute value.

[0094] Compared to existing technologies, the technical solution of this embodiment achieves the purpose of obtaining the corresponding complete attribute value by weighting and summing the weights of at least one associated connection edge according to the weights of at least one associated connection edge. This achieves the technical effect of determining the complete attribute value of the missing object attribute value based on at least one non-missing object attribute value associated with the missing object attribute value.

[0095] Figure 6 This is a structural block diagram of the data completion device provided in an embodiment of the present invention. Figure 6 As shown, the device includes:

[0096] The bipartite graph module 11 is used to create a bipartite graph based on the object identifier, attribute identifier, and non-missing object attribute value in the current data table; wherein, the object identifier and the attribute identifier correspond to the nodes in the bipartite graph, and the non-missing object attribute value corresponds to the weight of the connecting edge in the bipartite graph;

[0097] The effective path module 12 is used to determine the missing object attribute value of the current data table and to determine at least one effective path corresponding to the missing object attribute value in the bipartite graph. The effective path is a path that meets the set weight conditions and includes at least three connecting edges that connect the object identifier and the attribute identifier corresponding to the missing object attribute value.

[0098] The completion module 13 is used to determine the completion attribute value of the missing object attribute value based on the set weight of the at least one valid path.

[0099] In one embodiment, such as Figure 7 As shown, the device also includes:

[0100] Return module 14 is used to update the current data table, update the bipartite graph according to the updated current data table, and return the steps of determining the missing object attribute value of the current data table and determining at least one valid path corresponding to the missing object attribute value in the bipartite graph, until there is no missing object attribute value in the current data table or there is no valid path for the missing object attribute value in the current data table.

[0101] In one embodiment, the completion module includes:

[0102] The associated connection edge determination unit is used to determine the last connection edge of the valid path as the associated connection edge of the valid path, wherein one node of the last connection edge corresponds to the attribute identifier corresponding to the attribute value of the missing object.

[0103] The relevance determination unit determines the relevance of the associated connection edges of the at least one valid path corresponding to the missing object attribute value, wherein the relevance is used to represent the association strength between the corresponding associated connection edge and the missing object attribute value;

[0104] The completion unit is used to determine the completion attribute value of the missing object attribute value based on the weight and relevance of the associated connection edges of the at least one valid path.

[0105] In one embodiment, the relevance determination unit is used to determine the depth of the at least one valid path corresponding to the missing object attribute value; and to determine the relevance of the corresponding at least one associated connection edge based on the depth of the at least one valid path.

[0106] In one embodiment, the relevance determination unit is specifically used to take the reciprocal of the depth of each of the effective paths as the initial relevance of the corresponding associated connection edge; and to normalize the initial relevance of the at least one associated connection edge to obtain the relevance of the at least one associated connection edge.

[0107] In one embodiment, the completion module is used to use the relevance of the associated connection edge as the weight of the associated connection edge; and to perform a weighted summation of the weights of the associated connection edges of the at least one valid path according to the weights of the associated connection edges of the at least one valid path, so as to obtain the completion attribute value of the missing object attribute value.

[0108] In one embodiment, the effective path module includes:

[0109] The missing value determination unit is used to determine all missing object attribute values ​​in the current data table, determine the number of valid paths for each missing object attribute value based on the bipartite graph, and the confidence level of the valid paths corresponding to the number of valid paths; sort the confidence levels of all missing object attribute values ​​in descending order, and take the missing object attribute values ​​corresponding to the first set number of confidence levels in the descending sort result as at least one missing object attribute value in the current data table.

[0110] In one embodiment, the setting weight condition is: the difference between the weights of two adjacent connecting edges of each attribute identifier node is within a set threshold range.

[0111] In one embodiment, the effective path module includes:

[0112] The path determination unit is used to determine the object identifier corresponding to the missing object attribute value and the object node and attribute node corresponding to the attribute identifier in the bipartite graph, respectively, and to take the object node as the starting object node and the attribute node as the ending attribute node; to search all paths connecting the starting object node and the ending attribute node in the bipartite graph, and any path includes at least three connecting edges.

[0113] Compared to existing technologies, the data completion device provided in this invention provides a technical solution that improves the flexibility of determining the effective path by determining the effective path based on the bipartite graph to determine the effective path of the missing object attribute value. Since the weight of the connecting edge of the effective path corresponds to the non-missing object attribute value, the completion attribute value corresponding to the missing object attribute value is determined by setting the weight of all effective paths corresponding to the missing object attribute value. This achieves the technical effect of determining the completion attribute value of the missing attribute value based on the association between the missing object attribute value and the non-missing object attribute value, thereby improving the flexibility, accuracy and universality of data completion.

[0114] The technical solution of the present invention is that the data completion device provided in the embodiments of the present invention can execute the data completion method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0115] Figure 8 A schematic diagram of an electronic device 20 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0116] like Figure 8 As shown, the electronic device 20 includes at least one processor 21 and a memory, such as a read-only memory (ROM) 22 or a random access memory (RAM) 23, communicatively connected to the at least one processor 21. The memory stores computer programs executable by the at least one processor. The processor 21 can perform various appropriate actions and processes based on the computer program stored in the ROM 22 or loaded from storage unit 28 into the RAM 23. The RAM 23 can also store various programs and data required for the operation of the electronic device 20. The processor 21, ROM 22, and RAM 23 are interconnected via a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0117] Multiple components in electronic device 20 are connected to I / O interface 25, including: input unit 26, such as keyboard, mouse, etc.; output unit 27, such as various types of monitors, speakers, etc.; storage unit 28, such as disk, optical disk, etc.; and communication unit 29, such as network card, modem, wireless transceiver, etc. Communication unit 29 allows electronic device 20 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0118] Processor 21 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 21 performs the various methods and processes described above, such as data completion methods.

[0119] In some embodiments, the data completion method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 20 via ROM 22 and / or communication unit 29. When the computer program is loaded into RAM 23 and executed by processor 21, one or more steps of the data completion method described above may be performed. Alternatively, in other embodiments, processor 21 may be configured to perform the data completion method by any other suitable means (e.g., by means of firmware).

[0120] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] Computer programs for implementing the methods of the present invention can be written in any combination of at least one programming language. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0122] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include, based on an electrical connection of at least one wire, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0123] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0124] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0125] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0126] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0127] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data completion method provided in any embodiment of this application.

[0128] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A data completion method, characterized in that, include: A bipartite graph is created based on the object identifiers, attribute identifiers, and attribute values ​​of non-missing objects in the current data table; wherein the object identifiers and attribute identifiers correspond to nodes in the bipartite graph, and the attribute values ​​of non-missing objects correspond to the weights of the connecting edges in the bipartite graph; Determine the missing object attribute value of the current data table, and determine at least one valid path corresponding to the missing object attribute value in the bipartite graph. The valid path is a path that meets the set weight conditions, and the path includes at least three connecting edges that connect the object identifier and attribute identifier corresponding to the missing object attribute value. The completion attribute value of the missing object attribute value is determined based on the set weight of the at least one valid path; Determining the missing object attribute values ​​in the current data table includes: Determine all missing object attribute values ​​in the current data table, determine the number of valid paths for each missing object attribute value based on the bipartite graph, and use the number of valid paths for each missing object attribute value as the confidence level of the valid paths. Sort all the missing object attribute values ​​in descending order of confidence, and take the missing object attribute values ​​corresponding to the first set number of confidence values ​​in the descending sort result as at least one missing object attribute value of the current data table.

2. The method according to claim 1, characterized in that, After determining the completion attribute value of the missing object attribute value based on the set weight of the at least one valid path, the method further includes: Update the current data table, and update the bipartite graph based on the updated current data table; return to the steps of determining the missing object attribute values ​​of the current data table and determining at least one valid path corresponding to the missing object attribute values ​​in the bipartite graph, until there are no missing object attribute values ​​in the current data table or there is no valid path for the missing object attribute values ​​in the current data table.

3. The method according to claim 1, characterized in that, Determining the completion attribute value for the missing object attribute value based on the set weight of the at least one valid path includes: The last connected edge of the valid path is used as the associated connected edge of the valid path, and one node of the last connected edge corresponds to the attribute identifier corresponding to the attribute value of the missing object. The relevance of the associated connection edges of the at least one valid path corresponding to the missing object attribute value is determined respectively, wherein the relevance is used to represent the association strength between the corresponding associated connection edge and the missing object attribute value; The complete attribute value for the missing object attribute value is determined based on the weight and relevance of the associated connection edges of the at least one valid path.

4. The method according to claim 3, characterized in that, Determining the relevance of the associated connection edges of the at least one valid path corresponding to the missing object attribute value includes: Determine the depth of the at least one valid path corresponding to the missing object attribute value; The relevance of at least one associated connection edge is determined based on the depth of the at least one valid path.

5. The method according to claim 4, characterized in that, The step of determining the relevance of at least one associated connection edge based on the depth of the at least one valid path includes: The reciprocal of the depth of each effective path is used as the initial relevance of the corresponding associated edge; The initial relevance of the at least one associated connection edge is normalized to obtain the relevance of the at least one associated connection edge.

6. The method according to claim 3, characterized in that, The step of determining the completion attribute value for the missing object attribute value based on the weight and relevance of the associated connection edges of the at least one valid path includes: The relevance of the associated connection edge is used as the weight of the associated connection edge; Based on the weights of the associated edges of the at least one valid path, the weights of the associated edges of the at least one valid path are summed in a weighted manner to obtain the complete attribute value of the missing object attribute value.

7. The method according to claim 1, characterized in that, The conditions for setting the weights are as follows: The difference between the weights of two adjacent connecting edges of the node corresponding to each attribute identifier is within the set threshold range.

8. The method according to any one of claims 1-7, characterized in that, Determine the path corresponding to the missing object attribute value, including: In the bipartite graph, the object identifier and the object node and the attribute node corresponding to the missing object attribute value are determined respectively, and the object node is taken as the starting object node and the attribute node is taken as the ending attribute node. Search the bipartite graph for all paths connecting the starting object node and the ending attribute node, where each path includes at least three connecting edges.

9. A data completion device, characterized in that, include: The bipartite graph module is used to create a bipartite graph based on the object identifier, attribute identifier, and non-missing object attribute values ​​in the current data table; wherein, the object identifier and the attribute identifier correspond to the nodes in the bipartite graph, and the non-missing object attribute values ​​correspond to the weights of the connecting edges in the bipartite graph; The effective path module is used to determine the missing object attribute value of the current data table, and to determine at least one effective path corresponding to the missing object attribute value in the bipartite graph. The effective path is a path that meets the set weight conditions, and the path includes at least three connecting edges that connect the object identifier and the attribute identifier corresponding to the missing object attribute value. The completion module is used to determine the completion attribute value of the missing object attribute value based on the set weight of the at least one valid path; The effective path module includes: The missing attribute value determination unit is used to determine all missing object attribute values ​​in the current data table, determine the number of valid paths for each missing object attribute value based on the bipartite graph, and use the number of valid paths for each missing object attribute value as the confidence level of the valid path. Sort all the missing object attribute values ​​in descending order of confidence, and take the missing object attribute values ​​corresponding to the first set number of confidence values ​​in the descending sort result as at least one missing object attribute value of the current data table.

10. The apparatus according to claim 9, characterized in that, Also includes: The return module is used to update the current data table and update the bipartite graph based on the updated current data table; it returns the steps of determining the missing object attribute values ​​of the current data table and determining at least one valid path corresponding to the missing object attribute values ​​in the bipartite graph, until there are no missing object attribute values ​​in the current data table or there is no valid path for the missing object attribute values ​​in the current data table.

11. The apparatus according to claim 9, characterized in that, The completion module includes: The associated connection edge determination unit is used to determine the last connection edge of the valid path as the associated connection edge of the valid path, wherein one node of the last connection edge corresponds to the attribute identifier corresponding to the attribute value of the missing object. The relevance determination unit determines the relevance of the associated connection edges of the at least one valid path corresponding to the missing object attribute value, wherein the relevance is used to represent the association strength between the corresponding associated connection edge and the missing object attribute value; The completion unit is used to determine the completion attribute value of the missing object attribute value based on the weight and relevance of the associated connection edges of the at least one valid path.

12. The apparatus according to claim 11, characterized in that, The relevance determination unit includes: Determine the depth of the at least one valid path corresponding to the missing object attribute value; The relevance of at least one associated connection edge is determined based on the depth of the at least one valid path.

13. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data completion method according to any one of claims 1-8.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the data completion method according to any one of claims 1-8.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the data completion method according to any one of claims 1-8.

Images