Data processing method and device, computer readable storage medium, and terminal

Abstract

A data processing method and device, a computer readable storage medium and a terminal, the method comprising: obtaining inputted to-be-processed data, the to-be-processed data comprising a heterogeneous graph used to represent the association relationship between nodes in the to-be-processed data; dividing the nodes under a target node category according to the original association relationship of the heterogeneous graph to obtain a plurality of sub-blocks; for the nodes in each sub-block, calculating the association closeness between two nodes according to the number of nodes under a specified node category commonly associated with the two nodes, the specified node category being used to construct the association relationship of a homogeneous graph of the target node category; constructing a homogeneous sub-graph based on the association closeness between the nodes in each sub-block; splicing the homogeneous sub-graphs to obtain a homogeneous graph of the target node category; and performing data processing based on the homogeneous graph of the target node category. The above scheme can reduce the computational complexity in the data processing process and improve the accuracy of the data processing result.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method and apparatus, a computer-readable storage medium, and a terminal. Background Technology

[0002] When modeling and processing data on various aspects of life, a graph can be used as a paradigm: the objects themselves are called "nodes," and the relationships between them are called "edges." Various "nodes" and "edges" constitute a graph. Nodes can be concrete things such as people, companies, equipment, goods, and articles, or abstract concepts such as identity, conclusions, causes, and schools of thought. An edge connecting two nodes indicates the relationship between them, such as subordination, kinship, employment, or purchase. Each node and edge in a graph can have its own attributes. Graphs from the real world often contain multiple types of nodes and edges, and are called "heterogeneous graphs." Sometimes, we need to perform detection, classification, and prediction operations directly on massive and complex heterogeneous graph data. However, often, to solve a problem of interest, we only need to operate on a specific type of node in the graph. In this case, transforming the heterogeneous graph into a "homogeneous graph" containing only the target node category can significantly reduce the amount of data and the complexity of downstream algorithms.

[0003] Currently, a common method for converting heterogeneous graphs into isomorphic graphs in data processing is to count the number of paths between two target nodes and then set a threshold to construct the isomorphic graph. This method is simple and widely applicable. However, it requires calculating the number of paths between each pair of nodes, resulting in a huge computational burden, especially with large datasets. Furthermore, it can prevent some nodes from entering the isomorphic graph, leading to flaws in the definition of its strength and consequently lower accuracy in data processing based on isomorphic graphs. Summary of the Invention

[0004] The technical problems solved by the embodiments of the present invention are the large amount of computation and low accuracy of existing data processing.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a data processing method, comprising: acquiring input data to be processed, wherein the data to be processed includes a heterogeneous graph representing the relationships between nodes in the data to be processed; dividing nodes under a target node category into multiple sub-blocks based on the native relationships of the heterogeneous graph; calculating the correlation tightness between pairs of nodes in each sub-block based on the number of nodes in a specified node category that are commonly associated with each pair of nodes, wherein the correlation tightness is used to represent the similarity between nodes, and the specified node category is used to construct the relationships of the isomorphic graph of the target node category; constructing an isomorphic subgraph based on the correlation tightness between nodes in each sub-block; concatenating the isomorphic subgraphs to obtain an isomorphic graph of the target node category; and performing data processing based on the isomorphic graph of the target node category.

[0006] Optionally, the step of dividing the nodes under the target node category into multiple sub-blocks based on the native association relationships of the heterogeneous graph includes: partitioning each node under the target node category into multiple blocks according to the association relationships between each node under the target node category and other node categories, wherein the other node categories refer to node categories in the heterogeneous graph other than the target node category, and the other node categories associated with each node under the target node category are different in different blocks; taking the other node categories corresponding to each block as the designated node categories of each block, and for each block, dividing each node under the target node category into multiple sub-blocks according to the number of nodes under the designated node categories associated with each node under the target node category, wherein the numerical range of different sub-blocks is different.

[0007] Optionally, the step of dividing the nodes under the target node category into blocks according to the native association relationship of the heterogeneous graph includes: dividing each node under the target node category into blocks according to the number of nodes under the specified node category that each node under the target node category is associated with, to obtain multiple sub-blocks, and the numerical range of different sub-blocks is different.

[0008] Optionally, constructing an isomorphic subgraph based on the correlation between nodes in each sub-block includes: constructing an isomorphic subgraph for each adjacent sub-block based on the correlation between nodes in the adjacent sub-block.

[0009] Optionally, dividing the nodes under the target node category into blocks according to the number of nodes under the specified node category that each node under the target node category is associated with includes: dividing the nodes under the target node category into blocks according to the numerical range corresponding to each block and the number of nodes under the specified node category that each node under the target node category is associated with.

[0010] Optionally, the numerical range corresponding to each block can be determined in any of the following ways: based on the minimum and maximum values ​​of the number of nodes in the specified node category associated with each node in the target node category, the interval formed by the minimum value and the maximum value is divided equally to obtain the numerical range of each block; according to the distribution pattern of the number of nodes in the specified node category associated with each node in the target node category, the number of blocks and the numerical range of each block are determined.

[0011] Optionally, the association tightness can be calculated in the following manner: based on the number of nodes in a specified node category that are commonly associated with each pair of nodes and the number of nodes in the specified node category that each node is associated with, the crossover-union ratio of each pair of nodes is calculated, and the crossover-union ratio is used as the association tightness.

[0012] This invention also provides a data processing apparatus, comprising: an acquisition unit for acquiring input data to be processed, the data to be processed including a heterogeneous graph representing the relationships between nodes in the data to be processed; a segmentation unit for segmenting nodes under a target node category into multiple sub-blocks based on the original relationships of the heterogeneous graph; a calculation unit for calculating the correlation tightness between nodes in each sub-block based on the number of nodes in a specified node category that are commonly associated with each pair of nodes, the correlation tightness representing the similarity between nodes, the specified node category being used to construct the relationships of the isomorphic graph of the target node category; a construction unit for constructing an isomorphic subgraph based on the correlation tightness between nodes in each sub-block; an isomorphic graph obtaining unit for splicing the isomorphic subgraphs to obtain the isomorphic graph of the target node category; and a data processing unit for performing data processing based on the isomorphic graph of the target node category.

[0013] This invention also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when run by a processor, performs the steps of any of the above-described data processing methods.

[0014] This invention also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of any of the above-described data processing methods when running the computer program.

[0015] Compared with the prior art, the technical solution of the embodiments of the present invention has the following beneficial effects:

[0016] In this embodiment of the invention, input data to be processed is obtained, including a heterogeneous graph representing the relationships between nodes in the data. Nodes of the target node category are divided into blocks based on the native relationships of the heterogeneous graph. For nodes in each sub-block, the correlation strength between pairs of nodes is calculated based on the number of nodes in a specified node category that each pair of nodes is commonly associated with. Based on the correlation strength between nodes in each sub-block, a homogeneous sub-graph is constructed. The homogeneous sub-graphs are then concatenated to obtain a homogeneous graph of the target node category. By dividing nodes of the target node category into blocks based on the native relationships of the heterogeneous graph, and calculating the correlation strength between nodes within each block, it is not necessary to calculate the correlation strength between nodes in different blocks, i.e., it is not necessary to calculate the correlation strength across blocks, thereby effectively reducing the computational load. Furthermore, constructing isomorphic subgraphs based on the correlation between nodes in each sub-block helps to build isomorphic subgraphs based on nodes of the same number of sub-blocks. This ensures that nodes of appropriate strength can be integrated into the isomorphic graph, avoiding node omissions and improving the comprehensiveness and rationality of nodes integrated into the isomorphic graph. This, in turn, improves the accuracy of data processing results obtained when performing subsequent data processing based on the isomorphic graph. Therefore, the data processing method provided in this application can reduce computational load while improving the accuracy of data processing results. Moreover, reducing computational load can also improve data processing efficiency. Attached Figure Description

[0017] Figure 1 It provides a relationship diagram between data;

[0018] Figure 2 This is a flowchart of a data processing method according to an embodiment of the present invention;

[0019] Figure 3 This is another data relationship diagram in an embodiment of the present invention;

[0020] Figure 4 This is a schematic diagram of the structure of a data processing device according to an embodiment of the present invention. Detailed Implementation

[0021] As mentioned above, when modeling and processing data for various aspects of life, a common method for transforming heterogeneous graphs into isomorphic graphs is to count the number of paths between two target nodes and then set a threshold to construct the isomorphic graph. Let's take the graph formed by the relationships between authors, articles, and journals in an academic journal scenario as an example. For instance, describing relationships within academic journals typically involves an author writing an article, which is then published in a specific journal. The graph in this case contains three node categories: author, article, and journal. There are two types of edges: writing and publishing. Suppose our problem is to classify the disciplines to which articles belong. In this case, we focus on a specific type of node (i.e., article). We need to transform the original graph into an isomorphic graph and then classify the disciplines to which articles belong based on the transformed isomorphic graph. To simplify the problem, let's assume we only care about the "author-writing-article" relationship in the graph. (Refer to...) Figure 1 The diagram presents a relationship graph between data. For example, articles 1 and 2 can be connected by two paths through author 1 and author 2, while articles 3 and 4 can only be connected by one path through author 4. Therefore, when the threshold for the number of paths is 2, in the newly formed isomorphic graph, articles 1 and 2 are connected by an edge, while articles 3 and 4 are not connected by an edge.

[0022] However, for heterogeneous graphs where the attributes of nodes differ significantly, directly counting common paths leads to two problems: some nodes cannot enter the isomorphic graph; and even if they do, the definition of strength is flawed. For example, authors W1-W500 have written numerous articles, some of which are co-authored, denoted as A1-A100000. We want to construct a graph around author W, but the number of articles written by each author can vary considerably. For instance, some authors may have published dozens of articles, while others have only published 1-2. Using the path count mentioned above would prevent authors with fewer articles from being included in the isomorphic graph.

[0023] In current data processing, converting heterogeneous graphs into homogeneous graphs requires calculating the number of pathways between each pair of nodes, resulting in a massive computational load, especially with large datasets. Furthermore, this can prevent some nodes from entering the homogeneous graph, leading to flaws in the definition of its strength and consequently lower accuracy in data processing based on homogeneous graphs.

[0024] To address the aforementioned issues, in this embodiment of the invention, input data to be processed is acquired, including a heterogeneous graph characterizing the relationships between nodes within the data. Nodes of the target node category are divided into blocks based on the native relationships of the heterogeneous graph. For nodes within each sub-block, the correlation strength between pairs of nodes is calculated based on the number of nodes in a specified node category that each pair of nodes is commonly associated with. Based on the correlation strength between nodes in each sub-block, a homogeneous sub-graph is constructed. The homogeneous sub-graphs are then concatenated to obtain a homogeneous graph representing the target node category. By dividing nodes of the target node category into blocks based on the native relationships of the heterogeneous graph and calculating the correlation strength between nodes within each block, it is not necessary to calculate the correlation strength between nodes in different blocks, i.e., it is not necessary to calculate the correlation strength across blocks, thereby effectively reducing the computational load. Furthermore, constructing isomorphic subgraphs based on the correlation between nodes in each sub-block helps to build isomorphic subgraphs based on nodes of the same number of sub-blocks. This ensures that nodes of appropriate strength can be integrated into the isomorphic graph, avoiding node omissions and improving the comprehensiveness and rationality of nodes integrated into the isomorphic graph. This, in turn, improves the accuracy of data processing results obtained when performing subsequent data processing based on the isomorphic graph. Therefore, the data processing method provided in this application can reduce computational load while improving the accuracy of data processing results. Moreover, reducing computational load can also improve data processing efficiency.

[0025] To make the above-mentioned objectives, features and beneficial effects of the embodiments of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Reference Figure 2 The present invention provides a flowchart of a data processing method according to an embodiment of the invention, which may specifically include the following steps:

[0027] Step S21: Obtain the input data to be processed.

[0028] In practical implementation, during data processing, input data to be processed can be acquired. This data may include a heterogeneous graph representing the relationships between nodes within it. The heterogeneous graph includes nodes of multiple node categories and the native relationships between nodes of each category. This heterogeneous graph can be constructed based on collected raw data (e.g., text, audio / video, etc.), which records multiple analytical objects and the relationships between these objects (e.g., people, objects, etc. contained in text, audio / video). The nodes in the heterogeneous graph are constructed based on the analytical objects, and the native relationships are constructed based on the relationships between the analytical objects.

[0029] For example, a heterogeneous graph may contain nodes of three categories: category A, category B, category C, and category D. Category A includes nodes A1, A2, A3, and A4. Category B includes nodes B1, B2, and B3. Category C includes nodes C1 and C2. The relationship between category A and category B is E. The relationship between category B and category C is F.

[0030] For ease of understanding, let's take node category A as the author, node category B as the article, and node category C as the magazine as an example. The relationship between node category A and node category B is "writing". The relationship between node category B and node category C is "publication".

[0031] It should be noted that the above examples are merely illustrative for ease of understanding and do not limit the scope of protection of the embodiments of the present invention. In practice, the data to be processed differs in different application scenarios, resulting in different node types in different types of data to be processed, and the relationships between different node types are also different.

[0032] Step S22: Based on the original association relationship of the heterogeneous graph, the nodes under the target node category are divided into blocks to obtain multiple sub-blocks.

[0033] In practice, native relationships refer to the relationships between nodes in the heterogeneous graph of the acquired data to be processed, without any post-processing. Using native relationships can reduce the computational load of data processing and ensure the accuracy of the homogeneous graph of the target node category obtained subsequently based on the heterogeneous graph of the data to be processed.

[0034] The specific implementation of step S22 can include various methods, specifically:

[0035] In some non-limiting embodiments, the nodes under the target node category are partitioned according to their associations with other node categories, resulting in multiple blocks. The other node categories refer to node categories in the heterogeneous graph other than the target node category. The other node categories associated with the nodes under the target node category differ in different blocks. Each block's corresponding other node category is designated as its specified node category. For each block, the nodes under the target node category are further divided into multiple sub-blocks based on the number of nodes in the specified node category associated with each node under the target node category. The numerical ranges of these sub-blocks differ.

[0036] Other node categories can refer to all node categories in a heterogeneous graph other than the target node category, or they can refer to some node categories among all node categories in a heterogeneous graph other than the target node category.

[0037] Specifically, based on the native relationships in the heterogeneous graph of the data to be processed, we can obtain the other node categories that each node under the target node category is associated with. For each node under the target node category, we then partition the nodes under the target node category according to the other node categories they are associated with.

[0038] Each block is assigned a specific node category as its corresponding node category. Based on the native associations in the heterogeneous graph, the number of nodes in each specified node category associated with each node under the target node category is obtained. Then, based on the number of nodes in the specified node categories associated with each node under the target node category, and the numerical range of each sub-block, each node under the target node category is classified and assigned to a sub-block within its corresponding numerical range, thus completing the block division of nodes under the target node category for each block.

[0039] For example, a heterogeneous graph includes six node categories: a first node category, a second node category, a third node category, a fourth node category, a fifth node category, and a sixth node category. The first node category is designated as the target node category. All or some of the second, third, fourth, fifth, and sixth node categories are considered other node categories. Based on the native relationships between the target node category and other node categories in the heterogeneous graph of the data to be processed, if some nodes of the target node category are associated with the third node category, some nodes of the target node category are associated with the fourth node category, and some nodes of the target node category are associated with the sixth node category, then the nodes of the target node category associated with the third node category are assigned to the first block, the nodes of the target node category associated with the fourth node category are assigned to the second block, and the nodes of the target node category associated with the sixth node category are assigned to the third block.

[0040] Taking the first block as an example, the specific scheme for dividing nodes under the target node category in each block is explained. For some nodes of the target node category in the first block, based on the number of nodes of the third node category associated with each node of the target node category, and the numerical range of each sub-block, some nodes of the target node category are divided into multiple sub-blocks.

[0041] To facilitate understanding, examples are provided below. (Refer to...) Figure 3 Another data relationship diagram in an embodiment of the present invention is given. Combined with... Figure 3The heterogeneous graph contains nodes categorized as author U, article A, magazine Y, and topic S. The target node is article A. Articles A1, A2, A3, A4, and A5 have author information; articles A3, A4, and A5 have magazine information; and articles A5 and A6 have topic information. Articles A1, A2, A3, A4, and A5 are assigned to the first block; articles A3, A4, and A5 are assigned to the second block; and articles A5 and A6 are assigned to the third block. Based on the number of authors associated with each of articles A1, A2, A3, A4, and A5 in the first block, articles A1, A2, A3, A4, and A5 are further subdivided.

[0042] In actual data processing, there are situations where the data to be processed is missing. The relationships between a certain analytical object and other objects in the data to be processed should exist, but are missing due to various reasons (such as loss during data transmission or inability to obtain information for confidentiality). In the heterogeneous graph of the data to be processed, some edges that should exist will appear, but due to missing data, the complete heterogeneous graph cannot be seen. For example, in the heterogeneous graph "Author-Writing-Article-Published-Magazine," some magazines may not provide publication information for some articles due to their age, resulting in missing publication information for certain articles in the graph. When information is missing in the data to be processed, existing data processing methods, when converting a heterogeneous graph into a homogeneous graph, will not show nodes with missing data in the homogeneous graph. That is, data for some analytical objects will be lost during data processing due to missing data, leading to a significant deviation between the analysis results and the actual situation when analyzing the analytical objects in the data to be processed, affecting the accuracy of subsequent data processing results. However, using the block partitioning method provided in this embodiment of the invention, the nodes under the target node category are first partitioned according to the relationships between each node under the target node category and other node categories. For each partition, the nodes under the target node category are divided into blocks according to the number of nodes under the specified node category that each node under the target node category is associated with. In this way, the association relationship between each node under the target node category is determined by the association relationship with other node categories, thereby avoiding missing data and obtaining a more accurate isomorphic graph of the target node category.

[0043] In other non-limiting embodiments, each node in the target node category is divided into multiple sub-blocks according to the number of nodes in the specified node category that each node in the target node category is associated with, and the numerical range of different sub-blocks is different.

[0044] Furthermore, based on the numerical range corresponding to each block and the number of nodes in the specified node category associated with each node in the target node category, the nodes under the target node category are divided into blocks. This block division is based on the numerical range corresponding to each block and the number of nodes in the specified node category associated with each node in the target node category. Research has found that the probability of association between nodes of different sizes is low, while the probability of association between nodes of the same size is high. Even if nodes of different sizes are associated, the association strength is weak (i.e., the proportion of shared channels is small), and these nodes of different sizes with weak association are usually not the focus of attention. For example, when the target node category is "author," the probability of association between an author who has written one article and an author who has written 1000 articles is low. Even if an association exists, it would only involve one shared channel, and the association strength is weaker than the association probability between an author who has written 1000 articles and another author who has written 1000 articles. Therefore, by dividing the nodes under the target node category into blocks and calculating the correlation tightness of the nodes in each block, the embodiments of the present invention can effectively avoid comparing nodes of different sizes (i.e., different order of magnitude) during the self-construction of the isomorphic graph of the target node category. This significantly reduces the amount of computation while ensuring the accuracy of the nodes integrated into the isomorphic graph and ensuring that some nodes can be integrated into the isomorphic graph, thus avoiding node loss.

[0045] Specifically, based on the native associations of the heterogeneous graph, the number of nodes in each specified node category associated with each node under the target node category is obtained. Based on the number of nodes in the specified node categories associated with each node under the target node category, and the numerical range of each sub-block, each node under the target node category is classified, i.e., divided into sub-blocks within the corresponding numerical range, thus completing the block division of each node under the target node category. The number of specified nodes associated with each node under the target node category in each block all satisfy the numerical range of that block, thereby enabling nodes of the same magnitude under the target node category to be grouped into one block.

[0046] For ease of understanding, let's take a heterogeneous graph with node categories A, B, and C as an example. The target node category is A, which includes nodes A1, A2, ..., Ap, totaling p nodes, where p is a positive integer. The specified node category is B, and the target node category B includes nodes B1, B2, ..., Bq, totaling q nodes. The goal is to transform it into a homogeneous graph of category A. We count the number of nodes B1, B2, ..., Bq in the specified node category B associated with each node A1, A2, ..., Ap in the target node category A, where q is a positive integer. For example, node A1 is associated with 5 nodes in the specified node category B, node A2 is associated with 12 nodes in the specified node category B, and node Ap is associated with 20 nodes in the specified node category B. The numerical range of block 1 is [0, 10], and the numerical range of block 2 is (10, 20). During block partitioning, for each node in target node category A, nodes in the associated specified node category B with a number of nodes between [0, 10] are assigned to block 1, while nodes in the associated specified node category B with a number of nodes between (10, 20) are assigned to block 2. Therefore, node A1 is assigned to block 1, and nodes A2 and Ap are assigned to block 2.

[0047] In practice, the numerical range corresponding to each block can be determined in various ways. Examples are given below.

[0048] For example, based on the minimum and maximum number of nodes in the specified node category associated with each node under the target node category, the interval formed by the minimum and maximum values ​​is divided equally to obtain the numerical range of each block.

[0049] It is understandable that the number of blocks and the numerical range of the blocks can also be determined based on the minimum and maximum number of nodes in the specified node category that are associated with each node in the target node category, and the distribution of the number of nodes in the specified node category that are associated with each node in the target node category, so as to ensure the rationality of the numerical range of the blocks.

[0050] For example, the number of blocks and the numerical range of each block can be determined based on the distribution pattern of the number of nodes in the specified node category associated with each node in the target node category.

[0051] If the distribution of the number of nodes associated with each node under the target node category follows a normal distribution, in a normal distribution, the interval (or span) between the numerical ranges of the blocks at the two ends of the distribution is larger, while the interval between the numerical ranges of the blocks in the middle region is smaller. Therefore, when determining the numerical range of the blocks, larger intervals can be defined for the sparser distribution at the two ends, and smaller intervals can be defined for the more concentrated distribution in the middle region. In other words, the span of the numerical range of the blocks at the two ends of the ecological distribution should be greater than the span of the numerical range of the blocks in the middle region. The overall goal is to make the number of nodes in each block as even as possible.

[0052] Continue to refer to Figure 2 Step S23: For nodes in each sub-block, calculate the affinity between each pair of nodes based on the number of nodes in the specified node category that each pair of nodes is associated with.

[0053] In practice, the degree of association is used to characterize the similarity between nodes, and the specified node category is used to construct the association relationship of the isomorphic graph of the target node category.

[0054] In specific implementation, the association tightness can be calculated in the following way: For nodes in each sub-block, the intersection-union ratio of each pair of nodes is calculated based on the number of nodes in the specified node category that each pair of nodes is associated with and the number of nodes in the specified node category that each node is associated with. The intersection-union ratio is then used as the association tightness.

[0055] Furthermore, the sum of the number of nodes under the specified node category that each of the two nodes is associated with is subtracted from the number of nodes under the specified node category that both nodes are associated with to obtain a first result. The quotient of the number of nodes under the specified node category that both nodes are associated with and the first result is used as the intersection-union ratio of the two nodes, which is the correlation tightness.

[0056] For a given block, there is a native association between node type A and node type B. m nodes of type B (belonging to node type B) are associated with node Ai, n nodes of type B are associated with node Aj, and k nodes of type B are associated with both nodes Ai and Aj. The association tightness can be calculated using the following formula:

[0057]

[0058] Among them, IoU ij denoted as the degree of association between nodes Ai and Aj; k is the number of B-type nodes that are commonly associated with nodes Ai and Aj; n is the number of B-type nodes associated with node Aj; and m is the number of B-type nodes associated with node Ai.

[0059] Step S24: Construct isomorphic subgraphs based on the degree of association between nodes in each sub-block.

[0060] In practice, for each adjacent sub-block, an isomorphic subgraph is constructed based on the degree of association between the nodes in the adjacent sub-block.

[0061] Adjacent sub-blocks have similar numerical ranges and comparable orders of magnitude. Since nodes of the same order of magnitude have a higher overlap rate in samples and are more comparable, constructing isomorphic subgraphs for adjacent sub-blocks can reduce computational cost while ensuring the accuracy of nodes incorporated into the constructed isomorphic subgraphs.

[0062] In practice, for each sub-block, nodes with a correlation density exceeding the set value are filtered out, and isomorphic subgraphs are constructed based on the filtered nodes and their original correlation relationships.

[0063] Step S25: The isomorphic subgraphs are spliced ​​together to obtain the isomorphic graph of the target node category.

[0064] In specific implementation, when splicing the isomorphic subgraphs, the union of the association relationships of all isomorphic subgraphs is taken according to the association relationships between the nodes in each isomorphic subgraph to obtain the isomorphic graph of the target node category.

[0065] In specific implementation, if step S22, when dividing the nodes in the target node category into blocks based on the original association relationships of the heterogeneous graph, adopts the following block division method: Based on the association relationships between each node in the target node category and other node categories, the nodes in the target node category are partitioned into multiple blocks; for each block, based on the number of nodes in the specified node category associated with each node in the target node category, the nodes in the target node category are further divided into multiple sub-blocks. Then, in step S25, isomorphic subgraphs can be constructed for the sub-blocks in each block, and then isomorphic subgraphs corresponding to each block can be obtained. Finally, the isomorphic subgraphs corresponding to each block can be concatenated to obtain the isomorphic graph of the target node category.

[0066] By concatenating the isomorphic subgraphs corresponding to each block, the isomorphic graph of the target node category can be obtained. Based on the association relationships between nodes in the isomorphic subgraphs corresponding to each block, the union of the association relationships of all isomorphic subgraphs can be taken to obtain the isomorphic graph of the target node category.

[0067] Step S26: Perform data processing based on the isomorphic graph of the target node category.

[0068] Data processing based on isomorphic graphs of target node categories can include one or more data processing methods such as data classification, detection, prediction, statistical analysis, data cleaning, and data filtering.

[0069] In some embodiments, based on the isomorphic graph of the target node category, the nodes under the target node category are classified according to the degree of association between the nodes under the target node category.

[0070] Furthermore, classifying nodes within the target node category can create a category index. That is, after classifying each node, a corresponding index can be created and stored for each category. Nodes belonging to a specific category can then be found using the index corresponding to that category. This facilitates the storage and retrieval of data to be processed and improves the efficiency of data retrieval.

[0071] Furthermore, each node can be categorized and indexed based on the attributes of the target node category. These attributes can be obtained from the data to be processed. For example, if the target node category is "article," the article's attributes could include its subject matter and type (narrative, prose, argumentative, etc., with different types resulting from different classification methods). Similarly, if the target node category is "journal," its attributes could include the journal's category (e.g., core journals, non-core journals, general journals, academic journals, industry journals, indexable journals, domestic journals, foreign journals, national journals, local journals, etc., with different categories resulting from different classification methods). Likewise, if the target node category is "author," its attributes could include the author's preferred article type (style) and the type of journal in which they published their articles. It's understandable that the attributes will differ depending on the specific application scenario; configuration should be based on actual needs. The specific attributes for the target node category are not limited here.

[0072] In other embodiments, based on the isomorphic graph of the target node category, statistical analysis is performed on the nodes under the target node category according to the degree of association between the nodes under the target category. In some scenarios, trend predictions can also be made based on the degree of association, such as sales forecasting and audience prediction. The direction of trend prediction varies depending on the application scenario.

[0073] In other embodiments, based on the isomorphic graph of the target node category, data processing such as data filtering and data cleaning is performed according to the correlation strength between nodes under the target node category. Specifically, in the process of obtaining the isomorphic graph of the target node category, nodes of the target node category with a correlation strength greater than or equal to the set threshold are filtered out according to the relationship between the correlation strength and the set threshold, so as to clean or filter out nodes of the target node category with a correlation strength less than the set threshold, thereby realizing data processing such as data filtering or data cleaning.

[0074] As described above, the process involves acquiring input data to be processed, including a heterogeneous graph representing the relationships between nodes within the data. Nodes of the target node category are divided into blocks based on the native relationships of the heterogeneous graph. For each sub-block, the affinity between nodes is calculated based on the number of nodes in a specified node category that each pair of nodes is commonly associated with. Based on the affinity between nodes in each sub-block, a homogeneous sub-graph is constructed. These homogeneous sub-graphs are then concatenated to obtain a homogeneous graph representing the target node category. By dividing nodes of the target node category into blocks based on the native relationships of the heterogeneous graph and calculating the affinity between nodes within each block, the computational load can be effectively reduced. Nodes within different blocks do not require cross-block affinity calculations. Furthermore, constructing isomorphic subgraphs based on the correlation between nodes in each sub-block helps to build isomorphic subgraphs based on nodes of the same number of sub-blocks. This ensures that nodes of appropriate strength can be integrated into the isomorphic graph, avoiding node omissions and improving the comprehensiveness and rationality of nodes integrated into the isomorphic graph. This, in turn, improves the accuracy of data processing results obtained when performing subsequent data processing based on the isomorphic graph. Therefore, the data processing method provided in this application can reduce computational load while improving the accuracy of data processing results. Moreover, reducing computational load can also improve data processing efficiency.

[0075] This invention also provides a data processing apparatus, with reference to... Figure 4 A schematic diagram of a data processing apparatus according to an embodiment of the present invention is provided. The data processing apparatus 40 may include:

[0076] The acquisition unit 41 is used to acquire the input data to be processed, the data to be processed including a heterogeneous graph used to characterize the relationship between nodes in the data to be processed;

[0077] Segmentation unit 42 is used to segment nodes under the target node category according to the original association relationship of the heterogeneous graph to obtain multiple sub-blocks;

[0078] The calculation unit 43 is used to calculate the association tightness between nodes in each sub-block based on the number of nodes in a specified node category that are commonly associated with each pair of nodes. The association tightness is used to characterize the similarity between nodes. The specified node category is used to construct the association relationship of the isomorphic graph of the target node category.

[0079] Building unit 44 is used to construct isomorphic subgraphs based on the degree of association between nodes in each sub-block;

[0080] Isomorphic graph obtaining unit 45 is used to stitch together the isomorphic subgraphs to obtain the isomorphic graph of the target node category;

[0081] The data processing unit 46 is used to perform data processing based on the isomorphic graph of the target node category.

[0082] In specific implementation, the specific working principle and workflow of the data processing device 40 can be found in the description of the data processing method in the above embodiments, and will not be repeated here.

[0083] This invention also provides a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when run by a processor, executes the steps of the data processing method provided in any of the above embodiments of this invention.

[0084] The computer-readable storage medium may include non-volatile or non-transitory memory, and may also include optical discs, hard disk drives, solid-state drives, etc.

[0085] Specifically, in this embodiment of the invention, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0086] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0087] This invention also provides a data processing apparatus, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the steps of the data processing method provided in any of the above embodiments.

[0088] The memory and the processor are coupled, and the memory can be located inside or outside the data processing device. The memory and the processor can be connected via a communication bus.

[0089] Data processing devices may include, but are not limited to, terminal devices such as mobile phones, computers, and tablets, as well as servers and cloud platforms.

[0090] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatus, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0092] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or in a combination of hardware and software functional units. For example, for various devices or products applied to or integrated into a chip, each module / unit can be implemented using hardware such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware such as circuits; for various devices or products applied to or integrated into a chip module, each module / unit can be implemented using hardware such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0093] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0094] In the embodiments of this application, "multiple" refers to two or more.

[0095] The descriptions of "first," "second," "third," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0096] It should be noted that the sequence number of each step in this embodiment does not represent a limitation on the execution order of each step.

[0097] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A data processing method, characterized in that, include: The input data to be processed is obtained, and the data to be processed includes a heterogeneous graph used to characterize the relationship between nodes in the data to be processed; The heterogeneous graph includes nodes of multiple node categories and the native relationships between nodes of each node category. The heterogeneous graph is constructed based on the collected raw data, which includes text or audio / video. The raw data records multiple analysis objects and the relationships between them. The nodes in the heterogeneous graph are constructed based on the analysis objects, and the native relationships are constructed based on the relationships between the analysis objects. Based on the original association relationships of the heterogeneous graph, the nodes under the target node category are divided into blocks to obtain multiple sub-blocks; For nodes in each sub-block, the association tightness between pairs of nodes is calculated based on the number of nodes in a specified node category that each pair of nodes is commonly associated with. The association tightness is used to characterize the similarity between nodes. The specified node category is used to construct the association relationship of the isomorphic graph of the target node category. The association tightness is calculated as follows: the intersection-union ratio (IUU) of each pair of nodes is calculated based on the number of nodes in a specified node category that each pair of nodes is commonly associated with and the number of nodes in the specified node category that each node is individually associated with. The IUU is then used as the association tightness. Based on the degree of association between nodes in each sub-block, construct an isomorphic subgraph; The isomorphic subgraphs are spliced ​​together to obtain the isomorphic graph of the target node category; Data processing is performed based on the isomorphic graph of the target node category; The nodes under the target node category are divided into blocks based on the native association relationships of the heterogeneous graph, resulting in multiple sub-blocks, including: Based on the association relationships between each node under the target node category and other node categories, each node under the target node category is partitioned to obtain multiple blocks. The other node categories refer to the node categories in the heterogeneous graph other than the target node category. The other node categories associated with each node under the target node category in different blocks are different. Each block is assigned a different node category as its designated node category. Based on the original association relationships of the heterogeneous graph, the number of designated node categories associated with each node under the target node category is obtained. Based on the number of nodes under the designated node category associated with each node under the target node category, and the numerical range of each sub-block, each node under the target node category is classified into sub-blocks with corresponding numerical ranges. This completes the segmentation of each node under the target node category for each block, resulting in multiple sub-blocks with different numerical ranges for each sub-block.

2. The data processing method as described in claim 1, characterized in that, The construction of isomorphic subgraphs based on the correlation between nodes in each sub-block includes: For each adjacent sub-block, an isomorphic subgraph is constructed based on the degree of association between nodes in the adjacent sub-block.

3. The data processing method as described in claim 1, characterized in that, The step of dividing the nodes under the target node category into blocks according to the number of nodes under the specified node category that each node under the target node category is associated with includes: Based on the numerical range corresponding to each block and the number of nodes in the specified node category associated with each node in the target node category, the nodes in the target node category are divided into blocks.

4. The data processing method as described in claim 3, characterized in that, The numerical range corresponding to each block can be determined using any of the following methods: Based on the minimum and maximum number of nodes in the specified node category associated with each node under the target node category, the interval formed by the minimum and maximum values ​​is divided equally to obtain the numerical range of each block. Based on the distribution pattern of the number of nodes in the specified node category associated with each node under the target node category, the number of blocks and the numerical range of each block are determined.

5. A data processing apparatus, characterized in that, include: An acquisition unit is used to acquire input data to be processed, wherein the data to be processed includes a heterogeneous graph used to characterize the relationship between nodes in the data to be processed; The heterogeneous graph includes nodes of multiple node categories and the native relationships between nodes of each node category. The heterogeneous graph is constructed based on the collected raw data, which includes text or audio / video. The raw data records multiple analysis objects and the relationships between them. The nodes in the heterogeneous graph are constructed based on the analysis objects, and the native relationships are constructed based on the relationships between the analysis objects. The segmentation unit is used to segment nodes under the target node category into multiple sub-blocks based on the original association relationship of the heterogeneous graph. The calculation unit is used to calculate the association tightness between nodes in each sub-block based on the number of nodes in a specified node category that each pair of nodes is associated with. The association tightness is used to characterize the similarity between nodes. The specified node category is used to construct the association relationship of the isomorphic graph of the target node category. The association tightness is calculated in the following way: based on the number of nodes in a specified node category that each pair of nodes is associated with and the number of nodes in the specified node category that each node is associated with, the intersection-union ratio of each pair of nodes is calculated, and the intersection-union ratio is used as the association tightness. Building units are used to construct isomorphic subgraphs based on the degree of association between nodes in each sub-block; The isomorphic graph obtaining unit is used to stitch together the isomorphic subgraphs to obtain the isomorphic graph of the target node category; A data processing unit is used to perform data processing based on the isomorphic graph of the target node category; The segmentation unit is used to partition each node under the target node category according to the association relationship between each node under the target node category and other node categories, resulting in multiple blocks. The other node categories refer to node categories in the heterogeneous graph other than the target node category. The other node categories associated with each node under the target node category are different in different blocks. The other node categories corresponding to each block are used as the designated node categories of each block. According to the original association relationship of the heterogeneous graph, the number of designated node categories associated with each node under the target node category is obtained. According to the number of nodes under the designated node categories associated with each node under the target node category and the numerical range of each sub-block, each node under the target node category is classified, that is, divided into sub-blocks with corresponding numerical ranges. This completes the segmentation of each node under the target node category for each block, resulting in multiple sub-blocks with different numerical ranges for each sub-block.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by the processor to perform the steps of the data processing method according to any one of claims 1 to 4.

7. A terminal comprising a memory and a processor, wherein the memory stores a computer program capable of running on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the data processing method according to any one of claims 1 to 4.

Images