Entity similarity calculation method, device and related equipment

By dividing the original graph data into sub-graph data and calculating similarity scores in batches using multiple computing devices, the problems of high resource occupation and low efficiency in large-scale entity similarity calculations are solved, and efficient similarity calculation is achieved.

CN114996411BActive Publication Date: 2024-11-01HAINAN PUSHI INTELLIGENT TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202210766968.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-01
Publication Date
2024-11-01
Estimated Expiration
2042-07-01

AI Technical Summary

Technical Problem

In large-scale entity similarity calculation, the memory usage and low computing efficiency are too high, and memory overflow problems are prone to occur.

Method used

By dividing the original graph data into subgraph data and roaming, integrating and splitting paths based on the focus entity, multiple computing devices are used to calculate similarity scores in batches, reducing resource usage and improving computing efficiency.

Benefits of technology

It effectively reduces the resource usage in the calculation process of entity similarity, greatly improves the calculation efficiency, and supports the calculation of similarity of large data volumes under limited resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114996411B_ABST
    Figure CN114996411B_ABST
Patent Text Reader

Abstract

The present application discloses a method for calculating entity similarity, including dividing the original graph data according to preset screening conditions; for each subgraph data obtained by the division, taking the entity of interest as the benchmark, obtaining each subgraph path of the entity of interest by walking; splitting the entity path set obtained by integrating each subgraph path according to the preset broadcast quantity; broadcasting each walking path subset obtained by the splitting to each computing device in turn; for each subgraph data, using each computing device to calculate the initial similarity score based on the walking path subset; for each entity of interest, weighted summing the initial similarity score and the weight values ​​corresponding to each pre-acquired subgraph data to obtain the target similarity score. The method effectively reduces the resource occupation in the process of entity similarity calculation and improves the efficiency of entity similarity calculation. The present application also discloses a device, an equipment and a computer-readable storage medium, all of which have the above-mentioned beneficial effects.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of information recommendation, and in particular to a method for calculating entity similarity, and also to an entity similarity calculation device, equipment and computer-readable storage medium. Background Art

[0002] As a basic operation for similarity detection between entities, similar entity calculation technology is widely used in practical problems in various fields and industries, such as similar product recommendations on e-commerce platforms, similar patient groups in medical efficacy analysis, and similar users in the financial field. By using the attributes of the entity itself, the similarity between the attributes of the entities can be counted. In traditional similarity calculation, since it is necessary to calculate the similarity between any two entities among all entities, it is necessary to construct and calculate an n*n-dimensional numerical matrix. When the number of entities n is very large (in common business scenarios, the number of entities can often reach tens of millions or even hundreds of millions), the number of entities in the hundreds of millions requires at least 9094TB of memory under memory calculation. In this case, direct large-scale entity similarity calculation is prone to memory overflow, and there are problems such as excessive resource usage and low computing efficiency.

[0003] Therefore, how to effectively reduce the resource usage in the process of entity similarity calculation and improve the efficiency of entity similarity calculation is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0004] The purpose of this application is to provide a method for calculating entity similarity, which can effectively reduce the resource usage in the process of entity similarity calculation and improve the efficiency of entity similarity calculation; another purpose of this application is to provide an entity similarity calculation device, equipment and computer-readable storage medium, all of which have the above-mentioned beneficial effects.

[0005] In a first aspect, the present application provides a method for calculating entity similarity, comprising:

[0006] Determine the set of entities of interest in the original graph data;

[0007] Divide the original graph data according to preset screening conditions to obtain sub-graph data;

[0008] For each of the subgraph data, taking the concerned entity in the concerned entity set as a benchmark, walking according to a preset walking strategy, and obtaining each subgraph path of the concerned entity;

[0009] Integrate the subgraph paths of all the concerned entities in the concerned entity set to obtain an entity path set of all the concerned entities;

[0010] The entity path set is split according to a preset broadcast quantity to obtain each wandering path subset; wherein the product of the number of the wandering path subsets and the preset broadcast quantity is the total number of concerned entities in the concerned entity set;

[0011] Broadcasting each of the walking path subsets to each computing device in sequence;

[0012] For each subgraph data, using each of the computing devices to calculate an initial similarity score between the focused entity and other entities based on the wandering path subset;

[0013] Obtaining weight values ​​corresponding to each of the sub-graph data;

[0014] For each focused entity, a weighted sum is performed on the initial similarity score and the weight value to obtain a target similarity score between the focused entity and other entities.

[0015] Preferably, dividing the original graph data according to preset screening conditions to obtain each sub-graph data includes:

[0016] Dividing the original graph data according to the time series screening condition to obtain each initial sub-graph data;

[0017] The initial sub-graph data are screened according to the relationship screening conditions to obtain the sub-graph data.

[0018] Preferably, for each of the subgraph data, the concerned entity in the concerned entity set is taken as a benchmark, and walking is performed according to a preset walking strategy to obtain each subgraph path of the concerned entity, including:

[0019] For each of the subgraph data, taking the concerned entity as a benchmark, walking according to a random jump strategy or a probabilistic jump strategy with a preset walking number and a preset walking depth;

[0020] The results of multiple walks are de-duplicated and merged to obtain the subgraph path of the entity of interest.

[0021] Preferably, the subgraph paths of all the concerned entities in the concerned entity set are integrated to obtain an entity path set of all the concerned entities, including:

[0022] For each focused entity, the subgraph paths in the subgraph data are integrated to obtain an entity path subset of the current focused entity;

[0023] The entity path subsets of all the entities of interest in the entity set of interest are integrated to obtain the entity path set of all the entities of interest.

[0024] Preferably, before splitting the entity path set according to the preset broadcast quantity to obtain each wandering path subset, the method further includes:

[0025] Collect statistics on resource allocation information and the number of entities of interest;

[0026] The preset broadcast quantity is determined according to the resource configuration information, the quantity of the concerned entities, the preset screening condition and the preset roaming strategy.

[0027] Preferably, after obtaining the target similarity score between the focused entity and other entities, the method further includes:

[0028] Sorting the target similarity scores in descending order to obtain a similarity sequence;

[0029] Other entities corresponding to the first preset number of target similarity scores in the similarity sequence are used as high-similarity targets of the focused entity.

[0030] Preferably, after obtaining the target similarity score between the focused entity and other entities, the method further includes:

[0031] Other entities corresponding to target similarity scores exceeding a preset threshold are regarded as high-similarity targets of the focused entity.

[0032] In a second aspect, the present application further discloses an entity similarity calculation device, comprising:

[0033] A focus entity set determination module, used to determine a focus entity set in the original graph data;

[0034] The original graph division module is used to divide the original graph data according to the preset screening conditions to obtain each sub-graph data;

[0035] A subgraph path acquisition module is used to, for each of the subgraph data, take the concerned entity in the concerned entity set as a benchmark, perform roaming according to a preset roaming strategy, and obtain each subgraph path of the concerned entity;

[0036] A path integration module, used for integrating the subgraph paths of all the concerned entities in the concerned entity set to obtain an entity path set of all the concerned entities;

[0037] A path set splitting module, used to split the entity path set according to a preset broadcast quantity to obtain each wandering path subset; wherein the product of the number of wandering path subsets and the preset broadcast quantity is the total number of concerned entities in the concerned entity set;

[0038] A path broadcasting module, used for broadcasting each of the walking path subsets to each computing device in sequence;

[0039] A similarity score calculation module, configured to calculate, for each subgraph data, an initial similarity score between the focused entity and other entities based on the wandering path subset using each of the calculation devices;

[0040] A weight value acquisition module, used to obtain the weight values ​​corresponding to each of the sub-graph data;

[0041] The weighted summation module is used to perform weighted summation on the initial similarity score and the weight value for each focused entity to obtain a target similarity score between the focused entity and other entities.

[0042] In a third aspect, the present application further discloses an entity similarity calculation device, comprising:

[0043] Memory for storing computer programs;

[0044] A processor is used to implement the steps of any one of the entity similarity calculation methods described above when executing the computer program.

[0045] In a fourth aspect, the present application further discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any one of the entity similarity calculation methods described above are implemented.

[0046] The present application provides an entity similarity calculation method, including determining a set of concerned entities in original graph data; dividing the original graph data according to preset screening conditions to obtain each subgraph data; for each subgraph data, taking the concerned entity in the concerned entity set as a benchmark, wandering according to a preset wandering strategy to obtain each subgraph path of the concerned entity; integrating each subgraph path of all concerned entities in the concerned entity set to obtain an entity path set of all concerned entities; splitting the entity path set according to a preset broadcast quantity to obtain each wandering path subset; wherein the product of the number of wandering path subsets and the preset broadcast quantity is the total number of concerned entities in the concerned entity set; broadcasting each wandering path subset to each computing device in sequence; for each subgraph data, using each computing device to calculate an initial similarity score between the concerned entity and other entities based on the wandering path subset; obtaining a weight value corresponding to each subgraph data; for each concerned entity, weighted summing the initial similarity score and the weight value to obtain a target similarity score between the concerned entity and other entities.

[0047] By applying the technical solution provided in the present application, the original graph data is first divided into multiple sub-graph data according to preset screening conditions, and then each sub-graph data is processed. Based on the focused entity and according to the preset wandering strategy, each sub-graph path of the focused entity is obtained. By integrating the sub-graph paths, the entity path set of all focused entities is obtained. Finally, the entity path set is split according to the preset broadcast quantity to obtain each wandering path subset, and each wandering path subset is broadcasted to each computing device in turn. Multiple computing devices calculate the similarity score between the focused entity and other entities, thereby realizing entity similarity calculation. It can be seen that this method performs asynchronous operations in batches for a complete matrix operation task, supports the realization of similarity calculation of large data volumes under limited resources, and not only effectively reduces the resource occupation in the process of entity similarity calculation, but also greatly improves the calculation efficiency of entity similarity.

[0048] The entity similarity calculation device, equipment and computer-readable storage medium provided in this application all have the above-mentioned beneficial effects, which will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the prior art and the embodiments of the present application, the drawings required for use in the description of the prior art and the embodiments of the present application are briefly introduced below. Of course, the drawings described below in relation to the embodiments of the present application are only part of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work, and the obtained other drawings also belong to the protection scope of the present application.

[0050] Figure 1 A flowchart of an entity similarity calculation method provided in this application;

[0051] Figure 2 A schematic diagram of a random walk provided for this application;

[0052] Figure 3 A schematic diagram of the structure of an entity similarity calculation device provided by the present application;

[0053] Figure 4 A schematic diagram of the structure of an entity similarity calculation device provided in this application. DETAILED DESCRIPTION

[0054] The core of this application is to provide an entity similarity calculation method, which can effectively reduce the resource occupation in the entity similarity calculation process and improve the efficiency of entity similarity calculation; another core of this application is to provide an entity similarity calculation device, equipment and computer-readable storage medium, which also have the above-mentioned beneficial effects.

[0055] In order to describe the technical solutions in the embodiments of the present application more clearly and completely, the technical solutions in the embodiments of the present application will be introduced below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0056] An embodiment of the present application provides a method for calculating entity similarity.

[0057] Please refer to Figure 1 , Figure 1 A schematic diagram of a flow chart of an entity similarity calculation method provided in the present application, the entity similarity calculation method may include:

[0058] S101: Determine a set of entities of interest in original graph data.

[0059] Predetermine the set of entities of interest in the original graph data. It is understandable that graph data often has a lot of nodes. Taking the e-commerce scenario as an example, it will contain many entities such as [commodities, merchants, users, manufacturers, customer service...]. Here, the entity that is about to perform similarity calculation is called "entity of interest". For example, if you want to count the similarities between "users", then "user" is the "entity of interest" in this round; of course, you can also select multiple entities as entities of interest, that is, the number of entities of interest is not unique; and the screening of entities of interest can be configured in a more detailed manner, such as the attributes of "users" must meet certain conditions, etc.

[0060] S102: Divide the original graph data according to preset screening conditions to obtain sub-graph data.

[0061] This step aims to realize the division of the original graph data, that is, to divide the original data graph into multiple sub-graph data according to the preset screening conditions. Among them, the original graph data is the graph data that needs to perform entity similarity calculation, that is, to calculate the similarity between the entities in the original graph data, or the similarity between one or more entities and other entities. It can be understood that the original graph data is similar to the conventional graph data, including (point-entity, edge-relationship).

[0062] In addition, the specific content of the preset screening conditions is not unique and can be preset by technical personnel according to actual needs, and this application does not limit this. For example, the original graph data can be divided according to time information, or according to the relationship between entities, or according to a combination of various conditions.

[0063] S103: For each subgraph data, taking the focused entity in the focused entity set as a benchmark, wandering according to a preset wandering strategy to obtain each subgraph path of the focused entity.

[0064] This step aims to achieve path walking based on the focused entity, so as to obtain the walking path of the focused entity in each sub-graph data. Specifically, after obtaining each sub-graph data, further, for each sub-graph data, taking the focused entity as a benchmark, walking according to a preset walking strategy, the walking path of the focused entity can be obtained.

[0065] The number of walks can also be preset according to actual needs. If the number of walks is set to 1, the focused entity will obtain a walk path in the corresponding subgraph according to the preset walk depth, which will be used as the subgraph path corresponding to this subgraph. If the number of walks is set to N, the focused entity will be used as the starting point, and random walks will be repeated N times to obtain N paths, and the N paths will be de-overlapped and merged, and the de-overlapped and merged paths will be used as the subgraph path corresponding to this graph.

[0066] In addition, similar to the above-mentioned preset screening conditions, the specific content of the preset walking strategy is not unique and can be preset by technical personnel according to actual needs, and this application does not limit this. For example, it can be a random walking strategy or a walking strategy with preset limiting conditions.

[0067] S104: Integrate the subgraph paths of all the focused entities in the focused entity set to obtain an entity path set of all the focused entities.

[0068] After obtaining each subgraph path of the focused entity, the subgraph paths of all focused entities in the focused entity set are integrated to obtain the entity path set of all focused entities. Thus, a path set consisting of the subgraph paths corresponding to all focused entities that need similarity calculation in each subgraph data is obtained.

[0069] S105: Split the entity path set according to the preset broadcast quantity to obtain each wandering path subset.

[0070] The product of the number of walking path subsets and the preset broadcast number is the total number of focused entities in the focused entity set.

[0071] The number of broadcasts is preset, and the number of walking path subsets is determined according to the preset number of broadcasts and the total number of concerned entities in the concerned entity set, that is, the product of the number of walking path subsets and the preset number of broadcasts is the total number of concerned entities in the concerned entity set. After the entity path set of all concerned entities is obtained by integrating the subgraph paths of all concerned entities in the concerned entity set, the entity path set is split according to the preset number of broadcasts to obtain each walking path subset.

[0072] It can be understood that the purpose of splitting the entity path set here is to split the large set into multiple small sets, so as to split the complete computing task into multiple subtasks and improve the feasibility of computing under large data volumes.

[0073] S106: broadcast each walking path subset to each computing device in sequence.

[0074] After the entity path set is split according to the preset broadcast quantity and each wandering path subset is obtained, each wandering path subset is broadcasted to each computing device in sequence. By broadcasting each wandering path subset in sequence, the similarity calculation between the concerned entity and other entities involved in each wandering path subset is performed in batches, thus avoiding the influence of excessive data volume on the processing performance of each computing device.

[0075] S107: For each subgraph data, using each computing device, calculate the initial similarity score between the focused entity and other entities based on the wandering path subset.

[0076] After broadcasting each walking path subset to each computing device in sequence, for each subgraph data, each computing device is used to calculate the initial similarity score between the focused entity and other entities based on the walking path subset.

[0077] S108: Obtain the weight values ​​corresponding to each sub-graph data.

[0078] The weight values ​​corresponding to different sub-graph data are different. The weight values ​​corresponding to the sub-graph data are preset, and the weight values ​​corresponding to the sub-graph data are obtained.

[0079] S109: For each focused entity, perform a weighted sum of the initial similarity score and the weight value to obtain a target similarity score between the focused entity and other entities.

[0080] For each sub-graph data, each computing device is used to calculate the initial similarity score between the focus entity and other entities based on the wandering path subset, and after obtaining the weight value corresponding to each sub-graph data, for each focus entity, the initial similarity score and the weight value are weightedly summed to obtain the target similarity score between the focus entity and other entities.

[0081] It can be seen that the entity similarity calculation method provided in the present application first divides the original graph data into multiple sub-graph data according to the preset screening conditions, and then processes each sub-graph data, and obtains each sub-graph path of the focused entity according to the preset wandering strategy based on the focused entity, and obtains the entity path set of all focused entities by integrating the sub-graph paths, and finally splits the entity path set according to the preset broadcast quantity to obtain each wandering path subset, and broadcasts each wandering path subset to each computing device in turn, and multiple computing devices calculate the similarity score between the focused entity and other entities, thereby realizing entity similarity calculation. It can be seen that this method performs asynchronous calculations in batches for a complete matrix operation task, supports the realization of similarity calculations of large amounts of data under limited resources, and not only effectively reduces the resource occupation in the entity similarity calculation process, but also greatly improves the calculation efficiency of entity similarity.

[0082] In one embodiment of the present application, the above-mentioned dividing the original graph data according to preset filtering conditions to obtain each sub-graph data may include: dividing the original graph data according to time series filtering conditions to obtain each initial sub-graph data; filtering each initial sub-graph data according to relationship filtering conditions to obtain each sub-graph data.

[0083] It is worth noting that in the existing entity similarity calculation process, the analysis focus is often only placed on the attributes of the entity itself, while ignoring the entity's topological structure information. However, for graph data, topological structure information is also a very important feature. At present, there is little research on the differences in topological structures under different time and relationship constraints. Therefore, in order to further improve the accuracy of the entity similarity calculation results, the embodiment of the present application introduces time series screening conditions and relationship screening conditions to obtain the topological structure information of the entity as much as possible.

[0084] The topological structure information of an entity can be understood as the description of the current entity by the entities (neighboring nodes) around the entity. Taking the similar commodity scenario as an example, when the entity types of the graph data are only commodities and users, by analyzing the topological structure information of commodities under different restrictions, it is possible to achieve scenarios that are difficult to achieve by only analyzing the attributes of the entity itself, such as:

[0085] (1) Counting the topological structure information of all neighboring nodes within the first-order expansion relationship of the product (where the neighboring node currently expanded to the first order is the user), such as realizing statistics on similar situations of the purchasing groups of the product;

[0086] (2) Counting the topological structure information of all neighboring nodes within the second-degree expansion relationship of the product (where the neighboring nodes of the current first-degree relationship are users and the neighboring nodes of the second-degree expansion are products), for example, realizing the statistics of the similarity of products that the person who purchases the current product will also purchase.

[0087] Moreover, when time relationships or different constraints are introduced, the topological structure information of the main entity will also change, thereby realizing other more complex scenarios.

[0088] In the specific implementation scenario:

[0089] (1) Introducing time series filtering conditions: The collected original graph data is segmented according to time intervals, and the original graph data is segmented into several subsets of graph data (i.e., the initial subgraph data mentioned above). The significance of this is that if it is segmented into a single time period, the topological structure information around the entity that meets the time filtering conditions can be specifically extracted; if it is segmented into multiple time periods, the information of different topological structures of the entity in multiple time periods can be comprehensively extracted.

[0090] Taking the similarity of goods as an example, if the collected original graph data is the information of all goods and all users who purchased goods on an e-commerce platform in the five years (2016-2021), in the original graph data, the entity type is [goods, user], and the relationship is [user---(purchase)--->goods]. If the original graph data is sliced ​​based on the time interval [(2016-2017)(2017-2021)], the topological structure of the purchase users of the same product in the two time periods and the other goods purchased by the purchase users must be different. Then, the topological structure information of the entities extracted for the same product in the two time intervals must contain different features.

[0091] (2) Introducing relationship screening conditions: For the initial subgraph data, the neighboring node relationship conditions of the focused entity are used for screening to obtain the data of each subgraph. The significance of this is that the scope of the focused entity can be narrowed down according to the configuration information of the neighboring nodes as required. Similarly, this requirement is difficult to achieve by relying solely on the attributes of the entity itself.

[0092] Here we still take the similarity of goods as an example. In the initial subgraph data, the entity type is [merchant, commodity, user], and the relationship is [user---(purchase)---> commodity, merchant---(sell)---> commodity]. If the initial subgraph data is segmented based on the relationship filtering condition [the merchant type of the current commodity is "self-employed"], then all the entities of interest "commodity" can be narrowed down to those whose neighboring nodes contain one or more merchants that meet the merchant type of "self-employed".

[0093] The above-mentioned original data graph partitioning process based on time series filtering conditions and relationship filtering conditions can be implemented using a distributed framework to achieve rapid screening of original graph data and obtain sub-graph data that meets the conditions.

[0094] In one embodiment of the present application, for each subgraph data, the focus entity in the focus entity set is taken as a benchmark, and the roaming is performed according to a preset roaming strategy to obtain each subgraph path of the focus entity, which may include: for each subgraph data, taking the focus entity as a benchmark, roaming is performed according to a random jump strategy or a probabilistic jump strategy with a preset number of roaming times and a preset roaming depth; the results of multiple roaming are de-duplicated and merged to obtain the subgraph path of the focus entity.

[0095] The embodiment of the present application provides a preset wandering strategy to achieve the acquisition of wandering paths based on the preset wandering strategy.

[0096] As mentioned above, all edges of the original graph data can be pre-weighted to adapt to the biased walk scenarios that may occur in subsequent random walks. Figure 2 Take the scenario shown as an example. Figure 2 A random walk schematic diagram provided for this application, if the business personnel believe that, relative to the current focused entity, the importance of its neighboring nodes to its topological structure information is ranked as follows: customer>fund>company>bond, then the relationship can be weighted, for example: downstream-accounts receivable (weight 30%), holding A shares (weight 20%), issuing A shares (weight 10%)..., based on this, in the subsequent random walk process, it is more likely to jump to the next neighboring node along the edge with a higher weight, that is, the focused entity has a 30% probability of jumping to "company". Then, the original graph data can be divided into multiple sub-graph data according to the preset screening strategy.

[0097] On this basis, for each subgraph data, the implementation process of obtaining the wandering path based on the preset wandering strategy may include:

[0098] First, the neighbor node information of all entities is obtained. The information may include edge weight information and neighbor node attribute information. Based on the neighbor node information, the direction of entity movement can be controlled according to business needs.

[0099] Then, random walks are performed starting from each focused entity. Taking A, one of the focused entities in the first round of iterations, as an example, the neighboring node information of A is traversed, and the walk jumps are performed according to the corresponding business logic. The walk methods include random jumps and probabilistic jumps based on edge weights. If the result of the jump is entity B, the walk path of focused entity A is updated to {AB}, and in the next round of iterations, jumps are performed starting from B, and iterations are repeated until the number of walks reaches the maximum path depth D set in advance (i.e., the above-mentioned preset depth). In addition, the single path generated by random walks may not be sufficient to represent the topological structure information of the focused entity. Therefore, the number of paths N can be configured, and the above steps can be repeatedly performed starting from the focused entity to generate N paths starting from the same focused entity. The N paths are de-duplicated and merged, and the result is used as the walk path of the current focused entity under the current subgraph.

[0100] Therefore, the above-mentioned walking strategy is adopted for each concerned entity in each subgraph data to perform distributed walking, and the subgraph walking paths in all subgraphs of the concerned entity A can be obtained.

[0101] In one embodiment of the present application, the above-mentioned integration of the subgraph paths of all the focused entities in the focused entity set to obtain the entity path set of all the focused entities may include: for each focused entity, integrating the subgraph paths in each subgraph data to obtain the entity path subset of the current focused entity; integrating the entity path subsets of all the focused entities in the focused entity set to obtain the entity path set of all the focused entities.

[0102] The embodiment of the present application provides a method for counting walking paths. It is understandable that when the concerned entity walks multiple times in the same subgraph data, there are likely to be repeated walking paths. Therefore, in order to effectively reduce the amount of calculation and improve the calculation efficiency, in the process of merging all the walking paths of the concerned entity's multiple walks into the final walking path, all the walking paths can be merged and deduplicated first, that is, the repeated walking paths are deleted, and then the final walking path is generated.

[0103] In one embodiment of the present application, the above-mentioned splitting of the entity path set according to the preset broadcast quantity and obtaining each wandering path sub-item may also include: counting resource configuration information and the number of concerned entities; determining the preset broadcast quantity based on the resource configuration information, the number of concerned entities, preset screening conditions and the preset wandering strategy.

[0104] The embodiment of the present application provides a method for calculating the preset broadcast quantity, that is, the preset broadcast quantity can be determined by integrating multiple conditions such as resource configuration information, the number of concerned entities, preset screening conditions, and preset roaming strategies. Among them, the resource configuration information may include node attribute occupied resources, pre-configured operating resources, resource allocation coefficients, etc. For example, the calculation formula for the preset broadcast quantity n can be:

[0105] n=(S / (D*T*s))*ω;n≤M;

[0106] Among them, D is the maximum path depth, which can be determined based on the preset walking strategy; M is the number of concerned entities; T is the number of time periods, which can be determined based on the timing screening conditions; s is the resource occupied by the node attribute; S is the configured running resource, and ω is the resource allocation coefficient.

[0107] After calculating and obtaining the preset broadcast quantity, partial data of the final wandering path set can be broadcast to each computing device according to the preset broadcast quantity, so as to perform similarity calculation on the full set of the final wandering path set and partial data in a distributed manner, and the batch calculation is as follows:

[0108] For ease of understanding, in the following description, it is assumed that there are M entities of interest and M computing machines, that is, in the distributed computing process, each device has an entity path of an entity of interest. The elements enclosed in curly brackets {e1,e2,e3......en} represent the n entity paths of interest that are broadcast in one round of broadcasting.

[0109] First Batch:

[0110] (e1,{e1,e2,e3......en})--->(e1,e2), (e1,e3)......(e1,en)

[0111] (e2,{e1,e2,e3......en})

[0112] (e3,{e1,e2,e3......en}) ...

[0114] (eM,{e1,e2,e3......en})

[0115] Second batch:

[0116] (e1,{e1+n,e2+n,e3+n...en+n})--->(e1,e1+n), (e1,e2+n)...(e1,en+n)

[0117] (e2,{e1+n,e2+n,e3+n...en+n})

[0118] (e3,{e1+n,e2+n,e3+n...en+n}) ...

[0120] (eM,{e1+n,e2+n,e3+n...en+n})

[0121] The third batch:

[0122] (e1,{e1+2n,e2+2n,e3+2n...en+2n})--->(e1,e1+2n), (e1,e2+2n)...(e1,en+2n)

[0123] (e2,{e1+2n,e2+2n,e3+2n...en+2n})

[0124] (e3,{e1+2n,e2+2n,e3+2n...en+2n}) ...

[0126] (eM,{e1+2n,e2+2n,e3+2n...en+2n})

[0127] Finally, after all batches of data are calculated, they are merged using the element on the left in the above step as the key. Taking the focus entity e1 as an example, we can get the similarity scores of e1 relative to all other focus entities (e1,e2), (e1,e3), (e1,e4), (e1,e5)....(e1,en)......(e1,emM).

[0128] In one embodiment of the present application, after obtaining the target similarity score between the focus entity and other entities, the method may also include: sorting the target similarity scores in descending order to obtain a similarity sequence; and taking other entities corresponding to the first preset number of target similarity scores in the similarity sequence as high-similarity targets of the focus entity.

[0129] On the basis of completing the entity similarity calculation, the embodiment of the present application provides a similar entity recommendation method. Specifically, the similarities fed back by all computing devices can be obtained to obtain the similarities between the focused entity and other entities, and then all similarities are arranged in descending order according to the similarity values ​​to obtain a sorted similarity sequence, thereby selecting other entities corresponding to the first preset number of similarities in the similarity sequence as similar entities of the focused entity, that is, high similarity targets of the focused entity, and finally outputting them.

[0130] Among them, the specific value of the preset number (such as the first 100) does not affect the implementation of the present technical solution, and can be set by the technical personnel according to the actual needs, and this application does not limit this. Among them, the above-mentioned use of other entities corresponding to the first preset number of similarities in the similarity sequence as high similarity targets of the focused entity can also be changed to use other entities corresponding to the similarities of a preset proportion (such as the first 10%) in the similarity sequence as high similarity targets of the focused entity, and similar entity screening and recommendation can be achieved based on the calculated similarity.

[0131] In one embodiment of the present application, after obtaining the target similarity scores between the focused entity and other entities, the method may further include: taking other entities corresponding to target similarity scores exceeding a preset threshold as high similarity targets of the focused entity.

[0132] On the basis of completing the entity similarity calculation, the embodiment of the present application provides another similar entity recommendation method, that is, the similarity value can be directly used as a screening condition. Specifically, the similarity feedback from all computing devices can be obtained to obtain the similarity between the focused entity and other entities, and then other entities corresponding to the similarity exceeding the preset threshold can be directly used as similar entities of the focused entity, that is, high similarity targets of the focused entity, and finally outputted.

[0133] Similarly, the specific value of the above-mentioned preset threshold (such as 85%) does not affect the implementation of the present technical solution, and can be set by technical personnel according to actual needs, and this application does not impose any limitation on this.

[0134] This application also provides an entity similarity calculation device, please refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of an entity similarity calculation device provided by the present application. The entity similarity calculation device may include:

[0135] A focus entity set determination module 1 is used to determine a focus entity set in the original graph data;

[0136] The original graph division module 2 is used to divide the original graph data according to the preset screening conditions to obtain each sub-graph data;

[0137] The subgraph path acquisition module 3 is used to obtain each subgraph path of the focused entity by taking the focused entity in the focused entity set as a benchmark and performing roaming according to a preset roaming strategy for each subgraph data;

[0138] A path integration module 4 is used to integrate the subgraph paths of all concerned entities in the concerned entity set to obtain an entity path set of all concerned entities;

[0139] The path set splitting module 5 is used to split the entity path set according to the preset broadcast quantity to obtain each wandering path subset; wherein the product of the number of wandering path subsets and the preset broadcast quantity is the total number of concerned entities in the concerned entity set;

[0140] A path broadcast module 6, used to broadcast each walking path subset to each computing device in sequence;

[0141] A similarity score calculation module 7 is used to calculate, for each subgraph data, an initial similarity score between the focused entity and other entities based on the wandering path subset using each computing device;

[0142] The weight value acquisition module 8 is used to obtain the weight value corresponding to each sub-graph data;

[0143] The weighted summation module 9 is used to perform weighted summation of the initial similarity score and the weight value for each focused entity to obtain a target similarity score between the focused entity and other entities.

[0144] It can be seen that the entity similarity calculation device provided in the embodiment of the present application first divides the original graph data into multiple sub-graph data according to the preset screening conditions, and then processes each sub-graph data, and obtains each sub-graph path of the focused entity according to the preset wandering strategy based on the focused entity, and obtains the entity path set of all focused entities by integrating the sub-graph paths, and finally splits the entity path set according to the preset broadcast quantity to obtain each wandering path subset, and broadcasts each wandering path subset to each computing device in turn, and multiple computing devices calculate the similarity score between the focused entity and other entities, thereby realizing entity similarity calculation. It can be seen that this method performs asynchronous calculations in batches for a complete matrix operation task, supports the realization of similarity calculations of large data volumes under limited resources, and not only effectively reduces the resource occupation in the entity similarity calculation process, but also greatly improves the calculation efficiency of entity similarity.

[0145] In one embodiment of the present application, the original graph division module 2 can be specifically used to divide the original graph data according to the time series screening conditions to obtain each initial sub-graph data; and filter each initial sub-graph data according to the relationship screening conditions to obtain each sub-graph data.

[0146] In one embodiment of the present application, the subgraph path acquisition module 3 can be specifically used to perform roaming with a preset number of walks and a preset walking depth according to a random jump strategy or a probabilistic jump strategy for each subgraph data, based on the focus entity; de-duplication and merging the results of multiple walks to obtain the subgraph path of the focus entity.

[0147] In one embodiment of the present application, the path integration module 4 can be specifically used to integrate the subgraph paths in each subgraph data for each focus entity to obtain the entity path subset of the current focus entity; and to integrate the entity path subsets of all focus entities in the focus entity set to obtain the entity path set of all focus entities.

[0148] In one embodiment of the present application, the entity similarity calculation device may also include a preset broadcast quantity determination module, which is used to count resource configuration information and the number of concerned entities before splitting the entity path set according to the preset broadcast quantity to obtain each wandering path subset; determine the preset broadcast quantity based on the resource configuration information, the number of concerned entities, preset filtering conditions and preset wandering strategy.

[0149] In one embodiment of the present application, the entity similarity calculation device may also include a first recommendation module, which is used to sort the target similarity scores in descending order to obtain a similarity sequence after obtaining the target similarity scores between the focus entity and other entities as mentioned above; and use other entities corresponding to the first preset number of target similarity scores in the similarity sequence as high-similarity targets of the focus entity.

[0150] In one embodiment of the present application, the entity similarity calculation device may further include a second recommendation module for, after obtaining the target similarity score between the focus entity and other entities, taking other entities corresponding to target similarity scores exceeding a preset threshold as high similarity targets of the focus entity.

[0151] For an introduction to the device provided in this application, please refer to the above method embodiment, and this application will not go into details here.

[0152] This application also provides an entity similarity calculation device, please refer to Figure 4 , Figure 4 This is a schematic diagram of the structure of an entity similarity calculation device provided by the present application. The entity similarity calculation device may include:

[0153] Memory for storing computer programs;

[0154] The processor is used to implement the steps of any one of the above-mentioned entity similarity calculation methods when executing a computer program.

[0155] like Figure 4 , which is a schematic diagram of the composition structure of an entity similarity calculation device, the entity similarity calculation device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 communicate with each other through the communication bus 13.

[0156] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic devices, etc.

[0157] The processor 10 may call a program stored in the memory 11. Specifically, the processor 10 may execute operations in the embodiment of the entity similarity calculation method.

[0158] The memory 11 is used to store one or more programs, which may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 at least stores programs for implementing the following functions:

[0159] Determine the set of entities of interest in the original graph data;

[0160] Divide the original graph data according to the preset screening conditions to obtain each sub-graph data;

[0161] For each subgraph data, the concerned entity in the concerned entity set is taken as the benchmark, and the wandering is performed according to the preset wandering strategy to obtain each subgraph path of the concerned entity;

[0162] Integrate the subgraph paths of all concerned entities in the concerned entity set to obtain the entity path set of all concerned entities;

[0163] The entity path set is split according to the preset broadcast quantity to obtain each wandering path subset; wherein the product of the number of wandering path subsets and the preset broadcast quantity is the total number of focused entities in the focused entity set;

[0164] Broadcast each walking path subset to each computing device in sequence;

[0165] For each subgraph data, using each computing device to calculate the initial similarity score between the focused entity and other entities based on the walk path subset;

[0166] Get the weight value corresponding to each sub-graph data;

[0167] For each focused entity, the initial similarity score and the weight value are weighted and summed to obtain the target similarity score between the focused entity and other entities.

[0168] In a possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0169] In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0170] The communication interface 12 may be an interface of a communication module, and is used to connect to other devices or systems.

[0171] Of course, it should be noted that Figure 4 The structure shown does not constitute a limitation on the entity similarity calculation device in the embodiment of the present application. In actual applications, the entity similarity calculation device may include Figure 4 More or fewer components than shown, or combinations of certain components.

[0172] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned entity similarity calculation methods can be implemented.

[0173] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0174] For an introduction to the computer-readable storage medium provided in this application, please refer to the above method embodiment, and this application will not go into details here.

[0175] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0176] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0177] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0178] The technical solution provided by the present application is described in detail above. The principle and implementation method of the present application are described in detail using specific examples herein, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A method for calculating entity similarity, characterized in that: include: Determine the set of entities of interest in the original graph data; Divide the original graph data according to preset screening conditions to obtain sub-graph data; For each of the subgraph data, taking the concerned entity in the concerned entity set as a benchmark, walking according to a preset walking strategy, and obtaining each subgraph path of the concerned entity; Integrate the subgraph paths of all the concerned entities in the concerned entity set to obtain an entity path set of all the concerned entities; The entity path set is split according to a preset broadcast quantity to obtain each wandering path subset; wherein the product of the number of the wandering path subsets and the preset broadcast quantity is the total number of concerned entities in the concerned entity set; Broadcasting each of the walking path subsets to each computing device in sequence; For each subgraph data, using each of the computing devices to calculate an initial similarity score between the focused entity and other entities based on the wandering path subset; Obtaining weight values ​​corresponding to each of the sub-graph data; For each focused entity, a weighted sum is performed on the initial similarity score and the weight value to obtain a target similarity score between the focused entity and other entities.

2. The entity similarity calculation method according to claim 1, characterized in that: The step of dividing the original graph data according to the preset screening conditions to obtain each sub-graph data includes: Dividing the original graph data according to the time series screening condition to obtain each initial sub-graph data; The initial sub-graph data are screened according to the relationship screening conditions to obtain the sub-graph data.

3. The entity similarity calculation method according to claim 1, characterized in that: For each of the subgraph data, the concerned entity in the concerned entity set is taken as a benchmark, and walking is performed according to a preset walking strategy to obtain each subgraph path of the concerned entity, including: For each of the subgraph data, taking the concerned entity as a benchmark, walking according to a random jump strategy or a probabilistic jump strategy with a preset walking number and a preset walking depth; The results of multiple walks are de-duplicated and merged to obtain the subgraph path of the entity of interest.

4. The entity similarity calculation method according to claim 1, characterized in that: Integrate the subgraph paths of all the concerned entities in the concerned entity set to obtain an entity path set of all concerned entities, including: For each focused entity, the subgraph paths in the subgraph data are integrated to obtain an entity path subset of the current focused entity; The entity path subsets of all the entities of interest in the entity set of interest are integrated to obtain the entity path set of all the entities of interest.

5. The entity similarity calculation method according to claim 1, characterized in that: Before the entity path set is split according to the preset broadcast quantity to obtain each wandering path subset, the method further includes: Collect statistics on resource allocation information and the number of entities of interest; The preset broadcast quantity is determined according to the resource configuration information, the quantity of the concerned entities, the preset screening condition and the preset roaming strategy.

6. The entity similarity calculation method according to claim 1, characterized in that: After obtaining the target similarity score between the focused entity and other entities, the method further includes: Sorting the target similarity scores in descending order to obtain a similarity sequence; Other entities corresponding to the first preset number of target similarity scores in the similarity sequence are used as high-similarity targets of the focused entity.

7. The entity similarity calculation method according to claim 1, characterized in that: After obtaining the target similarity score between the focused entity and other entities, the method further includes: Other entities corresponding to target similarity scores exceeding a preset threshold are regarded as high-similarity targets of the focused entity.

8. An entity similarity calculation device, characterized in that: include: A focus entity set determination module, used to determine a focus entity set in the original graph data; The original graph division module is used to divide the original graph data according to the preset screening conditions to obtain each sub-graph data; A subgraph path acquisition module is used to, for each of the subgraph data, take the concerned entity in the concerned entity set as a benchmark, perform roaming according to a preset roaming strategy, and obtain each subgraph path of the concerned entity; A path integration module, used for integrating the subgraph paths of all the concerned entities in the concerned entity set to obtain an entity path set of all the concerned entities; A path set splitting module, used to split the entity path set according to a preset broadcast quantity to obtain each wandering path subset; wherein the product of the number of wandering path subsets and the preset broadcast quantity is the total number of concerned entities in the concerned entity set; A path broadcasting module, used for broadcasting each of the walking path subsets to each computing device in sequence; A similarity score calculation module, configured to calculate, for each subgraph data, an initial similarity score between the focused entity and other entities based on the wandering path subset using each of the calculation devices; A weight value acquisition module, used to obtain the weight values ​​corresponding to each of the sub-graph data; The weighted summation module is used to perform weighted summation on the initial similarity score and the weight value for each focused entity to obtain a target similarity score between the focused entity and other entities.

9. An entity similarity calculation device, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the entity similarity calculation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the entity similarity calculation method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Semantically sensitive knowledge graph random walk sampling method

    CN111444317A

  • Query Intent in Information Retrieval

    US20110289063A1

  • Named entity disambiguation using entity distance in a knowledge graph

    US20200342055A1

  • Method and apparatus for determining interest of user for information item

    US20210027146A1

  • Semantic similarity-based entity relation extraction method and apparatus, device and medium

    WO2021121198A1