Entity fusion method, device and equipment for sharing and exchanging in cross-domain data middle platform

Abstract

The application relates to an entity fusion method, device and equipment in cross-domain data middle platform sharing exchange. The method comprises the following steps: reading basic information, attribute tree and attribute value information of entities in a source domain and a target domain, and setting the importance of attributes; judging whether the entities are the same based on the basic information and the attribute tree of the entities; when it is judged that the entities are the same, performing similarity analysis on the attributes and the attribute values; and performing fusion processing based on the analysis result and the importance. The method can flexibly process various entity modification types, and improves the accuracy and consistency of cross-domain data fusion.

Description

Technical Field

[0001] This application relates to the field of data middleware technology, and in particular to a method, apparatus and equipment for entity fusion in cross-domain data middleware sharing and exchange. Background Technology

[0002] A data platform is a core architecture in digital transformation, integrating, governing, and standardizing internal and external data resources to form a reusable data asset system. During cross-domain data sharing and exchange, the data platform is deployed in different network domains (such as domain 1 and domain 2), with differences in access levels, system functions, network locations, and data usage. Entities, as core business objects abstracted from a business perspective, are frequently modified by different domains during cross-domain sharing and exchange, leading to discrepancies in the information of the same entity, such as changes to attribute names, attribute categories, attribute levels, the addition of sub-attributes, and updates to attribute values.

[0003] Existing entity fusion methods are mostly based on rules or traditional machine learning methods, which limit the types or scope of modifications to entity information, such as not changing the level or category of attributes within an entity. These methods lack flexibility, cannot be applied to all scenarios, and suffer from problems such as low fusion efficiency, insufficient accuracy, and inability to handle complex attribute structures. Specific shortcomings include: entity identity judgment relies on a single identifier, lacking comprehensive evaluation; attribute similarity analysis ignores semantic information; attribute value conflict resolution mechanisms are rigid; and they cannot handle modifications under multi-level attribute tree structures. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, and equipment for entity fusion in cross-domain data platform sharing and exchange that can improve the flexibility, accuracy, and applicability of fusion, addressing the aforementioned technical problems.

[0005] A method for entity fusion in cross-domain data platform sharing and exchange, the method comprising:

[0006] Read the basic entity information, attribute tree and attribute value information of entities in the source domain and the target domain. The basic entity information includes entity code, entity name and entity category. The attribute tree is constructed in a hierarchical structure, including first-level attributes, second-level attributes and third-level attributes. The upper-level attributes can contain the lower-level attributes or be directly assigned attribute values.

[0007] The importance of an attribute is determined based on the importance of that attribute within the entity.

[0008] Based on the entity's basic information and the attribute tree, determine whether the entities in the source domain and the target domain are the same entity;

[0009] When the entities are identified as the same entity, a similarity analysis is performed on the attributes and attribute values ​​of the entities to obtain the same attributes, similar attributes, newly added attributes, and the same attribute values, similar attribute values, and newly added attribute values.

[0010] Based on the similarity analysis results and the importance, the attributes and attribute values ​​of the entity are fused.

[0011] In one embodiment, the method further includes: if the entity has a unique code, then determining whether the entity is the same entity based on the unique code;

[0012] If the entity does not have a unique code, a comprehensive evaluation is performed based on the similarity between the entity name and the entity category. The entity name similarity is obtained using a text similarity analysis method, and the entity category similarity is 1 when the two entities are in the same category and 0.5 when the categories are different.

[0013] The formula for calculating the entity identity assessment value is as follows: ,in , These are the weights for entity name and category, respectively. , , These are name similarity and category similarity, respectively.

[0014] In one embodiment, the method further includes: comparing attribute names to obtain attribute pairs with the same name and a list of attributes with different names;

[0015] For attribute pairs with the same name, a comprehensive evaluation is conducted based on attribute category, level, parent attribute, attribute value, or child attribute to determine whether they are the same attribute;

[0016] For attribute pairs with inconsistent names, calculate the similarity of attribute names, select the attribute pair with the highest similarity, and conduct a comprehensive evaluation based on attribute category, level, parent attribute, attribute value, or child attribute.

[0017] The comprehensive evaluation of attributes uses a weighted scoring method, with the following weights for parent attribute, level, importance, attribute value, and child attribute: , , , , ;

[0018] When the evaluation value is greater than the threshold, the two attributes are determined to be the same attribute.

[0019] In one embodiment, the method further includes: when the attribute categories are different, determining whether the parent attributes of the two attributes are the same, whether the levels are the same, and whether the importance is similar, and calculating the similarity of the attribute values ​​or sub-attributes.

[0020] The criteria for determining similarity in importance are: ,in , These represent the importance of the two attributes, respectively.

[0021] When neither attribute has any child attributes, the similarity between the two attribute values ​​is calculated; when one attribute has child attributes and the other does not, both the similarity of attribute values ​​and the similarity of child attributes are set to 0; when both attributes have child attributes, the discrimination of child attributes is recursively performed according to the discrimination method of their parent attributes.

[0022] In one embodiment, the method further includes: when there are no entries for attribute values, using the BGE-LARGE-ZH vectorization model to encode the attribute name and attribute value, and converting the statement into a numerical vector in the same semantic space;

[0023] The similarity between vectors is calculated using cosine distance. The formula for cosine similarity is:

[0024] ;

[0025] in , Represents a vector;

[0026] Based on a set similarity threshold, determine whether the attribute values ​​are the same;

[0027] When an attribute has entries, one attribute of entity A has... There are several attribute values ​​for each item. One attribute of entity B has... The similarity of each item's attribute value is evaluated by cross-traversal comparison, resulting in a set of results:

[0028] ;

[0029] For sets Sort by numerical value in ascending order and select the most similar. ,if If the value is greater than the first threshold, then the entry is considered... With Entries If the attribute values ​​are the same, the two entries are considered similar if they are below the first threshold but above the second threshold; if they are below the second threshold, the two entries are considered not similar in attribute value.

[0030] In one embodiment, the method further includes: selecting one of the attributes that are determined to be the same and retaining it;

[0031] For attributes that are actually the same but have different names, select the corresponding attribute name of the entity corresponding to the domain with higher importance based on the importance of the domain.

[0032] For newly added sub-attributes, add the attribute and its sub-attributes or attribute values ​​to the corresponding position or level of the entity.

[0033] In one embodiment, the method further includes: for identical attribute values, selecting one to retain;

[0034] For similar attribute values, select the corresponding attribute value of the entity corresponding to the domain with higher importance based on the importance of the domain.

[0035] For similar attribute values ​​but with added content, the content of the two attribute values ​​is merged using a large language model;

[0036] For newly added attribute values, add them in the corresponding positions;

[0037] For different attribute values, if they are not divided into entries, they are merged as a whole using the large language model, or converted into entries; if they are divided into entries, the different attribute values ​​are added as new entries.

[0038] An entity fusion device for cross-domain data platform sharing and exchange, the device comprising:

[0039] The information reading module is used to read the basic entity information, attribute tree, and attribute value information of entities in the source domain and the target domain. The basic entity information includes entity code, entity name, and entity category. The attribute tree is constructed in a hierarchical structure, including first-level attributes, second-level attributes, and third-level attributes. The upper-level attributes can contain lower-level attributes or be directly assigned attribute values. The importance of the attributes is set according to the importance of the entity's attributes.

[0040] The entity determination module is used to determine whether the entities in the source domain and the target domain are the same entity based on the basic information of the entity and the attribute tree.

[0041] The similarity analysis module is used to perform similarity analysis on the attributes and attribute values ​​of the entity when they are determined to be the same entity, and to obtain the same attributes, similar attributes, newly added attributes, and the same attribute values, similar attribute values, and newly added attribute values.

[0042] The fusion processing module is used to perform fusion processing on the attributes and attribute values ​​of the entity based on the similarity analysis results and the importance.

[0043] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0044] Read the basic entity information, attribute tree and attribute value information of entities in the source domain and the target domain. The basic entity information includes entity code, entity name and entity category. The attribute tree is constructed in a hierarchical structure, including first-level attributes, second-level attributes and third-level attributes. The upper-level attributes can contain the lower-level attributes or be directly assigned attribute values.

[0045] The importance of an attribute is determined based on the importance of that attribute within the entity.

[0046] Based on the entity's basic information and the attribute tree, determine whether the entities in the source domain and the target domain are the same entity;

[0047] When the entities are identified as the same entity, a similarity analysis is performed on the attributes and attribute values ​​of the entities to obtain the same attributes, similar attributes, newly added attributes, and the same attribute values, similar attribute values, and newly added attribute values.

[0048] Based on the similarity analysis results and the importance, the attributes and attribute values ​​of the entity are fused.

[0049] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0050] Read the basic entity information, attribute tree and attribute value information of entities in the source domain and the target domain. The basic entity information includes entity code, entity name and entity category. The attribute tree is constructed in a hierarchical structure, including first-level attributes, second-level attributes and third-level attributes. The upper-level attributes can contain the lower-level attributes or be directly assigned attribute values.

[0051] The importance of an attribute is determined based on the importance of that attribute within the entity.

[0052] Based on the entity's basic information and the attribute tree, determine whether the entities in the source domain and the target domain are the same entity;

[0053] When the entities are identified as the same entity, a similarity analysis is performed on the attributes and attribute values ​​of the entities to obtain the same attributes, similar attributes, newly added attributes, and the same attribute values, similar attribute values, and newly added attribute values.

[0054] Based on the similarity analysis results and the importance, the attributes and attribute values ​​of the entity are fused.

[0055] The aforementioned entity fusion method, apparatus, and equipment for cross-domain data sharing and exchange, through complete attribute tree structure description, detailed similarity analysis process, conflict resolution mechanism based on domain importance, and large model-assisted text merging, can flexibly handle various entity modification types and significantly improve the accuracy and consistency of entity fusion in cross-domain data sharing and exchange. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of entity sharing and exchange in a cross-domain data platform in one embodiment;

[0057] Figure 2 This is a flowchart illustrating an entity fusion method in a cross-domain data platform sharing and exchange in one embodiment.

[0058] Figure 3 This is a schematic diagram of the entity attribute tree structure in one embodiment;

[0059] Figure 4 This is a schematic diagram of the overall entity fusion process in one embodiment;

[0060] Figure 5 This is a schematic diagram of the attribute similarity analysis process in one embodiment;

[0061] Figure 6 This is a structural block diagram of an entity fusion device in a cross-domain data platform sharing and exchange in one embodiment;

[0062] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] The entity fusion method provided in this application can be applied to, for example... Figure 1 The diagram illustrates a cross-domain data platform environment. The data platforms are deployed in Domain 1 and Domain 2, differing in level permissions, system functions, network locations, and data usage. They share and exchange data via the network. Entities, as core business objects, may be modified after inter-domain transmission, leading to inconsistencies.

[0065] In addition, several entities are stored and processed in domain 1 and domain 2. These entities have some overlap and some differences, such as Figure 1 As shown, due to business processing requirements, there is a process of entity sharing and exchange between the two domains. For example, entity A initially exists in domain 1 but not in domain 2. Later, as needed, it is transferred to domain 2 through the sharing and exchange mechanism. Domain 2 adds or modifies the information of entity A according to its own actual situation, which means that the information of the same entity A in domain 1 differs. Considering information consistency and to avoid information conflicts, regardless of whether domain 1 has modified the information of entity A during this time period, the information of entity A must be merged (synchronized and updated).

[0066] In one embodiment, such as Figure 2 As shown, a method for entity fusion in cross-domain data platform sharing and exchange is provided, and this method is applied to... Figure 1 Taking the data platform in the example, the explanation includes the following steps:

[0067] Step 202: Read the basic entity information, attribute tree, and attribute value information of entities in the source and target domains.

[0068] The basic information of an entity includes the entity code, entity name, and entity category. The attribute tree is constructed in a hierarchical structure, including first-level attributes, second-level attributes, and third-level attributes. A higher-level attribute may contain lower-level attributes or be directly assigned an attribute value.

[0069] Specifically, the basic information of the entity includes: entity code, such as EMP001; entity name, such as Zhang San; and entity category, such as employed.

[0070] Attribute tree information: such as Figure 3 As shown, entity attributes are organized in a tree structure. For example, basic information is a first-level attribute, which includes second-level attributes such as name and ID number; education experience is another first-level attribute, which can include multiple second-level attributes of academic qualifications, and each academic qualification can include third-level attributes such as school name, enrollment date, and graduation date.

[0071] Attribute value information: The specific numerical value or text corresponding to each terminal attribute.

[0072] Step 204: Set the importance of attributes based on the importance of the entity's attributes.

[0073] Next, assign importance to the retrieved attributes. For example, set the importance of the ID number to 0.95 (very high) and the importance of hobbies to 0.3 (low).

[0074] Step 206: Based on the entity's basic information and the attribute tree, determine whether the entities in the source domain and the target domain are the same entity.

[0075] The data platform first checks if both entities have entity codes. If both have codes and they are identical, they are determined to be the same entity. If one or both entities lack codes, a comprehensive evaluation is performed. For example, it calculates the similarity between the entity names Zhang San and Zhang San. =0.9, similarity between the entity categories of current employees and former employees. =0.5. Assuming name weight. =0.7, class weight If the threshold is 0.3, then the evaluation value E = 0.7 × 0.9 + 0.3 × 0.5 = 0.78. If the threshold is set to 0.75, then they are determined to be the same entity.

[0076] Step 208: When the entities are determined to be the same entity, perform similarity analysis on the entity's attributes and attribute values ​​to obtain the same attributes, similar attributes, newly added attributes, and the same attribute values, similar attribute values, and newly added attribute values.

[0077] Name comparison: Compare the names of all attributes of the two entities.

[0078] Same-name attribute analysis: For attributes with the same name, such as two entities both having a contact number, further comparison is made regarding their category, level, parent attribute, and importance. For example, if the contact number category in domain 1 is a string, while in domain 2 it is an encrypted string, then the categories are different. In this case, a weighted scoring method is used to comprehensively evaluate whether the parent attributes are both basic information, whether the levels are the same, and whether the importance is similar. For example, whether 0.98 × 0.8 ≤ 0.82 ≤ 1.02 × 0.8 is true, and whether the attribute values ​​are similar, etc., to finally determine whether they are the same attribute.

[0079] Heteronym analysis: For attributes with different names but potentially similar meanings, such as phone number and mobile phone number, calculate the semantic similarity between the names, and perform the above comprehensive evaluation on pairs with high similarity.

[0080] Attribute value similarity analysis:

[0081] No entry attribute value: such as personal profile text. The BGE-LARGE-ZH model is used to convert the two texts into vectors, and cosine similarity is calculated. If the similarity is higher than 0.9, they are considered identical; if it is between 0.7 and 0.9, they are considered similar.

[0082] There are entry attribute values, such as multiple educational records under Education Experience. All educational records for two entities are compared pairwise, and the similarity of each record pair is calculated. A threshold is used to determine which records are identical, similar, or different.

[0083] Step 210: Based on the similarity analysis results and the importance, the entity's attributes and attribute values ​​are fused.

[0084] Attribute fusion: For attributes determined to be identical, only one needs to be retained. For attributes determined to be similar but with different names, such as "phone number" and "mobile phone," the attribute name from the higher-importance domain is selected as the fused name based on preset domain importance (e.g., domain 1 is the core production domain and has higher importance than domain 2). For attributes determined to be newly added, such as the "emergency contact" attribute added to the entity in domain 2, it is fully added to the fused entity.

[0085] Attribute value fusion: For identical attribute values, retain one. For similar attribute values, if there is no substantial addition to the content (e.g., "Chaoyang District of Beijing" and "Chaoyang District"), select based on domain importance. For similar attribute values ​​but with added content (e.g., one version of a personal profile mentions proficiency in Java while another mentions proficiency in Python), input the two texts into a large language model, instruct it to merge the content, and output a merged text stating "proficient in Java" and "proficient in Python." For newly added attribute values, add them directly. For different attribute values, if they are in the form of separate entries (e.g., different educational records), retain all of them.

[0086] The entity fusion method in the cross-domain data sharing and exchange mentioned above, through complete attribute tree structure description, detailed similarity analysis process, conflict resolution mechanism based on domain importance, and large model-assisted text merging, can flexibly handle various entity modification types and significantly improve the accuracy and consistency of entity fusion in cross-domain data sharing and exchange.

[0087] In one embodiment, the entity determination step includes:

[0088] If an entity has a unique code, it is determined to be the same entity based on that unique code. If an entity does not have a unique code, a comprehensive evaluation is performed based on the similarity of its entity name and entity category. Entity name similarity is obtained using text similarity analysis methods, and entity category similarity is 1 when two entities belong to the same category and 0.5 when they belong to different categories. The formula for calculating the entity identity evaluation value is as follows: ,in , These are the weights for entity name and category, respectively. , , These are name similarity and category similarity, respectively.

[0089] In one embodiment, the similarity analysis step for entity attributes includes:

[0090] The attribute names are compared to obtain a list of attribute pairs with the same name and a list of attributes with different names. For attribute pairs with the same name, a comprehensive evaluation is performed based on attribute category, level, parent attribute, attribute value, or child attribute to determine whether they are the same attribute. For attribute pairs with different names, the attribute name similarity is calculated, and the attribute pair with the highest similarity is selected. A comprehensive evaluation is then performed based on attribute category, level, parent attribute, attribute value, or child attribute. The comprehensive evaluation of attributes uses a weighted scoring method, with the following weights for parent attribute, level, importance, attribute value, and child attribute: , , , , When the evaluation value is greater than the threshold, the two attributes are determined to be the same attribute.

[0091] like Figure 4 As shown, the specific steps are as follows:

[0092] Step 31: Filter out attributes that have the same attribute name, category, level, parent attribute, and attribute value (or child attribute), considering them to be the same attribute. When at least three attributes with the same name are found, if two attribute pairs with the same name remain after multi-factor analysis, proceed to Step 32 for further processing; if only one attribute remains, add it to the list of attributes to be compared for further analysis. If all are determined to be the same attribute and no attributes remain, the analysis ends.

[0093] Step 32: Sort the remaining attribute pairs by attribute level value and select the attribute pairs corresponding to the higher level values.

[0094] Step 33: Perform attribute pair analysis for different situations.

[0095] Step 331: Analysis and judgment under different categories.

[0096] If attributes differ in category, it is assumed that a domain may have modified the attribute categories within it. This determination is made by comprehensively evaluating factors such as parent attributes, level, and attribute values ​​(or child attributes).

[0097] Step 3311: Determine if the parent properties of the two properties are the same. If they are, the evaluation value is 1; otherwise, the evaluation value is 0.

[0098] Step 3312: Determine whether the two attributes belong to the same level. If so, increment the evaluation value by 1; otherwise, increment the evaluation value by 0.

[0099] Step 3313: Determine if the importance of the two attributes is similar. If so, increment the evaluation value by 1; otherwise, set the evaluation value to a certain difference value. (Importance values ​​range from [0,1]. Assume the importance values ​​of the two attributes are...) and The same judgment criteria are Otherwise, the difference value is );

[0100] Step 3314: Similarity determination of attribute values ​​or sub-attributes;

[0101] When neither attribute has any child attributes, calculate the similarity between the two attribute values, and set the similarity of the child attributes to 0.

[0102] When one attribute has child attributes while another attribute does not, both the attribute value similarity and the child attribute similarity are set to 0.

[0103] When both attributes have sub-attributes, the sub-attributes are recursively evaluated using the same method as their parent attributes to obtain similarity scores. The evaluation method employs a cross-tabulation comparison, taking the ratio of the number of sub-attributes deemed similar to the total number of sub-attributes as the similarity metric. Simultaneously, the attribute value similarity is set to 0.

[0104] Step 3315: Assume the weights of the parent attribute, level, importance, attribute value, and child attribute are as follows: , , , , A weighted scoring method is used to comprehensively evaluate attribute similarity.

[0105] Step 3316: If the evaluated value is greater than the threshold, the two attributes are determined to be the same attribute; otherwise, the two attributes are determined to be different attributes. The determination of sub-attributes is the same as that of attributes.

[0106] Step 332: Analysis and judgment under different levels.

[0107] Compare the parent attributes of the two attributes. If the parent attributes are the same (meaning the name, level, category, etc. are identical), there is no difference in level. If the parent attributes are different, it's possible that a domain has modified the name of an attribute, resulting in a name identical to the original attribute; a newly added attribute may have been named the same; or the attribute level may have been migrated. In all three cases, add the attribute to the comparison list for further evaluation with other attributes.

[0108] Step 333: Analysis and judgment under different parent attribute conditions.

[0109] There are three scenarios: first, an error occurs in determining the parent attribute's identity; second, a domain modifies the name of an attribute, resulting in an attribute with the same name; and third, a newly added attribute is set to the same name. The first scenario cannot be corrected during the analysis. For the second and third scenarios, the attribute is added to the list of attributes to be compared, pending further identification with other attributes.

[0110] Step 334: Analysis and judgment under different importance levels.

[0111] There are two scenarios: either the attribute was manually modified within a domain, or a new attribute was added or its name was modified. The handling methods are as follows:

[0112] Step 3341: For the first case, adopt the same comprehensive evaluation method as Step 331;

[0113] Step 3342: For the second case, add the attribute to the list of attributes to be compared, and leave it for further identification with other attributes.

[0114] Step 34: Remove the identified attribute pairs from the attribute list. If there are no attribute pairs left in the list, end the identification process; otherwise, go to Step 32.

[0115] In one embodiment, similarity analysis of entity attribute values ​​includes:

[0116] When there are no entries for attribute values, the BGE-LARGE-ZH vectorization model is used to encode the attribute name and attribute value, transforming the statement into a numerical vector within the same semantic space; the similarity between vectors is calculated using cosine distance, with the cosine similarity formula as follows:

[0117] ;

[0118] in , Represents a vector; based on a set similarity threshold, it determines whether attribute values ​​are the same; when there are entries for attribute values, one attribute of entity A has There are several attribute values ​​for each item. One attribute of entity B has... The similarity of each item's attribute value is evaluated by cross-traversal comparison, resulting in a set of results:

[0119] ;

[0120] For sets Sort by numerical value in ascending order and select the most similar. ,if If the value is greater than the first threshold, then the entry is considered... With Entries If the attribute values ​​are the same, the two entries are considered similar if they are below the first threshold but above the second threshold; if they are below the second threshold, the two entries are considered not similar in attribute value.

[0121] like Figure 5 As shown, the specific steps include:

[0122] 1. No entries for attribute content.

[0123] Step 11: Encoding of attribute names and attribute values.

[0124] Attribute names and values ​​are typically unstructured or semi-structured data in text form. Directly applying string-based matching methods (such as exact or fuzzy matching) to this text ignores semantic relationships. Encoding the data using a BGE-LARGE-ZH vectorization model effectively addresses this issue, transforming statements into numerical vectors within the same semantic space, capturing semantic information, and measuring semantic relationships between statements in subsequent tasks.

[0125] Step 12: Calculate the similarity of attribute names.

[0126] Similarity calculation quantifies the degree of matching between different entities on attribute names, providing a scientific basis for determining the same attribute. After attribute names or values ​​are encoded into high-dimensional vectors, the similarity between these vectors is calculated using cosine distance. The cosine similarity formula is as follows:

[0127] ;

[0128] in, Representing vectors I with vector J Cosine similarity between them, symbol " " represents the dot product operation of vectors, Represents the magnitude of a vector.

[0129] Step 13: Attribute value similarity determination.

[0130] Based on the set similarity threshold, it is determined whether these attribute values ​​are the same.

[0131] The same process described above can be used to determine the similarity of attribute names.

[0132] 2. The attribute content contains entries.

[0133] The items are cross-referenced and compared for evaluation. Then a comprehensive evaluation is performed. Assume entity A has m attribute values ​​for a certain attribute, and entity B has n attribute values ​​for a certain attribute. The similarity is evaluated through cross-reference, resulting in a set of results. The steps are as follows:

[0134] Step 21: Sort set S in ascending order of values.

[0135] Step 22: Select the most similar .if If it exceeds a certain first threshold, then the entry is considered... i With Entries jThe attribute values ​​are the same. If the values ​​are below the first threshold but above the second threshold, the two entries are considered similar, meaning they have the same attribute values ​​but also different attribute values. If the values ​​are below the second threshold, the two entries are considered not similar in attribute values.

[0136] Step 23: Remove from set S .

[0137] Step 24: If set S has a similarity value, proceed to Step 1; otherwise, end the analysis process.

[0138] The evaluation yielded identical attribute values, similar attribute values, and different attribute values.

[0139] In one embodiment, the attribute fusion process for entities includes: for attributes that are determined to be the same, one is selected and retained; for attributes that are actually the same but have different names, the corresponding attribute name of the entity corresponding to the domain with higher importance is selected according to the importance of the domain; for newly added sub-attributes, the attribute and sub-attributes or attribute values ​​are added to the corresponding position or level of the entity.

[0140] In another embodiment, the attribute value fusion process for entities includes: for identical attribute values, one is selected and retained; for similar attribute values, the corresponding attribute value of the entity corresponding to the domain with higher importance is selected based on the importance of the domain; for similar attribute values ​​but with added content, the content of the two attribute values ​​is merged using a large language model; for newly added attribute values, they are added in the corresponding position; for different attribute values, if they are not divided into entries, they are merged as a whole using a large language model, or converted into entries; if they are divided into entries, the different attribute values ​​are added as new entries.

[0141] It should be understood that, although Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0142] In one embodiment, such as Figure 6 As shown, an entity fusion device for cross-domain data platform sharing and exchange is provided, comprising:

[0143] The information reading module 602 is used to read the basic entity information, attribute tree, and attribute value information of entities in the source domain and the target domain. The basic entity information includes entity code, entity name, and entity category. The attribute tree is constructed in a hierarchical structure, including first-level attributes, second-level attributes, and third-level attributes. The upper-level attributes can contain lower-level attributes or be directly assigned attribute values. The importance of the attributes is set according to the importance of the entity's attributes.

[0144] The entity determination module 604 is used to determine whether the entities in the source domain and the target domain are the same entity based on the entity's basic information and the attribute tree.

[0145] The similarity analysis module 606 is used to perform similarity analysis on the attributes and attribute values ​​of the entity when they are determined to be the same entity, and to obtain the same attributes, similar attributes, newly added attributes, and the same attribute values, similar attribute values, and newly added attribute values.

[0146] The fusion processing module 608 is used to perform fusion processing on the attributes and attribute values ​​of the entity based on the similarity analysis results and the importance.

[0147] Specific limitations regarding the entity fusion device in cross-domain data platform sharing and exchange can be found in the limitations of the entity fusion method in cross-domain data platform sharing and exchange mentioned above, and will not be repeated here. Each module in the aforementioned entity fusion device in cross-domain data platform sharing and exchange can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0148] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores basic entity information, attribute trees, and attribute value information. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements an entity fusion method in a cross-domain data platform sharing and exchange.

[0149] Those skilled in the art will understand that Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0150] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the method described above.

[0151] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which is implemented when executed by a processor.

[0152] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0153] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An entity fusion method in cross-domain data middle platform sharing exchange, characterized in that, The method comprises: reading entity basic information, attribute tree and attribute value information of entities in a source domain and a target domain, the entity basic information comprising entity code, entity name and entity category, the attribute tree being constructed in a hierarchical structure and comprising first-level attributes, second-level attributes and third-level attributes, a superior attribute being capable of containing a subordinate attribute or directly assigning an attribute value; setting an importance of an attribute according to an importance of the attribute of the entity; judging whether the entities in the source domain and the target domain are the same entity based on the entity basic information and the attribute tree; when the entities are determined to be the same entity, performing similarity analysis on the attributes and attribute values of the entities to obtain same attributes, similar attributes, new attributes, same attribute values, similar attribute values and new attribute values; performing fusion processing on the attributes and attribute values of the entities based on the similarity analysis result and the importance; the similarity analysis on the attributes of the entities comprises: comparing attribute names to obtain attribute pairs with the same name and an attribute list with different names; based on attribute category, level, parent attribute, attribute value or child attribute, comprehensively evaluating the attribute pairs with the same name to determine whether they are the same attribute; calculating attribute name similarity, selecting the attribute pair with the greatest similarity, and comprehensively evaluating the attribute pairs with different names based on attribute category, level, parent attribute, attribute value or child attribute; The comprehensive evaluation of the attribute adopts a weighted scoring method, and the weights of the parent attribute, level, importance, attribute value, and child attribute are respectively , , , , ; when the evaluation value is greater than a threshold value, determining that the two attributes are the same attribute; the comprehensive evaluation of the attributes comprises: when the attribute categories are different, judging whether the parent attributes, the levels and the importance of the two attributes are the same, and calculating the similarity of the attribute values or child attributes; The criterion for determining the similarity of importance is: wherein , are the importance of the two attributes, respectively. when both attributes have no child attribute, calculating the similarity of the attribute values; when one attribute has a child attribute and the other attribute has no child attribute, the attribute value similarity and the child attribute similarity are both set to 0; when both attributes have child attributes, the judgment of the child attributes is recursively performed according to the judgment method of the parent attributes; the similarity analysis on the attribute values of the entities comprises: when the attribute value has no entry, using a BGE-LARGE-ZH vectorization model to encode the attribute name and the attribute value, and converting a sentence into a numerical vector in the same semantic space; calculating the similarity between vectors by a cosine distance, and a cosine similarity formula is: wherein , denotes a vector; based on a set similarity threshold value, determining whether the attribute values are the same; When the attribute value has entries, one attribute of entity A has entry attribute values, one attribute of entity B has entry attribute values, cross traversal comparison is performed to evaluate the similarity, and a result set is obtained: to the set ordered in ascending numerical order, the one with the greatest similarity is selected if is greater than a first threshold value, the entry is considered to have the same attribute value as entry ; if it is lower than the first threshold value and greater than a second threshold value, the two entries are considered to be similar in content; if it is lower than the second threshold value, the two entries are considered to have no similarity in attribute values.

2. The method of claim 1, wherein, the judgment of whether the entities in the source domain and the target domain are the same entity comprises: if the entity has a unique code, determining the same entity according to the unique code; if the entity has no unique code, comprehensively evaluating the similarity based on the entity name and the entity category; the entity name similarity is obtained by a text similarity analysis method, and the entity category similarity is 1 when the two entity categories are the same, and is 0.5 when the categories are different; The entity identity evaluation value calculation formula is: wherein , are the weights of the entity name and the category, respectively, , , are the name similarity and the category similarity, respectively.

3. The method of claim 1, wherein, the fusion processing on the attributes of the entities comprises: retaining one of the attributes determined to be the same; for the attributes actually the same but with different names, selecting the attribute name corresponding to the entity in the domain with high importance according to the importance of the domains; for the newly added child attributes, adding the newly added child attributes or attribute values at the corresponding positions or levels of the entities.

4. The method of claim 1, wherein, The attribute value of the entity is fused, including: For the same attribute value, one is selected to be reserved; For similar attribute values, according to the importance of the domain, the corresponding attribute value of the entity corresponding to the domain with higher importance is selected; For similar attribute values with content increase, the large language model is used to combine the contents of the two attribute values; For the new attribute value, it is added in the corresponding position; For different attribute values, if not divided into items, the large language model is used to combine the whole, or changed to the form of dividing items; if divided into items, the different attribute values are added as new items.

5. An entity fusion apparatus in cross-domain data hub sharing exchange, configured to implement the entity fusion method in cross-domain data hub sharing exchange according to any one of claims 1 to 4. The device comprises: An information reading module is configured to read entity basic information, attribute tree and attribute value information of entities in source and target domains, the entity basic information comprises entity code, entity name and entity category, the attribute tree is constructed in a hierarchical structure, comprising first-level attributes, second-level attributes and third-level attributes, upper-level attributes can contain lower-level attributes or directly assign attribute values; the importance of the attribute is set according to the importance of the attribute of the entity; An entity judgment module is configured to judge whether the entities in the source and target domains are the same entity based on the entity basic information and the attribute tree; A similarity analysis module is configured to analyze the similarity of the attributes and attribute values of the entity when it is determined to be the same entity, and obtain the same attribute, similar attribute, new attribute, same attribute value, similar attribute value and new attribute value; A fusion processing module is configured to fuse the attributes and attribute values of the entity based on the similarity analysis result and the importance. 6.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-5 when the computer program is executed by the processor. The processor executes the computer program to realize the steps of the method in any one of claims 1 to 4.

Images