A multi-source information-oriented cross-platform entity semantic disambiguation method and system

Abstract

The application provides a multi-source information-oriented cross-platform entity semantic disambiguation method and system, and relates to the technical field of semantic processing.The method comprises the following steps: step 1, obtaining multi-source information data, formulating a differentiated context extraction strategy according to the data characteristics of different information source platforms, extracting entity references and the text around the entity references from the original text as a context window to obtain the combination of the entity references and the context; and step 2, inputting the combination of the entity references and the context into a pre-trained language model for encoding, mapping to a high-dimensional semantic space, and generating a semantic fingerprint vector.The application improves the accuracy of entity recognition and the efficiency of information fusion by improving entity disambiguation from string matching to semantic space.

Description

Technical Field

[0001] This invention relates to the field of semantic processing technology, and in particular to a cross-platform entity semantic disambiguation method and system for multi-source intelligence. Background Technology

[0002] In the field of cyberspace security and intelligence analysis, multi-source intelligence fusion is an important step in building high-quality knowledge graphs. Entities with the same name that frequently appear on different platforms may refer to the same object or completely different individuals. Existing technologies often face some challenges in dealing with such cross-platform entity recognition problems.

[0003] Traditional intelligence processing methods mostly rely on string matching or simple rules for entity alignment, such as associating entities based on the literal consistency of their names. However, in practical applications, entities with the same name may have similar names on different platforms but point to different objects. Simply relying on string matching is often insufficient to accurately distinguish them. In addition, existing entity disambiguation methods mostly focus on the internal workings of a single platform and lack effective integration of cross-platform contextual information. The contextual information provided by a single platform is often limited and cannot fully characterize the implicit attributes of entities, thus affecting the accuracy of cross-platform entity association. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a cross-platform entity semantic disambiguation method and system for multi-source intelligence, which improves the accuracy of entity recognition and the efficiency of intelligence fusion by elevating entity disambiguation from string matching to semantic space.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: Firstly, a cross-platform entity semantic disambiguation method for multi-source intelligence, the method comprising: Step 1: Obtain multi-source intelligence data and formulate differentiated context extraction strategies based on the data characteristics of different intelligence source platforms. Extract entity references and surrounding text from the original text as context windows to obtain a combination of entity references and context. Step 2: Input the combination of entity reference and context into the pre-trained language model for encoding, map it to a high-dimensional semantic space, and generate a semantic fingerprint vector; Step 3: Based on semantic fingerprint vectors, entities are divided into candidate groups by referential association rules. Within each candidate group, entities are paired up and the cosine similarity of semantic fingerprint vectors between paired entities is calculated. A context-aware dynamic threshold strategy is used to retain candidate pairs with cosine similarity greater than the dynamic threshold, thus completing the coarse screening of entity pairs. Step 4: For each candidate pair obtained after coarse screening, input the context text of the two entities to be compared in the candidate pair into the two encoders with shared weights of the deep semantic matching model based on the dual-tower structure, and calculate the matching score of the two entities; compare the matching score with the preset judgment threshold, and when the matching score exceeds the judgment threshold, it is initially determined that the two entities refer to the same object, and a high-confidence matching result is obtained. Step 5: Use the high-confidence matching results as anchors to construct an entity association graph. Use the label propagation algorithm to perform weighted iterative updates on the confidence of other candidate pairs in the entity association graph, and assign a globally unique identifier to each independent entity.

[0006] Secondly, a cross-platform entity semantic disambiguation system for multi-source intelligence includes: The extraction module is used to acquire multi-source intelligence data. It formulates differentiated context extraction strategies based on the data characteristics of different intelligence source platforms. It extracts entity references and surrounding text from the original text as context windows to obtain a combination of entity references and context. The encoding module is used to input the combination of entity reference and context into the pre-trained language model for encoding, mapping to a high-dimensional semantic space, and generating semantic fingerprint vectors. The coarse screening module is used to divide entities into candidate groups based on semantic fingerprint vectors and referential association rules. Within each candidate group, entities are paired up and the cosine similarity of semantic fingerprint vectors between paired entities is calculated. A context-aware dynamic threshold strategy is used to retain candidate pairs with cosine similarity greater than the dynamic threshold, thus completing the coarse screening of entity pairs. The matching module is used to input the context text of the two entities to be compared in each candidate pair obtained after coarse screening into two encoders with shared weights in a deep semantic matching model based on a dual-tower structure, calculate the matching score of the two entities, compare the matching score with a preset judgment threshold, and preliminarily determine that the two entities refer to the same object when the matching score exceeds the judgment threshold, thus obtaining a high-confidence matching result. The allocation module is used to construct an entity association graph by using high-confidence matching results as anchors. It employs a label propagation algorithm to perform weighted iterative updates on the confidence of other candidate pairs in the entity association graph and assigns a globally unique identifier to each independent entity.

[0007] The above-described solution of the present invention has at least the following beneficial effects: Differentiated context extraction strategies are adopted to address the data characteristics of different intelligence source platforms, adapting to multi-source intelligence data with significant differences in text structure and expression style. Entity references and their effective contexts are extracted, reducing entity recognition errors caused by heterogeneous data sources. Semantic fingerprint vectors of entities are generated through a pre-trained language model, and candidate entities are coarsely screened by combining reference association rules and context-aware dynamic thresholds. This can quickly filter a large number of irrelevant entity pairs while ensuring recall, improving overall disambiguation efficiency. A deep semantic matching model with a shared weight dual-tower structure is adopted to fully learn the deep semantic information of entity context, measure the semantic similarity between different entity references, and construct an entity association graph using high-confidence matching results as anchors. The association confidence between entities is iteratively updated by combining a label propagation algorithm, realizing disambiguation decision-making from local matching to global uniformity. A globally unique identifier is assigned to each entity to ensure the consistency and uniqueness of entity objects under cross-platform and multi-source intelligence. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating a cross-platform entity semantic disambiguation method for multi-source intelligence provided by an embodiment of the present invention.

[0009] Figure 2 This is a schematic diagram of a cross-platform entity semantic disambiguation system for multi-source intelligence provided by an embodiment of the present invention. Detailed Implementation

[0010] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0011] like Figure 1 As shown, embodiments of the present invention propose a cross-platform entity semantic disambiguation method for multi-source intelligence, the method comprising the following steps: Step 1: Obtain multi-source intelligence data and formulate differentiated context extraction strategies based on the data characteristics of different intelligence source platforms. Extract entity references and surrounding text from the original text as context windows to obtain a combination of entity references and context. Step 2: Input the combination of entity reference and context into the pre-trained language model for encoding, map it to a high-dimensional semantic space, and generate a semantic fingerprint vector; Step 3: Based on semantic fingerprint vectors, entities are divided into candidate groups by referential association rules. Within each candidate group, entities are paired up and the cosine similarity of semantic fingerprint vectors between paired entities is calculated. A context-aware dynamic threshold strategy is used to retain candidate pairs with cosine similarity greater than the dynamic threshold, thus completing the coarse screening of entity pairs. Step 4: For each candidate pair obtained after coarse screening, input the context text of the two entities to be compared in the candidate pair into the two encoders with shared weights of the deep semantic matching model based on the dual-tower structure, and calculate the matching score of the two entities; compare the matching score with the preset judgment threshold, and when the matching score exceeds the judgment threshold, it is initially determined that the two entities refer to the same object, and a high-confidence matching result is obtained. Step 5: Use the high-confidence matching results as anchors to construct an entity association graph. Use the label propagation algorithm to perform weighted iterative updates on the confidence of other candidate pairs in the entity association graph, and assign a globally unique identifier to each independent entity.

[0012] In this embodiment of the invention, a differentiated context extraction strategy is adopted for the data characteristics of different intelligence source platforms. This strategy adapts to multi-source intelligence data with significant differences in text structure and expression style, extracts entity references and their effective context, and reduces entity recognition errors caused by heterogeneous data sources. Entity semantic fingerprint vectors are generated through a pre-trained language model, and candidate entities are coarsely screened by combining reference association rules and context-aware dynamic thresholds. This can quickly filter a large number of irrelevant entity pairs while ensuring recall, thus improving overall disambiguation efficiency. A dual-tower structure deep semantic matching model with shared weights is adopted to fully learn the deep semantic information of entity context, measure the semantic similarity between different entity references, construct an entity association graph using high-confidence matching results as anchors, and iteratively update the association confidence between entities using a label propagation algorithm. This achieves disambiguation decision-making from local matching to global uniformity, and assigns a globally unique identifier to each entity to ensure the consistency and uniqueness of entity objects across platforms and multi-source intelligence.

[0013] In a preferred embodiment of the present invention, step 1 above, which involves acquiring multi-source intelligence data and formulating differentiated context extraction strategies based on the data characteristics of different intelligence source platforms, extracting entity references and surrounding text from the original text as context windows to obtain a combination of entity references and context, may include: In this embodiment of the invention, step 110 involves collecting raw text data from multiple heterogeneous intelligence source platforms to obtain multi-source raw text. This multi-source raw text is then preprocessed to remove HTML tags, special symbols, and irrelevant noise data, resulting in cleaned text. Specifically, this includes: employing a combination of distributed web crawler technology and platform open API interface access technology, simultaneously connecting to multiple heterogeneous intelligence source platforms, including short text social platforms, long text news platforms, and long text interactive platforms. Following the text data storage format and data transmission specifications of each platform, all publicly available unstructured text stream data from each platform is collected, including posts, articles, comments, and dynamics containing all text content with entity references. All raw text data collected from different heterogeneous platforms is then uniformly summarized and formatted to form a multi-source raw text dataset containing text from multiple platforms and of multiple types. For the summarized multi-source raw text dataset, noise removal and data cleaning operations are performed text-by-text and character-by-character. First, a regular expression matching algorithm is used to identify and batch delete all HTML tags contained in the text, including... 、 , The text is preprocessed by removing various tags and invalid content within them. Then, a character matching library is used to identify and remove various meaningless special symbols from the text, including special symbols other than punctuation marks, garbled characters, platform-specific symbols, etc. Finally, a combination of manual and machine verification is used to remove irrelevant noise data from the text, including repetitive and redundant whitespace characters, semantically meaningless number strings, advertising copy that is irrelevant to entity reference and context, platform watermarks, etc. Only the core content with actual semantics and value for inferring entity attributes is retained. This completes the full-dimensional preprocessing of all multi-source original texts, and finally yields a cleaned text dataset that is noise-free, redundant, and semantically complete.

[0014] Step 111 involves identifying and extracting entity references from the cleaned text. Based on the platform type of the entity reference, a corresponding context window is extracted from the cleaned text. If the entity reference originates from a short text platform, the entire content of that text is used as the context window for that entity reference. If the entity reference originates from a long text platform, a text segment containing a predetermined number of tokens before and after the entity reference is extracted as the context window for that entity reference. Specifically, this includes using a pre-trained language model-based named entity recognition technology. The cleaned text dataset obtained in Step 110 is used as input to a named entity recognition model that has undergone fine-tuning of the Chinese corpus. This model captures entity features in the text through a multi-layer self-attention mechanism, identifying and locating various entity references such as people, organizations, regions, and institutions within the text. Simultaneously, it performs boundary delineation and type labeling on the identified entity references to avoid... The system identifies and corrects omissions and errors in entity reference extraction. Then, it extracts all entity references identified and labeled by the model, forming independent entity reference sets. Each entity reference is labeled with its corresponding original cleaned text and the information of its intelligence source platform, achieving a one-to-one correspondence between entity references, original text, and platform. For each entity reference in the entity reference set, a binary judgment is made based on its labeled intelligence source platform information, clearly classifying platforms into two categories: short text platforms and long text platforms. The judgment criteria are as follows: platforms with fragmented, short texts as their main content format, typically with fewer tokens per text and contextual information concentrated within a single text, are classified as short text platforms; platforms with long, structured texts as their main content format, typically with more tokens per text and contextual information scattered throughout the text, such as news websites, industry forums, and information portals, are classified as long text platforms.

[0015] For each entity reference whose platform type has been determined, a unique context window is extracted from its corresponding cleaned original text according to the extraction rules matching the platform type. The extraction process ensures the semantic relevance between the context window and the entity reference. Specifically, if the entity reference originates from a short text platform, the entire cleaned text containing the entity reference is used as the context window without any text truncation, ensuring the preservation of complete contextual information for the entity reference in the short text context, providing comprehensive short text context support for entity attribute inference. If the entity reference originates from a long text platform, the character position of the entity reference is first located in its corresponding cleaned original text. Using the center position of the entity reference as a reference, a text fragment of a preset number of tokens is extracted forward, and simultaneously, a text fragment of the same preset number of tokens is extracted backward. These symmetrically extracted text fragments are then concatenated to form a context window. A context window is formed for the entity reference. In this embodiment, the preset number is preferably 256, that is, a text segment of 256 tokens before and after the entity reference is fixed as the context window. This setting ensures that the context window can cover the core semantic information around the entity reference, maintain semantic coherence, and effectively control the text length of the context window, avoiding the problem of excessive computation and low efficiency in the semantic encoding process. Semantic integrity is checked for all extracted context windows. If the entity reference is located at the beginning or end of the text in the long text platform, and there is a situation where less than 256 tokens can be extracted before / after, the actual number of tokens that can be extracted is directly extracted with the beginning / end of the text as the boundary to ensure the integrity of the context window. At the same time, invalid content in the context window that has no semantic relationship with the entity reference is removed, and the final correction of the context window is completed, forming a unique and valid context window for each entity reference.

[0016] Step 112 involves combining each extracted entity reference with its corresponding context window to form an entity reference-context combination. Specifically, this includes: for each entity reference extracted in Step 111, establishing a one-to-one mapping relationship between the entity reference and its verified and corrected context window, using the text content of the entity reference as a unique identifier. This ensures that each entity reference corresponds to only one context window, and each context window serves only its corresponding entity reference, avoiding mapping confusion, one-to-many, or many-to-one situations. The mapped and bound individual entity references are then integrated with their corresponding context windows to form an independent, standardized entity reference-context window combination unit. In this unit, the entity reference is the core identifier, and the context window provides semantic context support for the entity reference. The combination unit also retains basic information such as the platform type and original text source of the entity reference. This integration operation is performed on all entity references in the entity reference set, summarizing all independent combination units to form an entity reference-context combination set containing all entity references and their corresponding context information.

[0017] By employing a multi-source collection method combining distributed crawlers and API calls, comprehensive and efficient collection of text data from various heterogeneous intelligence source platforms is achieved. This provides a complete and comprehensive data source foundation for cross-platform entity semantic disambiguation, ensuring that the disambiguation process can cover entity information from multiple platforms.

[0018] In a preferred embodiment of the present invention, step 2 above, which involves inputting the combination of entity reference and context into a pre-trained language model for encoding and mapping to a high-dimensional semantic space to generate a semantic fingerprint vector, may include: In this embodiment of the invention, step 220 involves combining the entity reference and its corresponding context window into an input sequence that meets the requirements of the pre-trained language model for each combination of entity reference and context. Specifically, this includes: determining the pre-trained language model and core input specifications; selecting the BERT-base-chinese model as the core pre-trained language model, which is a deep pre-trained model optimized for Chinese scenarios. Its input sequence has specific format and structural requirements. The core specification is that the input sequence starts with a special classification identifier [CLS] and uses a special separator identifier [SEP] as the separator for different text segments and the end position of the sequence. The total number of tokens in the input sequence must match the model's maximum processing length (512 tokens). (n) The input sequence needs to complete the full-dimensional feature transformation of tokenization, word vector encoding, position encoding, and segment encoding before it can be effectively encoded by the model; The basic text concatenation of entity reference and context window: For each independent entity reference-context window combination unit obtained in step 1, the complete text content of the entity reference and the complete text content of the context window in the unit are extracted separately. The text fragments are concatenated according to the fixed word order of [CLS] identifier + complete text of context window + [SEP] identifier + complete text of entity reference + [SEP] identifier to form an initial text sequence without redundancy and missing information. This ensures the semantic logic of contextual information first and entity core identifier second, allowing the model to capture contextual features first and then focus on entity features.

[0019] The attribute inference prompt enhancement text sequence extension (optional preferred scheme) is designed to further enhance the model's ability to represent the implicit attributes of entities. This embodiment sets a preferred implementation of the attribute inference prompt enhancement technology. After the basic initial text sequence, a concatenation operation is performed. The concatenated content is the attribute inference prompt question text + [SEP] identifier, forming an enhanced initial text sequence. The complete format after concatenation is [CLS] identifier + complete context window text + [SEP] identifier + complete entity reference text + [SEP] identifier + prompt question text + [SEP] identifier. The prompt question text is a standardized question about the implicit attributes of the entity. It is constructed as a fixed sentence structure, such as "What are the most likely occupations, regions, and associated organizations of {entity reference} mentioned in the text?". The placeholders for {entity reference} in the sentence structure are replaced with the actual entity reference text content in the corresponding combination unit, so that the prompt question corresponds one-to-one with the entity reference, guiding the model to focus on attribute feature mining.

[0020] Tokenization and segmentation processing of the initial text sequence. The basic initial text sequence or enhanced initial text sequence is input into the Chinese WordPieceToken tokenizer配套 with the BERT-base-chinese model to perform word-by-word and word-by-word discretization segmentation on the continuous text content, converting the text into a discrete Token sequence recognizable by the model. The segmentation process strictly follows Chinese semantic rules, avoiding splitting words with complete semantics. At the same time, all special identifiers [CLS] and [SEP] are treated as independent Tokens to participate in the segmentation, ensuring the integrity of the Token sequence; Length adaptation and completion of the Token sequence. The length of the segmented discrete Token sequence is verified. Based on the maximum processing length of 512 Tokens of the BERT-base-chinese model, length adaptation operations are performed. If the total length of the Token sequence exceeds 512 Tokens, truncation processing is performed starting from the end of the context window text, retaining the core preamble content of the context window, the entity reference Prompt question text (if any), and all special identifiers, ensuring that the core information related to the entity is not truncated; If the total length of the Token sequence is less than 512 Tokens, after the last [SEP] identifier in the sequence, the model-specific padding identifier [PAD] is added until the total length of the Token sequence reaches 512 Tokens, finally obtaining a discrete Token sequence with standardized length and normalized structure.

[0021] The full-dimensional feature encoding of the standardized token sequence involves generating and fusing three types of features—word vectors, positional encoding, and segmental encoding—required for model input after length adaptation. Specifically, word vector encoding matches each discrete token in the sequence with a 768-dimensional word vector pre-trained from the BERT-base-chinese model, resulting in a word vector feature representing the basic semantic information of the token. Positional encoding assigns a unique 768-dimensional positional feature vector to each token based on its position, representing its location within the sequence and addressing the model's inability to recognize text word order. Segmental encoding encodes different text segments (context windows, entity references, Prompt questions) within the token sequence. Each text segment is assigned a different 768-dimensional segment feature vector. All tokens within the same text segment share the same segment coding feature, while the segment coding features of different text segments are distinguished from each other, resulting in a segment coding feature corresponding to each token. This feature represents the text segment type to which the token belongs. Feature fusion involves adding the word vector feature, position coding feature, and segment coding feature corresponding to each token element-wise, i.e., the first dimension of the word vector plus the first dimension of the position coding plus the first dimension of the segment coding, and so on up to 768 dimensions, to obtain the fused feature vector of each token. The fused feature vectors of all tokens are arranged in sequence to form the final standardized input sequence, which is a 768-dimensional feature vector sequence that fully meets the input requirements of the BERT-base-chinese model. Each entity referencing and context combination unit generates a unique standardized input sequence.

[0022] Step 221 involves inputting the input sequence into a deep pre-trained language model. The multi-layer self-attention mechanism of the deep pre-trained language model maps discrete text symbols into continuous high-dimensional semantic space vectors. Specifically, the deep pre-trained language model is constructed based on the BERT-base-chinese model for task adaptation. The model's structure and parameters are set to include 12 Transformer encoding layers, each consisting of a multi-head self-attention mechanism module and a feedforward neural network connected in series. The hidden layer dimension is fixed at 768 dimensions, and the number of attention heads in the multi-head self-attention mechanism is set to 12, meaning the 768-dimensional feature vector is split into 12 independent attention heads (64 dimensions each) for parallel computation. The feedforward neural network includes two linear transformation layers with a GELU activation function introduced for non-linear mapping. The output layer dimension is the same as the hidden layer, at 768 dimensions. After model construction, the general weight parameters pre-trained on a large-scale Chinese corpus by the BERT-base-chinese model are loaded to ensure the model possesses basic Chinese semantic understanding and feature extraction capabilities.

[0023] To adapt the model to the encoding requirements of cross-platform entity semantic disambiguation, supervised contrastive learning fine-tuning of the model with general weights was performed using a multi-source intelligence entity disambiguation annotation dataset. Specifically, entity pair samples were collected from multiple platforms and divided into positive sample pairs (different platform references to the same entity) and negative sample pairs (references to different entities with the same name). The standardized input sequence of the sample pairs was input into the model. The optimization objective was to narrow the semantic vector distance of positive sample pairs and widen the semantic vector distance of negative sample pairs. A contrastive loss function was used to calculate the loss value between the model's predicted value and the sample label. The loss value was backpropagated layer by layer through the backpropagation algorithm to update the weight parameters of each Transformer encoding layer of the model. The training was iterated repeatedly until the loss value converged, resulting in a deep pre-trained language model that is task-adapted and specifically designed for cross-platform entity disambiguation. The standardized input sequence (768-dimensional feature vector sequence) obtained in step 220 is input into the finely tuned deep pre-trained language model. It then undergoes progressive deep semantic encoding through 12 Transformer encoding layers. Each encoding layer performs multi-head self-attention mechanism calculation and feedforward neural network transformation. Specifically, the intra-layer processing is as follows: For multi-head self-attention mechanism calculation, the input 768-dimensional feature vector sequence is first subjected to three linear transformations to obtain the query vector Q, key vector K, and value vector V, where Q, K, and V are all 768-dimensional. Q, K, and V are then split into 12 attention heads, each with 64 dimensions. Attention weights are calculated for each attention head, i.e., by performing matrix multiplication on the transpose of the query vector Q and the key vector K, and then dividing the result by a scaling factor. ( The dimension of each attention head is 64. (8) Finally, the result is normalized using the Softmax function to obtain the attention weights between tokens under each attention head. These weights represent the semantic association between a single token and all other tokens in the sequence. Then, the attention weights of each attention head are multiplied by the corresponding value vector V to obtain the feature output of each attention head. The feature outputs of the 12 attention heads are concatenated and then a linear transformation is performed to restore the 768-dimensional feature vector, resulting in the output features of the multi-head self-attention mechanism. This achieves the capture of semantic associations between tokens across positions and segments. Residual connection Layer normalization involves performing an element-wise addition residual concatenation operation between the output features of the multi-head self-attention mechanism and the original input features of the Transformer encoding layer. The result is then input into layer normalization to normalize each dimension of the feature vector, making the feature distribution more stable and avoiding the gradient vanishing problem during model training and inference. The feedforward neural network transformation inputs the layer-normalized feature vector into a feedforward neural network. First, a first-layer linear transformation maps the 768-dimensional features to 3072 dimensions. Then, a non-linear feature transformation is performed using the GELU activation function. Finally, a second-layer linear transformation maps the 3072-dimensional features to... The 768-dimensional features are restored to 768 dimensions, yielding the final output features of this Transformer encoding layer. Secondary residual connections and layer normalization are performed, adding element-wise residual connections between the output features of the feedforward neural network and the layer-normalized features output by the multi-head self-attention mechanism, followed by layer normalization, to obtain the final feature vector sequence encoded by this layer, which serves as the input to the next Transformer encoding layer. The mapping from discrete text symbols to a high-dimensional semantic space is achieved through a progressive encoding process across 12 Transformer encoding layers. The model transforms the discrete token feature vectors of the input sequence from... The initial basic text symbol space is gradually mapped to a 768-dimensional high-dimensional semantic feature space. Each layer of encoding performs semantic enhancement, association fusion, and attribute mining on the features. Finally, the feature vector sequence output by the 12th (last) Transformer encoding layer is a continuous high-dimensional semantic feature vector sequence that integrates contextual information, core entity referential information, and implicit entity attribute information (occupation, region, affiliated organization, etc.). The feature vector corresponding to each token is a 768-dimensional continuous real number vector, realizing a complete and accurate mapping from discrete text symbols to continuous high-dimensional semantic space vectors.

[0024] Step 222: Extract the CLS position vector from the output of the last layer of the pre-trained language model, and use this vector as the semantic fingerprint vector of the entity reference. Specifically, this includes: locating the CLS position feature vector. The output of the BERT-base-chinese model is a 768-dimensional feature vector sequence (512 feature vectors in total) with the same length as the input sequence. The first position in the sequence is the feature vector corresponding to the special classification identifier [CLS]. This vector is the core feature vector obtained by the model through 12 Transformer encoding layers, which globally aggregates and semantically fuses all token features of the entire input sequence. It fully represents all global semantic information of the context, entity reference, and latent attributes in the input sequence, and is the semantic core of the entire input sequence. This step involves the last layer (the 12th layer) of the Transformer model in step 221. The 768-dimensional high-dimensional semantic feature vector sequence output by the encoding layer is used to precisely locate the feature vector corresponding to the [CLS] identifier at the first position in the sequence, which serves as the core semantic feature vector of the entity to be identified. The formal determination and attribute representation of the semantic fingerprint vector directly determines the [CLS] position feature vector extracted above as the semantic fingerprint vector of the entity reference. This semantic fingerprint vector is a fixed 768-dimensional continuous high-dimensional real number vector, and its core characteristic is that it implicitly encodes key implicit attribute information such as occupation, region, and affiliated organization of the entity reference. Even if the attribute definition of the entity is not explicitly mentioned in the context window corresponding to the entity reference, the model can implicitly encode these attribute features into the semantic fingerprint vector through semantic mining of the associated descriptions such as code submissions, GitHub repositories, and version iterations in the context window, so that the vector has the dual ability of entity identification and attribute representation.

[0025] The semantic fingerprint vector validity verification and anomaly correction involves performing a full-dimensional validity verification on the extracted semantic fingerprint vectors to ensure their usability for subsequent similarity calculations and entity matching. The specific verification content and processing methods are as follows: Dimensional integrity verification: Checking if the vector has 768 dimensions. If dimensions are missing or redundant, immediately re-execute all operations 220-222 for the corresponding entity reference and context combination to regenerate the semantic fingerprint vector; Numerical validity verification: Calculating the magnitude of the semantic fingerprint vector, i.e., squaring each of the 768-dimensional values ​​of the vector, summing all the squared results, and finally taking the square root of the sum to obtain the vector magnitude; If the magnitude is 0 or infinitely close to 0, it is determined to be an invalid vector and immediately regenerated; if the magnitude is within a reasonable range, it is considered invalid. For valid vectors, feature distribution verification compares the semantic fingerprint vector with the feature distribution of positive sample vectors during model fine-tuning. If the deviation is too large, it is judged as a feature anomaly, and the vector is regenerated. For the association storage of entity references and semantic fingerprint vectors, a unique one-to-one mapping relationship is established between the semantic fingerprint vector that passes the validity verification and its corresponding entity reference. Basic information such as the text content of the entity reference, its intelligence source platform, context window text, and platform type are bound and stored with the 768-dimensional semantic fingerprint vector of that entity reference, forming an entity reference-semantic fingerprint vector mapping library. Each entity reference in the mapping library is assigned an independent index identifier, facilitating the rapid retrieval of the corresponding semantic fingerprint vector during the candidate pair coarse screening and fine ranking steps. This mapping library provides core feature data support for all subsequent entity matching operations.

[0026] Standardization is achieved throughout the entire process, from text concatenation and tokenization to feature encoding. Standardized attribute questions guide the model to focus on mining implicit attributes such as entity occupation and region, enabling the generated semantic fingerprint vector to have stronger attribute representation capabilities and improving the entity identity identification capability of the semantic fingerprint vector.

[0027] In a preferred embodiment of the present invention, step 3 above, based on semantic fingerprint vectors, divides entities into candidate groups using referential association rules, pairs entities within each candidate group, calculates the cosine similarity of semantic fingerprint vectors between paired entities, and uses a context-aware dynamic threshold strategy to retain candidate pairs with cosine similarity greater than the dynamic threshold, thus completing the coarse screening of entity pairs, may include: In this embodiment of the invention, step 330 involves grouping all entity references corresponding to the semantic fingerprint vector according to the reference association rules, and classifying entity references that meet the rules into the same candidate group to obtain multiple candidate groups. Specifically, this includes: determining the core rules for reference association, namely the literal consistency rule, the edit distance rule, and the alias relationship rule in the multi-source heterogeneous knowledge base. The three types of rules are judged in parallel. As long as an entity reference meets any one of the rules, it is judged as an associated entity and classified into the same candidate group, ensuring a high recall rate for the candidate group division and covering all potential associated entity references with the same name, similar spelling, and alias association; traversing all entity references in the constructed entity reference-semantic fingerprint vector mapping library, performing a word-by-word and symbol-by-symbol complete matching comparison of the text content of the entity references. If the text content of two or more entity references is completely consistent (including the normalization consistency of simplified and traditional characters), then these entity references are judged to meet the literal consistency rule, classified into the same candidate group, and a unique group identifier is assigned to the candidate group.

[0028] For the remaining entity references that have not passed the literal consistency rule grouping, the edit distance between any two entity references is calculated. The edit distance is defined as the minimum number of single-character editing operations (including insertion, deletion, and replacement) required to convert the text content of one entity reference into the text content of another entity reference. In this embodiment, the edit distance threshold is set to 2. If the edit distance between two entity references is less than or equal to 2, and the core vocabulary of the entity references is consistent, then the two entity references are determined to meet the edit distance rule and are grouped into the same candidate group. If any entity reference has been assigned to another candidate group, then the other entity reference is merged into that candidate group. A multi-source heterogeneous entity alias knowledge base is built. This knowledge base integrates the standard names, aliases, abbreviations, and colloquial names of entities such as institutions, people, and regions commonly used in the field of intelligence analysis. For entity references that have not been grouped... The text content is matched against all entity names and aliases in the knowledge base. If two or more entity references are the standard name and alias / abbreviation / common name of the same entity in the knowledge base, or different aliases of the same entity, then these entity references are determined to satisfy the alias relationship rule and are classified into the same candidate group. After completing the full judgment of the three types of rules, all candidate groups are sorted out, empty groups and duplicate groups are removed, and each valid candidate group is assigned a unique group code and group attribute label (the label indicates the association rule type of the group). At the same time, isolated entity references that do not satisfy any reference association rule are classified into independent candidate groups to ensure that all entity references are included in the candidate group system. Finally, a set of entity reference candidate groups containing multiple candidate groups is obtained. Each candidate group contains potential related entity references that satisfy the reference association rule, and there are no overlapping entity references between candidate groups.

[0029] Step 331: For each candidate group, pair all entity references within the candidate group to form multiple entity pairs. Each entity pair includes two entity references to be compared. Specifically, this includes: sequentially traversing all candidate groups obtained in step 330; for a single candidate group, first extracting all entity references within the group to form a subset of entity references for that group, recording the number of entity references in the subset as N (N≥1); for candidate groups with N≥2 entity references, using a combination pairing method to perform pairwise pairing of entity references within the group without repetition, that is, randomly selecting two different entity references from the N entity references to form an entity pair. The pairing process follows the principles of no repetition and no omission, avoiding positive pairings of the same entity pair. Reverse the generation process (e.g., if entity reference A and entity reference B are combined, entity reference B and entity reference A will not be generated again); for independent candidate groups with N=1 entity references, no pairing operation is performed, and no entity pairs are generated for this candidate group; for each generated entity pair, a unique entity pair identifier is assigned, and the entity pair is bound to the group code of its candidate group and the basic information of the two entity references (platform, context window, semantic fingerprint vector index), forming a standardized entity pair data structure containing entity pair identifier - group code - entity reference 1 to be compared - entity reference 2 to be compared - basic information of both entities; summarize the standardized entity pairs generated from all candidate groups to form an entity pair set.

[0030] Step 332: For each entity pair, extract the semantic fingerprint vectors corresponding to the two entity references in the entity pair from the semantic fingerprint vector set, calculate the cosine similarity between the two semantic fingerprint vectors, and use the cosine similarity as the identity consistency potential value of the entity pair; specifically, this includes: traversing the entity pair set obtained in step 331, and for a single entity pair, accurately retrieving the two corresponding semantic fingerprint vectors from the entity reference-semantic fingerprint vector mapping library based on the semantic fingerprint vector indices of the two entity references bound to it, denoted as vectors. sum vector Both vectors are 768-dimensional continuous high-dimensional real vectors generated in step 2, ensuring consistent vector dimensions and valid values. The cosine similarity is calculated step-by-step according to the cosine similarity formula. sum vector The cosine similarity formula is calculated step by step. ,in For vectors with vector dot product, For vectors The length of the mold, For vectors The modulus, specifically calculated by calculating the dot product. , will vector The first dimension of numerical and vector Multiplying the first dimension of the vector Second-dimensional numerical and vector Multiply the second dimension of the vector by the first, and so on, until all 768 dimensions have been multiplied. Then sum the 768 multiplication results to obtain the vector. with vector dot product; calculate vector Length of the module , will vector The 768-dimensional value is squared, then all the squared results are summed, and finally the sum is squared to obtain the vector. The magnitude of the vector; calculate the vector. Length of the module According to vector The method of calculating the modulus for vectors The 768-dimensional numerical values ​​are squared, summed, and squared in sequence to obtain a vector. The magnitude of the vector; using the dot product result obtained above as the dividend, the vector... Magnitude and vector The result of multiplying the magnitudes of the vectors is used as the divisor for division, and the quotient is the vector. with vector The cosine similarity is calculated, with a value ranging from [-1, 1]. The closer the value is to 1, the higher the similarity between the two semantic fingerprint vectors, and the greater the likelihood that the entities refer to the same object. The calculated cosine similarity value is directly determined as the identity consistency potential value of the entity pair. This value is bound to the corresponding entity pair, and intermediate values ​​such as the dot product and modulus are recorded during the calculation process for subsequent verification. After the cosine similarity calculation and potential value binding are completed for all entity pairs, an associated dataset containing entity pair-cosine similarity (identity consistency potential value) is obtained.

[0031] Step 333: For each candidate group, evaluate the richness of contextual information of the entity references within that group based on the context window length or contextual information entropy of all entity references within the group. Dynamically adjust the similarity threshold for that group based on the richness of contextual information. When contextual information is sparse, automatically increase the similarity threshold; when contextual information is rich, appropriately relax the similarity threshold. Specifically, this includes using two dimensions—context window length and contextual information entropy—to comprehensively evaluate the richness of contextual information of the entity references within the candidate group. These two indicators quantify the text volume and information density of the context, respectively, ensuring the objectivity and comprehensiveness of the evaluation results. The context window length is T of the context window corresponding to the entity reference. The token count and context information entropy are the information entropy of the context window text. Higher information entropy indicates greater information density and richer effective semantic information in the text. Iterate through all candidate groups. For a single candidate group, first extract the context windows corresponding to all entity references within the group, and calculate the context window length (token count) and context information entropy for each entity reference. The context information entropy is calculated by performing word frequency statistics on all characters in the context window text, calculating the probability of each character's occurrence, multiplying the probability of each character by its natural logarithm, summing all multiplication results, and taking the negative of the sum to obtain the information entropy of a single context window. After completing the index calculation for all entity references within the group, calculate... The mean context window length and mean context information entropy of the candidate groups are used to represent the information characteristics of the entire candidate group, avoiding the influence of the bias of indicators of individual entities on the overall evaluation. The richness of context information of the candidate groups is divided into three levels: information sparse, information moderate, and information rich, with clear level classification thresholds. The mean context window length and mean information entropy of all candidate groups are globally normalized to obtain a normalized value between 0 and 1. If the normalized value of the mean length and the normalized value of the mean information entropy are both less than 0.3, the candidate group is judged to be information sparse; if the normalized value is between 0.3 and 0.7, it is judged to be information moderate; if the normalized value of the mean length and the normalized value of the mean information entropy are both greater than 0.7, it is judged to be information rich. A mean normalized value > 0.7 indicates an information richness level. A basic similarity threshold range of 0.55 to 0.75 is set. Based on the information richness level of candidate groups, a dynamic threshold is assigned within this range to achieve personalized threshold adjustment. Specifically, if a candidate group is information sparse, the similarity threshold is automatically increased, and the dynamic threshold for that group is set to 0.75 to filter noise and avoid incorrect matches due to insufficient contextual information. If a candidate group is of medium information richness, a basic intermediate threshold is used, and the dynamic threshold for that group is set to 0.65, balancing recall and precision in the initial screening. If a candidate group is information rich, the similarity threshold is appropriately relaxed, and the dynamic threshold for that group is set to 0.55. By using a low threshold, all potentially related entity pairs are retained, avoiding the omission of high-potential candidate pairs. A dynamic threshold set for each candidate group is uniquely bound to its group code, forming a candidate group code-dynamic similarity threshold association table. This association table is then linked to the entity pair set, ensuring that each entity pair can be matched to its corresponding dynamic threshold based on its group code.

[0032] Step 334: For each entity pair, compare the cosine similarity of the entity pair with the dynamic threshold of the candidate group to which the entity pair belongs. Entity pairs with a cosine similarity greater than the dynamic threshold are retained as high-potential candidate pairs that pass the initial screening. Specifically, this includes: matching entity pairs with dynamic thresholds. Iterate through the entity pair-cosine similarity association dataset obtained in step 332. For a single entity pair, based on its candidate group code, match the dynamic threshold corresponding to the candidate group from the candidate group code-dynamic similarity threshold association table constructed in step 333, denoted as T. Denote the cosine similarity (identity consistency potential value) of the entity pair as S. Compare S with the dynamic threshold T. If S > T, the entity pair is determined to be a potentially related entity pair and retained; if S ≤ T, the entity pair is determined to be a non-related entity pair and removed. Removed entity pairs will no longer participate in the screening. The subsequent refined matching steps, after completing the threshold comparison and judgment of all entity pairs, summarize all the retained potential related entity pairs, remove duplicate and invalid entity pairs, reassign high-potential candidate pair identifiers to each retained entity pair, and retain all its basic information, cosine similarity value, dynamic threshold of the candidate group, etc., to form a high-potential candidate pair set. The results of the coarse screening are statistically analyzed, recording the total number of entity pairs, the number of retained high-potential candidate pairs, and the number of removed non-related entity pairs, and the coarse screening retention rate is calculated. At the same time, some high-potential candidate pairs are randomly selected for manual verification to verify the rationality of the cosine similarity and dynamic threshold judgment. If the verification finds that the proportion of missed or incorrect screening is too high, the information richness level threshold and dynamic threshold assignment rules of the candidate group are readjusted, and the operations of steps 333-334 are executed again to ensure the effectiveness of the coarse screening results.

[0033] The candidate group segmentation method, which uses three parallel rules—literal consistency, edit distance, and alias relationship—achieves comprehensive coverage of potential related entities with the same name, similar spelling, and alias association, ensuring a high recall rate for candidate group segmentation and avoiding missing any potential cross-platform related entity references.

[0034] In a preferred embodiment of the present invention, step 4 above, for each candidate pair obtained after coarse screening, inputs the context text of the two entities to be compared in the candidate pair into two encoders with shared weights in a deep semantic matching model based on a dual-tower structure, calculates the matching score of the two entities, compares the matching score with a preset judgment threshold, and when the matching score exceeds the judgment threshold, initially determines that the two entities refer to the same object, obtaining a high-confidence matching result, may include: In this embodiment of the invention, step 440 involves obtaining a set of candidate pairs after coarse screening, including multiple candidate pairs, each consisting of two entity references to be compared. Specifically, this includes: retrieving the set of high-potential candidate pairs obtained after coarse screening in step 3. This set is a dataset of entity pairs retained after reference association grouping, cosine similarity calculation, and context-aware dynamic threshold filtering, with no duplicate or invalid entity pairs in the set; performing data integrity verification on the retrieved set of high-potential candidate pairs, checking each candidate pair to see if it contains basic information such as the unique identifiers of the two entity references to be compared, the platform to which it belongs, the context window index, and the semantic fingerprint vector, and eliminating abnormal candidate pairs with missing information or incorrect identifiers; standardizing and organizing the valid set of high-potential candidate pairs after verification, assigning a continuous sequence identifier to each candidate pair to form a standardized data structure of sequence identifier - entity reference 1 to be compared - entity reference 2 to be compared - dual entity basic information index, which serves as the core object for subsequent processing in this step, ensuring that the processing flow of all candidate pairs is unified and traceable.

[0035] Step 441: For each candidate pair in the candidate pair set, extract the context text corresponding to the first entity reference and the context text corresponding to the second entity reference from the combination of entity reference and context. Specifically, this includes: traversing the standardized high-potential candidate pair set organized in step 440 by sequence identifier; for a single candidate pair, retrieving the combination set of entity reference and context constructed in step 1 based on the basic information index of the two entity references to be compared contained therein; and extracting the preprocessed and verified complete context window text corresponding to the first entity reference and the preprocessed and verified complete context window text corresponding to the second entity reference from the combination set of entity reference and context. The complete context window text after verification ensures that the extracted context text is noise-free, untruncated, and semantically complete. The two extracted context texts are formatted and normalized to uniformly remove redundant whitespace characters and standardize punctuation. At the same time, the corresponding entity reference identifiers are labeled for each text, forming text association data of candidate pair sequence identifier - entity reference 1 + corresponding context text - entity reference 2 + corresponding context text, to avoid confusion in the mapping between text and entity references during subsequent encoding. The text association data of all candidate pairs are summarized to form the context text extraction dataset, providing complete and standardized text data support for the encoding input of the dual-tower deep semantic matching model.

[0036] Step 442: Input the context text of the first entity reference into the first encoder of the pre-trained dual-tower deep semantic matching model, and simultaneously input the context text of the second entity reference into the second encoder of the dual-tower deep semantic matching model. The first encoder performs deep semantic encoding on the input context text to obtain the first semantic vector of the first entity reference, and the second encoder performs deep semantic encoding on the input context text to obtain the second semantic vector of the second entity reference. Specifically, the constructed dual-tower deep semantic matching model is a dual-tower structure model based on a pre-trained language model. Its core consists of three parts: a first encoder, a second encoder, and a similarity calculation layer. The core design uses a shared encoding method between the two encoders. The shared weight architecture is constructed as follows: First, the encoder base is selected, with both the first and second encoders built on the BERT-base-chinese model. The model structure consists of 12 Transformer encoding layers, 768-dimensional hidden layers, and a 12-head self-attention mechanism, maintaining consistency with the semantic encoding model architecture in step 2 to ensure semantic space uniformity. All layer parameters (including Transformer encoding layers, self-attention mechanism, feedforward neural network, etc.) are completely shared between the first and second encoders. This means both encoders use the same set of pre-trained weights and fine-tuned task-adaptive weights, ensuring that the two encoders fully understand the encoding logic and semantic mapping rules of the context text. Full consistency ensures that the encoded semantic vectors are in the same metric space, allowing for direct similarity comparison. The dual-tower model is fine-tuned for task adaptation using a multi-source intelligence entity disambiguation annotation dataset. Two context texts of positive sample pairs (same entity) are input into two separate encoders, and two context texts of negative sample pairs (same name, different entities) are input into two separate encoders. The optimization objective is to maximize the semantic similarity of positive sample pairs and minimize the semantic similarity of negative sample pairs. A binary classification cross-entropy loss function is used to calculate the loss value, and the shared weights are updated through backpropagation. Iterative training continues until the loss value converges, resulting in a pre-trained and task-adapted dual-tower deep semantic matching model. For each context text extracted in step 441, the following steps are performed: Step 220 involves consistent standardization processing, including tokenization segmentation using the BERT-base-chinese-compatible Chinese WordPieceToken segmenter, length adaptation (padding with [PAD] or truncating to 512 tokens), and generating a fused feature vector of word vectors + positional encoding + segment encoding to form a standardized feature sequence that meets the encoder input requirements, ensuring the standardization of text input. For a single candidate pair, the preprocessed contextual text feature sequence of the first entity reference is input into the first encoder of the dual-tower model, while the contextual text feature sequence of the second entity reference is input into the second encoder. The two encoders perform deep semantic encoding in parallel based on shared weights.The encoding process is consistent with step 221. The feature sequence is sequentially processed by a multi-head self-attention mechanism through 12 Transformer encoding layers, residual connections, layer normalization, and feedforward neural network transformation to achieve deep semantic mining of the context text and capture core information such as the implicit attributes and contextual associations of entities in the text. After the two encoders complete the encoding, the [CLS] position vector output by the last layer of their respective encoders is extracted as the core semantic vector, where the [CLS] position vector output by the first encoder is the first semantic vector of the first entity reference. The [CLS] position vector output by the second encoder is the second semantic vector of the second entity reference. Both semantic vectors are 768-dimensional continuous high-dimensional real vectors and reside in the same semantic metric space.

[0037] Step 443: Input the first semantic vector and the second semantic vector into the similarity calculation layer of the dual-tower deep semantic matching model. The similarity calculation layer calculates the similarity between the first semantic vector and the second semantic vector and maps the similarity to a matching score in the range of 0 to 1. Specifically, this includes: inputting the first semantic vector obtained in step 442 into the similarity calculation layer of the dual-tower deep semantic matching model. Second semantic vector The similarity calculation layer of the dual-tower model first normalizes the two vectors to ensure that the magnitude of the vectors is 1, eliminating the influence of vector scale on the similarity calculation. The basic calculation of cosine similarity is then performed in the similarity calculation layer according to the cosine similarity formula. and The basic similarity is calculated using the following formula: Since the vector has been normalized, and Since both are 1, the actual calculation is the dot product of the two vectors, i.e., taking... The first dimension value and Multiply the first-dimensional values ​​until the 768th dimension, then sum all the results to obtain the cosine similarity value, which ranges from [-1, 1]. Based on the basic cosine similarity, a temperature coefficient is introduced. With bias term b, the cosine similarity is mapped to a matching score in the interval of 0 to 1 using the Sigmoid activation function. In this embodiment, the temperature coefficient... The preferred value is 20, and the bias term b is preferably 0. The specific calculation formula is as follows: The specific calculation process is as follows: the first step is to compare the basic cosine similarity value with the temperature coefficient. The first step involves multiplying the cosine similarity value by 20 and adding a bias term of 0 to obtain an intermediate calculated value. The second step involves taking the negative of this intermediate calculated value to obtain a negative intermediate calculated value. The third step uses the natural constant... The first step involves performing an exponential operation on the negative intermediate calculated value to obtain the exponential result. The fourth step involves adding the exponential result to the number 1 to obtain the denominator value. The fifth step involves performing a division operation using the number 1 as the dividend and the above denominator value as the divisor. The quotient obtained is the matching score of the candidate pair. The score ranges from (0, 1). The closer the score is to 1, the higher the confidence that the two entities refer to the same object. The calculated matching score is uniquely bound to the corresponding candidate pair sequence identifier, and the precise value of the matching score is recorded. At the same time, process data such as cosine similarity and intermediate calculated values ​​are saved to form an associated dataset of candidate pair sequence identifier - matching score - process calculation data, which provides a quantitative basis for subsequent threshold determination.

[0038] Step 444: Based on the matching scores, each matching score corresponds to a high-potential candidate pair consisting of two entity references, and a preset judgment threshold is set. This judgment threshold is a value between 0 and 1, used to define whether the two entity references refer to the same real object. Specifically, this includes: performing global statistics on the matching scores of all candidate pairs obtained in step 443, analyzing the distribution characteristics of the scores, including the mean, median, extreme values, and the score concentration interval of positive sample pairs (manually labeled), etc., to provide data support for setting the judgment threshold; and, in combination with the high accuracy requirement of entity disambiguation, setting a preset judgment threshold within the matching score range of 0 to 1. In this embodiment, the decision threshold is preferably set to 0.85. This threshold is the final value verified through extensive experiments, which can maximize the accuracy of entity matching while ensuring recall, and effectively distinguish subtle semantic differences between highly similar but different entities. The preset decision threshold is standardized and defined as the core quantitative standard for determining whether two entity references the same real-world object. If the matching score of the candidate pair is greater than the threshold, it indicates that the confidence level of the two entities referring to the same object reaches a high level; if the matching score is less than or equal to the threshold, it indicates that the confidence level of the two entities referring to the same object is insufficient, and they are judged as different objects. The preset decision threshold is... =0.85 is embedded in the subsequent threshold determination process to form a fixed determination standard, ensuring that the determination rules for all candidate pairs are consistent and unbiased.

[0039] Step 445: For each high-potential candidate pair, compare the matching score corresponding to the candidate pair with a preset judgment threshold; specifically, this includes: traversing the candidate pair sequence identifier-matching score associated dataset obtained in step 443 by sequence identifier; for each high-potential candidate pair, extracting its corresponding matching score, denoted as S; and retrieving the preset judgment threshold set in step 444, denoted as S. =0.85; Perform a numerical comparison operation on each candidate pair, that is, compare the matching score S with the decision threshold. One-by-one comparisons are performed to determine the numerical relationship between the two pairs. Only three results exist: S > 0.85, S = 0.85, and S < 0.85. The comparison results of each candidate pair are recorded, and a judgment label is added indicating whether the score is greater than the threshold, equal to the threshold, or less than the threshold. This forms a comparison dataset of candidate pair sequence identifier, matching score, judgment threshold, and comparison result label, which provides a direct basis for subsequent consistency judgment of entity claims.

[0040] Step 446: For candidate pairs with a matching score greater than a preset judgment threshold, preliminarily determine that the two entities in the candidate pair refer to the same object, and mark the candidate pair as a high-confidence matching result. Specifically, this includes: from the comparison dataset in step 445, selecting all candidate pairs with a marked score greater than the threshold, i.e., candidate pairs with a matching score S > 0.85; for the selected candidate pairs, performing a preliminary judgment of the same entity to determine that the two entities to be compared in the candidate pair refer to the same real object; for each candidate pair judged to be the same entity, adding a unique label for a high-confidence matching result, and simultaneously marking the exact value of the matching score, the judgment threshold, the judgment time, etc., to establish a high-confidence matching result file for the candidate pair, ensuring the traceability and verifiability of the results; and preliminarily organizing all candidate pairs marked as high-confidence matching results, retaining all their basic information and judgment information to form an initial set of high-confidence matching results.

[0041] Step 447: For candidate pairs with matching scores less than or equal to a preset judgment threshold, the two entities in the candidate pair are determined to refer to different objects, and the candidate pair is not included in the high-confidence matching results. Specifically, this includes: from the comparison dataset in step 445, selecting all candidate pairs with labeled scores equal to or less than the threshold, i.e., candidate pairs with matching scores S≤0.85; for the selected candidate pairs, performing different entity judgment to determine that the two entities to be compared in the candidate pair refer to different real objects; for each candidate pair determined to be different entities, adding a unique label for non-matching results, labeling information such as matching score and judgment threshold, storing it as a negative sample result for entity disambiguation, which can be used for subsequent model iteration optimization; determining that this type of candidate pair is not included in the high-confidence matching results, no longer participating in the subsequent result summary of this step, and only retained as a negative sample to ensure the purity of the high-confidence matching results.

[0042] Step 448 involves aggregating all candidate pairs marked as high-confidence matches to form high-confidence matches. Specifically, this includes: validating the initial set of high-confidence matches obtained in Step 446 by checking whether each result's matching score is indeed greater than 0.85, whether the entity reference information is complete, and whether the judgment label is accurate, eliminating abnormal results caused by calculation or labeling errors; standardizing and summarizing the validated high-confidence matches, sorting them by matching score from highest to lowest, assigning a unique high-confidence match identifier to each result, and integrating the candidate pairs' basic information, matching scores, and judgment criteria to form a structured set of high-confidence matches; statistically analyzing the high-confidence match set, recording core indicators such as the total number of high-potential candidate pairs, the number of high-confidence matches, and the matching rate, and generating a statistical report to provide data support for subsequent trust propagation and iterative disambiguation steps; and associating the high-confidence match set with the entity reference-semantic fingerprint vector mapping library, labeling each matching result's entity reference with an associated identifier for the same object.

[0043] After initial screening, high-potential candidate pairs undergo data integrity verification and standardization to ensure that the encoding logic and semantic mapping rules of the two encoders are completely consistent, so that the encoded semantic vectors are in the same metric space, thus improving the accuracy of similarity calculation.

[0044] In a preferred embodiment of the present invention, step 5 above, which uses the high-confidence matching results as anchor points to construct an entity association graph, employs a label propagation algorithm to perform weighted iterative updates on the confidence of other candidate pairs in the entity association graph, and assigns a globally unique identifier to each independent entity, may include: Step 550: Select entity pairs with matching scores exceeding a preset high-confidence threshold as high-confidence anchor points. Use these anchor points as initial reliable connection edges and combine them with all entity nodes to be disambiguated to construct an entity association graph. Specifically, this includes: determining the high-confidence threshold value, setting the preset high-confidence threshold to 0.95. This threshold is the matching score judgment value in the 0-1 interval output by the dual-tower deep semantic matching model. It is an experimentally verified, extremely high-confidence threshold that can clearly determine that entity pairs are the same object. Entity pairs below this value are not included in the high-confidence anchor point selection range. Full traversal of high-confidence matching points... The result set is the set of all entity pairs whose matching scores exceed the 0.85 threshold after fine-tuning by the dual-tower model in step 4. For each entity pair in the result set, the two entity references, the corresponding cross-platform source information, and the specific matching score output by the dual-tower model are extracted to form a one-to-one correspondence list of entity pairs and matching scores. A high-confidence anchor point filtering operation is then performed, comparing the specific values ​​of the matching scores in the list with the 0.95 high-confidence threshold one by one. Only entity pairs with matching scores strictly greater than 0.95 are selected and defined as high-confidence anchor points. These anchor points are those processed by deep learning. After semantic matching, the combination of entities referring to the same real-world object can be 100% identified, which is the core and reliable basis for subsequent tag propagation. All entity nodes to be disambiguated are extracted, and all entity references involved in this cross-platform entity semantic disambiguation task are sorted out, covering all heterogeneous intelligence source platforms such as Weibo, news, and forums. Each independent entity reference is treated as a unique entity node, and each node is assigned a temporary identifier to form a complete set of entity nodes to be disambiguated, ensuring that no entity references are missed. Initial reliable connection edge attributes are assigned, and for all selected high-confidence anchor points, each anchor point is used as a connection between entity nodes. An initial reliable connection edge is formed, with each end of the edge representing one of the two entity nodes corresponding to the anchor point. The actual matching score of the anchor point is assigned to the corresponding connection edge as an edge weight, which represents the initial association confidence of the connection edge. The higher the weight value, the stronger the association confidence. An entity association graph is constructed using an undirected graph topology structure. All nodes in the complete set of entity nodes to be disambiguated are used as vertices of the graph, and the initial reliable connection edges corresponding to all high-confidence anchor points are used as edges of the graph. The edge weights are bound to the corresponding edges, ultimately forming an entity association graph containing entity nodes, initial reliable connection edges, and edge weights.

[0045] Step 551: In the entity association graph, assign reliability weights to each node and each edge, and perform label propagation; introduce a path decay factor during propagation so that the evidence strength between indirect entity pairs decreases as the propagation path length increases; specifically, this includes: calculating the reliability weights of entity nodes. This weight applies to any intermediate node in the entity relationship graph. Perform the calculation; the formula is as follows: ,in For nodes The information entropy connecting the probability distributions of all candidate objects is calculated as follows: statistical nodes. The initial association probability with all candidate entity pairs in the graph, which is the initial matching score output by the dual-tower model, forms the nodes. First, the set of associated probability distributions; second, according to the standard method for calculating information entropy, the information entropy of this set of probability distributions is calculated. The calculation shows that the larger the information entropy value, the more likely the node is to be affected. The more ambiguous the identity determination, the smaller the value represents the node. The clearer the identity determination; the more precise the natural constant. As the base, calculate The exponent value is used to obtain the node. Final reliability weight This step completes the assignment of reliability weights to all entity nodes in the graph, ensuring that each node has a unique corresponding weight value. It also completes the supplementation and confirmation of edge weights. For initial reliable edges in the graph, their weights, already assigned as the matching scores of the corresponding high-confidence anchor points in step 550, are fully validated in this step to ensure no weights are missing or incorrectly assigned. For temporary edges added during subsequent propagation, the initial edge weights are tentatively set to the initial matching scores of the corresponding entity pair's dual-tower model, to be updated in subsequent iterations. A path decay factor is also set. In this embodiment, the path attenuation factor This is a preset fixed constant coefficient, whose value has been experimentally verified to effectively suppress multi-hop propagation noise. This factor is used to characterize the degree of natural attenuation of confidence as the number of propagation hops increases. Its core function is to ensure that the evidence strength between indirectly connected entity pairs decreases with the increase of propagation path length. It initiates the basic tag propagation process, using the initial reliable connection edge corresponding to the high-confidence anchor point as the starting point for evidence propagation. High-confidence association evidence is transmitted from the initial connection edge to surrounding entity nodes in the graph that have not yet established connections. During the propagation process, for each indirectly connected propagation path, an evidence strength attenuation calculation is performed; that is, for every hop increase of 1 in the propagation path, the current evidence strength is multiplied by the path attenuation factor. If the propagation path is two hops, then the strength of the evidence is multiplied by [the factor]. × If it's 3 jumps, then multiply by × × This process continues to ensure that the longer the propagation path, the lower the strength of the evidence, effectively preventing noise from being amplified in multi-hop propagation and achieving reasonable transmission of related evidence; the propagation association probability between entity nodes is initially calculated, and during the label propagation process, for any two entities that are not directly connected... and entity Through their common intermediate node To transmit evidence, first calculate the association probability of a single path, i.e., the intermediate nodes. Reliability weight Multiply by path decay factor Multiply by entity With intermediate nodes Current association probability Multiply by entity Current association probability with intermediate node k Then for all achievable and Intermediate nodes in the transmission of evidence The calculation results are summed to obtain the entity. and The preliminary correlation probability values ​​obtained through graph propagation prepare for subsequent weighted fusion.

[0046] Step 552: Based on the current iteration number of tag propagation, dynamically adjust the balance coefficients and perform weighted fusion of the initial matching evidence and the evidence obtained through graph propagation; specifically, this includes: determining the balance coefficients. Dynamic adjustment rules, balance coefficient This is a core coefficient used to adjust the weight ratio between the initial matching evidence output by the dual-tower deep semantic matching model and the indirect evidence obtained through entity association graph propagation. Its value is not fixed, but varies with the number of iterations in label propagation. Dynamically increasing, the specific calculation formula is as follows: =0.6+0.1× The number of iteration rounds Starting the count from 0, the initial iteration ( When =0, the equilibrium coefficient =0.6 + 0.1 × 0 = 0.6, where the number of iterations is 0.6 for each complete iteration. The balance coefficient is increased by 0.1 with each increment until the iteration terminates. Initial matching evidence is extracted, and the initial semantic similarity probability of all candidate entity pairs in this disambiguation task is calculated using the full extraction of the dual-tower deep semantic matching model. This probability is the matching score in the 0-1 interval output by the dual-tower model in step 4, and is the candidate entity pair. and The basic basis for determining association is that the initial matching evidence remains constant throughout the entire tag propagation iteration process, and is not modified with the number of iterations. The comprehensive value of indirect evidence propagated through the graph is calculated for any candidate entity pair. and First, extract the entity. The set of neighboring nodes That is, all entities The graph contains nodes that are directly or indirectly connected; secondly, for the set Each intermediate node in Calculate the propagation contribution value of a single node, i.e., the intermediate node. Reliability weight Multiply by path decay factor Multiply by entity and The Current association probability Multiply by entity and The Current association probability For all intermediate nodes The single-node propagation contribution values ​​are summed to obtain the sum result; this sum result is then divided by the entity. Corresponding normalization factor Normalization factor For entities The sum of the probabilities associated with all neighboring nodes is used to ensure that the sum of the probability space after propagation is 1, ultimately yielding the entity pair. and The composite value of indirect evidence transmitted through the map.

[0047] The weighted fusion calculation of initial evidence and dissemination evidence is performed, and the core formula for the fusion calculation is as follows: ,in For the first Entity after +1 round of iterations With entity The correlation probability is calculated based on the current iteration round number. Determine the corresponding dynamic equilibrium coefficient Using dynamic balance coefficient Multiply by the initial matching evidence First, the weighted contribution value of the initial matching evidence is obtained; second, the dynamic balance coefficient is subtracted from the number 1. The weighting coefficient of the disseminated evidence is obtained. This coefficient is then multiplied by the comprehensive value of the indirect evidence disseminated through the graph to obtain the weighted contribution value of the disseminated indirect evidence. The weighted contribution value of the initial matching evidence is then added to the weighted contribution value of the disseminated indirect evidence to obtain the result. The association probability value after +1 iteration of fusion is used to complete the evidence fusion for a single candidate entity pair. This method is then applied to all candidate entity pairs in the graph to calculate the probability value for each pair, resulting in the first probability value for all entity pairs. The +1 round of fusion results in the set of associated probabilities, which temporarily stores the evidence fusion results, and includes the first-order probabilities of all candidate entity pairs. After the +1 round of fusion, the association probability value is bound to the corresponding entity pair one by one to form an iteratively updated association probability list, which prepares for subsequent confidence updates and ensures that the fusion result of each entity pair is traceable and verifiable.

[0048] Step 553: Based on the fusion results, the confidence of all candidate entity pairs is continuously updated through multiple iterations until the convergence condition is met or the preset maximum number of iterations is reached. Specifically, this includes setting iteration termination criteria, which include two main conditions. The iteration terminates when either condition is met. The first is the preset maximum number of iterations. In this embodiment, experimental verification shows that the maximum number of iterations is set to 3-5 rounds. This number represents the optimal number of iterations when the association probability in the entity association graph tends to stabilize, avoiding excessive iteration and wasting computational resources. The second is the convergence condition, specifically, after two consecutive iterations, the absolute value of the change in the association probability of all candidate entity pairs in the entity association graph is less than a preset minimum threshold (set to 0.001). That is, for any entity pair... and , its first +1 round association probability and the first The absolute value of the difference in round association probabilities <0.001 indicates that the association probability has stabilized and there is no need to continue iterating; perform the first round of confidence update operation, and update the confidence level obtained in step 552 for the first round ( =0 to =1) The fused association probability value is directly updated to the current confidence score of the corresponding candidate entity pair in the graph, replacing the original initial matching score, completing the confidence update of the first round of label propagation. At the same time, all confidence scores of the first round of iteration are recorded to form the first round confidence list. A multi-round iterative calculation is performed, using the updated confidence score as the basis for the next round of iteration. Following the process of recalculating node reliability weights, performing label propagation, adjusting the balance coefficient according to the new iteration round number, fusing initial matching evidence and propagated indirect evidence, calculating the fused association probability of the new round, and updating the confidence score, all operations from steps 551 to 552 are repeated to complete the second, third, and subsequent iterative calculations. In each round of iteration, the reliability weights of all nodes are recalculated. This ensures that node weights are dynamically adjusted as the probability of association between entities changes, while strictly adhering to the number of iteration rounds. Adjust the balance coefficient To ensure a reasonable transition in the weighting of evidence, an iteration termination check is performed. After each iteration is completed and the confidence score is updated, a termination condition check is immediately performed. If the current iteration number has reached the preset maximum number of iterations (3-5), an iteration termination command is directly triggered. If the maximum number of iterations has not been reached, the change in the confidence scores of all entity pairs in the current iteration and the previous iteration is calculated. It is checked whether the absolute value of all changes is less than the convergence threshold of 0.001. If the threshold is met, an iteration termination command is triggered. If the threshold is not met, the next iteration continues to determine the final confidence score. After the iteration terminates, the confidence scores of all candidate entity pairs updated in the last iteration are extracted to form a final confidence score list of all entity pairs. This list is the core basis for subsequent entity merging and identifier allocation, ensuring that all confidence scores are stable values ​​after global optimization.

[0049] Step 554: Based on the final confidence score obtained after iterative updates, assign a globally unique identifier to each independent entity to achieve semantic disambiguation of entities across platforms. Specifically, this includes: determining the final judgment criterion for entity identity. The final judgment threshold for entity identity is set to 0.85, which is consistent with the fine-ranking judgment threshold of the dual-tower deep semantic matching model in Step 4, forming a judgment criterion. For any candidate entity pair, if its final confidence score is strictly greater than 0.85, it is determined that the two entities in the entity pair refer to the same real object; if its final confidence score is less than or equal to 0.85, it is determined that the two entities in the entity pair refer to different real objects. Perform identity judgment on all entity pairs, traverse the list of final confidence scores obtained in Step 553, extract the two entity references and corresponding final confidence scores of each candidate entity pair one by one, and compare the final confidence score with 0.The system compares each entity pair against a threshold of 85, determining their identity according to the established criteria, resulting in two lists: identical entity pairs and different entity pairs. A cross-platform entity merging operation is then performed. Based on the identical entity pair list, a transitive merging rule is used to merge entity sets. If entity A and entity B are identical entity pairs, and entity B and entity C are identical entity pairs, then entities A, B, and C are merged into a single independent entity set. Following this rule, all entity references determined to be the same object are fully merged, forming multiple independent entity sets. All entity references within each set are cross-platform entities referring to the same real-world object. Individual entity references that do not form any entity pair are retained; these individual entity references are called independent entity units. A globally unique identifier allocation operation is then performed, establishing a unified global identifier coding rule. This rule is unique, exclusive, and scalable, applicable to the identifier allocation of newly added entities. Following this coding rule, all merged independent entity sets and all unmerged individual entity units are numbered sequentially, creating a unique identifier for each independent entity set. Each entity unit is assigned a unique global identifier, which serves as a unified cross-platform identification ID. All cross-platform entity references within the same independent entity set share the same global identifier, and global identifiers for different entity sets / units are unique. The binding and fusion of entity identifiers and attributes is completed, permanently binding each entity reference to its corresponding global identifier, forming a one-to-one correspondence between entity reference and global identifier. Simultaneously, the contextual attribute information of the entity reference across various platforms is integrated, including implicit attributes inferred from the context such as occupation, region, affiliated organization, and activity characteristics, as well as basic attributes such as the entity's platform origin and appearance time. All attribute information is bound to the corresponding global identifier, forming complete association data of global identifier - entity reference set - multi-source attribute information. This achieves the final implementation of cross-platform entity semantic disambiguation. All complete association data is integrated and stored, completing the semantic disambiguation of all cross-platform entity references in this task. This achieves accurate differentiation of entities with the same name and cross-platform global association of the same entity. Relevant information of a specific entity across all platforms can be queried through the global identifier, forming a panoramic view of the entities.

[0050] By dynamically adjusting the balance coefficient, the initial matching evidence and the graph propagation evidence are weighted and fused. The balance coefficient is relatively low in the initial iteration stage, making the algorithm more dependent on the global correlation information of the graph and fully exploring the indirect correlation between entities.

[0051] like Figure 2 As shown, embodiments of the present invention also provide a cross-platform entity semantic disambiguation system for multi-source intelligence, comprising: The extraction module is used to acquire multi-source intelligence data. It formulates differentiated context extraction strategies based on the data characteristics of different intelligence source platforms. It extracts entity references and surrounding text from the original text as context windows to obtain a combination of entity references and context. The encoding module is used to input the combination of entity reference and context into the pre-trained language model for encoding, mapping to a high-dimensional semantic space, and generating semantic fingerprint vectors. The coarse screening module is used to divide entities into candidate groups based on semantic fingerprint vectors and referential association rules. Within each candidate group, entities are paired up and the cosine similarity of semantic fingerprint vectors between paired entities is calculated. A context-aware dynamic threshold strategy is used to retain candidate pairs with cosine similarity greater than the dynamic threshold, thus completing the coarse screening of entity pairs. The matching module is used to input the context text of the two entities to be compared in each candidate pair obtained after coarse screening into two encoders with shared weights in a deep semantic matching model based on a dual-tower structure, calculate the matching score of the two entities, compare the matching score with a preset judgment threshold, and preliminarily determine that the two entities refer to the same object when the matching score exceeds the judgment threshold, thus obtaining a high-confidence matching result. The allocation module is used to construct an entity association graph by using high-confidence matching results as anchors. It employs a label propagation algorithm to perform weighted iterative updates on the confidence of other candidate pairs in the entity association graph and assigns a globally unique identifier to each independent entity.

[0052] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A cross-platform entity semantic disambiguation method for multi-source intelligence, characterized in that, The method includes: Step 1: Obtain multi-source intelligence data and formulate differentiated context extraction strategies based on the data characteristics of different intelligence source platforms. Extract entity references and surrounding text from the original text as context windows to obtain a combination of entity references and context. Step 2: Input the combination of entity reference and context into the pre-trained language model for encoding, map it to a high-dimensional semantic space, and generate a semantic fingerprint vector; Step 3: Based on semantic fingerprint vectors, entities are divided into candidate groups by referential association rules. Within each candidate group, entities are paired up and the cosine similarity of semantic fingerprint vectors between paired entities is calculated. A context-aware dynamic threshold strategy is used to retain candidate pairs with cosine similarity greater than the dynamic threshold, thus completing the coarse screening of entity pairs. Step 4: For each candidate pair obtained after coarse screening, input the context text of the two entities to be compared in the candidate pair into the two encoders with shared weights of the deep semantic matching model based on the dual-tower structure, and calculate the matching score of the two entities; compare the matching score with the preset judgment threshold, and when the matching score exceeds the judgment threshold, it is initially determined that the two entities refer to the same object, and a high-confidence matching result is obtained. Step 5: Use the high-confidence matching results as anchors to construct an entity association graph. Use the label propagation algorithm to perform weighted iterative updates on the confidence of other candidate pairs in the entity association graph, and assign a globally unique identifier to each independent entity.

2. The cross-platform entity semantic disambiguation method for multi-source intelligence according to claim 1, characterized in that, Acquire multi-source intelligence data and formulate differentiated context extraction strategies based on the data characteristics of different intelligence source platforms. Extract entity references and surrounding text from the original text as context windows to obtain a combination of entity references and context, including: Raw text data from multiple heterogeneous intelligence source platforms is collected to obtain multi-source raw text. The multi-source raw text is then preprocessed to remove HTML tags, special symbols, and irrelevant noise data, resulting in cleaned text. Entity references are identified and extracted from the cleaned text. Based on the type of platform where the entity reference is located, the corresponding context window is extracted from the cleaned text. If the entity reference comes from a short text platform, the entire content of the text is used as the context window of the entity reference. If the entity reference comes from a long text platform, the text fragments of a preset number of tokens before and after the entity reference are extracted as the context window of the entity reference. Each extracted entity reference is combined with its corresponding context window to form a combination of entity reference and context.

3. The cross-platform entity semantic disambiguation method for multi-source intelligence according to claim 2, characterized in that, The combination of entity reference and context is input into a pre-trained language model for encoding, mapped to a high-dimensional semantic space, and a semantic fingerprint vector is generated, including: For each combination of entity reference and context, the entity reference and the corresponding context window are combined into an input sequence that meets the requirements of the pre-trained language model. The input sequence is fed into a deep pre-trained language model, and the discrete text symbols are mapped into continuous high-dimensional semantic space vectors through the multi-layer self-attention mechanism of the deep pre-trained language model. Extract the CLS position vector from the output of the last layer of the pre-trained language model and use this vector as the semantic fingerprint vector of the entity reference.

4. The cross-platform entity semantic disambiguation method for multi-source intelligence according to claim 3, characterized in that, The semantic fingerprint vector implicitly encodes the occupation, region, and associated organizational attributes of the entity.

5. The cross-platform entity semantic disambiguation method for multi-source intelligence according to claim 4, characterized in that, Based on semantic fingerprint vectors, entities are divided into candidate groups using referential association rules. Within each candidate group, entities are paired, and the cosine similarity of the semantic fingerprint vectors between paired entities is calculated. A context-aware dynamic thresholding strategy is employed to retain candidate pairs with a cosine similarity greater than the dynamic threshold, thus completing the coarse screening of entity pairs, including: According to the referential association rules, all entity referentials corresponding to the semantic fingerprint vector are grouped and processed, and entity referentials that meet the rules are assigned to the same candidate group, resulting in multiple candidate groups; For each candidate group, all entity references within the candidate group are paired up to form multiple entity pairs, each entity pair including two entity references to be compared. For each entity pair, extract the semantic fingerprint vectors corresponding to the two entity references in the entity pair from the semantic fingerprint vector set, calculate the cosine similarity between the two semantic fingerprint vectors, and use the cosine similarity as the identity consistency potential value of the entity pair. For each candidate group, the richness of contextual information of the entity references in that group is evaluated based on the length of the context window or the contextual information entropy of all entity references within the candidate group. The similarity threshold of the group is dynamically adjusted based on the richness of contextual information. When the contextual information is sparse, the similarity threshold of the group is automatically increased; when the contextual information is rich, the similarity threshold of the group is appropriately relaxed. For each entity pair, the cosine similarity of the entity pair is compared with the dynamic threshold of the candidate group to which the entity pair belongs. Entity pairs with a cosine similarity greater than the dynamic threshold are retained as high-potential candidate pairs that pass the coarse screening.

6. The cross-platform entity semantic disambiguation method for multi-source intelligence according to claim 5, characterized in that, The referential association rules include literal consistency rules, edit distance rules, and alias relationship rules in multi-source heterogeneous knowledge bases.

7. The cross-platform entity semantic disambiguation method for multi-source intelligence according to claim 6, characterized in that, For each candidate pair obtained after coarse screening, the context text of the two entities to be compared in the candidate pair is input into two encoders with shared weights in a deep semantic matching model based on a dual-tower structure, and the matching score of the two entities is calculated, including: Obtain the set of candidate pairs after coarse screening, which includes multiple candidate pairs, each consisting of two entity references to be compared; For each candidate pair in the candidate pair set, extract the context text corresponding to the first entity reference and the context text corresponding to the second entity reference from the combination of entity reference and context. The context text of the first entity reference is input into the first encoder of the pre-trained dual-tower deep semantic matching model, and the context text of the second entity reference is input into the second encoder of the dual-tower deep semantic matching model. The first semantic vector of the first entity reference is obtained by deep semantic encoding of the input context text through the first encoder, and the second semantic vector of the second entity reference is obtained by deep semantic encoding of the input context text through the second encoder. The first semantic vector and the second semantic vector are input into the similarity calculation layer of the dual-tower deep semantic matching model. The similarity calculation layer calculates the similarity between the first semantic vector and the second semantic vector and maps the similarity to a matching score in the range of 0 to 1.

8. The cross-platform entity semantic disambiguation method for multi-source intelligence according to claim 7, characterized in that, The matching score is compared with a preset judgment threshold. When the matching score exceeds the judgment threshold, it is initially determined that the two entities refer to the same object, resulting in a high-confidence matching result, including: Based on the matching score, each matching score corresponds to a high-potential candidate pair consisting of two entity references, and a preset judgment threshold is set. This judgment threshold is a value between 0 and 1, which is used to define whether the two entity references point to the same real object. For each high-potential candidate pair, the matching score corresponding to the candidate pair is compared with a preset judgment threshold; For candidate pairs whose matching scores are greater than the preset judgment threshold, it is initially determined that the two entities in the candidate pair refer to the same object, and the candidate pair is marked as a high-confidence matching result. For candidate pairs whose matching scores are less than or equal to the preset judgment threshold, the two entities in the candidate pair are judged to be different objects, and the candidate pair is not included in the high-confidence matching results. All candidate pairs marked as high-confidence matches are aggregated to form high-confidence matches.

9. The cross-platform entity semantic disambiguation method for multi-source intelligence according to claim 8, characterized in that, The high-confidence matching results are used as anchors to construct an entity association graph. A label propagation algorithm is employed to iteratively update the confidence of other candidate pairs in the entity association graph using weighted methods. Each independent entity is assigned a globally unique identifier, including: Entity pairs with matching scores exceeding a preset high confidence threshold are selected as high confidence anchors. These anchors are then used as initial reliable connection edges, and combined with all entity nodes to be disambiguated, an entity association graph is constructed. In the entity association graph, each node and each edge is assigned a reliability weight and a label is propagated. During the propagation process, a path decay factor is introduced so that the evidence strength between non-directly connected entity pairs decreases as the propagation path length increases. The balance coefficient is dynamically adjusted based on the current iteration round of tag propagation, and the initial matching evidence is weighted and fused with the evidence obtained through graph propagation. Based on the fusion results, the confidence of all candidate entity pairs is continuously updated through multiple rounds of iteration until the convergence condition is met or the preset maximum number of iterations is reached. Based on the final confidence score obtained after iterative updates, a globally unique identifier is assigned to each independent entity to achieve semantic disambiguation of entities across platforms.

10. A cross-platform entity semantic disambiguation system for multi-source intelligence, the system implementing the method as described in any one of claims 1 to 9, characterized in that, include: The extraction module is used to acquire multi-source intelligence data. It formulates differentiated context extraction strategies based on the data characteristics of different intelligence source platforms. It extracts entity references and surrounding text from the original text as context windows to obtain a combination of entity references and context. The encoding module is used to input the combination of entity reference and context into the pre-trained language model for encoding, mapping to a high-dimensional semantic space, and generating semantic fingerprint vectors. The coarse screening module is used to divide entities into candidate groups based on semantic fingerprint vectors and referential association rules. Within each candidate group, entities are paired up and the cosine similarity of semantic fingerprint vectors between paired entities is calculated. A context-aware dynamic threshold strategy is used to retain candidate pairs with cosine similarity greater than the dynamic threshold, thus completing the coarse screening of entity pairs. The matching module is used to input the context text of the two entities to be compared in each candidate pair obtained after coarse screening into two encoders with shared weights in a deep semantic matching model based on a dual-tower structure, calculate the matching score of the two entities, compare the matching score with a preset judgment threshold, and preliminarily determine that the two entities refer to the same object when the matching score exceeds the judgment threshold, thus obtaining a high-confidence matching result. The allocation module is used to construct an entity association graph by using high-confidence matching results as anchors. It employs a label propagation algorithm to perform weighted iterative updates on the confidence of other candidate pairs in the entity association graph and assigns a globally unique identifier to each independent entity.

Images