Three-level entity normalization and cross-modal semantic anchoring method and device facing multi-modal knowledge base, electronic equipment, computer readable storage medium and system

Abstract

The application provides a three-level entity normalization and cross-modal semantic anchoring method, device, electronic equipment, computer readable storage medium and system for a multi-modal knowledge base. The method comprises: converting multi-modal data into an initial text block carrying a modal identifier, and generating a standard text block by alias replacement using a mapping dictionary; identifying entity names and associated description texts contained in the standard text block; when a historical node consistent with the entity name already exists in the knowledge base, appending the description text to the historical node, and when the historical node does not exist, creating a new node according to the entity name; calculating an initial similarity between a first feature sequence of the new node and a second feature sequence of a candidate node; when the modal identifiers of the new node and the candidate node are different, applying a positive adjustment parameter to the initial similarity to obtain a target similarity, and when the target similarity reaches a connectivity threshold, constructing an equivalent association edge. The application effectively alleviates the entity fragmentation problem in the construction of a cross-modal network.

Description

Technical Field

[0001] This application relates to the field of knowledge graph and data processing technology, and more specifically, to a three-level entity normalization and cross-modal semantic anchoring method, apparatus, electronic device, computer-readable storage medium and system for multimodal knowledge bases. Background Technology

[0002] As enterprises deepen their digital transformation, the data sources and types that knowledge management systems in governments and large enterprises need to handle are becoming increasingly diversified. A typical enterprise knowledge base needs to simultaneously incorporate document data, audio and video recordings, and structured data from business system databases. Integrating this heterogeneous multimodal data into a unified, searchable multimodal knowledge base is a core application scenario requirement in the current knowledge management field.

[0003] Existing knowledge graph construction schemes typically employ a processing architecture that directly and uniformly extracts multimodal source data and then performs a post-processing absolute comparison and merging. Specifically, this scheme first coarsely transforms the raw data of various modalities into plain text and segments it into multiple isolated text blocks. Then, an extraction model is used to independently identify entities within each text block to generate corresponding isolated entity nodes. Finally, in the graph merging stage, the system uniformly calculates the natural similarity of the feature vectors of each entity node. Only when this natural similarity exceeds a preset single absolute fixed threshold is the entity identified as the same and a deduplication and merging operation is performed.

[0004] However, this approach of direct extraction followed by absolute comparison has significant technical flaws. Because the same real-world entity can easily appear in different modalities with numerous unknown aliases or differentiated styles of expression, existing solutions lack a mechanism to uniformly standardize and replace multiple source texts before feature extraction. This results in severely fragmented and divergent entity nodes identified initially. Furthermore, existing solutions fail to consider the systematic impact of data modality on textual expression styles when performing similarity comparisons, relying solely on absolute and uniform fixed similarity thresholds. This makes it difficult to effectively overcome feature distance attenuation caused by differences in modal media. Consequently, originally equivalent cross-modal nodes fail to meet connectivity standards due to similarity slightly below the threshold, ultimately failing to establish effective equivalent semantic associations between cross-modal knowledge fragments and hindering the construction of a fragment-free unified graph network. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, computer-readable storage medium, and system for three-level entity normalization and cross-modal semantic anchoring for multimodal knowledge bases, so as to at least alleviate the above-mentioned technical problems.

[0006] A three-level entity normalization and cross-modal semantic anchoring method for multimodal knowledge bases includes: Multimodal data is converted into initial text blocks carrying modality identifiers, and the initial text blocks are aliased using a mapping dictionary to generate standard text blocks; Identify the entity names contained in the standard text block and extract the entity description text associated with the entity names; If a historical node with the same name as the entity already exists in the multimodal knowledge base, the entity description text is appended to that historical node; if no historical node with the same name as the entity exists in the knowledge base, a new node is created based on the entity name. Calculate the first feature sequence of the newly generated node and obtain the second feature sequence of the candidate node in the knowledge base, and calculate the initial similarity between the first feature sequence and the second feature sequence; When the modality identifier of the new node is different from that of the candidate node, a positive adjustment parameter is applied to the initial similarity to obtain the target similarity; when the target similarity reaches a preset connectivity threshold, an equivalent association edge is constructed between the new node and the candidate node.

[0007] Optionally, the step of using a mapping dictionary to perform alias replacement on the initial text block to generate a standard text block includes: Construct a mapping tree containing standard proper nouns and their aliases; Match the text sequence in the initial text block with the mapping tree; When a text fragment that matches any alias is found, that text fragment is identified as the target alias; Determine the starting position of the target alias in the text sequence; Extract the target standard proper nouns corresponding to the target aliases from the mapping tree; At the starting position, replace the target alias with the target standard proper noun; A replacement record is generated based on the original text of the target alias, the target standard proper noun, and the starting position; The replacement record is stored in the metadata area of ​​the initial text block to obtain the standard text block.

[0008] Optionally, after appending the entity description text to the history node, the method further includes: Insert a separator at the end of the original record paragraph in the historical node; The additional entity description text is concatenated after the delimiter to form an accumulating concatenated text; Determine the amount of storage space occupied by the accumulated concatenated text; When the storage space value is detected to exceed the preset redundancy threshold, the accumulated concatenated text is sent to the semantic extraction component; Receive the summary description paragraph generated by the semantic extraction component through compression and reconstruction of the accumulated concatenated text; Replace the accumulated cascading text stored in the history node with the entire summary description paragraph.

[0009] Optionally, after constructing an equivalence edge between the newly created node and the candidate node, the method further includes: Extract action relationships with intent-directed characteristics from multiple standard text blocks; Determine the first action association relationship starting with the entity name, and connect the starting point of the first action association relationship to the output of the corresponding historical node or the new node; Determine the second action association with the entity name as the termination point, and connect the endpoint of the second action association to the input of the corresponding historical node or the new node; By utilizing the connection paths within the nodes, a cross-modal traversal query path is constructed from the second action association to the first action association.

[0010] Optionally, the process of identifying entity names contained in the standard text block and extracting entity description text associated with the entity names further includes: Configure the entity recognition unit with a recognition probe for capturing external document citation formats; The recognition probe is driven to scan the standard text block and locate the document name string within the preset book title identifier and the adjacent citation verbs; Based on the document name string, generate an exogenous reference node; Based on the citation verb, generate citation relationship edges from the source node corresponding to the standard text block to the exogenous reference node; The entity name is determined based on the exogenous reference node, and the entity description text is generated based on the adjacent text fragments in the standard text block that are adjacent to the reference name string.

[0011] Optionally, the process of converting multimodal data into initial text blocks carrying modality identifiers further includes: Extract the generation time information of the multimodal data; Convert the generated time information into a standard timestamp in a unified format; The standard timestamp is stored in the metadata field corresponding to the initial text block; Upon receiving a business query instruction from an external device, multiple candidate text blocks matching the business query instruction are retrieved from the multimodal knowledge base. Extract the standard timestamp stored in each of the returned candidate text blocks; Multiple candidate text blocks with time differences within a preset range and carrying different modality identifiers are combined into a concurrent multimodal data cluster; The synchronous multimodal data cluster is pushed to the external presentation device to reconstruct a cross-modal event snapshot at a specified time.

[0012] Optionally, the step of converting multimodal data into initial text blocks carrying modality identifiers includes: When the multimodal data is audio data, the speech activity detector is used to locate the speech pause points, the audio data is segmented into independent speech segments based on the speech pause points, and speech recognition is performed on the independent speech segments to obtain spoken text. Audio modality identifiers are added to the spoken text to obtain the initial text block. When the multimodal data is structured data, the annotation information of the header column attributes and the foreign key dependency information representing the hierarchical relationship between multiple tables are extracted. The annotation information and the foreign key dependency information are concatenated into a natural language description paragraph through a semantic generation network. The structured data modality identifier is added to the natural language description paragraph to obtain the initial text block.

[0013] Optionally, calculating the initial similarity between the first feature sequence and the second feature sequence includes: The first feature sequence is mapped to a preset vector space to obtain a first semantic vector representing its semantics; The second feature sequence is mapped to the vector space to obtain a second semantic vector representing its semantics. Calculate the cosine of the angle between the first semantic vector and the second semantic vector in the vector space; A quantitative score for indicating the degree of semantic content overlap is determined based on the cosine value of the included angle; The quantified score is assigned as the initial similarity.

[0014] Optionally, when the modality identifier of the newly generated node differs from the modality identifier of the candidate node, applying a positive adjustment parameter to the initial similarity to obtain the target similarity includes: Identify the first underlying medium type indicated by the modal identifier of the newly generated node; Identify the second underlying media type indicated by the modal identifier of the candidate node; When the first underlying media type and the second underlying media type correspond to written text media and spoken text transcription media respectively, the preset cross-media style difference compensation value is retrieved from the configuration parameter table; The cross-media style difference compensation value is determined as the positive adjustment parameter; The target similarity is obtained by adding the positive adjustment parameter to the initial similarity.

[0015] A three-level entity normalization and cross-modal semantic anchoring device for multimodal knowledge bases includes: The modality conversion module is used to convert multimodal data into initial text blocks carrying modality identifiers; The alias replacement module is used to replace the aliases of the initial text block using a mapping dictionary to generate a standard text block. An entity recognition module is used to identify entity names contained in the standard text block and extract entity description text associated with the entity names; The historical node processing module is used to append the entity description text to the historical node when a historical node with the same name as the entity already exists in the multimodal knowledge base. The new node creation module is used to create a new node based on the entity name when there is no historical node with the same name as the entity in the knowledge base. The feature acquisition module is used to calculate the first feature sequence of the newly generated node and to acquire the second feature sequence of the candidate node in the knowledge base; The similarity calculation module is used to calculate the initial similarity between the first feature sequence and the second feature sequence; An adjustment module is used to apply a positive adjustment parameter to the initial similarity when the modality identifier of the new node is different from that of the candidate node, so as to obtain the target similarity. The association construction module is used to construct equivalent association edges between the new node and the candidate node when the target similarity reaches a preset connectivity threshold.

[0016] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the aforementioned three-level entity normalization and cross-modal semantic anchoring method for multimodal knowledge bases.

[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned three-level entity normalization and cross-modal semantic anchoring method for multimodal knowledge bases.

[0018] A three-level entity normalization and cross-modal semantic anchoring system for multimodal knowledge bases includes: Multimodal data acquisition equipment, used to acquire multimodal data; And the aforementioned three-level entity normalization and cross-modal semantic anchoring device for multimodal knowledge bases, wherein the device is communicatively connected to the multimodal data acquisition device and is used to receive the multimodal data and perform three-level entity normalization and cross-modal semantic anchoring.

[0019] The technical advantages of the technical solution provided in this application are: This application presents a three-level entity normalization and cross-modal semantic anchoring method, apparatus, electronic device, computer-readable storage medium, and system for multimodal knowledge bases. These methods address the technical shortcomings of traditional post-processing direct comparison and merging schemes, such as severe initial fragmentation and divergence of cross-modal entities, and the difficulty in establishing connectivity between equivalent semantic nodes due to the lack of consideration for differences in modal media.

[0020] First, this application effectively solves the problem of isolated and divergent entity nodes caused by the lack of source intervention in traditional solutions by converting multimodal data into initial text blocks carrying modality identifiers and then using a mapping dictionary to replace the aliases of these initial text blocks to generate standard text blocks. Compared to the traditional architecture that relies on independent extraction followed by blind comparison, this application proactively introduces a pre-emptive mapping dictionary standardization mechanism before identifying the entity names contained in the standard text blocks and extracting the entity description text associated with those entity names. This allows differentiated and non-standard alias representations under various modalities to be uniformly switched to standard text in advance. Based on this unified standard, when a historical node with the same entity name already exists in the multimodal knowledge base, the entity description text can be accurately appended to that historical node. Only when a historical node with the same entity name does not exist in the knowledge base is a new node created based on the entity name. This mechanism significantly eliminates entity splitting caused by the divergence of aliases across different modalities from the input source, and significantly improves the aggregation degree and knowledge attribution consistency of multi-source entity fragments in cross-modal scenarios.

[0021] Furthermore, after calculating the first feature sequence of the newly generated node and obtaining the second feature sequence of the candidate node in the knowledge base, and calculating the initial similarity between the first and second feature sequences, this application applies a positive adjustment parameter to the initial similarity when the modal identifier of the newly generated node differs from that of the candidate node, thereby obtaining the target similarity. This solves the technical defect of traditional schemes that make it difficult to connect cross-modal equivalent nodes due to feature loss caused by the use of a single absolutely fixed threshold. Compared with the traditional rigid one-size-fits-all judgment system, this application actively identifies the differences in the underlying modal identifiers and adaptively applies a positive adjustment parameter, reasonably compensating for the systematic feature distance deviation caused by the natural differences in expression styles between different modal media. Therefore, the corrected target similarity can more accurately reflect the true semantic equivalence relationship between cross-modal segments, and thus, when the target similarity reaches the preset connectivity threshold, it can more robustly construct equivalent association edges between the newly generated node and the candidate node. This greatly enhances the ability to establish effective equivalent connections between cross-modal knowledge fragments across media barriers, providing strong technical support for building a semantically coherent and fragment-free unified multimodal graph network. Attached Figure Description

[0022] Figure 1 This application provides an embodiment of a three-level entity normalization and cross-modal semantic anchoring scenario for a multimodal knowledge base. Figure 2 This application provides a three-level entity normalization and cross-modal semantic anchoring method for multimodal knowledge bases. Figure 3 This application provides an embodiment of a three-level entity normalization and cross-modal semantic anchoring device for multimodal knowledge bases. Figure 4 This is an electronic device according to an embodiment of the present application.

[0023] Figure 5 This is a computer-readable storage medium according to an embodiment of the present application.

[0024] Figure 6 This application provides an embodiment of a three-level entity normalization and cross-modal semantic anchoring system for multimodal knowledge bases. Detailed Implementation

[0025] like Figure 1 As shown, this application provides an embodiment of a three-level entity normalization and cross-modal semantic anchoring scenario for a multimodal knowledge base; Figure 2 As shown in the figure, this application provides a three-level entity normalization and cross-modal semantic anchoring method for multimodal knowledge bases, which includes the following steps: Multimodal data is converted into initial text blocks carrying modality identifiers, and the initial text blocks are aliased using a mapping dictionary to generate standard text blocks; Identify the entity names contained in the standard text block and extract the entity description text associated with the entity names; If a historical node with the same name as the entity already exists in the multimodal knowledge base, the entity description text is appended to that historical node; if no historical node with the same name as the entity exists in the knowledge base, a new node is created based on the entity name. Calculate the first feature sequence of the newly generated node and obtain the second feature sequence of the candidate node in the knowledge base, and calculate the initial similarity between the first feature sequence and the second feature sequence; When the modality identifier of the new node is different from that of the candidate node, a positive adjustment parameter is applied to the initial similarity to obtain the target similarity; when the target similarity reaches a preset connectivity threshold, an equivalent association edge is constructed between the new node and the candidate node.

[0026] Optionally, the step of using a mapping dictionary to perform alias replacement on the initial text block to generate a standard text block includes: Construct a mapping tree containing standard proper nouns and their aliases; Match the text sequence in the initial text block with the mapping tree; When a text fragment that matches any alias is found, that text fragment is identified as the target alias; Determine the starting position of the target alias in the text sequence; Extract the target standard proper nouns corresponding to the target aliases from the mapping tree; At the starting position, replace the target alias with the target standard proper noun; A replacement record is generated based on the original text of the target alias, the target standard proper noun, and the starting position; The replacement record is stored in the metadata area of ​​the initial text block to obtain the standard text block.

[0027] Preferably, the process of constructing a mapping tree containing standard proper nouns and their aliases begins with pre-set domain dictionary data. This domain dictionary is essentially a structured terminology lookup table, stored as data records in persistent media. Each record contains a standard proper noun selected as the normalization benchmark, and a list of aliases with equivalent referential relationships to that standard proper noun. In the context of a government and enterprise knowledge base, the standard proper noun is typically selected from the full name used in authoritative documents, formal regulations, or industry standards, such as "People's Bank of China"; while the alias list compiles variant forms of the entity appearing in everyday writing, spoken communication, or business abbreviations, such as "Central Bank," "People's Bank of China," or "People's Bank of China." The construction process first traverses all records in the domain dictionary, extracting the standard proper noun and its corresponding alias set for each record, and initializes an empty multi-way search tree structure in memory, using the standard proper noun as the primary key and the alias set as the association value. The technical essence of this multi-way search tree is a finite state automaton based on character sequences. Each node represents a character, and the characters along the path from the root node to a given node are concatenated sequentially to form a complete alias or prefix of a standard proper noun. Specifically, during construction, for each standard proper noun in a record and each alias in the record's alias list, an insertion operation is performed into the multi-way search tree: starting from the root node, with each individual character constituting the name as the granularity, it checks whether the current character already exists as a child node of the current node. If not, a new child node is created, and the current node's pointer is moved down to that child node. After all characters in the name have been sequentially inserted, a termination flag is marked on the node corresponding to the last character, and a reference pointer to the corresponding standard proper noun is appended to this termination node. When all standard proper nouns and aliases in the domain dictionary have been inserted in this way, the resulting multi-way search tree is a mapping tree. The root node of the mapping tree does not carry semantic characters; its child nodes represent the first character of each name. Each node marked with a termination flag uniquely corresponds to a complete name, and through the reference pointer attached to the termination node, it is possible to reverse-address from any alias to its unique standard proper noun. If an alias corresponds to multiple standard proper nouns, disambiguation can be achieved by attaching a list of reference pointers to the termination node. However, it is preferable to ensure a one-to-one mapping between aliases and standard proper nouns during the dictionary maintenance phase. This tree structure allows for multi-pattern matching in the text sequence to discover all possible aliases simultaneously with a single left-to-right scan, without needing to initiate an independent string search for each alias, greatly reducing the time complexity of the matching process.

[0028] Preferably, the matching of the text sequence in the initial text block with the mapping tree is completed through a single traversal scan driven by a finite state machine. The initial text block is a text segment carrying a modal identifier, obtained after modal transformation and block processing of the original multimodal data. The text sequence stored within it is the character sequence to be standardized into aliases. The matching process uses this text sequence as input, setting a current state pointer pointing to the root node of the mapping tree and a scanning cursor pointing to the starting position of the text sequence. As the scanning cursor moves character by character along the text sequence from front to back, the matching logic simultaneously attempts to perform state transitions along the child nodes in the mapping tree corresponding to the current character: if the node pointed to by the current state pointer has a child node that matches the character pointed to by the scanning cursor, the state pointer is moved to that child node, and the scanning cursor is moved to the next character; if the current state pointer has already pointed to a node marked with an end flag, it indicates that the continuous subsequence formed from the starting character of this match to the character before the current character is exactly consistent with an alias recorded in the mapping tree. At this point, the matching logic does not terminate immediately, but continues to attempt to move along deeper child nodes to capture potentially longer alias matches. This follows the longest match priority principle, ensuring that in alias inclusion relationships such as "People's Bank of China" and "People's Bank of China Monetary Policy Committee," longer aliases are identified first to avoid fragmented replacements. When the character pointed to by the scanning cursor cannot find a match in any child node of the current state node, if the current state node is a terminal node, the text segment from the start of this match to the character before the scanning cursor is recorded as a hit alias match; if the current state node is not a terminal node, the current matching path is invalidated, the state pointer backs to the root node, the scanning cursor backs to the next character position after the start of this match, and a new round of matching begins. The entire matching process continues until the scanning cursor has traversed all characters of the text sequence. Each alias hit information recorded during the matching process includes the literal string of the alias itself and the starting and ending character indices of that alias string in the text sequence. Since the mapping relationship tree has embedded the associations of all aliases and their corresponding standard proper nouns in the reference pointers of the terminal nodes during the construction phase, once an alias is hit during the scanning process, the reference of the target standard proper noun corresponding to that alias can be obtained synchronously without the need for a second table lookup after the matching is completed.

[0029] Preferably, when a text segment matching any alias is found, the determination operation of identifying that text segment as the target alias directly relies on the matching information recorded at the terminal node during the aforementioned single-pass scan. Specifically, during the scanning cursor traversal of the text sequence, whenever the state pointer enters a node with a terminal marker and subsequently cannot continue to move along deeper child nodes, the scanning logic marks the continuous character subsequence from the starting character index of this round of matching to the character index before the current scanning cursor as a successfully matched text segment. This text segment is completely consistent with a pre-set alias in the mapping tree in terms of character content, and therefore is identified as the target alias hit in the current round. The essence of the technical concept of the target alias is a literal instance of a non-standard expression that actually appears in the initial text block and needs to be replaced in subsequent steps. It is different from the static alias template stored in the mapping tree and is a dynamic manifestation of the static alias template in a specific context. To accurately locate the boundaries of target aliases in a text sequence, two key offsets are continuously maintained during the matching process: the match start offset, which records the character index of the scanning cursor at the start of the current matching round; and the match end offset, which records the character index of the scanning cursor's final position decremented by one when the state transition fails and rollback occurs. These two offsets together define a closed interval. All characters in the text sequence located within this closed interval, when connected in their original order, constitute the literal content of the target alias. When multiple text segments in the text sequence match different alias templates, the matching logic discovers and marks them sequentially according to the natural traversal order of the scanning cursor, and generates a matching event record for each successfully matched text segment, containing its original alias text, start offset, and end offset. Furthermore, to avoid replacement conflicts caused by overlapping matches, after marking a target alias, the matching logic sets the starting point of the next round of matching for the scanning cursor directly to the next character position after the end offset of that target alias. This ensures that the matched character interval is not reused to generate new alias matches, thereby guaranteeing the spatial exclusivity and determinism of the replacement operation.

[0030] Preferably, the operation of determining the starting position of the target alias in the text sequence relies on character-level index offset information continuously tracked during the matching process. Internally, the text sequence is represented as a linearly arranged array of characters in display order, with each character occupying an index position, incrementing sequentially from zero. After a round of alias matching is successful and the target alias is determined, the system extracts the matching start offset recorded at the start of the matching from the matching context of that round. This offset is the index position of the first character of the target alias string within the character array of the text sequence. The technical essence of this index position is the spatial positioning anchor point of the target alias in the text sequence. It precisely limits subsequent replacement actions to the one-dimensional character coordinate space of the text sequence, ensuring that the replacement only operates on the character range occupied by the target alias and does not affect the character content of other parts of the text sequence. Specifically, before each new matching attempt, the matching logic first saves the character index value of the current scanning cursor as a variable named "current matching start point." Subsequently, as the state transition progresses, the scanning cursor moves backward continuously, but the value of the current matching start point remains unchanged until the end of the current matching round. When a match in this round is determined to be a successful match for an alias, the index value stored at the current matching starting point is identified as the starting position of the target alias. When a match in this round fails and rolls back, the current matching starting point is discarded and assigned a new value when the next match begins. The record of the starting position and the literal content of the target alias are stored in the match event record in a one-to-one correspondence, providing an immutable spatial basis for subsequent replacement execution and replacement audit log generation. Since the text sequence may contain multiple target aliases that need to be replaced, these target aliases may be adjacent to each other or separated in the sequence. Their respective starting positions are calculated independently and do not interfere with each other, thus providing an accurate index basis for parallel replacement operations of multiple aliases.

[0031] Preferably, the extraction of the target standard proper noun corresponding to the target alias from the mapping tree is accomplished using reference pointers pre-embedded in the terminal nodes of the mapping tree. When constructing the mapping tree, each terminal node representing the complete character path of the alias holds a data field called a standard name reference. This field stores a direct index or pointer to a standard proper noun string object in memory. When the matching process identifies a target alias and backtracks to the corresponding terminal node, the matching logic directly reads the string content pointed to by the standard name reference field of that terminal node. This content is the target standard proper noun with an equivalent referential relationship to the target alias. The essence of the target standard proper noun is the unified baseline name assigned to the concept to which the alias belongs in the domain dictionary; it is the identifying text of that concept as the unique master node in the knowledge graph. Since the mapping tree has already fixed the mapping relationship between the alias and its standard proper noun in the reference pointers of the terminal nodes when inserting the alias, the extraction action only involves one pointer addressing and string copying operation, without the need for additional string comparison or database queries. When a target alias is uniquely matched in the mapping tree, the extracted target standard proper noun is also unique. If redundancy in dictionary maintenance leads to an alias being associated with multiple standard proper nouns, a reference list can be stored in the terminal node. In this case, the extraction action will return a candidate list, from which a pre-defined disambiguation rule will select one, such as prioritizing standard proper nouns that are ranked higher in the dictionary or have a longer character length. In government and enterprise knowledge base applications, since domain dictionaries are usually carefully maintained by domain experts, a strict one-to-one mapping is maintained between aliases and standard proper nouns. Therefore, the extraction action does not require additional disambiguation and directly obtains a unique target standard proper noun. After extraction, the string of the target standard proper noun, along with the original text of the target alias and its starting position, will be encapsulated in a replacement descriptor data structure. This replacement descriptor serves as the input parameter for subsequent text replacement operations and the generation of audit records.

[0032] Preferably, the operation of replacing the target alias with the target standard proper noun at the starting position is implemented through a single index-based partial text rewriting. The operation target is the text sequence in the initial text block, which is loaded as a variable-length character buffer, supporting deletion and insertion operations within a specified index range. The processing action first receives the replacement descriptor, from which it extracts the starting position of the target alias, the character length of the target alias, and the string value of the target standard proper noun. The character length of the target alias is obtained by subtracting the starting position from the ending offset of the target alias and adding one, which indicates how many characters are spanned from the starting position to form the complete alias text. The replacement operation uses the starting position as the anchor point. First, it sets the continuous character range in the text sequence starting from the starting position and with a length equal to the character length of the target alias as the area to be deleted, and performs a deletion operation, removing all original characters located within this range from the character buffer. Then, at the same starting position, the string value of the target standard proper noun is inserted into the character buffer character by character as the new content. After a combination of deletion and insertion operations, the character range originally occupied by the target alias in the text sequence is completely replaced with the target standard proper noun. The total length of the text sequence may change due to the difference between the alias length and the standard proper noun length, and all character indices after the starting position are automatically offset accordingly. To ensure that when multiple target aliases need to be replaced in the text sequence, the starting position of subsequent aliases is not affected by changes in text length caused by previous replacements, the replacement operation is performed in reverse order from the starting position of all target aliases in the text sequence. The basis for reverse execution is that later replacement operations only change the character indices of the later part of the text sequence, without affecting the starting position indices of those aliases located before them. When performing each replacement in reverse order, the starting position of the target alias uses its original starting position value determined in the original text sequence. This value remains valid when processing earlier aliases in reverse order, thus ensuring that all target aliases are accurately hit and replaced without character index errors. After the replacement is completed, all aliases in the text sequence that are included in the domain dictionary are standardized into their corresponding standard proper noun expressions, forming an intermediate text sequence that has undergone alias standardization.

[0033] Preferably, the operation of generating replacement records based on the original text of the target alias, the target standard proper noun, and the starting position is accomplished by constructing a structured audit data entry. The original text of the target alias is the raw string extracted from the text sequence during the matching phase; the target standard proper noun is the normalized name string extracted from the mapping tree; and the starting position is the starting character index of the target alias in the original text sequence. When generating replacement records, a data structure in the form of a key-value pair set is created using the above three information items as the core payload. This data structure contains at least three fields: an alias original text field, used to store the original text content of the target alias; a standard name field, used to store the target standard proper noun used for replacement; and a replacement position field, used to store the integer value of the starting position, and may also include the character length or ending position of the target alias to fully identify the range of the replaced interval. Optionally, the replacement record may also include contextual information such as a replacement timestamp and an initial text block identifier to enhance the spatiotemporal traceability of the audit. The generation action occurs after each alias is successfully identified and the replacement descriptor is constructed, and is performed asynchronously or synchronously with the deletion and insertion operations of the text sequence. The generation of replacement records does not alter the text sequence itself, but rather binds it to the text block as supplementary data, recording all the modification details the text block undergoes during alias standardization. By recording the original text of the target alias and the target standard proper noun in the same entry, the replacement record constructs a complete mapping trajectory from the original non-standard expression to the standard expression. This allows any subsequent processing stage that relies on the standardized text to retrospectively query whether the appearance of a certain standard name in the text block is due to alias replacement, and what the original word was before the replacement. Furthermore, the record of the starting position in the replacement record precisely locks the spatial coordinates of each modification in the original text sequence. Even if the text content has changed after multiple rounds of replacement, the original appearance before replacement can still be reproduced at the corresponding position in the original text through reverse verification of the starting position and the original alias text.

[0034] Preferably, the operation of storing replacement records in the metadata area of ​​the initial text block involves injecting the replacement records generated in the previous step as additional descriptive information into the structured metadata container of the initial text block. The initial text block is a composite data object containing text content, source modality identifiers, and extensible metadata fields. Its metadata area is logically a collection of key-value pairs, capable of holding any descriptive, auditing, or control information related to the text block. The storage operation targets the metadata area of ​​the initial text block, serializing the generated replacement records into an ordered set of metadata entries. Each metadata entry exists as a pair of string keys and structured values. Specifically, a metadata key named "Alias ​​Replacement Log" can be defined for the replacement record set. Its corresponding value is a list structure consisting of all replacement record objects, where each element contains subfields such as the alias original text field, the standard name field, and the replacement position field. When multiple rounds of alias matching and replacement generate multiple replacement records on the same initial text block, these replacement records are appended to the same list in the order in which the replacements occurred, forming a complete replacement history archive. The technical purpose of storing replacement records in the metadata area instead of discarding them is to provide downstream knowledge graph construction pipelines with complete alias replacement audit and traceability capabilities. Subsequent stages such as entity relationship extraction, entity alignment, and knowledge fusion, when consuming standardized text within standard text blocks, can read the alias replacement logs stored in the metadata area to determine which standard proper nouns in the current text were replaced by aliases, their original meanings, and their original positions within the text. This supports advanced functions such as knowledge tracing, error correction and rollback, and manual review. The metadata area, as an integral part of the text block, is transmitted and persisted along with the text block, ensuring that the lifespan of the replacement records is consistent with that of the text block and is not lost as the processing progresses. Once the initial text block has undergone the aforementioned alias replacement operation and successfully stored the replacement records in its metadata area, the text block is transformed into a standard text block. The technical essence of a standard text block is that all aliases covered by the domain dictionary in the text content it carries have been replaced with unified standard proper nouns, and its metadata area fully records the audit information of the original text, standard name and location of each replacement. This not only achieves terminology standardization at the text level, but also preserves a complete chain of evidence for standardization, laying a clean and traceable data foundation for the subsequent stages of multi-level entity normalization.

[0035] Optionally, after appending the entity description text to the history node, the method further includes: Insert a separator at the end of the original record paragraph in the historical node; The additional entity description text is concatenated after the delimiter to form an accumulating concatenated text; Determine the amount of storage space occupied by the accumulated concatenated text; When the storage space value is detected to exceed the preset redundancy threshold, the accumulated concatenated text is sent to the semantic extraction component; Receive the summary description paragraph generated by the semantic extraction component through compression and reconstruction of the accumulated concatenated text; Replace the accumulated cascading text stored in the history node with the entire summary description paragraph.

[0036] Preferably, the process of inserting a separator identifier at the end of the original record segment in the historical node applies to historical nodes whose entity names have been normalized and already exist in the multimodal knowledge base. In the graph storage structure, a historical node corresponds to an entity container, which holds a text field for recording the semantic description of the entity. This text field is written with the initial entity description text when the historical node is first created, and new entity description text is continuously appended during the merging of nodes with the same name, thus forming the original record segment. The end position of the original record segment is determined by calculating the length of the string stored in the current text field and using this length as a character offset index. This offset index points to the next writable position after the last valid character in the string. The technical essence of the separator identifier is a boundary delimitation sequence that does not carry entity semantics. Its function is to embed clear segment boundary markers in the linear text stream, making entity description texts appended from different sources and at different times physically continuous but logically isolated, providing definite segmentation anchors for subsequent possible segment location, source tracing, or segmented summarization. The reason for inserting this delimiter at the end of the original record paragraph instead of relying on implicit breakpoints is that entity description text from different modalities may inherently contain regular punctuation marks such as periods and line breaks in natural language. If only regular punctuation marks are used as implicit segment boundaries, it will be impossible to distinguish the true appended boundaries from the punctuation marks in the entity description text itself when subsequent reverse parsing of each appended entity description text is required, thus losing the separability and auditability of the entity description text source. The delimiter can be selected using a type of non-printable control character or a combination thereof that does not appear in regular text content, such as the unit delimiter character in the unified character encoding standard, and the encoded value of this character can be recorded in the metadata field of the historical node as the agreed delimiter to ensure consistency and unique identification throughout the entire knowledge graph construction pipeline. After determining the end position of the original record paragraph, the processing action directly writes the delimiter string to that position, so that the last character of the original record paragraph is immediately followed by the delimiter. At this time, the text field content in the history node consists of the original record paragraph and the delimiter that follows, indicating that the entity description text to be added will have a clear left boundary identifier.

[0037] Preferably, the operation of appending the entity description text to the delimiter follows immediately after the delimiter insertion operation. This operation processes the entity description text currently identified and extracted from the standard text block, which matches the entity name of the historical node. The purpose of appending the entity description text to the delimiter is to maintain the temporal order and traceability of the appended entity description text. The newly arrived entity description text is placed at the end of the existing content of the historical node, forming a record structure that appends linearly according to arrival time. Simultaneously, the new content and old content are clearly separated by the aforementioned inserted delimiter, thus constructing a logically segmented but physically connected description sequence within a single text field. The appending operation is achieved by completely copying the string of entity description text and appending it to the current end position of the historical node text field. After appending, the total length of the text field is equal to the length of the original record segment plus the length of the delimiter plus the length of the entity description text. The continuous text sequence formed after appending is the cascading text. The technical essence of cascaded text accumulation is a sequential text aggregate containing multiple descriptive fragments. Each fragment corresponds to the entity description text extracted from a specific standard text block or modality. The fragments are concatenated by a unified delimiter. The entire cascaded text accumulation can be viewed as a complete log of entity semantic information across modalities and documents, unfolded chronologically or in processing order. This structure facilitates downstream components in quickly separating individual descriptive fragments using delimiters. This allows for importance assessment and information deduplication at the fragment level when summarizing is required, and for locating the specific source of a descriptive fragment when auditing and tracing are needed. Simultaneously, cascaded text accumulation also serves as a direct storage format for historical node text fields, supporting the display of entity node information in the graph and entity retrieval functions based on entity description text.

[0038] Preferably, determining the amount of storage space occupied by the accumulated concatenated text is for the purpose of quantitatively monitoring the redundancy of the text fields in historical nodes, thereby driving subsequent compression and reconstruction decisions. The technical essence of the storage space value is the number of bytes or characters occupied by the accumulated concatenated text in the storage medium or memory buffer; it is a direct measure of information redundancy and the degree of text field expansion. In scenarios where strings are stored using fixed-length encoding, the storage space value can be obtained by calculating the number of characters in the accumulated concatenated text string and multiplying it by the number of bytes occupied by each character. For example, when using a unified character encoding standard where each character occupies two or four bytes, the storage space value is the product of the total number of characters and the number of bytes per character. In scenarios using variable-length byte encoding, the string byte length acquisition function provided by the underlying runtime environment can be directly called to traverse the underlying byte sequence of the string to count the total number of bytes actually occupied. The timing for determining the storage space value can be after each appending of entity description text to a historical node, or during the unified inspection phase after batch appending. In government and enterprise knowledge base applications, a frequently occurring core entity, such as "People's Bank of China," may be repeatedly mentioned in dozens of policy documents and meeting recordings, resulting in various forms of entity description text. As the knowledge graph construction progresses, these entity description texts are continuously added to the text field of the same historical node, causing the length of the accumulated cascading text to grow linearly. Excessively long accumulated cascading texts not only consume storage resources in the graph database but also reduce the efficiency of operations such as historical node serialization, deserialization, and similarity calculation based on entity description text. Therefore, it is necessary to monitor storage space usage to trigger necessary simplification operations.

[0039] Preferably, the operation of sending the accumulated concatenated text to the semantic extraction component when the detected storage space exceeds a preset redundancy threshold constitutes a control link from redundancy monitoring to proactive simplification. The preset redundancy threshold is an integer threshold value pre-configured in the knowledge graph construction parameters. It is set based on the expected reasonable length of text fields in historical nodes. This reasonable length can be obtained through statistical analysis of the length distribution of text fields in normalized historical nodes in the knowledge base. For example, a certain multiple of the upper quartile or median of the length distribution can be used as the threshold, or an empirical value (e.g., four thousand characters) can be directly specified in the initial configuration of the construction pipeline. The technical essence of the preset redundancy threshold is a trigger condition parameter that divides the scale of the accumulated concatenated text into two states: when it is below the threshold, it is in a normal accumulation state, allowing continued addition of entity description text; when it reaches or exceeds the threshold, it enters a compression state, requiring the initiation of the summary extraction process to suppress further expansion. When the detected storage space is greater than or equal to the preset redundancy threshold, the processing action uses the accumulated concatenated text in the current historical node as input data and transmits it to the semantic extraction component. The semantic extraction component is a text processing unit configured to compress and reorganize information from multi-paragraph entity description text. Its core technology lies in its ability to extract core semantic elements from redundant, repetitive, and stylistically diverse entity description texts and reorganize them into a compact and coherent text. This component's design does not rely on a single summarization algorithm and is compatible with both extractive and generative summarization approaches. In extractive summarization scenarios, the component uses delimiters to segment the accumulating concatenated text into discrete description fragments. For each fragment, it calculates its semantic novelty score and entity coverage score relative to selected fragments, iteratively selecting the fragments with the highest scores to form a summary set, and then arranging the selected fragments according to their original temporal or logical relationships. In generative summarization scenarios, the component uses a pre-trained sequence-to-sequence neural network to encode the entire accumulating concatenated text into a sequence of context vectors. A decoder then generates a summary text sequence word by word, focusing on key entities and relational descriptions during the generation process through an attention mechanism. Regardless of the approach used, the sending action always transfers the complete accumulating concatenated text as input context and waits for the component to return the compressed and reconstructed result.

[0040] Preferably, receiving the summary description paragraph generated by compressing and reconstructing the cascaded text from the semantic extraction component marks the transformation of the text field in the historical node from a redundant state to a concise state. The specific implementation process of compression and reconstruction is executed internally by the semantic extraction component. Its core lies in analyzing the information content and repetition relationships of each description fragment in the cascaded text, and constructing a shorter but semantically more comprehensive summary description paragraph based on the principles of removing redundant information, merging similar expressions, and retaining different and complementary information. In the extractive summarization approach, compression and reconstruction involve selecting a subset of descriptive fragments from the cascaded text and inserting necessary transitional connectors according to the logical connections between the selected fragments, thus forming a readable, continuous text as the summary description paragraph. In the generative summarization approach, compression and reconstruction involves generating entirely new descriptive text as the summary description paragraph in an end-to-end manner based on a deep semantic understanding of the entire cascaded text. This summary description paragraph may use completely different wording from the individual descriptive fragments in the cascaded text, but it must cover the main entity attributes, key relationships, and timeliness information mentioned in the individual descriptive fragments. The technical essence of the summary description paragraph is a semantically equivalent, concise expression of the cascaded text. It is significantly shorter in character length than the cascaded text, but still maintains the explanatory power of the core semantics of the entities in terms of information retention. In the context of government and enterprise knowledge bases, the generation of summary description paragraphs also emphasizes the preservation of numerical information with decision-making value (such as budget amounts and timelines) and legally grounded citation information (such as policy document numbers) to avoid the loss of key facts due to compression and reconstruction. The receiving action caches the summary description paragraph text returned by the semantic extraction component in memory and associates it with the corresponding historical node identifier for use in subsequent replacement and writing operations.

[0041] Preferably, the operation of replacing the accumulated concatenated text stored in the historical node with a summary description paragraph aims to switch the content of the text field in the historical node from the bloated accumulated concatenated text to the concise summary description paragraph, thereby freeing up storage space and improving subsequent processing efficiency while maintaining semantic usability. The reason for using a complete replacement instead of appending the summary description paragraph to the accumulated concatenated text is that the existence of the accumulated concatenated text itself constitutes information redundancy. The information carried by each descriptive fragment within it has already been covered and refined by the summary description paragraph. Continuing to retain the accumulated concatenated text would cause the length of the text field to continue to increase while the semantic increment decreases, violating the original intention of the simplification operation. The complete replacement is achieved by clearing all the current content of the text field in the historical node and then writing the complete string of the summary description paragraph into the text field. After the replacement, the content stored in the text field of the historical node is completely changed to the summary description paragraph, and no longer directly holds the original accumulated concatenated text. To preserve the possibility of audit traceability and recovery from accidental compression, the replacement operation can serialize the original accumulating concatenated text and write it to a cold storage area used only for archiving before execution. The metadata field of the historical node records the timestamp of the archive reference and the replacement operation, ensuring that the complete accumulating concatenated text can still be viewed based on the archive reference if needed later. However, for the main text field of the historical node's online service, its content has become a compact summary description paragraph. Subsequently, when another entity description text with the same name is extracted from the new standard text block and needs to be appended, the new append operation will be based on the summary description paragraph, inserting a delimiter at the end of the summary description paragraph and concatenating the new entity description text, thus starting a new round of accumulation and simplification. This lifecycle management mechanism for descriptive information, consisting of accumulation, monitoring, refinement, and replacement, ensures that during the continuous growth of the multimodal knowledge base, the entity description text of core entity nodes can continuously absorb new cross-modal knowledge without spiraling out of control due to unlimited appending. This provides an adaptive lifecycle management capability for descriptive information for large-scale, long-cycle knowledge graph construction systems.

[0042] Preferably, the sequence-to-sequence neural network used in the above-mentioned generative summarization scenario consists of an embedding mapping layer, a stacked self-attention encoder, a context-gated attention bridging module, a stacked cross-attention decoder, and an output projection layer connected sequentially. The embedding mapping layer, as the entry point of the sequence-to-sequence neural network, maps each discrete character or sub-word unit in the accumulated cascaded text to a dense vector of fixed dimensions, forming the input embedding sequence. Simultaneously, it maps structured knowledge such as entity type annotations and relation type annotations to corresponding knowledge embedding vectors, aligns them with the input embedding sequence dimensionally, and then adds them element-wise to obtain the encoder input sequence that incorporates prior knowledge constraints. The stacked self-attention encoder consists of multiple structurally identical but parameter-independent encoder sublayers. Each encoder sublayer contains a multi-head self-attention computation unit and a position-sensitive feedforward network. The multi-head self-attention computation unit performs a linear transformation on the input encoder sequence to generate a query vector matrix, a key vector matrix, and a value vector matrix. An attention weight matrix is ​​calculated by scaling the dot product of the query and key vectors. This attention weight matrix is ​​then used to weight and sum the value vector matrices to capture long-distance semantic dependencies between any two positions within the cascading text. The position-sensitive feedforward network performs two linear mappings and one non-linear activation on the output of the multi-head self-attention computation unit to enhance the non-linear expressive power of the encoder sublayer. The stacked self-attention encoder outputs a context vector sequence, where each position vector incorporates semantic information from its corresponding input position and its global context.

[0043] Preferably, the context-gated attention bridging module is located between the stacked self-attention encoder and the stacked cross-attention decoder. Its technical function is to perform importance filtering and information density reconfiguration on the context vector sequence, enabling the stacked cross-attention decoder to prioritize regions in the accumulated concatenated text that are highly relevant to entity attributes and relational descriptions during word-by-word generation, while suppressing attention to insinuations and repetitive expressions. This context-gated attention bridging module receives the context vector sequence and simultaneously receives an entity mention mask sequence pre-annotated from the accumulated concatenated text or obtained through entity links. This entity mention mask sequence marks at each position whether the character at that position belongs to a certain entity mention text segment. Internally, the context-gated attention bridging module includes a learnable gating parameter vector and a position-by-position gating network. The position-by-position gating network concatenates each position vector in the context vector sequence with the corresponding entity mention marker in the entity mention mask sequence, and then calculates the gating coefficient for that position through a fully connected layer with a non-linear activation function. This gating coefficient is a continuous value between zero and one, representing the degree to which information at that position is allowed to enter the stacked cross-attention decoder. The gating coefficients are multiplied element-wise with the context vectors at the corresponding positions to selectively allow the context vector sequence, resulting in a gated context vector sequence. The gating coefficients corresponding to the text fragments mentioned by entities and the positions of relation trigger words are pushed to one to preserve the information at these positions with a higher strength, while the gating coefficients corresponding to meaningless stop words or repetitive and redundant content positions are pushed to zero, thereby reducing the interference of redundant information on the generation process of the stacked cross-attention decoder.

[0044] Preferably, the stacked cross-attention decoder consists of multiple structurally identical but parameter-independent decoder sublayers. Each decoder sublayer sequentially includes a masked multi-head self-attention computation unit, a multi-head cross-attention computation unit, and a position-sensitive feedforward network. The masked multi-head self-attention computation unit takes the preceding text sequence already generated by the stacked cross-attention decoder as input. By introducing a future position masking mechanism, it ensures that the word at the current position can only focus on the historical generation results before the current word, thereby maintaining the autoregressive property of text generation. The multi-head cross-attention computation unit receives the decoder self-attention representation output by the masked multi-head self-attention computation unit as a query vector matrix and receives the gated context vector sequence output by the context-gated attention bridging module as the key vector matrix and value vector matrix, performing a cross-sequence attention computation. This allows the stacked cross-attention decoder to dynamically retrieve and focus on the text fragment most relevant to the currently generated word from the global semantic information provided by the stacked self-attention encoder and the gated enhanced entity relation context when generating each word. The position-sensitive feedforward network has the same structure as the position-sensitive feedforward network in the encoder sublayer, further enhancing the representation capability of the decoder sublayer. After layer-by-layer processing of all decoder sub-layers, the stacked cross-attention decoder outputs a sequence of decoder hidden states. Each position vector in this sequence incorporates the generated preceding text, the context of the current position to be generated, and entity relationship information extracted from the accumulated concatenated text through cross-attention.

[0045] Preferably, the output projection layer receives the decoder hidden state sequence, maps each hidden state vector in the sequence to an unnormalized probability vector of the same size as the vocabulary using a linear transformation matrix, and then normalizes it using a flexible maximum function to obtain the generation probability distribution of each word in the vocabulary at the current position. When calculating the generation probability distribution, the output projection layer also receives an entity coverage feedback vector, which records the cumulative frequency distribution of entity mentions in the generated text segments covered by the stacked cross-attention decoder. The entity coverage feedback vector is dynamically updated at each generation time step through a coverage attention mechanism. This mechanism is calculated as follows: the cross-attention weights generated by the multi-head cross-attention calculation units of the stacked cross-attention decoder in the previous time step are summed along the direction of the cascading text length to obtain a cumulative coverage vector. Each element in this cumulative coverage vector represents the cumulative attention level of the stacked cross-attention decoder to the corresponding position in the input text during historical generation. The cumulative coverage vector and the cross-attention weights of the current time step are then input into a coverage avoidance network. This network linearly maps the cumulative coverage vector using a learnable transformation matrix and adds the mapping result as a bias term to the log-probability of the current cross-attention calculation. This prevents the stacked cross-attention decoder from repeatedly focusing on the same already covered text interval, prompting it to shift to insufficiently covered entity mention text segments when generating subsequent text, thus avoiding the omission of important entities in the generated summary text sequence. The final generated summary text sequence is obtained by word-by-word sampling or greedy decoding from the generation probability distribution. This summary text sequence serves as the summary description paragraph fed back by the semantic extraction component.

[0046] Preferably, the training process of the sequence-to-sequence neural network described above is optimized using a large-scale entity description text summarization dataset constructed on a government and enterprise knowledge base corpus. Each training sample in this dataset consists of a simulated cumulative concatenated text and a reference summary description paragraph. During training, the simulated cumulative concatenated text is input into the sequence-to-sequence neural network for forward propagation to generate a predicted summary description paragraph. The difference loss between the predicted and reference summary description paragraphs is calculated, and the weight parameters of the sequence-to-sequence neural network are iteratively updated with the goal of minimizing this difference loss. The difference loss is composed of a main loss term and an entity coverage auxiliary loss term, which are weighted and summed. The main loss term adopts word-by-word cross-entropy loss, which is calculated as follows: for each word position in the reference summary description paragraph, the probability distribution of the word at that position predicted by the sequence-to-sequence neural network is used to calculate the element-by-element cross-entropy with the unique hot vector of the word that actually appears at that position in the reference summary description paragraph. The average cross-entropy loss value is obtained by averaging the loss across all word positions in the entire reference summary description paragraph. This average cross-entropy loss value measures the degree of deviation between the predicted summary description paragraph and the reference summary description paragraph in terms of language fluency and content relevance. The entity coverage auxiliary loss term is used to enhance the completeness of the sequence-to-sequence neural network's coverage of entity mention text fragments appearing in the accumulated concatenated text. It is calculated as follows: all entity mention text fragments are pre-extracted from the accumulated concatenated text and deduplicated to form an entity mention set. During the generation process of the stacked cross-attention decoder, the cumulative sum of the cross-attention weights at each time step at the corresponding input positions of all entity mention text fragments is recorded. The difference between this cumulative sum and one is averaged over the entire sequence to obtain the entity coverage average omission loss. This entity coverage average omission loss value measures the degree of omission of entity mention text fragments appearing in the accumulated concatenated text when the sequence-to-sequence neural network generates the summary description paragraph. When the main loss term and the entity coverage auxiliary loss term are weighted and summed, the weight coefficient of the entity coverage auxiliary loss term is set to a small positive number (e.g., 0.1) to introduce entity coverage constraints more gently without excessively compromising the fluency of the generated text. The entire sequence-to-sequence neural network employs a mini-batch-based adaptive moment estimation optimization algorithm for gradient backpropagation and parameter updates. After multiple rounds of iterative training until the difference loss converges, the trained sequence-to-sequence neural network weights are obtained. When receiving accumulated concatenated text, the trained sequence-to-sequence neural network can generate summary descriptions with high coverage and complete preservation of core semantics, making it particularly suitable for the stringent requirements of government and enterprise knowledge base scenarios where key entity information must not be lost.

[0047] Optionally, after constructing an equivalence edge between the newly created node and the candidate node, the method further includes: Extract action relationships with intent-directed characteristics from multiple standard text blocks; Determine the first action association relationship starting with the entity name, and connect the starting point of the first action association relationship to the output of the corresponding historical node or the new node; Determine the second action association with the entity name as the termination point, and connect the endpoint of the second action association to the input of the corresponding historical node or the new node; By utilizing the connection paths within the nodes, a cross-modal traversal query path is constructed from the second action association to the first action association.

[0048] Preferably, the process of extracting action-related relationships with intent-directed characteristics from multiple standard text blocks uses a set of standard text blocks that have already undergone entity recognition and relationship extraction, generating entity names and entity descriptions, as the processing object. After entity relationship extraction, the structured extraction results carried within the standard text blocks contain not only entity names and entity descriptions, but also several relation triples. Each relation triple consists of a head entity name, a relation predicate, and a tail entity name. These relation triples constitute the action-related relationships. The technical essence of action-related relationships is to describe a directional semantic interaction between two entities. It consists of a head entity name as the starting point, a relation predicate as the action type, and a tail entity name as the ending point, and can be formally represented as a directed edge from the head entity name to the tail entity name. The intent-oriented characteristic is determined by semantically classifying relational predicates: a list of intent-oriented relational predicates is pre-maintained in the knowledge graph construction pipeline. This list includes verbs or verb phrases with clear action-oriented meanings, such as "issue," "revise," "require," "approval," "allocation," "notification," and "reporting." The action extraction process iterates through the relational triples of each standard text block, matching the relational predicate of each triple with the list of intent-oriented relational predicates. If a relational predicate matches any entry in the list, the triple is considered to have intent-oriented characteristics, marked as an intent-oriented action association, and extracted. If no match is found, the triple is considered a descriptive association and is not extracted in this process. This extraction method, based on relational predicate category filtering, differs from the traditional approach of indiscriminately incorporating all relational triples into the graph. It can filter out action-related relationships that carry decision-making, information, and action command flows from massive amounts of inter-entity relationships, providing a precise relational framework for subsequently constructing cross-modal, penetrating query paths. In the application scenario of government and enterprise knowledge bases, the "issue" and "revise" relationships in policy documents, the "requirement" and "report" relationships in meeting recordings, and the "allocation" relationship in database table structures are all typical examples of intent-to-action relationship associations.

[0049] Preferably, the operation of determining the first action association starting with the entity name processes the set of intent-directed action associations extracted in the previous step. The technical essence of the starting point is the end that initiates the action in a directed action association, i.e., the head entity name in the relation triple. Starting with the entity name means that the head entity name of the relation triple is completely identical to the target entity name currently undergoing normalization and anchoring processing. The determination process involves traversing the set of intent-directed action associations, extracting each association one by one, and performing a string equality comparison between the head entity name and the target entity name. If a match is found, the intent-directed action association is determined as the first action association. The technical essence of the first action association is a cross-modal semantic action record with the target entity as the action initiator. It represents a relation instance where the target entity plays an active role in the knowledge graph, such as the relation "People's Bank of China issues interest rate adjustment notice" with "People's Bank of China" as the head entity. Next, the processing of connecting the starting point of the first action relationship to the output end of the corresponding historical node or new node is executed according to the node persistence status of the target entity in the multimodal knowledge base: if the target entity already exists in the multimodal knowledge base as a historical node, the internal identifier of the historical node is obtained through the node reference mechanism of the graph database; if the target entity has not yet been persisted and is still in the state of a new node, the internal identifier of the new node is obtained through the transaction buffer of the graph database. The technical essence of the output end is that the entity node is used in the directed edge model of the knowledge graph to carry all outgoing edges starting from the entity. In the implementation of the graph database, the output end corresponds to the outgoing edge adjacency list structure stored in the entity node object. The connection action is implemented by adding a directed edge record pointing to the tail entity node included in the first action relationship to the outgoing edge adjacency list of the entity node. The directed edge record simultaneously writes the relation predicate and the source text block identifier, thereby completing the graph physical attachment of the first action relationship.

[0050] Preferably, the operation of determining the second action association with the entity name as the termination end forms a symmetrical process with the operation of determining the first action association in the previous step. The technical essence of the termination end is the end that acts as the action recipient in a directed action association, i.e., the tail entity name in the relation triple. Using the entity name as the termination end means that the tail entity name of the relation triple is completely identical to the current target entity name. The determination process also traverses the set of intent-pointing action associations, performing a string equality comparison between the tail entity name of each intent-pointing action association and the target entity name. If a match is found, the intent-pointing action association is determined as the second action association. The technical essence of the second action association is a cross-modal semantic action record with the target entity as the action recipient. It represents a relation instance in the knowledge graph where the target entity plays a passive receiving role, such as "People's Bank of China" as the tail entity node in "Director Zhang requested the People's Bank of China". The process of connecting the endpoint of the second action association to the input end of the corresponding historical node or new node similarly obtains the internal identifier of the corresponding historical node or new node based on the node landing status of the target entity. The technical essence of the input end is that entity nodes in the directed edge model of a knowledge graph serve as logical connection ports carrying all incoming edges reaching that entity. In the implementation of the graph database, the input end corresponds to the incoming edge adjacency list structure stored in the entity node object. The connection action is achieved by adding a directed edge record from the head entity node to the entity node in the incoming edge adjacency list. This directed edge record also contains the relation predicate and the source text block identifier, thus completing the graph physical attachment of the second action association relationship. Through the paired execution of the outgoing edge connection of the first action association relationship and the incoming edge connection of the second action association relationship, the target entity simultaneously obtains outward intention-directing relationships and inward intention-directing relationships in the graph. The sources of the two relationships may belong to different modalities and different standard text blocks, but they converge structurally by sharing the same entity node.

[0051] Preferably, the operation of constructing a cross-modal traversal query path from the second action association to the first action association through the internal connection path of the node uses the entity node with the incoming and outgoing edges already attached in the previous step as the core hub. The internal connection path of the node is a data routing mechanism inherent in the entity node in the graph database storage structure: in the physical storage record of the entity node, the incoming edge adjacency table and the outgoing edge adjacency table are organized in the same data page or the same document, and are associated together by the internal identifier of the entity node as the primary key. Any record in the incoming edge adjacency table can jump directly to any record in the outgoing edge adjacency table through the internal identifier of the entity node. This traversal path from the incoming edge to the outgoing edge does not rely on any external index lookup, and can be completed only by jumping through the pointer or offset inside the node, hence it is called the internal connection path of the node. The action of constructing a cross-modal penetration query path first filters out all incoming edge records marked as second action relationships from the incoming edge adjacency list of the entity node. Each incoming edge record contains a source header entity node reference, a relation predicate, and a source text block identifier, and the source text block identifier carries the modal identifier of the standard text block in which the relationship was extracted. Next, it filters out all outgoing edge records marked as first action relationships from the outgoing edge adjacency list of the entity node. Each outgoing edge record contains a destination tail entity node reference, a relation predicate, and a source text block identifier. Then, for each pair consisting of an incoming edge record of a second action relationship and an outgoing edge record of a first action relationship, it checks whether the modal identifier carried by the source text block identifier of the incoming edge record is different from that carried by the source text block identifier of the outgoing edge record. If they are different, it is determined that this pair of incoming and outgoing edges constitutes a cross-modal penetration query path. The technical essence of cross-modal penetration query paths is to start from the action context in which an entity in one modality acts as a passive receiver, penetrate the internal connection path of that entity node, and reach the action context in another modality in which the same entity acts as an active initiator. This connects two intentional actions that were originally isolated in different modal data fragments into a complete information flow chain through a shared entity. In the retrieval scenario of government and enterprise knowledge bases, when a user queries "What requirements has the People's Bank of China recently received and what has it issued accordingly?", the graph query engine uses cross-modal penetration query paths to start from the incoming edge of the second action relationship "Director Zhang requested the People's Bank of China" extracted from the meeting recording, and through the internal connection path of the "People's Bank of China" entity node, directly locate the outgoing edge of the first action relationship "The People's Bank of China issued an interest rate adjustment notice" extracted from the policy document. Thus, it returns complete cross-modal action chain information within a single query.This cross-modal association method, which differs from existing technologies that rely on multi-hop path exploration and full graph traversal, uses local connection paths within entity nodes as high-speed channels. During a query, it only needs to complete the adjacency jump from the incoming edge to the outgoing edge within a single entity node, avoiding costly cross-node path searches in large-scale knowledge graphs. This significantly reduces the response latency of cross-modal penetration queries. At the same time, the constructed cross-modal penetration query path itself is also persistently stored as a composite knowledge structure in the graph for reuse in subsequent queries.

[0052] Optionally, the process of identifying entity names contained in the standard text block and extracting entity description text associated with the entity names further includes: Configure the entity recognition unit with a recognition probe for capturing external document citation formats; The recognition probe is driven to scan the standard text block and locate the document name string within the preset book title identifier and the adjacent citation verbs; Based on the document name string, generate an exogenous reference node; Based on the citation verb, generate citation relationship edges from the source node corresponding to the standard text block to the exogenous reference node; The entity name is determined based on the exogenous reference node, and the entity description text is generated based on the adjacent text fragments in the standard text block that are adjacent to the reference name string.

[0053] Preferably, the processing of configuring the entity recognition unit with a recognition probe for capturing external document citation formats occurs before entity recognition and entity description text extraction of the standard text block, representing a targeted extension of the entity recognition unit's capabilities. The entity recognition unit is a functional component in the knowledge graph construction pipeline responsible for identifying entity names and extracting entity description text from standard text blocks. Technically, it can be a hybrid processor integrating a pattern matching rule engine and a deep semantic analysis model. After receiving the text sequence of the standard text block, it determines the entity boundaries and entity types appearing in the text through a combination of rule-based matching and model semantic understanding. To support the capture of cross-document citation relationships in explicit anchoring mechanisms, a recognition probe needs to be dynamically loaded within the entity recognition unit. Technically, the recognition probe is a set of pre-compiled regular expressions for matching specific document citation text patterns and a corresponding dictionary of citation verbs. It is designed to be injected as a plug-in into the pre-processing or parallel processing channels of the entity recognition unit without altering its core processing logic. The specific implementation of the configuration operation is as follows: During the configuration phase of the knowledge graph construction pipeline initialization, a preset citation pattern definition is read from an external configuration store. This definition includes at least one book title identifier pair, such as a boundary symbol pair consisting of Chinese left and right book title marks, and may also include other optional boundary symbol pairs such as double quotation marks and single book title marks. Simultaneously, a list of citation verbs is read, which contains verbs or verb phrases representing semantics such as "based on," "according to," "referencing," "executing," "forwarding," "revising," and "abolishing," etc. The configuration logic converts the book title identifier pairs into corresponding regular expression fragments, converts the list of citation verbs into regular expression fragments connected by OR operations, and combines them into a complete bibliographic citation pattern regular expression according to the order of the citation verbs and book title identifiers and the allowed interval distance (e.g., an interval window of zero to five arbitrary characters). The regular expression is compiled into a finite state automaton, which, along with a reference verb dictionary, is encapsulated into a recognition probe data structure. This recognition probe data structure is then injected into the pattern matching buffer inside the entity recognition unit, so that the entity recognition unit will drive the recognition probe to scan the text sequence synchronously in parallel when processing each standard text block.The reason for configuring a special identification probe to capture the citation format of external documents is that in the application scenarios of government and enterprise knowledge bases, there are many explicit citation expressions in policy documents, regulations, and meeting minutes, such as "according to Article 3 of the XX Measures". Such expressions carry the basis, revision and association between documents. However, conventional entity extraction models may only identify the XX Measures as a work name entity, and cannot explicitly promote it to an external document node that forms a citation relationship with the current document node. Therefore, by configuring the identification probe in a targeted manner, we can ensure that this kind of key structured association is reliably extracted.

[0054] Preferably, the operation of driving the recognition probe to scan the standard text block to locate the document name string within the preset book title identifier and the adjacent citation verb is the specific execution process of applying the configured recognition probe to the current standard text block. The text sequence of the standard text block is used as the scanning object and is fed character by character into the finite state automaton maintained inside the recognition probe. This finite state automaton defines multiple states according to pre-compiled regular expressions, with the initial state being the waiting citation verb state. When the scanning cursor moves forward along the text sequence, the automaton compares the character subsequence in the current scanning window with the entries in the citation verb list. Once a match is found, the automaton enters the citation verb confirmation state and records the start and end positions of the citation verb. Subsequently, the automaton uses the end position of the citation verb as the starting point and searches for the left book title identifier within the allowed interval window range. If the left book title identifier is found within the preset interval window length (e.g., five characters), the automaton enters the document name capture state; otherwise, the automaton returns to the initial state and continues scanning. After entering the document title capture state, the automaton records all characters after the left title identifier sequentially until it encounters the right title identifier. The captured continuous character subsequence at this point constitutes the document title string, and the start and end positions of this string are recorded. After the right title identifier, the automaton completes a successful pattern match, packaging the matched citation verb, the document title string, and their respective position indices into a citation instance description structure, outputting it to the matching result list. The automaton then resets to its initial state and continues scanning the remaining part of the text sequence from the current scanning position. This scanning mechanism can capture all citation instances matching the preset pattern in a single traversal, even if multiple citations appear in the same standard text block, such as "according to Notice 1 and Notice 2," two independent citation instance description structures can be generated for each. Throughout the scanning process, the specific characters of the preset book title identifier are passed to the recognition probe through parameters in the configuration phase. The preset method is to specify them in the configuration file of the pipeline in the form of key-value pairs. For example, "left_bookmark" corresponds to the left book title mark and "right_bookmark" corresponds to the right book title mark. When generating regular expressions, the pattern string is read from the configuration and concatenated into it, so that the recognition probe has the ability to adapt to the reference boundaries in different languages ​​or different document formats.

[0055] Preferably, the operation of generating exogenous reference nodes based on the document name string is a knowledge graph node creation step performed after the citation instance description structure is successfully captured from the standard text block. The technical essence of an exogenous reference node is that it is an entity node representing an external document in the multimodal knowledge base. Unlike conventional entities such as institution names and personal names that appear directly in the content of the standard text block, it represents an independent document cited by the current text and existing in the external document space. Its type in the graph is marked as document type. The generation action takes the document name string stored in the citation instance description structure as input. First, boundary normalization processing is performed on the document name string to remove potentially misincluded leading and trailing whitespace characters, invisible control characters, and redundant characters remaining in the context of the book title itself, resulting in a normalized document name string. Then, using this normalized document name string as the entity name, a node existence check is performed in the multimodal knowledge base, that is, to query whether a historical node with the exact same entity name and document type already exists in the current knowledge base. If an exogenous reference node exists, its internal identifier is directly retrieved as the exogenous reference node for this current citation to avoid duplicate creation. If it does not exist, a new entity node is constructed using the node creation interface of the graph database. Its entity name field is assigned the normalized document name string, and its entity type field is assigned a preset document type enumeration value. The internal identifier of this new node is returned as the result. The created or retrieved exogenous reference node is temporarily attached to a temporary graph fragment corresponding to the current text block, awaiting subsequent edge connections and descriptive text filling. This exogenous reference node itself also participates in the merging of nodes with the same name and vector similarity deduplication in the subsequent entity normalization process. Therefore, if the same document is cited in multiple different standard text blocks, the exogenous reference nodes generated from them will eventually be merged into a unified document entity node, thus forming a unique and complete external document view in the knowledge graph.

[0056] Preferably, the operation of generating citation edges from the source node corresponding to the standard text block to the exogenous reference node based on the citation verb is executed immediately after the exogenous reference node is generated. Its purpose is to explicitly establish a directed semantic relationship between the current document and the cited reference in the graph. The source node corresponding to the standard text block is an entity node representing the source document of the current standard text block. During the segmentation and parsing phases, the metadata of each standard text block records its source document identifier, which corresponds to an already created document type entity node in the knowledge graph, i.e., the source node. When generating citation edges, a directed edge is created using the source node as the starting node, the exogenous reference node as the target node, and the citation verb as the predicate label. Specifically, through the edge creation interface provided by the graph database, the internal identifier of the source node, the internal identifier of the target node, the type label of the edge, and an attribute dictionary are passed in. This attribute dictionary contains at least three fields: the original text of the citation verb, the start and end positions of the citation verb in the standard text block, and the source text block identifier. The edge type labels are converted from the original text of the citation verbs to a unified citation relationship type through a mapping function. For example, verbs such as "based on," "according to," and "refer to" are mapped to the "based on" relationship type; "revise" and "repeal" are mapped to the "revise" or "repeal" relationship type; and "cite" and "see also" are mapped to the "cite" relationship type. The generated citation relationship edges not only record the existence of citations but also retain the original lexical evidence and location information, providing fine-grained anchors for audit tracing. The reason for generating these citation relationship edges is that in the multimodal scenario of government and enterprise knowledge bases, the implicit anchoring of shared entity nodes alone is insufficient to reveal the direct logical dependencies between documents. For example, if a policy document is based on a certain regulation, this relationship is key knowledge for understanding the policy-making background and the applicability of the regulation. Through explicit citation relationship edges, documents of different modalities (such as PDF policy documents and regulatory index records in the database) can be connected through citation paths, forming a cross-document navigation network, significantly enhancing the reasoning and interpretation capabilities of the knowledge graph.

[0057] Preferably, the operation of determining entity names based on exogenous reference nodes and generating entity description text based on adjacent text fragments in the standard text block that correspond to the reference name string completes the entityification of external reference citation knowledge and the extraction of contextual information. The action of determining the entity name involves adding the entity name of the exogenous reference node as an entity name output from this identification process of the standard text block to the entity name list of the standard text block, thus enabling it to participate in subsequent processes of merging and normalizing nodes with the same name. This is achieved by appending the entity name of the exogenous reference node, along with its internal node identifier, to the entity extraction result set corresponding to the standard text block. The determination of adjacent text segments is anchored by the positions of the quoting verbs and the document name string recorded in the citation instance description structure: Extending leftward, take up to a certain number of characters (e.g., twenty characters) before the quoting verb or until the sentence boundary punctuation is encountered, as the left context segment; extending rightward, take up to a certain number of characters after the end of the document name string or until the sentence boundary, as the right context segment; then, the left context segment, the original quoting verb, the document name string, and the right context segment are concatenated into a continuous descriptive text. However, to highlight the document description rather than repeating the document name itself, the document name string can be replaced with a preset placeholder or simplified expression, or the entire quoted sentence can be retained as context. In government and enterprise scenarios, for example, if a standard text block contains "According to the requirements of Chapter 3 of the 'People's Bank of China Monetary Policy Implementation Report (Fourth Quarter of 2025)'", then adjacent text segments can be taken as the entire sentence or slightly trimmed to form a description, such as "Requirements are made in accordance with Chapter 3 of the 'People's Bank of China Monetary Policy Implementation Report (Fourth Quarter of 2025)'". The action of generating entity description text associates this adjacent text fragment with the exogenous reference node, appending it to the node's description field. Since a document may be cited in multiple text blocks, generating multiple description fragments, these fragments are concatenated by a separator during subsequent merging of nodes with the same name, forming a cross-document contextual description set for the document entity. In this way, the exogenous reference node not only connects to the source node through citation edges but also acquires rich contextual semantic descriptions, making it an entity node with independent semantic information in the knowledge graph. This supports the document citation network in the explicit anchoring mechanism, further enhancing the depth and query completeness of cross-modal knowledge association.

[0058] Optionally, the process of converting multimodal data into initial text blocks carrying modality identifiers further includes: Extract the generation time information of the multimodal data; Convert the generated time information into a standard timestamp in a unified format; The standard timestamp is stored in the metadata field corresponding to the initial text block; Upon receiving a business query instruction from an external device, multiple candidate text blocks matching the business query instruction are retrieved from the multimodal knowledge base. Extract the standard timestamp stored in each of the returned candidate text blocks; Multiple candidate text blocks with time differences within a preset range and carrying different modality identifiers are combined into a concurrent multimodal data cluster; The synchronous multimodal data cluster is pushed to the external presentation device to reconstruct a cross-modal event snapshot at a specified time.

[0059] Preferably, the process of extracting the generation time information of multimodal data occurs during the preprocessing stage of converting multimodal data into initial text blocks carrying modality identifiers. The object of this processing is standardized document format data that has not yet been segmented into text blocks. Each piece of raw data from different modalities, when parsed into a standardized document format, carries time metadata in its file system attributes or data content header that identifies its generation time; this time metadata constitutes the generation time information of the multimodal data. The specific implementation of the extraction action adopts a differentiated extraction strategy based on the different source modalities of the multimodal data: For document-type data, such as PDF or Word files converted into structured text by a document parsing engine, the generation time information is first read from the creation time or last modification time field of the document attribute dictionary. If the document attribute dictionary does not contain a valid time value, it falls back to the file creation timestamp of the file system. For audio data, the generation time information is extracted from the session start time of the transcribed text with timestamps and speaker identifiers output by the speech recognition engine. This start time is usually recorded during the speech activity detection stage and written into the metadata header of the transcribed text. For structured data, such as table data from a business system database, the generation time information is read from the system time column of the database table or the timestamp field of the export operation. If there is no explicit time field in the table, the database query execution time is used as the generation time information. The reason for actively extracting the generation time information of multimodal data during the initial text block generation stage is that in the application scenario of government and enterprise knowledge bases, the time dimension of knowledge content has the same important retrieval and association value as the semantic dimension. The release time of a policy document, the recording time of a meeting recording, and the update time of a database record together constitute a time sequence chain around a specific event or decision-making process. Default time information will cause the subsequent simultaneous multimodal aggregation of time windows to lose its time anchor, making the timestamp anchoring mechanism between multimodal knowledge fragments unable to operate.

[0060] Preferably, the operation of converting generated time information into a standardized timestamp in a unified format takes the extracted raw generated time information as input and transforms it into time data with a unified representation through a time format normalization pipeline. Due to differences in source modalities and storage systems, the raw generated time information may exhibit various format variations. For example, the time in document attributes may be stored in the operating system's local time format, the time in audio-to-text transcription may be represented as a relative second value since the speech recognition engine started, and database time fields may be stored in the Coordinated Universal Time (UTC) standard format. These heterogeneous time representations cannot be directly used for subsequent time difference calculations and time window aggregation. The conversion operation first performs format sniffing on the string representation of the raw generated time information. A pre-compiled list of time format regular expressions is used to sequentially match the raw time string to determine its corresponding time format template. This list of time format regular expressions contains more than ten variations of common date-time combination patterns, such as "year-month-day hour:minute:second", "year / month / day hour:minute", and "month / day / year hour:minute:second". Once a match is successful, the conversion logic captures six time components—year, month, day, hour, minute, and second—from the original time string according to a format template. Missing components, such as seconds, are filled with the default value of zero. Then, the captured six time components are fed into a timestamp constructor to generate a long integer value based on the Coordinated Universal Time (UTC) time zone and the number of seconds or milliseconds since the reference time. This long integer value is the standard timestamp. The technical essence of the standard timestamp is an absolute time scalar value that is independent of time zones and the original format. It eliminates the time representation differences between multimodal data sources, allowing any two time pieces from different modalities to directly obtain a precise time interval through integer subtraction. This provides a unified quantification basis for subsequent calculation of the time difference of candidate text blocks and the determination of time windows for contemporaneous multimodal data clusters. In government and enterprise scenarios, a meeting recording might contain the raw string "9:30 AM on March 15, 2026," while a policy document might contain the standardized format "2026-03-14T16:00:00Z." Through the conversion via a time format normalization pipeline, both are mapped to two standard timestamp values ​​under the same time base, allowing for a quick determination that they differ by approximately 17.5 hours and are within the same time window.

[0061] Preferably, the operation of storing the standard timestamp into the metadata field of the corresponding initial text block is performed immediately after the standard timestamp is generated. Its processing object is the initial text block data structure carrying the original text content and modal identifier. The initial text block is a composite data object in memory, containing a text content field, a modal identifier field, and an expandable set of metadata fields. Logically, the metadata field set is a key-value pair container; each metadata entry consists of a string key and a polymorphic value, capable of holding any type of additional descriptive information. The storage operation first checks the metadata field set of the initial text block to see if a metadata entry with the key name "standard timestamp" already exists. If it does not exist, a new metadata entry is created, its key is set to the preset standard timestamp key name (e.g., "standard timestamp"), and its value is set to the long integer value of the standard timestamp generated in the previous step. If it already exists, the value of the existing standard timestamp metadata entry is updated to the currently generated standard timestamp value to overwrite any outdated values ​​that may have been generated due to time information correction. The metadata fields of the corresponding initial text block are determined through the mapping between the unique hash identifier generated after the initial text block is segmented and the standard timestamp: when the initial text block is created and assigned a unique hash identifier, this unique hash identifier is registered in a global text block index table, and metadata storage space is reserved for this text block. When the standard timestamp needs to be written, the corresponding metadata field set of the initial text block is located in the text block index table through this unique hash identifier, and the write is performed. Storing the standard timestamp in the metadata fields of the initial text block, rather than passing it only as a temporary variable, aims to make time information a persistent attribute of the initial text block. It is carried throughout the subsequent stages of the pipeline, such as segmentation, entity recognition, relation extraction, and graph merging, without being lost. This ensures that no matter which stage in the knowledge graph construction pipeline needs to obtain the time context of the text block, the standard timestamp can be read directly from the metadata field without having to backtrack to the original data source to re-extract the time information, thus ensuring the reliable execution of the timestamp anchoring mechanism during the retrieval stage.

[0062] Preferably, upon receiving a business query instruction from an external device, the operation of retrieving multiple candidate text blocks matching the business query instruction from the multimodal knowledge base is the initiation step of the timestamp anchoring mechanism during the retrieval phase. The business query instruction from the external device is a retrieval request consisting of a natural language question or structured query conditions. It is decomposed into two parts by a query parsing frontend: one part is semantic matching conditions, used to retrieve text blocks semantically related to the query intent in the multimodal knowledge base; the other part is time anchor information, used to limit the time range of the retrieval results. The technical essence of returning candidate text blocks is a batch of standard text blocks initially matched by semantic retrieval in the multimodal knowledge base after first-level entity normalization and second-level merging of identically named nodes. These blocks, as a candidate set, have not yet undergone time-dimensional filtering and modal-dimensional combination optimization. The retrieval operation first performs a hybrid search in the inverted index and vector index of the text block in the multimodal knowledge base using semantic matching conditions to obtain a batch of initially matched text blocks. Then, for each initially matched text block, the standard timestamp is read from its metadata field. If the absolute difference between the standard timestamp of the initially matched text block and the time anchor specified in the business query instruction is within a preset range, the initially matched text block is marked as a candidate text block for return; otherwise, it is excluded. In the retrieval scenario of the government and enterprise knowledge base, a user may issue a query such as "discussion and decision-making status regarding budget adjustments in mid-March 2026". The query parsing front end extracts "budget adjustment" as a semantic matching condition and parses "mid-March 2026" as a time anchor. Among all the semantically matched text blocks, the retrieval operation only retains those text blocks whose standard timestamps fall between March 11 and March 20, 2026, as candidate text blocks for return. This further superimposes the time dimension constraint on top of semantic relevance, filtering out knowledge fragments closely related to the query time range.

[0063] Preferably, the operation of extracting the standard timestamp stored in each candidate text block is performed on the list of candidate text blocks retrieved in the previous operation. Each candidate text block in this list is a standard text block instance carrying metadata fields. The extraction action iterates through the list of candidate text blocks, and for each candidate text block, reads the value of the metadata entry with the standard timestamp as the key from its metadata field. This value is the long integer value of the standard timestamp stored in the construction phase of that candidate text block. This extracted standard timestamp and the standard timestamp stored in the metadata field of the corresponding initial text block in the previous steps are the same data in different processing stages: the former is the time attribute written and persisted in the multimodal knowledge base during the construction phase, and the latter is the same time attribute read from the multimodal knowledge base during the retrieval phase. The two are completely identical in value and are the same metadata entry in both the storage and retrieval directions on the timeline. The extracted standard timestamps of all candidate text blocks are collected into a timestamp list, and each timestamp in the list uniquely corresponds to a candidate text block. In the multimodal scenario of government and enterprise knowledge bases, these candidate text blocks may come from three different modalities: policy document PDFs, transcripts of meeting recordings, and database table descriptions. Their standard timestamps may differ numerically, but they all fall within the time window specified by the business query command. At this point, the timestamp list serves as the direct input for subsequent time difference calculations and contemporaneity determination. Internally, it stores absolute time scalar values ​​that have been standardized in a unified format, supporting rapid numerical comparison and difference calculations without requiring further format parsing and time zone conversion.

[0064] Preferably, the core aggregation step of the timestamp anchoring mechanism is the operation of combining multiple candidate text blocks with time differences within a preset range and carrying different modal identifiers into a concurrent multimodal data cluster. This operation takes the candidate text blocks and their corresponding standard timestamps as input. First, it pairs all standard timestamps in the timestamp list and calculates the absolute time difference between each pair. This absolute time difference is the absolute value of the difference between the two standard timestamps, measured in seconds or minutes. The preset range is a pre-configured time window threshold, set based on the maximum reasonable time interval between multimodal records of the same event in the government and enterprise knowledge base. For example, the time between the recording time of a meeting and the issuance time of the policy discussed at the meeting usually does not exceed several calendar days; therefore, the preset range can be configured to several hours or several days (e.g., forty-eight hours). For each pair of candidate text blocks whose time difference is within a preset range, the modal identifiers stored in the metadata fields of the two candidate text blocks are further examined to see if they are different. The modal identifiers are written during the initial text block generation stage according to the source type of the multimodal data. For example, document data is marked with a document modal identifier, audio transcription data is marked with an audio modal identifier, and database description data is marked with a database modal identifier. If two candidate text blocks carry different modal identifiers, they are grouped into the same contemporaneous multimodal data cluster. The technical essence of the contemporaneous multimodal data cluster is a cross-modal text block group that is associated with both time and entity dimensions. The meaning of "contemporary" does not require that the standard timestamps of all members be exactly equal, but rather that their time differences are within a preset range, thus forming a closely adjacent event window on the timeline, which can reflect the performance of the same event or decision process in different modal data sources. The combination action is achieved through a connected component discovery algorithm in graph theory: using candidate text blocks as nodes and edge construction conditions such as time differences within a preset range and different modal identifiers, an undirected temporal graph is constructed. Then, the connected component discovery algorithm is run on this undirected temporal graph, and all nodes in each connected component constitute a contemporaneous multimodal data cluster. The reason for requiring different modal identifiers for combination is that the technical goal of timestamp anchoring is to piece together the whole picture of the same event from different modal perspectives. If two text blocks have the same modality (e.g., two PDFs), their temporal proximity is more likely to belong to a regular document publication sequence rather than a multimodal complementary relationship. Forcibly aggregating them would introduce redundant noise and reduce the information density and readability of cross-modal event snapshots.

[0065] Preferably, the operation of pushing the concurrent multimodal data cluster to the external presentation end to reconstruct the cross-modal event snapshot at a specified time is the final presentation stage of the timestamp anchoring mechanism in delivering results to the user. The concurrent multimodal data cluster serves as the input object, containing a set of standard text block references from different modalities that are temporally adjacent and involve the same entities. The specified time is determined from the time anchor information carried in the business query command. This time anchor is extracted and converted into a specified time value in a standard timestamp format at the query parsing front end, representing the time point of the event the user wants to know. The reconstruction action first starts from each standard text block reference in the concurrent multimodal data cluster, reading the complete text content, entity name list, and relation triplet list corresponding to the standard text block reference from the multimodal knowledge base, and grouping this information according to modality identifiers. Then, centered on the specified time, all grouped information is sorted in ascending order of the difference between the standard timestamp and the specified time, forming a cross-view data structure with the time axis as the vertical axis and the modality type as the horizontal axis. This cross-view data structure is the cross-modal event snapshot. The technical essence of cross-modal event snapshots is an aggregated presentation structure of knowledge related to the same entity from different modal data sources near a specific point in time. Unlike traditional search results that present documents from various modalities in a mixed list format, cross-modal snapshots align and arrange the content of each modality using a timeline index. This allows users to see at a glance how the entity was described in policy documents, discussed in meeting recordings, and updated or statistically analyzed in database records near a specified time. It is called "cross-modal" because this snapshot spans at least two different data modalities, including documents, audio transcriptions, and structured data descriptions, reassembling fragmented knowledge from modal media into a continuous panoramic view of information based on time and entity dimensions. In the retrieval scenario of the government and enterprise knowledge base, when a user queries "discussions and decisions regarding budget adjustments in mid-March 2026," the external presentation terminal will ultimately display a cross-modal event snapshot centered on March 15, 2026. This snapshot may simultaneously include a summary of policy documents released on March 14, dialogue snippets from meeting recordings on March 15 mentioning budget adjustments, and change records for relevant entries in the database budget table on March 15. Users can obtain a complete multimodal understanding of the same event without having to search document libraries, audio libraries, and databases separately. The push notification renders this cross-modal event snapshot through the external presentation terminal's application programming interface (API) or web interface, completing the end-to-end processing from multimodal data to a unified knowledge view.

[0066] Optionally, the step of converting multimodal data into initial text blocks carrying modality identifiers includes: When the multimodal data is audio data, the speech activity detector is used to locate the speech pause points, the audio data is segmented into independent speech segments based on the speech pause points, and speech recognition is performed on the independent speech segments to obtain spoken text. Audio modality identifiers are added to the spoken text to obtain the initial text block. When the multimodal data is structured data, the annotation information of the header column attributes and the foreign key dependency information representing the hierarchical relationship between multiple tables are extracted. The annotation information and the foreign key dependency information are concatenated into a natural language description paragraph through a semantic generation network. The structured data modality identifier is added to the natural language description paragraph to obtain the initial text block.

[0067] Preferably, when the multimodal data is audio data, the process involves using a speech activity detector to locate speech pauses and segmenting the audio data into independent speech segments based on these pauses. The processing object is the raw audio data stream acquired from an external device or read from a storage medium. This audio data stream is physically a sequence of pulse-code modulation sample values ​​arranged in chronological order. The speech activity detector is a signal processing component that performs dual threshold judgments on the time and frequency domain characteristics of the audio signal. Internally, it maintains a short-time energy calculation channel and a short-time zero-crossing rate calculation channel. These two channels continuously move across the audio data stream using sliding windows with preset frame lengths (e.g., 20 milliseconds) and preset frame shifts (e.g., 10 milliseconds), respectively. The short-time energy calculation channel calculates the sum of squares of all sample values ​​within each frame to obtain the short-time energy value of that frame and continuously updates a dynamic noise energy threshold. The short-time zero-crossing rate calculation channel counts the number of sign changes between adjacent sample values ​​within each frame to obtain the zero-crossing rate value of that frame. The determination of speech pause points is accomplished by threshold decision logic: when the short-time energy values ​​of several consecutive frames are lower than the dynamic noise energy threshold and the zero-crossing rate value is lower than the preset zero-crossing rate threshold, the decision logic determines that the current time interval is a speech pause interval, and the center or starting position of this speech pause interval is marked as a speech pause point. The time interval between speech pause points is the speech activity interval, and its boundary is defined by two adjacent speech pause points. The segmentation based on speech pause points is achieved by establishing a segment index on the sampled value sequence of the audio data stream: starting from the first speech pause point, the sampled value interval between every two adjacent speech pause points is extracted into an independent audio segment. This audio segment does not contain long silences or background noise segments, but only retains the acoustic signal of a single speech turn or a single semantically coherent segment. Each audio segment obtained through this segmentation is an independent speech segment. The technical essence of an independent speech segment is that it is the smallest acoustic unit carrying a complete spoken expression, formed by cutting the original continuous audio data according to semantic pause boundaries. This differs from audio blocks that are rigidly cut according to fixed durations. It maintains the natural boundaries and semantic integrity of spoken expression, thus providing higher-quality acoustic input for subsequent speech recognition and avoiding recognition errors and semantic breaks caused by the cut points being in the middle of words. In the application scenario of government and enterprise knowledge bases, a continuous recording of a departmental work meeting may contain multiple speakers alternating, intermittent pauses, and background ambient noise. By locating pause points and segmenting independent speech segments using a speech activity detector, each speaker's continuous statements, questions, or responses can be encapsulated into independent acoustic input units, which can then be fed into the speech recognition engine segment by segment to obtain the corresponding transcribed text paragraphs.

[0068] Preferably, the operation of performing speech recognition on independent speech segments to obtain spoken-to-text transcripts and adding audio modality identifiers to the spoken-to-text transcripts to obtain initial text blocks is performed immediately after the independent speech segments are segmented. The process of performing speech recognition on independent speech segments involves feeding the acoustic feature sequences of the independent speech segments into a pre-trained end-to-end speech recognition model. This end-to-end speech recognition model consists of two parts: an acoustic encoder and a text decoder. The acoustic encoder receives the Mel-frequency cepstral coefficient feature sequence of the independent speech segments and compresses the acoustic features into a low-frame-rate acoustic hidden state sequence through a multi-layered stacked two-dimensional convolutional network and a temporally separable convolutional network. The text decoder uses the acoustic hidden state sequence as a condition and generates the corresponding text character sequence word by word through a multi-head attention mechanism and autoregressive decoding. The generated text character sequences are concatenated into a natural language text, which is the spoken-to-text transcript. The spoken-to-text transcript fully records the speaker's spoken expression in the independent speech segments but retains features such as omissions, repetitions, and interjections. After speech recognition is completed, the system performs a modal identifier injection operation on the spoken-to-text text: It retrieves the audio modal identifier value representing the audio transcription data from a pre-defined set of modal identifier enumeration values ​​and structurally binds this audio modal identifier value to the spoken-to-text text. The technical essence of the audio modal identifier is a classification label. It acts as an immutable marker indicating the data form source of the text block in all subsequent processing stages of the knowledge graph construction pipeline. Any processing stage that receives the text block can read the audio modal identifier to determine that the underlying data of the text block comes from audio transcription, thereby enabling differentiated processing strategies based on the text style characteristics of the audio modality in subsequent vector similarity correction, timestamp anchoring, and time window aggregation. The spoken-to-text text bound with the audio modal identifier is encapsulated into an initial text block data structure. This initial text block data structure includes a text content field, a modal identifier field, and a reserved metadata field. The text content field stores the spoken-to-text text, and the modal identifier field stores the audio modal identifier. At this point, the transformation from raw audio data to an initial text block carrying modal identifiers is complete. This initial text block will then enter subsequent pipeline stages such as text segmentation, alias replacement, and entity extraction.

[0069] Preferably, when the multimodal data is structured data, the processing of extracting annotation information from the header column attributes and foreign key dependency information representing hierarchical relationships between multiple tables uses structured table data from the business system database as the processing object. Structured table data is physically organized in the form of two-dimensional tables, each consisting of several columns and rows. The name of each column in the header row is the header column attribute. Database management systems typically add annotation information to the header column attributes when defining the table structure. This annotation information is stored in the form of comment statements in Structured Query Language or memo fields in a data dictionary table, providing natural language explanations of the meaning, value unit, and business semantics of the data within the column. The specific implementation of extracting the annotation information of the header column attributes is as follows: A metadata query operation is performed through the database connection interface, calling the introspection interface provided by the database management system to obtain a list of column definition tuples for the target table. Each tuple in this list contains four fields: column name, column data type, whether the column can be nullable, and column comment text. The list of column definition tuples is traversed, and each column name and corresponding column comment text are extracted to form a list of header column attribute annotations. Foreign key dependency information, representing hierarchical relationships between multiple tables, refers to the referential integrity relationship between tables defined by foreign key constraints in database design. Technically, it means that one or more columns in one table reference a primary key column in another table, thus establishing a parent-child or referential-referenced relationship at the data level. The specific implementation for determining foreign key dependency information involves retrieving the target table's foreign key constraint definition list through the database introspection interface. Each definition in this list contains four fields: foreign key name, a list of foreign key column names in the current table, the name of the referenced table, and a list of referenced column names in the referenced table. This list is then traversed, and each definition is parsed into a foreign key dependency tuple. This tuple records which column (or columns) in the current table references which primary key column of which parent table, and based on this, the hierarchical relationship semantics between the two tables are inferred. For example, a project number column in a project management table references a project number column in the project master table, forming a hierarchical relationship where the project master table contains records from multiple project management tables. The reason for extracting the annotation information and foreign key dependency information of the column header attributes is that in the application scenario of government and enterprise knowledge base, the structured data stored in the database itself lacks the readable context of natural language. The simple column names and values ​​cannot be effectively understood by the downstream large language model. The annotation information just supplements the semantic interpretation of the column level, and the foreign key dependency information reveals the business entity relationship between multiple tables. Together, they constitute the semantic raw materials necessary to translate structured data into natural language description.

[0070] Preferably, the core technical step of structured data modality conversion is the operation of concatenating annotation information and foreign key dependency information into a natural language description paragraph through a semantic generation network and adding structured data modality identifiers to this natural language description paragraph to obtain an initial text block. The semantic generation network is a multi-layered neural text generation component designed to reorganize structured meta-information into coherent natural language. Internally, it consists of three sequentially connected modules: a template generation layer, a semantic filling layer, and a coherence verification layer. The template generation layer takes column names from the header column attribute annotation list as input. It matches a basic sentence template describing the semantics of each column name using a pre-loaded sentence template library. This sentence template library is a set of patterns automatically mined and clustered from government and enterprise database document corpora, storing fillable sentence skeletons such as "a certain column name field in a certain table name represents a certain placeholder." The template generation layer outputs a sequence of basic sentence skeletons filled with column names and table names column by column. The semantic fill layer receives the basic sentence skeleton sequence output by the template generation layer as the syntactic skeleton, and receives the column annotation text from the column attribute annotation list in the table header as semantic fill material. According to the correspondence of column names, each column annotation text is filled into the placeholder position of the corresponding basic sentence skeleton to form a column description statement with specific semantics. At the same time, the semantic fill layer also receives a list of foreign key dependency tuples, selects a set of preset foreign key relationship description templates for each foreign key dependency tuple, such as the expression template "a certain current table name is associated with a certain primary key column name of a certain parent table name through a certain foreign key column name, indicating that the current table is a child table of the parent table", and fills in the table name and column name in the foreign key dependency tuple to generate a foreign key relationship description statement. The coherence verification layer receives all column description statements and foreign key relationship description statements output by the semantic filling layer. It concatenates these statements into a sentence sequence, prioritizing column descriptions over foreign key relationships. Then, a pre-trained text coherence scoring model is invoked to score the semantic coherence between adjacent statements in this sequence. For adjacent statement pairs with coherence scores below a preset coherence threshold, the coherence verification layer selects a transitional connector or supplementary explanatory phrase from a lightweight connector word generation lexicon that semantically matches the adjacent statement pair and inserts it between the two statements. Finally, all statements, along with the inserted transitional connectors, are merged into a coherent plain text segment, which is the natural language description paragraph. The technical essence of the natural language description paragraph is to reorganize the table structure metadata and inter-table relation metadata, originally stored in a relational and discretized form, into a text fragment that unfolds linearly according to human natural language habits and contains complete business semantics. This allows database knowledge, which was originally only accessible through structured query statements, to be understood and indexed in a unified way by downstream large language models and text retrieval systems.After obtaining the natural language description paragraph, the system performs a modality injection operation on it: It retrieves a structured data modality identifier value representing the structured data description from a pre-defined set of modality identifier enumeration values ​​and structurally binds this value to the natural language description paragraph. The technical essence of the structured data modality identifier is the same as that of the aforementioned audio modality identifier; it is also a classification label that marks the underlying data form of the text block as a structured data description. Therefore, in subsequent vector similarity correction, when it is necessary to determine the modality source difference between two candidate entities, this structured data modality identifier serves as one of the bases for determining modality differences and participates in the decision-making process for retrieving cross-media style difference compensation values. The natural language description paragraph, after being bound with the structured data modality identifier, is encapsulated into an initial text block data structure. The text content field of this initial text block data structure stores the natural language description paragraph, and the modality identifier field stores the structured data modality identifier. At this point, the transformation from structured table data to initial text blocks carrying modal identifiers is complete. These initial text blocks will then enter the subsequent chunking and entity extraction pipeline, enabling structured knowledge such as project numbers, budget amounts, and foreign key relationships in the database to be normalized and associated with entity descriptions from documents and audio in the multimodal knowledge base.

[0071] Optionally, calculating the initial similarity between the first feature sequence and the second feature sequence includes: The first feature sequence is mapped to a preset vector space to obtain a first semantic vector representing its semantics; The second feature sequence is mapped to the vector space to obtain a second semantic vector representing its semantics. Calculate the cosine of the angle between the first semantic vector and the second semantic vector in the vector space; A quantitative score for indicating the degree of semantic content overlap is determined based on the cosine value of the included angle; The quantified score is assigned as the initial similarity.

[0072] Preferably, the operation of mapping the first feature sequence to a preset vector space to obtain a first semantic vector representing its semantics takes the first feature sequence corresponding to the new node as the processing object. The technical essence of the first feature sequence is an ordered list of real numbers, which is output by the Embedding model after forward inference calculation of the entity description text of the new node. The Embedding model is a pre-trained deep neural network containing multiple layers of transformer encoding layers. Each layer converts the character sequence of the input text into a context-dependent hidden state vector. The last layer aggregates the hidden states of the entire text into a fixed-dimensional numerical sequence, which is the first feature sequence. Its dimension is usually several hundred dimensions (e.g., 768 or 1024 dimensions). Each value in the sequence represents the activation intensity of the entity description text in a certain semantic dimension. The preset vector space is a high-dimensional real-number vector space loaded into memory during the initialization phase of the knowledge graph construction pipeline. The preset method is as follows: a dimension value is specified in the pipeline configuration parameters, which strictly matches the dimension of the feature sequence output by the Embedding model. Then, a real-number vector space data structure with this dimension value as a parameter is created in memory using a mathematical library. This real-number vector space data structure provides basic linear algebra operation interfaces such as vector storage, dot product operation, modulus calculation, and cosine value calculation. The specific implementation process of the mapping is to fill each real-number element contained in the first feature sequence into the corresponding dimension position of a new vector instance in the vector space according to its index order in the sequence, such that the zeroth dimension element of the new vector instance is equal to the first real-number element of the first feature sequence, the first dimension element is equal to the second real-number element of the first feature sequence, and so on until all dimensions are filled. The resulting filled vector instance is the first semantic vector. The technical essence of the first semantic vector is a coordinate representation of the entity description text of the newly generated node in a predefined vector space. It compresses the lexical semantics, contextual semantics, and relational semantics of the entity description text into a set of geometrically comparable numerical coordinates. The geometric distance and directional difference between two semantic vectors in the vector space can approximately reflect the degree of semantic difference between the corresponding two entity description texts, thus providing a unified algebraic basis for subsequent initial similarity calculations.

[0073] Preferably, the operation of mapping the second feature sequence to a vector space to obtain a second semantic vector representing its semantics is technically symmetrical to the operation of mapping the first feature sequence to a vector space. Its processing object is the second feature sequence corresponding to the candidate node. The second feature sequence is also output by the embedding model after forward inference calculation of the entity description text of the candidate node. Its numerical dimension is completely consistent with the first feature sequence, and it also internally stores a set of real numbers that encode the semantic information of the entity description text of the candidate node. The mapping operation fills each real element of the second feature sequence into a new vector instance in the same preset vector space as the one used to generate the first semantic vector, according to the same index correspondence rules as described above. This results in a second semantic vector that is equal in dimension to the first semantic vector but stored independently in coordinate position. The technical essence of the second semantic vector is the coordinate representation of the entity description text of the candidate node in the same preset vector space. It shares the same vector space basis as the first semantic vector, therefore the vector operations between the two are geometrically comparable. The reason for mapping the first and second semantic vectors to the same preset vector space is that only under the same basis in the same vector space can the cosine of the angle between the two semantic vectors accurately reflect the semantic direction similarity between the two entity description texts, without being affected by the measurement distortion caused by the transformation of different vector space basis. In the application scenario of government and enterprise knowledge base, the entity description text of a new node may come from the standard written expression of a policy document, while the entity description text of a candidate node may incorporate colloquial supplementary information transcribed from meeting recordings and field annotations from database tables. Although the two texts have different expression styles and word usage habits, after being encoded into feature sequences by the same embedding model and mapped to the same vector space, their geometric positions in the vector space have directly comparable algebraic properties, providing a reliable underlying similarity measurement benchmark for subsequent cross-modal adaptive similarity correction.

[0074] Preferably, the operation of calculating the cosine of the angle between the first semantic vector and the second semantic vector in the vector space involves using the first and second semantic vectors as inputs and performing a calculation of the cosine of the angle in a preset vector space. The calculation process consists of three sub-operations: First, the dot product of the first and second semantic vectors is calculated. The dot product operation involves multiplying the element values ​​of the first and second semantic vectors pairwise in each same dimension, and then summing the results of the multiplications in all dimensions to obtain a scalar dot product value. This scalar dot product value reflects the combined effect of the consistency of the two semantic vectors in direction and the magnitude of their magnitudes. Next, the magnitudes of the first and second semantic vectors are calculated separately. The magnitude operation involves squaring the element values ​​of each dimension of the semantic vectors, summing them, and then taking the square root to obtain the first and second magnitude values, respectively. The magnitude values ​​reflect the absolute semantic information content intensity of the semantic vectors. Finally, the scalar dot product value is divided by the product of the first and second magnitude values ​​to obtain the cosine of the angle. The cosine of the angle is a dimensionless real number between negative one and positive one. Physically, it represents the cosine of the angle between the directions of two semantic vectors in vector space. When the two semantic vectors have identical directions, the cosine is positive one; when their directions are completely opposite, the cosine is negative one; and when their directions are orthogonal and unrelated, the cosine is zero. The advantage of this calculation method is that it eliminates the influence of modal length caused by differences in the length and detail of the two entity descriptions, retaining only the pure similarity of the two semantic vectors in their semantic direction. This perfectly suits the reality that in cross-modal scenarios, the description of the same entity often has different information content in different modalities but similar semantic directions.

[0075] Preferably, the operation of determining the quantization score used to indicate the degree of semantic content overlap based on the cosine value of the included angle involves taking the cosine value calculated in the previous step as input and transforming the range of the cosine value from negative one to positive one through a preset mapping function to a value range that conforms to the intuitive expression habits of similarity. The technical essence of the quantization score is a numerical indicator conventionally defined as ranging from zero and one or from zero to one hundred. It is designed to monotonically increase with the deepening of semantic content overlap, facilitating direct threshold comparison by the downstream threshold decision logic. One direct mapping method for determining the quantization score is to assign the quantization score to the cosine value of the included angle itself (in this case, the value range of the quantization score is consistent with the cosine value). This method retains all the numerical resolution accuracy of the cosine value, but requires the downstream threshold decision logic to be compatible with negative values. In government and enterprise knowledge base applications, a more common mapping method is to map the cosine of the angle to a range between zero and one using a linear scaling and translation transformation: The cosine of the angle is incremented by one and divided by two to obtain a quantized score between zero and one. When the cosine is positive one, the quantized score is one, indicating that the two entity descriptions are semantically identical; when the cosine is zero, the quantized score is 0.5, indicating that the semantic directions are orthogonal and unrelated; when the cosine is negative one, the quantized score is zero, indicating that the semantic directions are completely opposite. Regardless of whether direct assignment mapping or scaling and translation mapping is used, a strictly monotonically increasing functional relationship is maintained between the quantized score and the cosine of the angle, ensuring that the qualitative correspondence that a higher degree of semantic content overlap corresponds to a larger quantized score is not disrupted.

[0076] Preferably, assigning the quantized score as the initial similarity is the final step in the initial similarity calculation process. It passes the quantized score determined in the previous operation to a variable or data structure field named "initial similarity" in the form of a numerical assignment. The initial similarity is the input parameter for the vector similarity deduplication and equivalent association edge construction process in the third-level normalization. It is recorded in the candidate entity pair description structure between the new node and the candidate node, serving as one of the inputs for the subsequent cross-modal similarity correction mechanism. The assignment operation itself is a simple numerical transfer, without any numerical transformation or rounding. All valid bits of the quantized score are completely preserved in the initial similarity, thus ensuring that subsequent modal identification difference determination and positive adjustment parameter application operations can be performed based on the original precision equivalent to the cosine value of the included angle, avoiding the introduction of additional quantization errors due to intermediate truncation. Thus, the complete technical chain is completed, starting from the entity description text of the newly generated node, encoding it into the first feature sequence through the Embedding model, mapping it into the first semantic vector, calculating the cosine of the angle between the first feature vector and the second semantic vector of the candidate node, and finally assigning the initial similarity. In the multimodal entity normalization scenario of government and enterprise knowledge base, this technical chain, combined with the first-level domain dictionary standardization and the second-level homonymous nodes, forms the final numerical calculation basis of the three-level progressive normalization system: for entity pairs not covered by the domain dictionary and with mismatched homonyms, the initial similarity, as a purely semantic similarity measure, serves as the sole basis for matching. When the initial similarity is too low due to differences in cross-modal expression styles, the subsequent positive adjustment parameters will compensate and correct the initial similarity modally adaptively, so that the target similarity can more accurately reflect the true semantic relationship of cross-modal equivalent entities.

[0077] Preferably, the training process of the above-mentioned embedding model is based on a constructed large-scale entity description text similarity dataset. The weight parameters within the model are iteratively updated through an optimized contrastive loss function until the model's semantic discrimination ability on the validation set reaches the convergence criterion. The embedding model's structure employs a stacked transformer encoder as the backbone network, specifically configured as a twelve-layer transformer encoder with identical but independent parameters. Each transformer encoder layer contains a multi-head self-attention sublayer and a position-wise fully connected feedforward sublayer, both employing residual connections and hierarchical normalization. The input text first passes through a character embedding layer, converting each discrete character into a dense vector of fixed dimensions (e.g., 768 dimensions), and then superimposing positional encoding vectors to inject sequence order information, forming the input embedding sequence. The input embedding sequence passes through twelve transformer encoding layers sequentially. In each layer, a multi-head self-attention sublayer calculates the semantic dependency weights between any two positions in the input sequence using twelve parallel attention heads. The outputs of the twelve attention heads are then concatenated and linearly projected to restore the dimension. A position-wise fully connected feedforward sublayer independently performs two linear mappings and one non-linear activation on the vector at each position to enhance the model's non-linear expressive power. Finally, in the output sequence of the twelfth transformer encoding layer, the hidden state vector corresponding to the special classification label position at the beginning of the sequence is taken and fed into a projection head consisting of two fully connected layers. The projection head maps this hidden state vector into a feature sequence of a preset dimension, which becomes either the first or second feature sequence output by the embedding model.

[0078] The loss function used to train the embedding model is a contrastive loss function based on in-batch negative samples. In each training batch, several sets of entity description texts are sampled from the training dataset. Each set contains one anchor entity description text, one positive entity description text pointing to the same real-world entity as the anchor entity description text, and multiple negative entity description texts pointing to different real-world entities than the anchor entity description text. The anchor entity description text, positive entity description text, and all negative entity description texts are fed into the embedding model to obtain the corresponding anchor feature sequence, positive feature sequence, and each negative feature sequence. Subsequently, following the same process as mapping the feature sequences to a preset vector space and converting them into semantic vectors, the sets of anchor semantic vectors, positive semantic vectors, and negative semantic vectors are obtained. The loss function is calculated as follows: first, the cosine of the angle between the anchor semantic vector and the positive semantic vector is calculated, which is used as the positive similarity; then, the cosine of the angle between the anchor semantic vector and each negative semantic vector is calculated, which is used as the negative similarity. Then, using the positive example similarity as the numerator and the sum of the exponential functions of the positive example similarity and all negative example similarities as the denominator, a ratio is constructed. Taking the negative natural logarithm of this ratio yields the contrastive loss value for the anchor entity description text. The contrastive loss values ​​of all anchor entity description texts within the batch are averaged to obtain the batch average contrastive loss value. Minimizing this batch average contrastive loss value is the optimization objective. An adaptive moment estimation optimization algorithm is used to update all weight parameters within the Embedding model through gradient backpropagation. This process is repeated for multiple training epochs until the difference between the mean positive example similarity and the mean negative example similarity calculated on the reserved entity description text validation set tends to stabilize, indicating that the model training is complete. The trained Embedding model can bring entity description texts pointing to the same entity from different modalities closer together in the vector space, while keeping entity description texts pointing to different entities further apart, providing a feature sequence with strong cross-modal semantic discriminative power for subsequent vector similarity deduplication.

[0079] Optionally, when the modality identifier of the newly generated node differs from the modality identifier of the candidate node, applying a positive adjustment parameter to the initial similarity to obtain the target similarity includes: Identify the first underlying medium type indicated by the modal identifier of the newly generated node; Identify the second underlying media type indicated by the modal identifier of the candidate node; When the first underlying media type and the second underlying media type correspond to written text media and spoken text transcription media respectively, the preset cross-media style difference compensation value is retrieved from the configuration parameter table; The cross-media style difference compensation value is determined as the positive adjustment parameter; The target similarity is obtained by adding the positive adjustment parameter to the initial similarity.

[0080] Preferably, the operation of identifying the first underlying media type indicated by the modal identifier of a new node takes the modal identifier carried by the new node as the processing object. The modal identifier of a new node is a classification label injected by the knowledge graph construction pipeline when converting multimodal data into initial text blocks. Its technical essence is an enumerated data field. The value space of this enumerated data field is predefined in the pipeline initialization phase as a set containing at least three enumerated values. These three enumerated values ​​correspond one-to-one with document text, spoken text, and structured data description text, respectively. During the creation of a new node, the modal identifier has been read from the metadata field of the initial text block and written into the node attribute set of the new node. Therefore, the identification operation only needs to perform a key-value query on the node attribute set of the new node, using the key name of the modal identifier as the query key, to retrieve the corresponding enumerated value from the node attribute set. This enumerated value is the first underlying media type. The technical essence of the first underlying media type is the classification identifier of the original physical carrier and expression generation method of the descriptive information carried by the entity node. It is not a content summary of the descriptive information itself, but a retrospective mark of the underlying media environment on which the descriptive information depends when it is generated. The reason for identifying the first underlying media type is that in the multimodal fusion scenario of government and enterprise knowledge bases, the same entity name appears with different expression styles in written documents, spoken recordings, and database structures. The entity description text under the written document medium has the characteristics of complete sentence structure, standardized wording, and rigorous logic, while the entity description text under the spoken recording medium usually contains elliptical sentences, interjections, colloquial abbreviations, and even speech recognition errors. The entity description texts produced by these two underlying media types naturally have a systematic distance offset in the semantic vector space. If the underlying media type is not explicitly identified and distinguished, the subsequent similarity judgment will incorrectly attribute this offset caused by media differences to semantic dissimilarity, thus causing the omission of cross-modal equivalent entity associations.

[0081] Preferably, the operation of identifying the second underlying media type indicated by the modal identifier of a candidate node is technically symmetrical to the operation of identifying the first underlying media type. Its processing object is the modal identifier carried by the candidate node. The modal identifier of a candidate node is also a classification label injected into the knowledge graph construction pipeline when converting multimodal data into initial text blocks. Its technical essence is the same as that of the modal identifier of a newly generated node; it is also an enumerated data field, and its value space also includes three enumerated values: document text, spoken-to-written text, and structured data description text. When retrieving a candidate node from the multimodal knowledge base, the complete node attribute set of the candidate node is loaded into memory, which includes the modal identifier field. The identification operation performs the same key-value query on the node attribute set of the candidate node to retrieve the enumerated value of the modal identifier. This enumerated value is the second underlying media type. The technical essence of the second underlying media type is the classification of the underlying media environment in which the entity description text of the candidate node was initially generated. It is independent of the entity name and entity description text content of the candidate node, and only records the modal data from which the entity description text aggregated by the node was initially converted. Since candidate nodes may have undergone second-level merging of nodes with the same name, the aggregated entity description text within them may originate from multiple initial text blocks of different modalities. Therefore, the modality identifier field of a candidate node stores the enumerated value of the modality identifier that appears most frequently among all merged initial text blocks. This allows the second underlying media type to reflect the dominant narrative style of the candidate node, ensuring that the candidate node has a uniquely determined modality attribution at the modality identifier level. By identifying the first and second underlying media types respectively, the data source modal features of both new nodes and candidate nodes are converted into two comparable classification label values, providing clear input parameter pairs for subsequent modality difference determination and compensation parameter retrieval.

[0082] Preferably, when the first underlying media type and the second underlying media type correspond to written text media and spoken text transcription media respectively, the operation of retrieving the preset cross-media style difference compensation value from the configuration parameter table is the condition triggering and parameter acquisition link of the entire cross-modal similarity correction mechanism. The technical essence of written text media refers to entity description text originating from manually written, formatted, and proofread text documents, such as policy documents (PDF), regulatory documents (Word), or formal reports. This type of text is characterized by complete sentence structure, formal wording, and standardized punctuation. Its semantic vector in the high-dimensional vector space typically exhibits a distribution with high information density and relatively concentrated direction. The technical essence of spoken text transcription media refers to entity description text originating from transcribed text obtained after speech recognition of audio recordings. This type of text is characterized by colloquial, fragmented, and highly repetitive expression. Its semantic vector in the high-dimensional vector space exhibits a distribution with diffuse direction and a systematically large angle with the written text vector due to the presence of a large number of filler words, repetitive segments, and recognition noise. When the first underlying media type is written text and the second underlying media type is spoken-to-text (or vice versa), the newly generated node and the candidate node are determined to form a cross-media entity pair. For this cross-media entity pair, a preset cross-media style difference compensation value is retrieved from the configuration parameter table. The configuration parameter table is a key-value mapping table loaded into memory during the knowledge graph construction pipeline initialization phase. Technically, it is a hash mapping structure with a combination of two modality identifier enumeration values ​​as the primary key and a real-valued compensation value as the associated value. The configuration parameter table is configured as follows: multiple modality pairs and their corresponding compensation values ​​are predefined in the pipeline configuration file. For example, the combination of written text and spoken-to-text media can be used as the key, and an empirical compensation value (e.g., 0.05) determined through prior statistics can be used as the value. During pipeline initialization, these correspondences are read and parsed into the memory data structure of the configuration parameter table. The technical essence of the preset cross-media style difference compensation value is a pre-defined numerical bias used to correct the systematically low similarity of semantic vectors in entity description texts in cross-media scenarios. Its preset basis can be determined through statistical analysis of the vector similarity distributions of a batch of known cross-modal equivalent entity pairs and known non-equivalent entity pairs: a critical compensation range is selected that allows the mean similarity of equivalent entity pairs to exceed a preset connectivity threshold while the mean similarity of non-equivalent entity pairs remains below the preset connectivity threshold. In the application scenario of government and enterprise knowledge bases, since the stylistic differences between written policy texts and transcribed meeting recordings are relatively fixed and statistically stable, using the preset cross-media style difference compensation value in the configuration parameter table can effectively alleviate the similarity decay problem caused by cross-media, without needing to re-estimate statistically each time a judgment is made.

[0083] Preferably, the operation of determining the cross-media style difference compensation value as a positive adjustment parameter involves assigning the cross-media style difference compensation value retrieved from the configuration parameter table to a variable named "positive adjustment parameter." This positive adjustment variable will be used as one of the operators in subsequent addition operations. The technical essence of the positive adjustment parameter is a numerical increment that adds compensation based on the initial similarity. Its sign is positive, indicating that a positive enhancement operation is applied to the initial similarity in cross-media scenarios to offset the similarity decay caused by differences in expression style resulting from different underlying media types. In the process of determining the cross-media style difference compensation value as the positive adjustment parameter, the first step is to check whether the first and second underlying media types are the same. If they are the same, the correction is skipped, and the positive adjustment parameter is zero. This means that no additional compensation is applied between entity pairs of the same media type; equivalence is determined solely by the initial similarity. If the first and second underlying media types are written text and spoken-to-text text, respectively, the cross-media style difference compensation value is read from the configuration parameter table and assigned as the positive adjustment parameter. If the first and second underlying media types are written text and structured data description text, or spoken-to-text text and structured data description text, respectively, the corresponding other set of cross-media style difference compensation values ​​is read from the configuration parameter table and assigned as the positive adjustment parameter. Each set of cross-media style difference compensation values ​​is configured independently based on the statistical characteristics of the modal pair, and they are independent of each other. Through this unidirectional, configurable compensation mechanism, the similarity of cross-modal entity pairs is reasonably improved to match the degree of modal difference without introducing additional mismatch noise.

[0084] Preferably, the operation of adding the positive adjustment parameter to the initial similarity to obtain the target similarity is the final numerical synthesis step of the cross-modal similarity correction mechanism. Its processing objects are the initial similarity calculated in the aforementioned steps and the assigned positive adjustment parameter. The addition operation is completed by a single floating-point addition instruction, accumulating the value of the positive adjustment parameter to the initial similarity value to obtain a corrected similarity value, which is the target similarity. The reason for using addition instead of multiplication or other nonlinear transformations is that the similarity attenuation caused by differences in cross-media expression styles manifests numerically as a relatively stable additional bias rather than proportional scaling: In the semantic vector space, the main attenuation of the cosine of the angle between two different modal entity descriptions pointing to the same entity originates from the irrelevant directional components introduced by noise components in the spoken text. These irrelevant directional components, through vector addition, cause the semantic vector of the spoken text to deviate from the semantic vector direction of the written text. This deviation is reflected in the cosine of the angle as approximately a fixed negative offset. Therefore, additive compensation corresponds to this numerically. After additive compensation, the target similarity may slightly exceed the upper limit of one; in this case, it can be truncated to one to maintain consistency with the concept of semantic similarity. After obtaining the target similarity, the downstream decision logic of the knowledge graph construction pipeline compares the target similarity with a preset connectivity threshold. If the target similarity reaches or exceeds the preset connectivity threshold, an equivalent connection edge is constructed between the new node and the candidate node. If it is lower than the preset connectivity threshold, the new node and the candidate node are determined to point to different real-world entities, and no equivalent connection edge is constructed. In the application scenario of government and enterprise knowledge base, although the "People's Bank of China" in a policy document and the "People's Bank" in a meeting recording have been replaced with the standard name by the alias replacement in the first-level domain dictionary, under the pure vector similarity judgment without compensation, the vector deviation caused by the difference in expression style may make the initial similarity (e.g., 0.78) slightly lower than the preset connectivity threshold (e.g., 0.80), resulting in the omission of equivalent entities. After applying a positive adjustment parameter (e.g., 0.05), the target similarity reaches 0.83, successfully exceeding the preset connectivity threshold, thereby establishing an equivalent connection edge between the two and ensuring the connectivity of cross-modal knowledge fragments in the graph. This cross-modal similarity correction mechanism, which is based on addition operations, can effectively reduce the missing associations of equivalent entities caused by differences in modal media in the multimodal entity normalization scenario of government and enterprise knowledge bases, compared with the traditional scheme that uses a fixed threshold, and improve the integrity of cross-modal knowledge fusion.

[0085] like Figure 3 As shown in the figure, this application provides a three-level entity normalization and cross-modal semantic anchoring device for a multimodal knowledge base, comprising: The modality conversion module is used to convert multimodal data into initial text blocks carrying modality identifiers; The alias replacement module is used to replace the aliases of the initial text block using a mapping dictionary to generate a standard text block. An entity recognition module is used to identify entity names contained in the standard text block and extract entity description text associated with the entity names; The historical node processing module is used to append the entity description text to the historical node when a historical node with the same name as the entity already exists in the multimodal knowledge base. The new node creation module is used to create a new node based on the entity name when there is no historical node with the same name as the entity in the knowledge base. The feature acquisition module is used to calculate the first feature sequence of the newly generated node and to acquire the second feature sequence of the candidate node in the knowledge base; The similarity calculation module is used to calculate the initial similarity between the first feature sequence and the second feature sequence; An adjustment module is used to apply a positive adjustment parameter to the initial similarity when the modality identifier of the new node is different from that of the candidate node, so as to obtain the target similarity. The association construction module is used to construct equivalent association edges between the new node and the candidate node when the target similarity reaches a preset connectivity threshold.

[0086] like Figure 4 As shown, an electronic device according to an embodiment of this application includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The feature is that when the processor executes the program, it implements the method described in any one of the claims of this application.

[0087] like Figure 5 As shown, a computer-readable storage medium according to an embodiment of this application stores a computer program thereon, characterized in that the program, when executed by a processor, implements the method described in any one of the claims of this application.

[0088] like Figure 6 As shown in the figure, a three-level entity normalization and cross-modal semantic anchoring system for multimodal knowledge bases is provided in an embodiment of this application, comprising: Multimodal data acquisition equipment, used to acquire multimodal data; And the apparatus as described in this application, which is communicatively connected to the multimodal data acquisition device, for receiving the multimodal data and performing three-level entity normalization and cross-modal semantic anchoring.

[0089] The above Figures 3-6For an exemplary description, please refer to the above. Figure 1 This will not be elaborated upon here.

Claims

1. A three-level entity normalization and cross-modal semantic anchoring method for multimodal knowledge bases, characterized in that, include: Multimodal data is converted into initial text blocks carrying modality identifiers, and the initial text blocks are aliased using a mapping dictionary to generate standard text blocks; Identify the entity names contained in the standard text block and extract the entity description text associated with the entity names; If a historical node with the same name as the entity already exists in the multimodal knowledge base, the entity description text is appended to that historical node; if no historical node with the same name as the entity exists in the knowledge base, a new node is created based on the entity name. Calculate the first feature sequence of the newly generated node and obtain the second feature sequence of the candidate node in the knowledge base, and calculate the initial similarity between the first feature sequence and the second feature sequence; When the modality identifier of the new node is different from that of the candidate node, a positive adjustment parameter is applied to the initial similarity to obtain the target similarity; When the target similarity reaches a preset connectivity threshold, an equivalent association edge is constructed between the new node and the candidate node.

2. The method according to claim 1, characterized in that, The step of using a mapping dictionary to perform alias replacement on the initial text block to generate a standard text block includes: Construct a mapping tree containing standard proper nouns and their aliases; Match the text sequence in the initial text block with the mapping tree; When a text fragment that matches any alias is found, that text fragment is identified as the target alias; Determine the starting position of the target alias in the text sequence; Extract the target standard proper nouns corresponding to the target aliases from the mapping tree; At the starting position, replace the target alias with the target standard proper noun; A replacement record is generated based on the original text of the target alias, the target standard proper noun, and the starting position; The replacement record is stored in the metadata area of ​​the initial text block to obtain the standard text block.

3. The method according to claim 1, characterized in that, After appending the entity description text to the history node, the method further includes: Insert a separator at the end of the original record paragraph in the historical node; The additional entity description text is concatenated after the delimiter to form an accumulating concatenated text; Determine the amount of storage space occupied by the accumulated concatenated text; When the storage space value is detected to exceed the preset redundancy threshold, the accumulated concatenated text is sent to the semantic extraction component; Receive the summary description paragraph generated by the semantic extraction component through compression and reconstruction of the accumulated concatenated text; Replace the accumulated cascading text stored in the history node with the entire summary description paragraph.

4. The method according to claim 1, characterized in that, After constructing an equivalent association edge between the newly created node and the candidate node, the process further includes: Extract action relationships with intent-directed characteristics from multiple standard text blocks; Determine the first action association relationship starting with the entity name, and connect the starting point of the first action association relationship to the output of the corresponding historical node or the new node; Determine the second action association with the entity name as the termination point, and connect the endpoint of the second action association to the input of the corresponding historical node or the new node; By utilizing the connection paths within the nodes, a cross-modal traversal query path is constructed from the second action association to the first action association.

5. The method according to claim 1, characterized in that, The process of identifying entity names contained in the standard text block and extracting entity description text associated with the entity names further includes: Configure the entity recognition unit with a recognition probe for capturing external document citation formats; The recognition probe is driven to scan the standard text block and locate the document name string within the preset book title identifier and the adjacent citation verbs; Based on the document name string, generate an exogenous reference node; Based on the citation verb, generate citation relationship edges from the source node corresponding to the standard text block to the exogenous reference node; The entity name is determined based on the exogenous reference node, and the entity description text is generated based on the adjacent text fragments in the standard text block that are adjacent to the reference name string.

6. The method according to claim 1, characterized in that, The process of converting multimodal data into initial text blocks carrying modality identifiers also includes: Extract the generation time information of the multimodal data; Convert the generated time information into a standard timestamp in a unified format; The standard timestamp is stored in the metadata field corresponding to the initial text block; Upon receiving a business query instruction from an external device, multiple candidate text blocks matching the business query instruction are retrieved from the multimodal knowledge base. Extract the standard timestamp stored in each of the returned candidate text blocks; Multiple candidate text blocks with time differences within a preset range and carrying different modality identifiers are combined into a concurrent multimodal data cluster; The synchronous multimodal data cluster is pushed to the external presentation device to reconstruct a cross-modal event snapshot at a specified time.

7. A three-level entity normalization and cross-modal semantic anchoring device for multimodal knowledge bases, characterized in that, include: The modality conversion module is used to convert multimodal data into initial text blocks carrying modality identifiers; The alias replacement module is used to replace the aliases of the initial text block using a mapping dictionary to generate a standard text block. An entity recognition module is used to identify entity names contained in the standard text block and extract entity description text associated with the entity names; The historical node processing module is used to append the entity description text to the historical node when a historical node with the same name as the entity already exists in the multimodal knowledge base. The new node creation module is used to create a new node based on the entity name when there is no historical node with the same name as the entity in the knowledge base. The feature acquisition module is used to calculate the first feature sequence of the newly generated node and to acquire the second feature sequence of the candidate node in the knowledge base; The similarity calculation module is used to calculate the initial similarity between the first feature sequence and the second feature sequence; An adjustment module is used to apply a positive adjustment parameter to the initial similarity when the modality identifier of the new node is different from that of the candidate node, so as to obtain the target similarity. The association construction module is used to construct equivalent association edges between the new node and the candidate node when the target similarity reaches a preset connectivity threshold.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

10. A three-level entity normalization and cross-modal semantic anchoring system for multimodal knowledge bases, characterized in that, include: Multimodal data acquisition equipment, used to acquire multimodal data; And the apparatus as described in claim 10, wherein the apparatus is communicatively connected to the multimodal data acquisition device, for receiving the multimodal data and performing three-level entity normalization and cross-modal semantic anchoring.

Images