Intelligent data processing system based on AI large model

Abstract

The invention relates to the field of data processing, and discloses an intelligent data processing system based on an AI large model, which comprises the following steps: when detecting that a plurality of candidate entities exist in a text, judging whether the candidate entities need to be subjected to semantic disambiguation processing or not according to context semantic relevancy and entity historical reference frequency; constructing a cross-domain context representation vector, introducing a pre-training language model to encode the context, generating deep semantic representation of candidate entities, and judging whether a clustering result has ambiguity or not; whether the candidate entities have conflicts or not is judged by fusing language model embedding and structuring knowledge graph information; constructing an entity relationship graph based on a graph neural network, performing causality, temporal and attribute dependence reasoning on the relationship between the existing candidate entities, and judging whether the relationship between the candidate entities should be combined or split; and carrying out label replacement on the candidate entities in the target text in combination with the context and the reasoning result. The method has the advantage of improving the accuracy of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and specifically to an intelligent data processing system based on an AI large model. Background Art

[0002] With the development of artificial intelligence technology, the application of AI big models in data intelligent processing is becoming increasingly widespread, especially in multi-source data fusion, semantic understanding and reasoning. However, when applying AI big models to data intelligent processing systems, existing technologies still have some fine-grained problems that need to be solved. When faced with synonymous entities or terms with similar semantics in the context, the system lacks refined entity disambiguation capabilities, which causes confusion in the model during the semantic fusion process. For example, when processing cross-domain texts or multilingual materials, the same entity may be recognized as different objects by the system due to different expressions, or different entities may be misjudged as the same object, thereby affecting the accuracy of data analysis. Although some systems use named entity recognition and word vector matching technologies to try to solve this problem, these methods are limited in the depth of context understanding and reasoning capabilities, especially in scenarios with dense professional terms or ambiguous contexts, and are unstable, seriously affecting the subsequent modeling effect. Therefore, it is very necessary to design a data intelligent processing system based on AI big models to improve the accuracy of data processing. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the present invention provides a data intelligent processing system based on an AI large model, which has the advantage of improving the accuracy of data processing and solves the problems in the above-mentioned background technology.

[0004] To achieve the above-mentioned purpose of improving the accuracy of data processing, the present invention provides the following technical solution: a data intelligent processing system based on an AI large model, comprising: Semantic analysis module: When multiple candidate entities are detected in the text, the module determines whether the candidate entity needs semantic disambiguation based on the contextual semantic relevance and the entity's historical citation frequency. If so, the module enters the semantic modeling module; if not, the candidate entity is directly determined as a disambiguated entity and sent to the semantic output module; Semantic modeling module: Constructs a cross-domain context representation vector and introduces a pre-trained language model to encode the context, generates a deep semantic representation of the candidate entity, and performs semantic clustering to determine whether the clustering results are ambiguous. If so, it enters the entity verification module; if not, it sends the clustering results as disambiguated entities to the semantic output module; Entity Verification Module: By integrating language model embedding with structured knowledge graph information, it determines whether there is any conflict in the candidate entities. If so, it enters the relational reasoning module; if not, the verified candidate entity set is sent to the semantic output module; Relationship reasoning module: Builds a candidate entity relationship graph based on a graph neural network, performs causal, temporal, and attribute dependency reasoning on the existing candidate entity relationships, and determines whether the entity relationships should be merged or split. If so, it enters the semantic fusion output module; if not, the original entity set is sent to the semantic output module; Semantic output module: Combines the context and reasoning results to replace the annotations of candidate entities in the target text and outputs the standardized semantic results after disambiguation.

[0005] Preferably, the process of determining whether a candidate entity needs semantic disambiguation is as follows: Calculate the semantic relevance value between the candidate entity and the context; Combine the citation frequency of candidate entities in historical texts to generate a semantic ambiguity score; If the semantic ambiguity score is higher than the set ambiguity judgment threshold, it is determined that semantic disambiguation processing is required.

[0006] Preferably, the process of constructing the cross-domain context representation vector is: Extract sentences and context windows containing candidate entities from the detected text and identify keywords related to the candidate entities; Combining syntactic dependencies to structurally enhance the identified keywords related to the candidate entity, and generating structurally enhanced contextual information; A cross-domain language model is used to semantically encode the contextual information after structural enhancement to obtain the contextual semantic representation vector of the candidate entity; After the semantic encoding is completed, the position indicator of the candidate entity is introduced to explicitly mark the position of the candidate entity; The encoded context semantic representation vector with entity location indication tags is used as the cross-domain context representation vector of the candidate entity.

[0007] Preferably, the contextual semantic representation vector of the candidate entity is obtained as follows: A cross-domain pre-trained language model based on a multi-layer structure encodes the contextual information after structural enhancement; The hidden state vector corresponding to the candidate entity position in the encoding result is extracted as the contextual semantic representation vector of the candidate entity.

[0008] Preferably, the process of semantic clustering is: Perform weighted fusion on the hidden state vectors of different layers to generate candidate entity semantic embedding vectors that integrate multi-layer semantic information. Use a density-based clustering algorithm to cluster the embedding vector set obtained by summarizing all candidate entity semantic embedding vectors. Calculate the cosine similarity of the semantic embedding vectors of the candidate entities in each cluster, and use the average cosine similarity between all candidate entities in the cluster as the semantic consistency score; If the semantic consistency score is less than the set clustering reliability threshold, the clustering result is judged to be semantically ambiguous; If there is a semantic consistency score greater than or equal to the set clustering reliability threshold, it is determined that the clustering result has no semantic ambiguity.

[0009] Preferably, the process of determining whether a candidate entity conflicts is as follows: Extract the semantic embedding vector of the candidate entity, retrieve the structured attribute information of the corresponding candidate entity node from the knowledge graph, and generate a structured entity vector through the attribute embedding method; Adopting feature concatenation, attention mechanism or nonlinear mapping, the semantic embedding vector of the candidate entity is fused with the structured entity vector to form a unified fused representation vector of the candidate entity. Calculate the semantic difference of the fused representation vectors of all candidate entities to obtain the original semantic conflict score; Combining the semantic domain to which the candidate entity belongs, the sensitivity of the context in which it is located, and the consistency of the candidate entity attributes in the knowledge graph, the original semantic conflict score is multi-dimensionally weighted and integrated to obtain the final conflict risk score; If the final conflict risk score is greater than the preset conflict judgment threshold, it is determined that there is a semantic conflict between the candidate entities; If the final conflict risk score is less than or equal to the preset conflict judgment threshold, it is determined that there is no semantic conflict between the candidate entities.

[0010] Preferably, the process of reasoning about causality, temporality, and attribute dependency between existing candidate entities is as follows: Build an entity relationship graph containing candidate entities, and add context connection edges and knowledge dependency edges between candidate entities in the entity relationship graph; Utilize graph neural networks to propagate semantic information between candidate entities, aggregate information, and perform semantic updates on each candidate entity node. Based on the connection strength, path length and node attributes between candidate entities, it is determined whether there is a causal relationship, time sequence or attribute dependency between candidate entities.

[0011] Preferably, the process of determining whether the relationship between candidate entities should be merged or split is as follows: Based on the constructed entity relationship graph, multiple rounds of graph neural network information propagation are performed on the candidate entity nodes to obtain the context-enhanced semantic representation of each candidate entity; Determine the pairing relationship between candidate entities through co-occurrence in the same context, semantic clustering results, or proximity in the semantic embedding space to form a set of candidate entity pairs; The similarity between the context-enhanced semantic representations of each pair of candidate entities is calculated, and the relationship reasoning analysis is performed based on the path relationship and attribute connection between entities in the graph structure. If the analysis results show that the candidate entities have a strong dependency relationship and the semantic similarity is higher than the candidate entity merging threshold, the candidate entity merging operation is performed; If the analysis results show that the semantic differences of the candidate entities are significant and the independence is higher than the candidate entity splitting threshold, the candidate entity splitting operation is performed.

[0012] Preferably, the process of outputting the standardized semantic result after disambiguation is: Generate a mapping table between candidate entities and standard entity labels based on the results of semantic analysis, modeling, verification, and reasoning. Replace the ambiguous candidate entity names in the original text with the corresponding standard entity labels; Output disambiguated structured semantic text.

[0013] Compared with the existing technology, the present invention provides a data intelligent processing system based on AI large model, which has the following beneficial effects: 1. When multiple candidate entities are detected in a text, the system uses two dimensions, contextual semantic relevance and entity historical citation frequency, to quantitatively assess ambiguity, thereby accurately identifying semantic disambiguation requirements. This effectively screens truly ambiguous candidate entities, avoiding unnecessary disambiguation of semantically clear entities. This reduces the system's computational burden, improves overall processing efficiency, and enhances the accuracy of entity recognition and replacement, demonstrating its practicality and engineering application value.

[0014] 2. By constructing a cross-domain context representation vector and introducing a pre-trained language model to deeply encode the context, the semantic expression ability of candidate entities in different semantic scenarios can be effectively enhanced; combined with an improved semantic clustering method to achieve consistency judgment of the semantics of candidate entities, it helps to identify entities with ambiguous semantics or expression differences, improve the accuracy and robustness of the entity disambiguation process, and provide a more reliable semantic foundation for subsequent entity verification and relationship reasoning.

[0015] 3. By integrating the dual information sources of language model and knowledge graph, the entity verification module can effectively improve the semantic discrimination accuracy in the candidate entity recognition process. It can not only identify entities that are highly similar in semantics but conflicting in structure, but also avoid misjudgments caused by language ambiguity or insufficient context, thereby improving the accuracy and stability of the overall disambiguation system and laying the foundation for the subsequent refined reasoning of entity relationships.

[0016] 4. By building an entity relationship graph based on a graph neural network and introducing multi-dimensional reasoning mechanisms such as causality, temporality, and attribute dependency, it is possible to comprehensively model the deep semantic relationships between entities, effectively determining the strength and diversity of associations between candidate entities. It accurately identifies entities that need to be merged or split, resolving ambiguities caused by semantic overlap, weak contextual associations, or attribute conflicts. This improves the semantic integrity and structural rationality of entity recognition, providing a high-quality reasoning basis for subsequent semantic fusion output.

[0017] 5. By combining contextual information with relational reasoning, it can accurately identify semantically ambiguous entities in text and output standardized labels based on deep semantic understanding and entity reasoning, effectively completing the entity disambiguation task. This improves the accuracy and consistency of entity recognition and understanding in natural language processing, giving the system stronger semantic resolution capabilities when handling synonymous entities, polysemous words, and cross-domain entities, further optimizing the semantic input quality for downstream tasks such as information extraction, knowledge graph construction, and intelligent question-answering systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the structure of the present invention; Figure 2 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] Example 1 See also Figure 1 As shown, an AI big model-based data intelligent processing system according to an embodiment of the present invention includes: Semantic analysis module: When multiple candidate entities are detected in the text, the module determines whether the candidate entity needs semantic disambiguation based on the contextual semantic relevance and the entity's historical citation frequency. If so, it enters the semantic modeling module; if not, it directly determines the candidate entity as a disambiguated entity and sends it to the semantic output module.

[0021] The process of determining whether a candidate entity needs semantic disambiguation processing in the semantic analysis module is as follows: Calculate the semantic relevance between the candidate entity and the current context; identify the candidate entity and the complete sentence or paragraph in which it is located from the target text as the contextual semantic environment; use the pre-trained language model to semantically encode the candidate entity and the context respectively; Combine the citation frequency of the candidate entity in the historical text to calculate the semantic fuzziness score; retrieve all occurrence records of the candidate entity from the corpus or historical text database, and extract the corresponding context sentences to form a historical context set , using the pre-trained language model for each historical context Encode and obtain the corresponding context vector; use the clustering algorithm to perform semantic clustering on all context vectors to obtain the number of clusters k, each cluster represents a semantic type; for each cluster , count the number of citations included , calculate the probability distribution of each semantic cluster: , where F is the total number of citations; information entropy is used to measure the dispersion of entity semantic distribution: Where k is the number of clusters; A higher score indicates that the candidate entity is used more evenly in multiple semantics and has ambiguity.

[0022] It is understandable that the purpose of determining whether a candidate entity requires semantic disambiguation is to: Function 1: Processing only entities with high semantic ambiguity can more accurately focus on the context where ambiguous content exists, effectively improving the quality of final entity disambiguation.

[0023] Function 2: In real-time data processing, quickly screening out candidate entities with clear semantics helps speed up semantic recognition and output efficiency.

[0024] When multiple candidate entities are detected in a text, the system uses two dimensions, contextual semantic relevance and the entity's historical citation frequency, to quantitatively assess ambiguity, thereby accurately identifying semantic disambiguation requirements. This effectively screens truly ambiguous candidate entities, avoiding unnecessary disambiguation of semantically clear entities. This reduces the system's computational burden, improves overall processing efficiency, and enhances the accuracy of entity recognition and replacement, demonstrating its practicality and engineering application value.

[0025] Semantic modeling module: Constructs a cross-domain context representation vector and introduces a pre-trained language model to encode the context, generates a deep semantic representation of the candidate entity, and performs semantic clustering to determine whether the clustering result is ambiguous. If so, it enters the entity verification module; if not, the clustering result is sent to the semantic output module as a disambiguated entity.

[0026] The process of constructing the cross-domain context representation vector in the semantic modeling module is as follows: Extract sentences and context windows containing candidate entities from the detected text and identify keywords related to the candidate entities; Combining syntactic dependencies to structurally enhance the identified keywords related to the candidate entity, and generating structurally enhanced contextual information; A cross-domain language model is selected to semantically encode the contextual information after structural enhancement to obtain the contextual semantic representation vector of the candidate entity; the cross-domain language model can understand and represent concepts and terms in multiple professional fields, avoiding semantic understanding barriers or deviations in non-training field corpus; the semantic representation in one field is migrated and adapted to the context of another field to achieve cross-domain knowledge sharing and contextual reasoning; it has a stronger ability to discern the meaning of domain-specific terms in different contexts and can generate more accurate contextual semantic representation vectors; it has the ability to model the semantic differences when the same entity appears in multiple fields, and effectively handles semantic ambiguity or entity polysemy; it can still maintain a high level of understanding accuracy and representation accuracy when faced with non-standard expressions, mixed domains or cross-domain references.

[0027] After the semantic encoding is completed, the position indicator of the candidate entity is introduced to explicitly mark the position of the candidate entity; The encoded context semantic representation vector with entity location indication tags is used as the cross-domain context representation vector of the candidate entity.

[0028] The contextual semantic representation vector of the candidate entity is obtained as follows: A cross-domain pre-trained language model based on a multi-layer structure encodes the contextual information after structural enhancement; The hidden state vector corresponding to the candidate entity position in the encoding result is extracted as the contextual semantic representation vector of the candidate entity.

[0029] The process of semantic clustering in the semantic modeling module is as follows: Perform weighted fusion on the hidden state vectors of different layers to generate candidate entity semantic embedding vectors that integrate multi-layer semantic information. Use a density-based clustering algorithm to cluster the embedding vector set obtained by summarizing all candidate entity semantic embedding vectors. Calculate the cosine similarity of the semantic embedding vectors of the candidate entities in each cluster, and use the average cosine similarity between all candidate entities in the cluster as the semantic consistency score; If the semantic consistency score is less than the set clustering reliability threshold, the clustering result is judged to be semantically ambiguous; If there is a semantic consistency score greater than or equal to the set clustering reliability threshold, it is determined that the clustering result has no semantic ambiguity.

[0030] Perform clustering on the semantic embedding vectors of the candidate entities to obtain multiple semantic clusters; obtain the semantic embedding vector of each candidate entity encoded in the context by the pre-trained language model to form a set of candidate entity embedding vectors; perform normalization on the set of candidate entity embedding vectors to keep the scale consistent in the vector space; select a clustering algorithm for semantic clustering, the clustering algorithm includes but is not limited to K-means clustering, hierarchical clustering or density clustering; determine the cluster center or initial clustering condition based on the preset number of clusters, vector density or silhouette coefficient and other indicators; execute the clustering process to divide the candidate entities into corresponding semantic clusters to obtain multiple clusters representing different semantic categories; output the set of candidate entities contained in each semantic cluster, and use the clustering results as input for subsequent ambiguity determination and entity verification modules; Sample several vector samples in each semantic cluster, calculate the cosine similarity and generate the cluster consistency index; for each semantic cluster, based on the total number and distribution density of vectors, randomly or equidistantly sample several semantic embedding vectors according to a preset sampling ratio or a fixed number to form a sampling vector subset; for any pair of vectors in each sampling vector subset, calculate the cosine similarity value: Where A and B are two semantic embedding vectors, are the modulus lengths of the two semantic embedding vectors respectively; All calculated cosine similarity values are summarized, and the average and variance are calculated to form the cluster consistency index of the semantic cluster; the cluster consistency indexes of all semantic clusters are compared. If the consistency index of a semantic cluster is lower than the preset consistency threshold, it is determined that the semantic cluster has semantic divergence or ambiguity; the consistency index of each semantic cluster is output, and the ambiguous cluster is passed to the entity verification module for further verification and splitting judgment.

[0031] By constructing a cross-domain context representation vector and introducing a pre-trained language model to deeply encode the context, the semantic expression ability of candidate entities in different semantic scenarios can be effectively enhanced; combined with the improved semantic clustering method to achieve consistency judgment of the semantics of candidate entities, it helps to identify entities with ambiguous semantics or expression differences, improve the accuracy and robustness of the entity disambiguation process, and provide a more reliable semantic foundation for subsequent entity verification and relationship reasoning.

[0032] Entity verification module: By integrating language model embedding and structured knowledge graph information, it determines whether there is any conflict in the candidate entities. If so, it enters the relational reasoning module; if not, the verified candidate entity set is sent to the semantic output module.

[0033] The process of determining whether a candidate entity conflicts in the entity verification module is as follows: Extract the semantic embedding vector of the candidate entity, retrieve the structured attribute information of the corresponding candidate entity node from the knowledge graph, and generate a structured entity vector through the attribute embedding method; Adopting feature concatenation, attention mechanism or nonlinear mapping, the semantic embedding vector of the candidate entity is fused with the structured entity vector to form a unified fused representation vector of the candidate entity. Calculate the semantic difference of the fused representation vectors of all candidate entities to obtain the original semantic conflict score; Combining the semantic domain to which the candidate entity belongs, the sensitivity of the context in which it is located, and the consistency of the candidate entity attributes in the knowledge graph, the original semantic conflict score is multi-dimensionally weighted and integrated to obtain the final conflict risk score; If the final conflict risk score is greater than the preset conflict judgment threshold, it is determined that there is a semantic conflict between the candidate entities; If the final conflict risk score is less than or equal to the preset conflict judgment threshold, it is determined that there is no semantic conflict between the candidate entities.

[0034] Extract the deep semantic representation vector of the candidate entity and fuse it with the structured information embedding of the corresponding entity in the knowledge graph; explicitly mark the position of the candidate entity in the context with special tags, extract the hidden layer vector corresponding to the candidate entity position in the language model output as the context-dependent semantic representation of the entity, recorded as the semantic representation vector; locate the node that matches the candidate entity name in the knowledge graph, extract adjacent nodes, attribute information and relationship triples; use the graph embedding algorithm to encode the structured information into the graph embedding vector of the entity; if an entity corresponds to multiple graph node candidates, use the entity name matching score, context relevance, etc. for weighted aggregation; fuse the semantic representation vector with the graph embedding vector, including but not limited to the following methods: send it to the fully connected layer after splicing; learn weights through the attention mechanism; use the gating mechanism to dynamically adjust the contribution of the two; the final output fusion vector is the deep representation of the candidate entity after fusing the knowledge semantics in the current context; Calculate the semantic conflict score between the fused vectors. Extract the fused vectors of the two candidate entities, representing them as the semantic representations of entity A and entity B, respectively. Use cosine similarity to calculate the similarity between the two vectors in the semantic space. The more consistent the direction of the two fused vectors, the higher the similarity, indicating stronger semantic consistency; conversely, the lower the similarity, the greater the semantic difference between the two. Convert the similarity to a semantic distance by subtracting the cosine similarity from 1. This yields a distance value where a larger value indicates a more significant semantic conflict.

[0035] The technical solution of this embodiment is: using the language model to deeply embed the semantic representation of the candidate entity in the context, and simultaneously extracting the attribute information and category labels of the entity in the structured knowledge graph to generate a unified fusion vector representation, and by calculating the semantic conflict score of the fusion vector between the candidate entities, combining the type, relationship and attribute consistency in the knowledge graph to determine whether there is a semantic conflict, thereby verifying the identity or distinctiveness between the candidate entities. By integrating the dual information sources of the language model and the knowledge graph, the entity verification module can effectively improve the semantic discrimination accuracy in the candidate entity recognition process. It can not only identify entities that are highly similar in semantics but conflicting in structure, but also avoid misjudgments caused by language ambiguity or insufficient context, improve the accuracy and stability of the overall disambiguation system, and lay the foundation for the subsequent refined reasoning of entity relationships.

[0036] Example 2 like Figure 1 As shown, a data intelligent processing system based on AI big model also includes the following modules: Relationship reasoning module: Builds an entity relationship graph based on graph neural network, performs causal, temporal and attribute dependency reasoning on the existing entity relationships, and determines whether the entity relationships should be merged or split. If so, it enters the semantic fusion output module; if not, the original entity set is sent to the semantic output module.

[0037] The process of reasoning about causality, temporality, and attribute dependency of the existing inter-entity relationships in the relational reasoning module is as follows: Construct an entity relationship graph containing candidate entities, and add contextual connection edges and knowledge dependency edges between candidate entities in the entity relationship graph; extract all candidate entities from the input text, and construct each candidate entity as a node in the graph; for each pair of candidate entities, judge the degree of semantic association based on contextual factors such as whether they have a co-occurrence relationship in the original text, whether they belong to the same semantic paragraph, and whether they share contextual keywords; if the degree of association exceeds the set threshold, add contextual connection edges between the corresponding candidate entity nodes to indicate the semantic association of the candidate entities in the context; query the standard entity and structured information such as attributes, categories, and hierarchical relationships corresponding to each candidate entity from the external knowledge graph; if two candidate entities have an explicit semantic relationship in the knowledge graph, add a knowledge dependency edge to the graph to indicate the semantic dependency between the two in the knowledge structure.

[0038] Utilize graph neural networks to propagate semantic information between candidate entities, aggregate information, and perform semantic updates on each candidate entity node. Based on the connection strength, path length and node attributes between candidate entities, it is determined whether there is a causal relationship, time sequence or attribute dependency between candidate entities.

[0039] Construct an entity relationship graph and establish contextual connection edges and knowledge dependency edges between entities in the graph; extract identified candidate entities from the target text and construct a node in the graph for each entity, each node contains the entity's unique identifier, context position, language model embedding vector and entity in the knowledge graph; for entity nodes that are adjacent or contextually close, set a connection threshold based on their position in the text, co-occurrence frequency and context window distance. If two entities appear in the same context window, a contextual connection edge is established between them; call the known semantic relationship between entities in the structured knowledge graph, and if there is a direct or indirect structural connection between two entities in the graph, a knowledge dependency edge is constructed between the nodes; all nodes and edges are integrated to form a weighted directed graph, in which different edge types can be set as multi-channels. The graph structure will then be used as input to the graph neural network model to perform reasoning on causal, temporal and attribute dependencies between entities.

[0040] Graph neural network models are used to propagate semantic information between nodes. Each entity node is initialized as a high-dimensional feature vector, which is a concatenation or weighted combination of the following components: the language model embedding of the text context, the structured embedding of the entity in the knowledge graph, and an optional entity attribute vector. Edges are divided into different types, each with a different weight or relationship label to distinguish the degree of influence during propagation. For each round of propagation in the graph, all nodes receive information from neighboring nodes and update their own representations. Perform causal, temporal, and attribute dependency analysis on the connection strength and path information in the entity relationship diagram to determine whether the relationships between entities should be merged or split.

[0041] The process of determining whether the relationship between entities should be merged or split in the relationship reasoning module is as follows: Based on the constructed entity relationship graph, multiple rounds of graph neural network information propagation are performed on the candidate entity nodes to obtain the context-enhanced semantic representation of each candidate entity; Determine the pairing relationship between candidate entities through co-occurrence in the same context, semantic clustering results, or proximity in the semantic embedding space to form a set of candidate entity pairs; The similarity between the context-enhanced semantic representations of each pair of candidate entities is calculated, and the relationship reasoning analysis is performed based on the path relationship and attribute connection between entities in the graph structure. If the analysis results show that the candidate entities have a strong dependency relationship and the semantic similarity is higher than the candidate entity merging threshold, the candidate entity merging operation is performed; If the analysis results show that the semantic differences of the candidate entities are significant and the independence is higher than the candidate entity splitting threshold, the candidate entity splitting operation is performed.

[0042] In the graph, each node represents a candidate entity, and each edge represents a relationship between entities. The edge types are divided into: Causal edge: Entity A is the result or cause of entity B; Time series edge: the temporal sequence or logical sequence of entities in the text; Attribute dependency edge: entities share or depend on the same attribute features; The edge weights of the graph are generated jointly through structured knowledge graph and context analysis; Using graph neural networks, we perform multiple rounds of information aggregation and semantic updates on entity nodes, so that the vector representation of each entity node integrates contextual and structural information. For each pair of candidate entities, the following reasoning analysis is included but not limited to: Determine whether there is a logical causal path between two entities: If there is a bidirectional causal or circular path between the two entities, split them; If there is a clear one-way causal path between the two entities, then merge them.

[0043] Compare the chronological order of two entities in context: If the time overlaps or is consistent, then merge; If there is a time conflict or the span is large, split it.

[0044] Vector encode the attribute features of the entity and calculate the attribute similarity: If the attributes are highly similar, merge them; If there are conflicting attributes, split them.

[0045] By building an entity relationship graph based on a graph neural network and introducing multi-dimensional reasoning mechanisms such as causality, temporality, and attribute dependency, we can comprehensively model the deep semantic relationships between entities, effectively determining the strength and diversity of associations between candidate entities. This allows us to accurately identify entities that need to be merged or split, resolving ambiguities caused by semantic overlap, weak contextual associations, or attribute conflicts. This improves the semantic integrity and structural rationality of entity recognition, providing a high-quality reasoning basis for subsequent semantic fusion output.

[0046] Semantic output module: Combines the context and reasoning results to replace the entities in the target text and outputs the standardized semantic results after disambiguation.

[0047] The process of outputting the disambiguated standardized semantic results in the semantic output module is as follows: Generate entity disambiguation mapping table based on the results of semantic analysis, modeling, verification and relationship reasoning; Replace ambiguous original entities in the text with standard entity tags; Output structured semantic text results.

[0048] By combining contextual information with relational reasoning, it can accurately identify semantically ambiguous entities in text and output standardized labels based on deep semantic understanding and entity reasoning, effectively completing the entity disambiguation task. This improves the accuracy and consistency of entity recognition and understanding in natural language processing, giving the system stronger semantic resolution capabilities when handling synonymous entities, polysemous words, and cross-domain entities, further optimizing the semantic input quality for downstream tasks such as information extraction, knowledge graph construction, and intelligent question-answering systems.

[0049] Example 3 See also Figure 2 As shown, a data intelligent processing method based on an AI large model includes the following steps: S1: When multiple candidate entities are detected in the text, determine whether the candidate entity needs to be semantically disambiguated based on the contextual semantic relevance and the entity's historical citation frequency.

[0050] S2: Construct a cross-domain context representation vector and introduce a pre-trained language model to encode the context, generate a deep semantic representation of the candidate entity, and perform semantic clustering to determine whether the clustering results are ambiguous.

[0051] S3: By fusing language model embedding with structured knowledge graph information, determine whether there is a conflict between candidate entities.

[0052] S4: Build an entity relationship graph based on graph neural network, perform causal, temporal and attribute dependency reasoning on the existing relationships between entities, and determine whether the relationships between entities should be merged or split.

[0053] S5: Combine the context and reasoning results to replace the entities in the target text with annotations, and output the standardized semantic results after disambiguation.

[0054] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0055] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A data intelligent processing system based on AI big model, characterized by: include: Semantic analysis module: When multiple candidate entities are detected in the text, the module determines whether the candidate entity needs semantic disambiguation based on the contextual semantic relevance and the entity's historical citation frequency. If so, the module enters the semantic modeling module; if not, the candidate entity is directly determined as a disambiguated entity and sent to the semantic output module; Semantic modeling module: Constructs a cross-domain context representation vector and introduces a pre-trained language model to encode the context, generates a deep semantic representation of the candidate entity, and performs semantic clustering to determine whether the clustering results are ambiguous. If so, it enters the entity verification module; if not, it sends the clustering results as disambiguated entities to the semantic output module; Entity Verification Module: By integrating language model embedding with structured knowledge graph information, it determines whether there is any conflict in the candidate entities. If so, it enters the relational reasoning module; if not, the verified candidate entity set is sent to the semantic output module; Relationship reasoning module: Builds a candidate entity relationship graph based on a graph neural network, performs causal, temporal, and attribute dependency reasoning on the existing candidate entity relationships, and determines whether the entity relationships should be merged or split. If so, it enters the semantic fusion output module; if not, the original entity set is sent to the semantic output module; Semantic output module: Combines the context and reasoning results to replace the annotations of candidate entities in the target text and outputs the standardized semantic results after disambiguation.

2. The data intelligent processing system based on AI big model according to claim 1 is characterized in that: The process of determining whether a candidate entity needs semantic disambiguation is as follows: Calculate the semantic relevance between the candidate entity and the context, and generate a semantic fuzziness score based on the citation frequency of the candidate entity in historical texts. If the semantic ambiguity score is higher than the set ambiguity judgment threshold, it is determined that semantic disambiguation processing is required.

3. The data intelligent processing system based on AI big model according to claim 2 is characterized in that: The process of constructing a cross-domain context representation vector is as follows: Extract sentences and context windows containing candidate entities from the detected text and identify keywords related to the candidate entities; Combining syntactic dependencies to structurally enhance the identified keywords related to the candidate entity, and generating structurally enhanced contextual information; A cross-domain language model is used to semantically encode the contextual information after structural enhancement to obtain the contextual semantic representation vector of the candidate entity; After the semantic encoding is completed, the position indicator of the candidate entity is introduced to explicitly mark the position of the candidate entity; The encoded context semantic representation vector with entity location indication tags is used as the cross-domain context representation vector of the candidate entity.

4. The data intelligent processing system based on AI big model according to claim 3 is characterized in that: The contextual semantic representation vector of the candidate entity is obtained as follows: A cross-domain pre-trained language model based on a multi-layer structure encodes the contextual information after structural enhancement; The hidden state vector corresponding to the candidate entity position in the encoding result is extracted as the contextual semantic representation vector of the candidate entity.

5. The data intelligent processing system based on AI big model according to claim 4 is characterized in that: The process of semantic clustering is: Perform weighted fusion on the hidden state vectors of different layers to generate candidate entity semantic embedding vectors that integrate multi-layer semantic information. Use a density-based clustering algorithm to cluster the embedding vector set obtained by summarizing all candidate entity semantic embedding vectors. Calculate the cosine similarity of the semantic embedding vectors of the candidate entities in each cluster, and use the average cosine similarity between all candidate entities in the cluster as the semantic consistency score; If the semantic consistency score is less than the set clustering reliability threshold, the clustering result is judged to be semantically ambiguous; If there is a semantic consistency score greater than or equal to the set clustering reliability threshold, it is determined that the clustering result has no semantic ambiguity.

6. The data intelligent processing system based on AI big model according to claim 5 is characterized in that: The process of determining whether a candidate entity conflicts is as follows: Extract the semantic embedding vector of the candidate entity, retrieve the structured attribute information of the corresponding candidate entity node from the knowledge graph, and generate a structured entity vector through the attribute embedding method; Adopting feature concatenation, attention mechanism or nonlinear mapping, the semantic embedding vector of the candidate entity is fused with the structured entity vector to form a unified fused representation vector of the candidate entity. Calculate the semantic difference of the fused representation vectors of all candidate entities to obtain the original semantic conflict score; Combining the semantic domain to which the candidate entity belongs, the sensitivity of the context in which it is located, and the consistency of the candidate entity attributes in the knowledge graph, the original semantic conflict score is multi-dimensionally weighted and integrated to obtain the final conflict risk score; If the final conflict risk score is greater than the preset conflict judgment threshold, it is determined that there is a semantic conflict between the candidate entities; If the final conflict risk score is less than or equal to the preset conflict judgment threshold, it is determined that there is no semantic conflict between the candidate entities.

7. The data intelligent processing system based on AI big model according to claim 6 is characterized in that: The process of reasoning about causality, temporality, and attribute dependency between existing candidate entities is as follows: Build an entity relationship graph containing candidate entities, and add context connection edges and knowledge dependency edges between candidate entities in the entity relationship graph; Utilize graph neural networks to propagate semantic information between candidate entities, aggregate information, and perform semantic updates on each candidate entity node. Based on the connection strength, path length and node attributes between candidate entities, it is determined whether there is a causal relationship, time sequence or attribute dependency between candidate entities.

8. The data intelligent processing system based on AI big model according to claim 7 is characterized in that: The process of determining whether the relationship between candidate entities should be merged or split is as follows: Based on the constructed entity relationship graph, multiple rounds of graph neural network information propagation are performed on the candidate entity nodes to obtain the context-enhanced semantic representation of each candidate entity; Determine the pairing relationship between candidate entities through co-occurrence in the same context, semantic clustering results, or proximity in the semantic embedding space to form a set of candidate entity pairs; The similarity between the context-enhanced semantic representations of each pair of candidate entities is calculated, and the relationship reasoning analysis is performed based on the path relationship and attribute connection between entities in the graph structure. If the analysis results show that the candidate entities have a strong dependency relationship and the semantic similarity is higher than the candidate entity merging threshold, the candidate entity merging operation is performed; If the analysis results show that the semantic differences of the candidate entities are significant and the independence is higher than the candidate entity splitting threshold, the candidate entity splitting operation is performed.

9. The data intelligent processing system based on AI big model according to claim 8 is characterized in that: The process of outputting the standardized semantic results after disambiguation is: Generate a mapping table between candidate entities and standard entity labels based on the results of semantic analysis, modeling, verification, and reasoning. Replace the ambiguous candidate entity names in the original text with the corresponding standard entity labels; Output disambiguated structured semantic text.

Images