A computer system-based data expansion and interaction method

Abstract

This invention relates to the field of data processing technology, and more particularly to a data expansion and interaction method based on a computer system. It includes: extracting raw multidimensional data from multimodal interactive input; normalizing the raw multidimensional data to obtain normalized data; performing semantic preprocessing and entity recognition on the normalized data to obtain a set of semantic relation triples; constructing a user local semantic graph based on the set of semantic relation triples, and generating expanded data by associating and matching it with a built-in global semantic graph; performing semantic unification on the expanded data, then performing semantic conflict identification and fusion processing to generate a semantic fusion graph structure; introducing a local caching mechanism, and using a graph neural network to perform information propagation and semantic clustering on the expanded data nodes; and enabling user interaction through a visualized graph. This method solves the technical problems of inaccurate processing and analysis of data from different sources.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data expansion and interaction method based on a computer system. Background Technology

[0002] With the rapid development of artificial intelligence technology, especially the advancements in natural language processing, computer vision, and speech recognition, more and more systems are able to process data from different modalities, such as text, speech, and image data. However, most current data processing systems face a series of technical challenges when handling multimodal data. Different modalities of data have different representations and structures; text data typically relies on natural language processing, image data on computer vision, and speech data on speech recognition. The semantic information between these modalities is not easily directly integrated and fused, thus requiring a unified semantic framework to integrate them for effective analysis and processing.

[0003] In the context of multimodal data processing, traditional semantic expansion methods typically rely on rule or keyword matching, which falls short when faced with complex user intent and contextual information. Although existing technologies can perform semantic expansion to some extent through keywords or rules, these methods cannot accurately capture the deep relationships between semantics, nor can they handle complex interactions between multiple semantic levels.

[0004] At the same time, the existing technologies mentioned above also have technical problems such as inaccurate processing and analysis of data from different sources. Summary of the Invention

[0005] This invention provides a data expansion and interaction method based on a computer system to solve the technical problems of inaccurate processing and analysis of data from different sources.

[0006] The present invention provides a data expansion and interaction method based on a computer system, specifically including the following technical solutions:

[0007] A data expansion and interaction method based on a computer system includes the following steps:

[0008] S1. Extract raw multidimensional data from the multimodal interactive input sent by the user, perform normalization processing on the raw multidimensional data to obtain normalized data; perform semantic preprocessing and entity recognition on the normalized data to obtain a set of semantic relation triples; construct a user local semantic graph based on the set of semantic relation triples, and generate extended data by associating and matching with the built-in global semantic graph;

[0009] S2. After unifying the semantics of the extended data, semantic conflict identification and fusion processing are performed to generate a semantic fusion graph structure. Based on the semantic fusion graph structure, a local caching mechanism is introduced, and a graph neural network is used to perform information propagation and semantic clustering of the extended data nodes. User interaction is performed through a visualized graph.

[0010] Preferably, S1 specifically includes:

[0011] A multi-relation path enhancement matching algorithm is introduced to associate and match the user's local semantic graph with the built-in global semantic graph.

[0012] Preferably, S1 specifically includes:

[0013] In the implementation of the multi-relation path enhancement matching algorithm, starting from the user's local semantic graph, the vector representation of any local node in the pre-trained semantic vector space is calculated.

[0014] Preferably, S1 specifically includes:

[0015] In the implementation of the multi-relation path enhancement matching algorithm, starting from any local node, all reachable paths from it to the target node in the global semantic graph are traversed. Based on the modeling requirements of the influence of paths on semantic associations in the global semantic graph, graph structure theory, natural language semantic embedding and path decay mechanism are integrated to define the path enhancement representation.

[0016] Preferably, S1 specifically includes:

[0017] In the implementation of the multi-path similarity enhancement matching algorithm, the cumulative score of multi-path similarity is calculated as the matching degree; after normalizing the final matching degree, it is determined whether the normalized final matching degree exceeds the global node based on the threshold preset by the expert experience method.

[0018] Preferably, S1 specifically includes:

[0019] In the implementation of the multi-relation path enhancement matching algorithm, new extended triples are constructed based on the attribute structure, superordinate class label and historical context of the nodes matched from the global semantic graph. An extended graph is obtained based on the new extended triples. At the same time, the extended data and the local semantic graph are merged into an enhanced semantic graph structure as extended data.

[0020] Preferably, S2 specifically includes:

[0021] The extended data undergoes unified semantic interpretation and format normalization. During the operation, each semantic node in the extended data is standardized and mapped, and near-synonymous expressions are merged and classified. The action relation semantics in the predicate are decomposed into verb phrases and reassigned to standard semantic labels according to syntactic roles, resulting in a set of semantically unified nodes.

[0022] Preferably, S2 specifically includes:

[0023] After completing the semantic unification process, the semantic conflict identification and fusion stage begins. Semantic conflict weights are introduced into the semantic unification node set to perform attribute difference analysis on all nodes with the same label, calculate the attribute difference degree, and compare the attribute difference degree with a preset threshold. When the attribute difference degree is lower than the preset threshold, a fusion rule mechanism is introduced to fuse the semantic data in the node.

[0024] Preferably, S2 specifically includes:

[0025] After semantic unification and conflict fusion processing are completed, a semantic fusion graph structure is obtained. Based on the semantic fusion graph structure, a local caching mechanism is introduced to determine the set of semantic hot paths. After the local caching mechanism is completed and used for path optimization, the semantic fusion graph structure is fed into a graph neural network to cluster the structured semantic nodes generated in the semantic fusion graph structure.

[0026] The beneficial effects of the technical solution of the present invention are:

[0027] 1. By associating and matching the user's local semantic graph with the built-in global semantic graph using a multi-relational path enhancement matching algorithm, the system can dynamically expand data based on the user's input local semantic graph. The data expansion process is not limited to simple matching; it also considers factors such as hierarchical relationships and semantic similarity between nodes in the graph through path modeling, generating expanded data that highly matches the user's needs. The path enhancement representation and the calculation of the final matching degree make the expanded data more accurate and richer, providing higher precision in semantic depth and contextual consistency.

[0028] 2. During the data expansion process, the precise identification and fusion processing of redundant nodes and semantic conflicts effectively avoids semantic errors caused by data duplication or attribute conflicts; semantically redundant nodes are removed, and for semantic nodes with different contextual backgrounds, a context-driven strategy is adopted for fusion, retaining the most representative and relevant nodes, thus avoiding information loss or erroneous merging during the node fusion process. Attached Figure Description

[0029] Figure 1 This is a flowchart of a data expansion and interaction method based on a computer system according to the present invention. Detailed Implementation

[0030] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0032] The following description, in conjunction with the accompanying drawings, details a specific scheme for a data expansion and interaction method based on a computer system provided by the present invention.

[0033] See attached document Figure 1 The diagram illustrates a flowchart of a data expansion and interaction method based on a computer system according to an embodiment of the present invention. The method includes the following steps:

[0034] S1. Extract raw multidimensional data from the multimodal interactive input sent by the user, perform normalization processing on the raw multidimensional data to obtain normalized data; perform semantic preprocessing and entity recognition on the normalized data to obtain a set of semantic relation triples; construct a user local semantic graph based on the set of semantic relation triples, and generate extended data by associating and matching with the built-in global semantic graph;

[0035] First, raw multidimensional data is extracted from the multimodal interactive inputs from the user. These inputs include text input (natural language), voice commands (transcribed via ASR speech recognition), and image recognition results (OCR or object recognition). The raw multidimensional data is then standardized using existing techniques well-known to those skilled in the art. The standardization process is as follows: voice commands are converted to text via an ASR (Automatic Speech Recognition) system; image recognition results are used to extract text via OCR, or objects in the image are identified and described based on a CV model, such as "This displays a flowchart with 4 modules," resulting in standardized data. This standardized data is then subjected to semantic preprocessing and entity recognition using existing techniques well-known to those skilled in the art, including word segmentation and part-of-speech tagging, using methods such as LTP or Spacy; named entity recognition (NER) to identify key domain entities such as "device name," "operation verb," ​​and "parameter name"; and dependency parsing to identify the subordinate relationships between entities, such as "read" being the root verb and "parameter X" being the object in "read the value of parameter X." After the above semantic preprocessing and entity recognition, a set of semantic relation triples is obtained:

[0036] S sem ={(e i r ij e j )}

[0037] Among them, S sem It is a set of semantic relation triples, representing structured semantic relation expressions between key semantic entities extracted from multimodal interactive input from users. Essentially, it is a set of basic semantic units used to construct local semantic graphs; e i It is the i-th semantic entity, representing the object, operation, concept, parameter, or role appearing in the user input; e j It is the j-th semantic entity, representing e. i The object that is affected or associated with; r ij From semantic entity e i Pointing to semantic entity e j Dependency relationships, such as "belonging to," "controlling," "depending on," "operating," and "causally," are e i e j ∈ Semantic entity set, r ij ∈ Dependency type.

[0038] Furthermore, a user local semantic graph G is constructed based on the extracted set of semantic relation triples. localEach semantic entity is treated as a local node, and dependencies are treated as edges; simultaneously, the user's local semantic graph is compared with the existing built-in global semantic graph G. global It includes concepts, attributes, operations, domain terms, and their semantic edges, such as hierarchical relationships, similarity relationships, causal chains, etc., and performs association matching through a multi-relational path enhancement matching algorithm; the association matching process is as follows: from the user's local semantic graph G local Starting with the user's local semantic graph G local Each semantic entity node in the data performs the following operations:

[0039] The first step is to compute the vector representation of any local node in the existing pre-trained semantic vector space (such as BERT-CLS embedding);

[0040] The second step is to use any local node n i Starting from this point, iterates through the target nodes in the global semantic graph. set of all reachable paths Based on the need to model the impact of paths on semantic associations in the global semantic graph, this paper integrates graph structure theory, natural language semantic embedding, and path decay mechanisms to define a path enhancement representation:

[0041]

[0042] Among them, E p It is the enhanced semantic representation vector of path p; p is the set of paths. Any element of the graph; L is the number of hops in the current path p, which is the path length calculated by existing graph traversal algorithms (such as DFS, BFS); The edge type r is the edge connected to the l-th hop in the path. l The importance weight for overall semantic matching is determined based on expert experience, and its value range is (0, 1]; r l The type of relation edge used in the l-th hop of the path, such as "is-a", "cause-of", "part-of", etc., belongs to the set R of relation edge types in the global semantic graph; v l It is the intermediate node traversed by the l-th hop in the path; Enc(·) is the intermediate node v of the l-th hop in the path. l Semantic vector representations, such as those generated by models like BERT, RoBERTa, and TransE; It is a path length suppression factor, used to reduce the contribution of long paths to the overall semantics;

[0043] The third step is to calculate the cumulative score of multi-path similarity as the matching degree;

[0044] For each node Its set of all reachable paths is Define the final match degree as:

[0045]

[0046] in, It is the final matching degree; γ p This is the path confidence factor, which assigns weights to the importance of each path to control its impact on the final matching degree. It is determined by the path information entropy and the degree of the target node; Enc(n i ) is a local node n i The semantic vector representation; ||·|| is the modulo operation;

[0047] The fourth step is to normalize the final matching degree and then determine whether the normalized final matching degree exceeds the global node based on the threshold preset by the expert experience method.

[0048] If the final matching degree after normalization exceeds the global node with a threshold preset according to the expert experience method (such as 0.85), it is treated as an extended node, and the original edge relationship is preserved.

[0049] For example, "parameter X" matches semantic neighbors such as "current threshold" and "upper limit of ambient temperature" in the global semantic graph; "read" matches operation-type nodes such as "extract," "call," and "retrieve." The matched content is then extended to the local semantic graph G. local In the nodes.

[0050] Furthermore, new extended triples are constructed from the nodes matched from the global semantic graph based on their attribute structure, hyperclass labels, and historical context, and the extended graph G is obtained based on the new extended triples. expand Simultaneously, the extended data and local semantic graphs are merged into an enhanced semantic graph structure G. augmented =G local ∪G expand That is, the final enhanced semantic graph structure G. augmented This will be used as extended data.

[0051] S2. After unifying the semantics of the extended data, semantic conflict identification and fusion processing are performed to generate a semantic fusion graph structure. Based on the semantic fusion graph structure, a local caching mechanism is introduced, and a graph neural network is used to perform information propagation and semantic clustering of the extended data nodes. User interaction is performed through a visualized graph.

[0052] First, the extended data undergoes unified semantic interpretation and format normalization to ensure that attribute descriptions, predicate labels, naming rules, and data granularity of nodes from different sources have consistent expressive capabilities within the same semantic space. This provides an operational basis for subsequent conflict detection and fusion. Specifically, each semantic node in the extended data is standardized and mapped using existing semantic alignment algorithms. Existing nested entity renaming strategies are employed to merge and classify near-semantic expressions. The action relationship semantics in predicates are decomposed into verb phrases and reassigned to standard semantic labels based on their syntactic roles, forming a unified (subject-predicate-object) expression element. This results in a semantically unified node set, achieving unified semantic processing.

[0053] After semantic unification processing is completed, the semantic conflict identification and fusion stage begins. The technical objective of this stage is to identify and eliminate semantically redundant nodes, contradictory entities, or attribute-biased nodes in the extended data. This avoids duplicate data consuming graph structure storage resources, affecting subsequent reasoning efficiency, or causing semantic errors. The specific process includes: firstly, introducing semantic conflict weights into the semantically unified node set to perform attribute difference analysis on all nodes with the same label, examining whether the naming origin, semantic path length, and concept membership chain of the nodes are consistent. Attribute dissimilarity is calculated using expert experience and compared with a preset threshold. If the attribute dissimilarity exceeds the threshold, nodes with low dissimilarity are retained, while nodes with high dissimilarity are deleted. If the dissimilarity is below the threshold, semantic data within the nodes is fused. A fusion rule mechanism is introduced, employing a context-driven strategy: nodes with denser adjacency and higher historical reference frequency are designated as master nodes, while other nodes are added to the master node structure as attributes. Simultaneously, existing graph edges are replaced and redirected to ensure the structure remains connected. If two nodes are semantically similar but have completely different context paths and represent logical conflicts (e.g., one representing "upper limit voltage" and the other "actual voltage"), the context conflict judgment engine retains both nodes but creates a "semantic conflict edge" in the graph structure, assigning a priority marker for these edges for processing during subsequent graph neural network propagation.

[0054] After the semantic unification and conflict fusion processing are completed, a semantic fusion graph structure with a unified semantic structure, redundant node removal, and contextual consistency is obtained. This semantic fusion graph structure will serve as the backbone carrier for the entire data propagation and interactive response. Based on the semantic fusion graph structure, a local caching mechanism is introduced. The technical purpose is to reduce the response latency caused by repeated calculations of high-frequency semantic paths in the graph model, thereby improving the response speed and smoothness of user requests. By recording the user's historical interaction behavior trajectory, including each clicked node, query path, feedback evaluation, etc., interaction frequency statistics are performed. Combined with the semantic density and response frequency of each node in the semantic fusion graph structure in the user's current query window, a "semantic popularity path set" is determined. The top K paths in the semantic popularity path set are cached and encapsulated, and all nodes contained in the path and their contextual relationships are serialized and stored in the cache graph pool in a graph structure manner. Each time a new user query request is initiated, the request is first quickly matched to determine whether it hits a path in the cache. If it does, the extended data is returned directly from the cache, skipping the semantic extension and graph construction process, thereby significantly reducing processing time. If it does not hit, the normal process is executed and the cache strategy is updated.

[0055] After the local caching mechanism is completed and can be used for path optimization, the semantic fusion graph structure is fed into a graph neural network for node information propagation and semantic region partitioning. The technical objective is to cluster the structured semantic nodes generated in the semantic fusion graph structure, allowing semantically similar nodes to aggregate into subgraph regions logically belonging to the same interaction block, further improving the interaction clarity and semantic organization of the graph. In practice, the semantic fusion graph structure is loaded as the input model of the graph neural network. Each node in the semantic fusion graph structure is accompanied by a multi-dimensional feature vector, including semantic label embedding, entity type, context path density, and ontology category. A hierarchical graph propagation mechanism is introduced, propagating the semantic information of each node to its neighboring nodes, accumulating contextual features during propagation. This ensures that after each round of propagation, the feature vector of each node incorporates information from its surrounding nodes, thereby achieving semantic consensus updates. After propagation through a predetermined number of rounds based on expert experience, semantic distance calculation is performed on the final node features, and semantically similar nodes are automatically grouped into a semantic cluster group. Based on this, an independent subgraph is generated for each semantic cluster group, and different colors, edge styles or layer labels are assigned to the graph to facilitate subsequent visualization.

[0056] Finally, the semantically clustered graph structure is visualized and rendered for user interaction.

[0057] In summary, a data expansion and interaction method based on a computer system has been developed.

[0058] The order of the embodiments is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0059] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0060] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A data expansion and interaction method based on a computer system, characterized in that, Includes the following steps: S1. Extract raw multidimensional data from the multimodal interactive input from the user, and normalize the raw multidimensional data to obtain normalized data; perform semantic preprocessing and entity recognition on the normalized data to obtain a set of semantic relation triples; construct a user local semantic graph based on the set of semantic relation triples, introduce a multi-relation path enhancement matching algorithm, and perform association matching between the user local semantic graph and the built-in global semantic graph; starting from the user local semantic graph, calculate the vector representation of any local node in the pre-trained semantic vector space; starting from any local node, traverse all reachable paths from it to the target node in the global semantic graph, and based on the modeling requirements of the influence of paths on semantic association in the global semantic graph, integrate graph structure theory, natural language semantic embedding and path decay mechanism to define path enhancement representation; Calculate the cumulative score of multi-path similarity as the matching degree; After normalizing the final matching degree, determine whether the normalized final matching degree exceeds the global node with a threshold preset according to the expert experience method; The nodes matched from the global semantic graph are used to construct new extended triples based on their attribute structure, superordinate class labels, and historical context. An extended graph is obtained based on the new extended triples. At the same time, the extended data is merged with the local semantic graph to form an enhanced semantic graph structure, which is used as the extended data. S2. After unifying the semantics of the extended data, semantic conflict identification and fusion processing are performed to generate a semantic fusion graph structure. Based on the semantic fusion graph structure, a local caching mechanism is introduced, and a graph neural network is used to perform information propagation and semantic clustering of the extended data nodes. User interaction is performed through a visualized graph.

2. The data expansion and interaction method based on a computer system according to claim 1, characterized in that, S2 specifically includes: The extended data undergoes unified semantic interpretation and format normalization. During the operation, each semantic node in the extended data is standardized and mapped, and near-synonymous expressions are merged and classified. The action relation semantics in the predicate are decomposed into verb phrases and reassigned to standard semantic labels according to syntactic roles, resulting in a set of semantically unified nodes.

3. The data expansion and interaction method based on a computer system according to claim 2, characterized in that, S2 specifically includes: After completing the semantic unification process, the semantic conflict identification and fusion stage begins. Semantic conflict weights are introduced into the semantic unification node set to perform attribute difference analysis on all nodes with the same label, calculate the attribute difference degree, and compare the attribute difference degree with a preset threshold. When the attribute difference degree is lower than the preset threshold, a fusion rule mechanism is introduced to fuse the semantic data in the node.

4. The data expansion and interaction method based on a computer system according to claim 3, characterized in that, S2 specifically includes: After semantic unification and conflict fusion processing are completed, a semantic fusion graph structure is obtained. Based on the semantic fusion graph structure, a local caching mechanism is introduced to determine the set of semantic hot paths. After the local caching mechanism is completed and used for path optimization, the semantic fusion graph structure is fed into a graph neural network to cluster the structured semantic nodes generated in the semantic fusion graph structure.

Images