A multi-source cross-domain data query method and system

Abstract

The present invention belongs to the field of information retrieval technology, and discloses a multi-source cross-domain data query method and system. The method of the present invention includes the following steps: first, using a large language model to identify, extract, and classify data to construct a multi-source knowledge graph; second, based on the large language model, deep aggregation and reconstruction of multi-source cross-domain data are performed to obtain a homologous knowledge line graph; then, confidence calculation of the query node and the subgraph to which it belongs is performed according to the query requirements, low-quality nodes are eliminated, and credible candidate subgraphs are extracted from the knowledge graph to ensure that the multi-domain retrieval query results are valid; finally, the screened credible candidate subgraphs are embedded in the prompt word template, and data query is performed using a vertical large model. The present invention is based on a multi-source knowledge fusion and query method based on a large language model, which integrates cross-domain data from different sources to form domain knowledge, and uses a vertical large model to achieve accurate and efficient query of multi-source cross-domain data.

Description

Technical Field

[0001] The present invention belongs to the technical field of information retrieval, and relates to a multi-source cross-domain data query method and system. Background Art

[0002] With the rapid development of information technology, the digitalization process across industries is accelerating, leading to an explosive growth in data. Processing multi-source, cross-domain data has become a key challenge. Multi-source, cross-domain data originates from diverse industries and fields, and its data models, structures, semantics, and quality standards vary widely. This data comes from a wide range of sources, has varying quality, is decentralized, and is heterogeneous. While research on multi-source, cross-domain data has progressed, most rely on expert engines and rule-based systems, or are modeled as semantic matching problems. These approaches rely on large amounts of labeled data or expert knowledge, resulting in significant limitations. Faced with diverse data standards, multimodal and multi-layered data formats, and the demand for massive amounts of high-quality data across numerous industries, traditional approaches struggle to meet current data governance requirements. Processing multimodal data is particularly crucial when training large, vertical language models. Complex dependencies between multi-source, cross-domain data can reduce the model's inference performance. Furthermore, data security, privacy, and normalization are pressing challenges, severely hindering the full realization of data value. Furthermore, large language models, with their powerful natural language processing capabilities, can extract valuable insights from heterogeneous data sources, promoting effective data utilization and aiding decision-making and business optimization. As a key achievement of knowledge engineering in the big data era, knowledge graphs provide an effective approach for the knowledge-based organization and intelligent application of multi-source, cross-domain data. They can process complex, unstructured, cross-domain, and multimodal data, generating high-quality structured data graphs. They alleviate problems such as hallucinations, catastrophic forgetting, and incorrect knowledge editing in large language models, enhance data standardization and interpretability, broaden application scenarios, and improve governance efficiency. Summary of the Invention

[0003] The purpose of the present invention is to propose a multi-source cross-domain data query method, which proposes a multi-source knowledge fusion and query method based on a large language model, integrates cross-domain data from different sources to form domain knowledge, and thus uses a vertical large model to achieve accurate and efficient query of multi-source cross-domain data.

[0004] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions:

[0005] A multi-source cross-domain data query method includes the following steps:

[0006] Step 1. Retrieve multi-source, cross-domain data from different sources and storage formats within the domain, and use a large language model to identify, extract, and classify the data to build a multi-source knowledge graph.

[0007] Step 2. Reconstruct and aggregate the multi-source knowledge graph, merging and reorganizing nodes with similar semantics scattered across the multi-source knowledge graph to obtain a homologous knowledge line graph;

[0008] Step 3. Based on the query requirements, use the large language model to score the confidence of the query node in the homologous knowledge line graph, calculate the source credibility threshold of the corresponding homologous linear subgraph, and screen the credible candidate subgraphs related to the query;

[0009] Step 4. Embed the core content and key related information of the selected credible candidate subgraphs into the prompt word template, and then use the vertical large model to filter out data results that highly match the query intent as the data query results.

[0010] In addition, based on the above multi-source cross-domain data query method, the present invention also proposes a corresponding multi-source cross-domain data query system, which adopts the following technical solutions:

[0011] A multi-source cross-domain data query system includes the following modules:

[0012] A multi-source knowledge graph construction module is used to retrieve multi-source cross-domain data from different sources and storage formats in the field, and use a large language model to identify, extract, and classify data to build a multi-source knowledge graph;

[0013] The homologous knowledge line graph construction module is used to reconstruct and aggregate the obtained homologous knowledge line graph, merge and reorganize the nodes with similar semantics scattered in the multi-source knowledge graph to obtain the homologous knowledge line graph;

[0014] The trusted candidate subgraph screening module is used to use the large language model to perform confidence scoring of query nodes in the homologous knowledge line graph, calculate the source credibility threshold of the corresponding homologous linear subgraph, and screen the trusted candidate subgraphs related to the query;

[0015] And a data query module, which is used to embed the core content and key related information of the screened credible candidate subgraphs into the prompt word template, and then use the vertical large model to screen out data results that are highly matched with the query intent.

[0016] The present invention has the following advantages:

[0017] As described above, the present invention describes a multi-source cross-domain data query method and system based on the collaboration of domain knowledge and vertical large models. In terms of data processing efficiency, the present invention proposes a homologous knowledge line graph structure, which greatly optimizes the data storage and query architecture. Compared with the traditional method that relies on expert engines and rule systems and requires a large amount of labeled data, it can locate and obtain the required data information more quickly, significantly improve the overall efficiency of data processing, and reduce processing time costs. In terms of data accuracy, the present invention integrates multi-dimensional factors to calculate the confidence of entity nodes and eliminates untrustworthy nodes, effectively eliminating the interference of errors and low-quality data in the data query and screening process, thereby providing more accurate and reliable candidate subgraphs and data results. In dealing with complex data relationships, the present invention makes full use of the advanced capabilities of large language models and vertical large models to effectively handle the complex dependencies between multi-source cross-domain data, overcome the problem that the existing technology causes loss of large language model reasoning performance when processing such data, ensure the stability and reliability of data processing, and make it applicable to a wider range of and more complex data scenarios. The method of the present invention improves the query performance and speed of multi-source cross-domain data by utilizing a large language model combined with knowledge graph optimization construction. At the same time, by utilizing a vertical large model, it can perform accurate and efficient data queries on data in specific fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the process of multi-source cross-domain data query method according to an embodiment of the present invention;

[0019] Figure 2 Schematic diagram of the transformation of a linear knowledge graph in a multi-source cross-domain data query method according to an embodiment of the present invention;

[0020] Figure 3 A flowchart for constructing a homologous knowledge line graph according to an embodiment of the present invention;

[0021] Figure 4 Schematic diagram of the process of a trusted candidate subgraph matching method in an embodiment of the present invention;

[0022] Figure 5 Schematic diagram of the process of multi-source cross-domain data query method in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0024] Example 1

[0025] This embodiment 1 describes a multi-source cross-domain data query method, which uses a large language model to deeply aggregate multi-source cross-domain data in a domain to reconstruct a dense homologous knowledge line graph; and performs credibility analysis and screening on the obtained homologous knowledge line graph to match and obtain credible candidate subgraphs; finally, the screened credible candidate subgraphs are returned to the vertical large model, thereby realizing efficient and accurate data query functions.

[0026] like Figure 1 As shown, this embodiment describes a multi-source cross-domain data query method based on the collaboration of domain knowledge and vertical large models, which uses multi-source cross-domain knowledge fusion and vertical large models to query multi-source cross-domain data. The steps are as follows:

[0027] Step 1. Retrieve multi-source cross-domain data from different sources and storage formats in the domain, and use a large language model to identify, extract, and classify the data to build a multi-source knowledge graph.

[0028] Specifically, we first acquire multi-source cross-domain data from different data sources, integrate the data, and construct a knowledge graph. The multi-source cross-domain data types include structured tables, semi-structured XML documents, and unstructured text.

[0029] Use large language models to extract entities and relationships, and leverage the general semantic understanding and analysis capabilities of large language models to deeply identify key information, sort out entity relationships, and obtain entities and relationships between entities in multi-source cross-domain data.

[0030] Use a unified data storage method to initially integrate multi-source, cross-domain data. First, design a data conversion parser for different data formats to integrate multi-source, cross-domain data and unify its storage format. For practical application scenarios, directly acquire data from multiple data formats with different sources and convert it into a unified, standardized representation.

[0031] Specifically, the LLMGraphTransformer library in the Python programming language is used to extract entities and relationships from multi-source knowledge graphs. With its powerful natural language processing capabilities, the library can perform in-depth semantic analysis on data from different data sources, accurately identify key elements such as entity information, such as people, places, events, and things, and simultaneously extract the relationships between them, such as causal relationships, affiliation, and association relationships. After that, the file name and file metadata of the data are collected, and the domain where the file is located is divided. The data content is then parsed and stored in the JSON-LD format and converted into linked data. Finally, the present invention assigns a unique identifier to the data to form standardized data.

[0032] In step 1, the process of preliminary fusion of multi-source cross-domain data is as follows:

[0033] Different data conversion parsers are used to perform data analysis on each different data type, and the analysis of structured data, semi-structured data and unstructured data is distinguished during data analysis.

[0034] For structured data, parsing is equivalent to storing tabular data in JSON format. The attribute variables in the file are stored using a columnar storage model (DSM), so that all attribute information can be extracted using column indexes for consistency verification. For semi-structured data, parsing is equivalent to storing tree data in JSON format. The file is stored in a multi-layer nested JSON format. This format does not have columnar indexes and fast retrieval, and tree / graph retrieval algorithms (such as DFS) are used for retrieval. For unstructured data, this unstructured data is text data, and the content is stored directly. Later, LLM is considered for entity / relationship extraction tasks to obtain the corresponding information. Ultimately, the fusion of multi-source cross-domain data is expressed as the following formula:

[0035] .

[0036] in, Represents the fused multi-source cross-domain data, Represents data conversion parsers for structured data, semi-structured data, and unstructured data, respectively. Data sets representing structured data, semi-structured data, and unstructured data respectively.

[0037] The present invention is based on a multi-source cross-domain data extraction and storage method to obtain a multi-source knowledge graph architecture, which can provide basic support for the subsequent optimization of step 2 and the query of step 4.

[0038] Step 2. Reconstruct and aggregate the multi-source knowledge graph obtained in step 1, merge and reorganize the nodes with similar semantics scattered in the multi-source knowledge graph to achieve a denser storage structure and improve query efficiency. After reconstructing and aggregating the multi-source knowledge graph, a dense homologous knowledge graph is obtained, which is defined as a homologous knowledge line graph, such as Figure 2 shown.

[0039] Specifically, first, the entities and relationships extracted in step 1 are used to match and align the same or similar entities in different data sources through a text similarity model, and the relationships between entities are sorted and fused to obtain an aggregated homologous data group; then, a homologous knowledge line graph is constructed based on the homologous data group, where each node in the homologous knowledge line graph represents a triple, and the associated edges represent the relationship between triples.

[0040] Based on the multi-source knowledge graph initially constructed in step 1, it is necessary to traverse all graphs to obtain all cross-domain homologous data group sets SVs and all isolated point sets , and finally complete the aggregation and reconstruction of the knowledge graph.

[0041] like Figure 3 As shown in the figure, the construction process of the homology knowledge line graph is as follows:

[0042] Step 2.1. Initialize the unvisited node set, homologous data set, and isolated point set, including initializing the unvisited node set , homologous data set SVs , isolated point set .

[0043] This initialization process lays the foundation for subsequent traversal and data grouping.

[0044] Step 2.2. Traverse each node in the unvisited node set. For the currently traversed node A, initialize the variables used to store the set of homologous nodes, the set of edges, and related variables of the homologous nodes.

[0045] Traverse the other nodes B in the unvisited node set except the current node A again. When it is determined that the current node A and other nodes B meet the homology relationship, perform the following operations:

[0046] Add the current node A and other nodes B that satisfy the homology relationship to the homology node set respectively and edge sets ; and remove the current node A and other nodes B from the unvisited node set to avoid repeated access and processing.

[0047] Step 2.3. After determining the homologous nodes and edges of a node, make the following judgment:

[0048] If the same source node set If it is an empty set, add the current node A to the isolated point set In the data set, it is managed separately as a special data point independent of the homologous data group;

[0049] If the same source node set If it is not an empty set, and the corresponding edge set As a homogenous data set , added to the homologous data set In order to conduct unified analysis and processing of data with homology relationships in the future.

[0050] Step 2.4. Repeat steps 2.2 and 2.3 until all nodes in the unvisited node set are traversed and processed, completing the construction process of the entire homology knowledge graph, and finally outputting the homology data set SVs and the isolated point set .

[0051] Step 2.5. For homogenous data sets Each homologous data set in , which contains the set of homologous nodes and homologous edge sets Constructing homologous linear subgraphs ;

[0052] All homologous linear subgraphs and isolated point sets Perform aggregation to obtain a homologous knowledge line graph .

[0053] It should be noted that the homology knowledge line diagram in the present invention It is only used for confidence calculation and retrieval query of homologous data. Queries for other types of data are still performed in the multi-source knowledge graph in step 1.

[0054] This paper aggregates data based on a text similarity model. This involves deeply analyzing nodes and edges in a knowledge graph to identify groups of nodes with similar semantic features and potential connections. Using text similarity calculations, these scattered, similar semantic nodes are merged and reorganized to form a denser storage model, ensuring efficient subsequent querying and filtering.

[0055] Step 3. Based on the query requirements, use the large language model to perform confidence scoring of query nodes in the homologous knowledge line graph, calculate the source credibility threshold of the corresponding homologous linear subgraph, and screen the credible candidate subgraphs related to the query.

[0056] Specifically, the query requirement is parsed into a search instruction, and the homologous knowledge line graph is traversed according to the confidence of the node to screen and match the query-related subgraphs. The obtained subgraph is called a trusted candidate subgraph; the confidence of the entity node is calculated by the mutual information entropy confidence calculation method, and the reliability of the node source, the node authority and the historical authority factors are comprehensively considered, and the probability statistical model and the large language model are used to calculate the score. The present invention introduces a confidence calculation method based on mutual information entropy to evaluate the confidence of the homologous line graph. The core idea of this method is that if two nodes with the same attributes in the homologous line graph are close to the same in content, their similarity is high, and thus the confidence is also high; otherwise, the confidence is low.

[0057] Define the homologous linear subgraph to which the current node belongs The corresponding confidence value is the source confidence threshold, which is calculated by The similarity of all node pairs in the source graph is determined by averaging the similarities. Untrusted nodes and their associated edges whose confidence is lower than the source confidence threshold need to be eliminated, and finally a trusted candidate subgraph is returned.

[0058] like Figure 4 As shown in the figure, the present invention designs a credible candidate subgraph matching algorithm to calculate the data source credibility of homologous subgraphs to ensure the quality of the knowledge graph embedded in the LLM. The specific implementation steps are as follows:

[0059] Step 3.1. Graph the homology knowledge lines The confidence of all data nodes, data authority, and historical credibility estimation of data nodes, The corresponding source confidence threshold and the trusted candidate subgraph T-sub are initialized;

[0060] Traversal , for each data node , initialize the variable N to be equal to the value corresponding to the current element The cardinality of the collection; for the current element Corresponding For each node in , execute steps 3.2 to 3.4.

[0061] Step 3.2. Compute homologous linear subgraphs The consistency score between two data nodes in .

[0062] set up for The node set in; for any two nodes with the same attribute , defining the similarity between them A calculation method based on mutual information entropy is used.

[0063] Mutual Information Entropy It measures the mutual dependence of the attribute contents of two nodes, and its formula is:

[0064] (1)

[0065] in, and Node and The attribute value set of is a node Get attribute value and nodes Get attribute value The joint probability distribution of and They are and The marginal probability distribution of .

[0066] Node similarity It is defined as the normalized form of mutual information entropy to ensure that its value is in the interval [0, 1]:

[0067] (2)

[0068] in and Node and The entropy of the attribute value set, the node is unified with The calculation formula is:

[0069] (3)

[0070] in, Representative Node and , represent and .

[0071] Next, calculate the node consistency score , which reflects the consistency of nodes in different data sources, and uses mutual information entropy to calculate the similarity between node pairs. , and its consistency score is expressed as:

[0072] (4)

[0073] in, Is with the node A collection of nodes with the same attributes. , is the similarity between nodes.

[0074] Finally, the source confidence threshold is calculated :

[0075] (5)

[0076] in, represents the number of nodes in the homologous linear subgraph, Represents the similarity between node pairs.

[0077] Step 3.3. Calculate the authority score of the node.

[0078] The authority score of the node Calculated by the following formula:

[0079] (6)

[0080] in, is a weight coefficient used to balance the contribution of LLM evaluation authority and historical authority, satisfying ; is the node authority score evaluated by LLM, The historical authority of the node.

[0081] Based on the global influence and local connection strength of the node Evaluation; LLM calculates the credibility of knowledge by integrating the strength of association between entities, entity type information, and multi-step path information:

[0082] (7)

[0083] in, LLM is the node Provided authoritative ratings, is a parameter that controls the steepness of the rating curve.

[0084] The historical authority of the node It is an authoritative score based on the node's historical data. The formula is as follows:

[0085] (8)

[0086] in, yes The number of entities provided for all historical queries, yes historical credibility, is the correct answer set, is the query-related data obtained from the homologous linear subgraph; Represents the entity node in the correct answer set, Indicates the authority score of the entity node.

[0087] Step 3.4. Add formula (4) and formula (6) to obtain the final confidence score of the node; if the final confidence score of the node is lower than the source confidence threshold, delete the node from the current homology subgraph.

[0088] Step 3.5. Repeat steps 3.2 to S3.4 until all elements in the data source are traversed, thus completing the matching process of the entire trusted candidate subgraph, and finally outputting the data source credibility and the trusted candidate subgraph T-sub.

[0089] The method of the present invention screens and matches credible candidate subgraphs according to the query requirements through the above step 3, eliminates low-quality nodes, and extracts credible candidate subgraphs from the knowledge graph, thereby ensuring the validity of the multi-domain search query results.

[0090] Step 4. Embed the core content and key related information of the selected credible candidate subgraphs into the prompt word template, and then use the vertical large model to filter out data results that highly match the query intent as the data query results.

[0091] Specifically, we first design a suitable prompt word template based on the input requirements of the vertical large model and the data characteristics of the query. Then, we embed the key information of the trusted candidate subgraph into the prompt word template, and call on the vertical large model to quickly filter out data results that highly match the query intent.

[0092] like Figure 5 As shown, the present invention integrates the first three steps and combines them with a vertical large model to perform accurate query of multi-source cross-domain data, including the following steps:

[0093] Step 4.1. Construct a prompt word containing a small number of reference examples, perform intent recognition and entity extraction tasks based on the query requirements, and thus obtain a query node set; initialize the query node set and the trusted answer set.

[0094] Step 4.2. According to step 2, obtain the homology knowledge line graph; according to step 3, calculate the confidence score of the query node and the source credibility threshold of the corresponding homology linear subgraph, and obtain the credible candidate subgraph.

[0095] Step 4.3. Store the source confidence threshold and the trusted candidate subgraph into the trusted answer set. Construct corresponding prompt words based on the query requirements and the trusted answer set, input them into the vertical large model to ask questions, and obtain the final query answer.

[0096] The method of the present invention achieves efficient fusion of multi-source cross-domain data with the assistance of a large language model, screens and matches trusted nodes for the homologous knowledge line graph obtained by fusion, obtains trusted candidate subgraphs and uses them for data retrieval queries of vertical large models, providing an effective way for multi-source cross-domain data retrieval queries.

[0097] Example 2

[0098] This embodiment 2 describes a multi-source cross-domain data query system, which is based on the same inventive concept as the multi-source cross-domain data query system in the above embodiment 1.

[0099] The multi-source cross-domain data query system in this embodiment includes the following modules:

[0100] A multi-source knowledge graph construction module is used to retrieve multi-source cross-domain data from different sources and storage formats in the field, and use a large language model to identify, extract, and classify data to build a multi-source knowledge graph;

[0101] The homologous knowledge line graph construction module is used to reconstruct and aggregate the obtained homologous knowledge line graph, merge and reorganize the nodes with similar semantics scattered in the multi-source knowledge graph to obtain the homologous knowledge line graph;

[0102] The trusted candidate subgraph screening module is used to use the large language model to perform confidence scoring of query nodes in the homologous knowledge line graph, calculate the source credibility threshold of the corresponding homologous linear subgraph, and screen the trusted candidate subgraphs related to the query;

[0103] And a data query module, which is used to embed the core content and key related information of the screened credible candidate subgraphs into the prompt word template, and then use the vertical large model to screen out data results that are highly matched with the query intent.

[0104] It should be noted that the implementation process of the functions and effects of each functional module in the multi-source cross-domain data query system is detailed in the implementation process of the corresponding steps in the multi-source cross-domain data query method in the above embodiment 1, and will not be repeated here.

[0105] Example 3

[0106] This embodiment 3 describes a computer device comprising a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it implements the multi-source cross-domain data query method described above. In this embodiment, the computer device is any device or apparatus capable of data processing, and will not be further described here.

[0107] Example 4

[0108] This embodiment 4 describes a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, it is used to implement the above-mentioned multi-source cross-domain data query method.

[0109] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc. equipped on the device.

[0110] Of course, the above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above-mentioned embodiments. It should be noted that all equivalent substitutions and obvious deformation forms made by any technician familiar with this field under the guidance of this specification fall within the substantive scope of this specification and should be protected by the present invention.

Claims

1. A multi-source cross-domain data query method, characterized in that: The steps include: Step 1: Retrieve multi-source cross-domain data from different sources and storage formats in the domain, and use a large language model to identify, extract, and classify the data to build a multi-source knowledge graph; Multi-source cross-domain data types include structured tables, semi-structured XML documents, and unstructured text; Use different data conversion parsers to parse each different data type; Step 2. Reconstruct and aggregate the multi-source knowledge graph, merge and reorganize the nodes with similar semantics scattered in the multi-source knowledge graph, and obtain a homologous knowledge line graph; The entities and relationships extracted using the large language model in step 1 are used to match and align the same or similar entities in different data sources through a text similarity model, and to sort out and fuse the relationships between entities to obtain an aggregated homologous data set. Based on the homologous data set, a homologous knowledge line graph is constructed. The process is as follows: Step 2.

1. Initialize the unvisited node set, the homologous data set, and the isolated point set, including initializing the unvisited node set unvisited = {V}, the homologous data set Isolated point set Step 2.

2. Traverse each node in the unvisited node set. For the currently traversed node A, initialize the set of nodes and edges, and the variables related to the nodes. Traverse the other nodes B in the unvisited node set except node A again. If it is determined that node A and other nodes B have a homology relationship, perform the following operations: Add the current node A and other nodes B that satisfy the homology relationship to the homology node set U sg and edge set E sg ; And remove the current node A and other nodes B from the set of unvisited nodes to avoid repeated access and processing; Step 2.

3. After determining the homologous nodes and edge sets of a node, make the following judgment: If the homologous node set U sg If it is an empty set, add the current node A to the isolated point set In the data set, it is managed separately as a special data point independent of the homologous data group; If the homologous node set U sg If it is not an empty set, then the homologous node set U sg And the corresponding edge set E sg The whole is taken as a homologous data group SV and added to the homologous data set SVs; Step 2.

4. Repeat steps 2.2 and 2.3 until all nodes in the unvisited node set are traversed and processed, completing the construction process of the entire homologous knowledge graph, and finally outputting the homologous data set SVs and the isolated point set Step 2.

5. For each homologous data set SV in the homologous data set SVs i , which contains the homologous node set U sg and homologous edge set E sg Constructing homologous linear subgraphs All homologous linear subgraphs and isolated point sets Perform aggregation to obtain a homologous knowledge line graph Step 3. Based on the query requirements, use the large language model to score the confidence of the query node in the homologous knowledge line graph, calculate the source credibility threshold of the corresponding homologous linear subgraph, and screen the credible candidate subgraphs related to the query; The confidence of the entity node is calculated by the mutual information entropy confidence calculation method, taking into account the reliability of the node source, the node authority and the historical authority factors, and using the probability statistical model and the large language model to calculate the score; Step 4. Embed the core content and key related information of the screened credible candidate subgraphs into the prompt word template, and then use the vertical large model to screen out data results that highly match the query intent as the data query results.

2. The multi-source cross-domain data query method according to claim 1, characterized in that: The step 1 is specifically as follows: First, multi-source cross-domain data is obtained from different data sources and storage media; the multi-source cross-domain data types include structured tables, semi-structured XML documents, and unstructured text; Use a large language model to extract entities and relationships to obtain entities and relationships between entities in multi-source cross-domain data; use a unified data storage method to preliminarily integrate multi-source cross-domain data to obtain a multi-source knowledge graph.

3. The multi-source cross-domain data query method according to claim 2, characterized in that: In step 1, the process of preliminary fusion of multi-source cross-domain data is as follows: Different data conversion parsers are used to parse each different data type, namely structured data, semi-structured data and unstructured data; For structured data, its parsing is equivalent to storing table data in JSON format, and the attribute variables in the file are stored using the column storage model DSM; For semi-structured data, its parsing is equivalent to using JSON format to store tree data. The file is stored in a multi-layer nested JSON format. This format of data does not have column indexing and fast retrieval, and tree / graph retrieval algorithms are used for retrieval. For unstructured data, which is text data, the content is directly stored, and LLM is considered to be used for entity / relationship extraction tasks to obtain corresponding information in the future; Finally, the fusion of multi-source cross-domain data is expressed as the following formula: Among them, FusionData represents the fused multi-source knowledge graph, Ada stru 、Ada semi-s 、Ada unstru Denotes the data conversion parser for structured data, semi-structured data and unstructured data respectively, D stru 、D semi-s 、D unstru Data sets representing structured data, semi-structured data, and unstructured data respectively.

4. The multi-source cross-domain data query method according to claim 1, characterized in that: The step 3 is specifically as follows: The query requirement is parsed into a search instruction, and the homologous knowledge line graph is traversed according to the confidence of the node to screen and match the query-related subgraphs. The obtained subgraph is called a trusted candidate subgraph. The process is as follows: The confidence of the entity node is calculated by the mutual information entropy confidence calculation method, taking into account the reliability of the node source, the node authority and the historical authority factors, and using the probability statistical model and the large language model to calculate the score; Define the homologous linear subgraph to which the current node belongs The corresponding confidence value is the source confidence threshold, which is calculated by calculating the homologous linear subgraph The average similarity of all node pairs in is determined; Untrusted nodes and their associated edges whose confidence is lower than the source confidence threshold are removed, and finally a trusted candidate subgraph is returned.

5. The multi-source cross-domain data query method according to claim 4 is characterized in that: In step 3, the specific process of querying and matching trusted candidate subgraphs is as follows: Step 3.

1. Graph the homologous knowledge lines The confidence of all data nodes, data authority, and historical credibility estimation of data nodes, The corresponding source confidence threshold and the trusted candidate subgraph T-sub are initialized; Traversal For each data node υ, initialize the variable N to be equal to the U corresponding to the current element sg The cardinality of the set; corresponding to the current element υ For each node in , execute steps 3.2 to 3.4; Step 3.

2. Calculate homologous linear subgraphs The consistency score between two data nodes in ; set up for The node set in; for any two nodes with the same attribute Define the similarity between them I(υ i ,υ j ) uses a calculation method based on mutual information entropy, and its formula is: Among them, V i and V j They are node υ i and υ j The attribute value set of node υ, p(x,y) is i Take the attribute value x and node υ j Take the joint probability distribution of attribute value y, where p(x) and p(y) are the marginal probability distributions of x and y respectively; Node similarity S(υ i ,υ j ) is defined as the normalized form of mutual information entropy to ensure that its value is in the interval [0,1]: Where H(υ i ) and H(υ j ) are nodes υ i and υ j The entropy of the attribute value set, where nodes are uniformly represented by υ, is calculated as follows: Among them, v represents the node υ i and υ j , V stands for V i and V j ; Next, calculate the node consistency score S n (υ), which reflects the consistency of nodes in different data sources. The mutual information entropy is used to calculate the similarity between node pairs. For node υ, its consistency score is expressed as: Among them, N(v) is the set of nodes with the same attributes as node v, u∈N(v), S(v,u) is the similarity between nodes; Finally, the source confidence threshold is calculated in, Indicates the number of nodes in the homologous linear subgraph, S(υ i ,v j ) represents the similarity of node pairs; Step 3.

3. Calculate the authority score of the node; The authority score A(v) of a node is calculated using the following formula: A(υ)=α*Auth LLM (v)+(1-a)Auth hist (v) (6) Among them, α is the weight coefficient, which is used to balance the contribution of LLM evaluation authority and historical authority, satisfying 0≤α≤1; LLM (υ) is the node authority score evaluated by LLM, hist (υ) is the historical authority of the node; Auth based on the node's global influence and local connection strength LLM (υ) Evaluation: LLM calculates the credibility of knowledge by integrating the strength of association between entities, entity type information, and multi-step path information: Among them, C LLM (υ) is the authority score provided by LLM for node v, and β is the parameter that controls the steepness of the score curve; Node's historical authority Auth hist (υ) is the authoritative score based on the node's historical data, and the formula is as follows: in, yes The number of entities provided for all historical queries, yes The historical credibility of D v [q] is the set of correct answers, is the query-related data obtained from the homologous linear subgraph; p Represents the entity node in the correct answer set, A(v p ) represents the authority score of the entity node; Step 3.

4. Add formula (4) and formula (6) to obtain the final confidence score of the node; if the final confidence score of the node is lower than the source confidence threshold, delete the node from the current homology subgraph; Step 3.

5. Repeat steps 3.2 to 3.4 until all elements in the data source are traversed, thereby completing the entire trusted candidate subgraph matching process, and finally outputting the data source credibility and the trusted candidate subgraph T-sub.

6. The multi-source cross-domain data query method according to claim 1, characterized in that: The step 4 is specifically as follows: Step 4.

1. Construct a prompt word containing reference examples, perform intent recognition and entity extraction tasks based on the query requirements, and thus obtain a query node set; initialize the query node set and the trusted answer set; Step 4.

2. According to step 2, obtain a homologous knowledge line graph; and according to step 3, calculate the confidence score of the query node and the source credibility threshold of the corresponding homologous linear subgraph, and obtain a credible candidate subgraph; Step 4.

3. Store the source confidence threshold and the trusted candidate subgraph into the trusted answer set; According to the query requirements and the set of credible answers, corresponding prompt words are constructed, and the vertical large model is input to ask questions to obtain the final query answer.

7. A multi-source cross-domain data query system for implementing the multi-source cross-domain data query method according to claim 1, characterized in that: The multi-source cross-domain data query system includes the following modules: A multi-source knowledge graph construction module is used to retrieve multi-source cross-domain data from different sources and storage formats in the field, and use a large language model to identify, extract, and classify data to build a multi-source knowledge graph; The homologous knowledge line graph construction module is used to reconstruct and aggregate the obtained homologous knowledge line graph, merge and reorganize the nodes with similar semantics scattered in the multi-source knowledge graph to obtain the homologous knowledge line graph; The trusted candidate subgraph screening module is used to use the large language model to perform confidence scoring of query nodes in the homologous knowledge line graph, calculate the source credibility threshold of the corresponding homologous linear subgraph, and screen the trusted candidate subgraphs related to the query; And a data query module, which is used to embed the core content and key related information of the screened credible candidate subgraphs into the prompt word template, and then use the vertical large model to screen out data results that are highly matched with the query intent.

Images