Distributed rdf data semantic retrieval method and system based on large model

Abstract

The application discloses a distributed RDF data semantic retrieval method and system based on a large model and belongs to the technical field of semantic association. The method comprises the following steps: based on a large language model, extracting an entity keyword group from a user question; constructing a vector database based on RDF data resources, and obtaining a first external knowledge text of the large model through similarity matching between the user question and vector data in the vector database; obtaining RDF data information related to the entity keyword group through federated query of SPARQL, and expanding the RDF data information to obtain a second external knowledge text of the large model; integrating the user question, the first external knowledge text and the second external knowledge text, and submitting the integration result to the large model to obtain an answer to the user question. The application can make up for the deficiencies of traditional RDF retrieval, such as lack of semantics and inability to effectively give complex RDF retrieval results to the large model.

Description

Technical Field

[0001] This invention belongs to the field of semantic association technology, and in particular relates to a distributed RDF data semantic retrieval method and system based on a large model. Background Technology

[0002] Large Language Models (LLMs) are deep learning models trained on massive amounts of text data. They can not only generate natural language text but also deeply understand the meaning of text and handle various natural language tasks, such as text summarization, question answering, and translation. Resource Description Frameworks (RDFs) are used to describe the characteristics of Web resources and the relationships between resources.

[0003] Large language models are undeniably powerful in processing natural language, but due to limited computing resources, the training datasets cannot cover all the knowledge, resulting in irrelevant or even incorrect information in the generated text.

[0004] RDF data has many characteristics such as semantic clarity, flexibility, and support for data integration from different sources, but its semantic retrieval capabilities are currently weak, and achieving semantic retrieval is highly complex.

[0005] Currently, there are still many prominent problems in RDF data retrieval that urgently need to be solved, specifically as follows:

[0006] 1. RDF data is an open, easily integrated, and highly semantically descriptive type of data. However, there is currently a lack of a distributed RDF data semantic retrieval method that can support user input of natural language questions and deeply mine the semantic content in RDF data.

[0007] 2. RDF retrieval emphasizes the semantic relationships between data, rather than simple word matching. RDF data typically requires exact matching. How to use similarity queries and how to textualize RDF data without losing semantic meaning, and then feed the processed text data into large models, is a technical challenge.

[0008] 3. RDF data contains a large number of relationships. How to handle multi-level and multi-type relationships, how to expand the scope of RDF data retrieval to enrich data information, and how to provide the results to a large model are technical challenges. Summary of the Invention

[0009] To address the aforementioned prominent issues, this invention proposes a distributed RDF data semantic retrieval method and system based on a large model, which can overcome the shortcomings of traditional RDF retrieval in lacking semantics and inability to effectively provide complex RDF retrieval results to a large model.

[0010] To achieve the above-mentioned objectives, the technical solution of the present invention includes the following:

[0011] A distributed RDF data semantic retrieval method based on a large model, the method comprising:

[0012] Based on a large language model, entity keyword groups are extracted from user questions;

[0013] A vector database is constructed based on RDF data resources, and the first external knowledge text of the large model is obtained by matching the similarity between user questions and vector data in the vector database.

[0014] By using SPARQL federated queries to obtain RDF data information related to entity keyword groups, and by performing semantic parsing of associated resources, semantic mining of general relations, retrieval of international data sources, and supplementary explanations of ontology information on this RDF data information, a second external knowledge text of the large model is obtained.

[0015] The user's question, the first external knowledge text, and the second external knowledge text are integrated, and the integrated result is submitted to the large model to obtain the answer to the user's question.

[0016] Furthermore, the construction of a vector database based on RDF data resources includes:

[0017] The RDF data resources are retrieved by parsing the distributed SPARQL endpoints.

[0018] Transform RDF data resources into text without losing attribute semantics;

[0019] Add appropriate auxiliary words to the text to make the converted document more readable.

[0020] The converted text is vectorized and saved to a vector database.

[0021] Furthermore, the process of converting RDF data resources into text without losing attribute semantics includes:

[0022] Retrieve the name attribute and other attributes from an RDF data resource;

[0023] Use the value of the name attribute as the subject;

[0024] Use other attributes as predicates;

[0025] Use the values ​​of other attributes as objects.

[0026] Furthermore, the extension of the associated resource semantic parsing of the RDF data information includes:

[0027] Retrieve the value range of attributes in RDF data;

[0028] For attributes whose value range is resource type, further analysis is performed, and the semantic information of the RDF data information is improved based on the attribute values ​​of the resource obtained from the analysis.

[0029] Furthermore, the extension of the general relational semantic mining of the RDF data information includes:

[0030] Obtain the common relation terms from the RDF data information; wherein, the common relation terms include: the common attribute owl:sameAs and the common attribute rdfs:subClassOf;

[0031] The search is performed using common relational terms, and the semantic information of the RDF data is improved based on the search results.

[0032] Furthermore, the extension of performing international data source association retrieval on the RDF data information includes:

[0033] Obtain international open data nodes that conform to the SPARQL specification;

[0034] The semantic information of the RDF data is improved based on the information in the international open data nodes.

[0035] Furthermore, the expansion of the ontology information supplementation for the RDF data information includes:

[0036] Retrieve detailed information about the classes and attributes of RDF data using rdfs:comment;

[0037] The semantic information of the RDF data is improved based on the detailed information of the class and attributes.

[0038] A distributed RDF data semantic retrieval system based on a large model, the system comprising:

[0039] The entity extraction module is used to extract entity keyword groups from user questions based on a large language model.

[0040] The first knowledge generation module is used to build a vector database based on RDF data, and obtain the first external knowledge text of the large model by matching the similarity between user questions and vector data in the vector database.

[0041] The second knowledge generation module is used to obtain RDF data information related to entity keyword groups through SPARQL federated queries, and to expand the RDF data information by performing semantic parsing of associated resources, semantic mining of general relations, retrieval of international data sources, and supplementary explanation of ontology information to obtain the second external knowledge text of the large model.

[0042] The answer retrieval module integrates the user's question, the first external knowledge text, and the second external knowledge text, and submits the integrated result to the large model to obtain the answer to the user's question.

[0043] A computer device, characterized in that the computer device comprises: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the distributed RDF data semantic retrieval method based on a large model as described above.

[0044] A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the distributed RDF data semantic retrieval method based on a large model as described above.

[0045] Compared with the prior art, the present invention has at least the following technical advantages:

[0046] This invention targets RDF data, supporting user input of natural language questions to deeply mine the semantic content within the RDF data. It focuses on textualizing RDF data without sacrificing semantics and enabling SPARQL extended queries. Leveraging the contextual understanding and language organization capabilities of a large model, it summarizes the search results to make them more readable. This invention overcomes the shortcomings of traditional RDF retrieval, such as its lack of semantics and its inability to effectively provide complex RDF search results to large models. Attached Figure Description

[0047] Figure 1 A flowchart of the distributed RDF data semantic retrieval method of the present invention.

[0048] Figure 2 The flowchart of RDF data text similarity query of the present invention.

[0049] Figure 3 The flowchart of the SPARQL extended query of this invention. Detailed Implementation

[0050] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below through specific embodiments and in conjunction with the accompanying drawings.

[0051] This invention first converts RDF data into a plain string format, then vectorizes it using a text vector model, and finally saves it to a vector database. When a user asks a question to retrieve information, a large language model is used to process the user's question and extract entity information text from the question. This entity information text is then used for similarity matching in the vector database and to retrieve relevant background information from distributed SPARQL endpoints. Finally, the user's question and the retrieved background information are processed by the large language model to generate the final answer. This method can parse user questions to obtain keyword phrases at low cost, while leveraging the characteristics of RDF data to expand information and reduce the possibility of text illusions generated by the large language model. The overall process of this method is as follows: Figure 1 As shown.

[0052] 1. Extracting entity keyword phrases from a large language model

[0053] Large language models can understand natural language and perform a wide range of tasks. Now, by combining large language models, we can simplify the work of extracting specific phrases from user questions. For example, if a user asks, "What are the morphological features of 'land-incompatible'?", after LangChain+Kor entity extraction, we get the two keyword phrases "land-incompatible" and "morphological features".

[0054] 2. RDF data text similarity query

[0055] RDF data text similarity query is one of the innovations of this invention. It transforms RDF data into text at the resource entity level. By parsing the relevant attributes of RDF data resources retrieved from distributed SPARQL endpoints, the RDF data is converted into text without semantic loss. Appropriate auxiliary words are added to make the converted text more fluent and human-readable. A text vector model is used to vectorize the converted text, which is then stored in a vector database. Similarity matching is performed with user queries, and the list of texts with the highest similarity is used as external knowledge for the larger model. This lossless semantic conversion of RDF data into text, and the quantification examples, are further elaborated. Figure 2 As shown.

[0056] Convert the following triples into vectors.

[0057] @prefix rdfs:<http: / / www.w3.org / 2000 / 01 / rdf-schema#>

[0058] @prefix xtipc:<http: / / xtipc.semweb.csdb.cn / resource / >

[0059] @prefix semanticscience:<http: / / semanticscience.org / resource / >

[0060] @prefix ioz:<http: / / ioz.semweb.csdb.cn / resource / >

[0061] @prefix owl:<http: / / www.w3.org / 2002 / 07 / owl#sameAs>

[0062] xtipc:Herb_Stephania_epigaea_Lo rdfs:label "Stephania epigaea root tuber".

[0063] xtipc:Herb_Stephania_epigaea_Lo semanticscience:SIO_000225 "The tuberous roots of this species are well-known traditional Chinese herbs in Yunnan. They are bitter and pungent, cool in nature, slightly poisonous, and have the functions of clearing heat and detoxifying, calming, regulating qi, and relieving pain.".

[0064] xtipc:Herb_Stephania_epigaea_Lo semanticscience:SIO_000061 "It is produced in the eastern, central and western parts of Yunnan, and the western and southern parts of Sichuan.".

[0065] For the above triples, the present invention first obtains the name attribute and other attributes in the RDF data. The rdfs:label attribute is the name attribute, and its value is used as the subject, and the word "的" is added after the subject as a particle. Other attributes are used as the predicate, and their values are used as the object, and ";" is added after the object to indicate the end of a group of attributes. A group of RDF resource entities is transformed into text. After the text is vectorized through a text vector model, it is saved in a vector database.

[0066] 3. SPARQL Extended Query

[0067] The SPARQL extended query is one of the innovations of the present invention. First, the large model extracts the entities in the question sentence, and then obtains all the RDF data information related to the entities through the federated query of SPARQL. For example, when the large model extracts the entity "Stephania epigaea root tuber", all the RDF data with the name "Stephania epigaea root tuber" are simultaneously queried in the SPARQL endpoints "DBpedia" and "LinkedGeoData". The SPARQL query statement is as follows:

[0068]

[0069] However, the RDF data information in the results is not comprehensive enough. It can be expanded through the following four key aspects to achieve in-depth semantic mining of RDF data, making the content provided to the large model more comprehensive.

[0070] 3.1 Semantic parsing of associated resources

[0071] The search results yield target resources containing multiple attributes. The attribute value range can be either a literal constant type or a resource type. For attributes with a resource type value range, further parsing is possible.

[0072] For example, the value of the ioz:eatenBy attribute of the target resource ioz:Species_Stephania_japonica is the resource type, and the corresponding values ​​are ioz:Species_Calyptra_thalictri,

[0073] `ioz:Species_Oraesia_excavata` and `ioz:Species_Eudocima_salaminia` are used to parse the rdfs:label attribute of the resource and obtain its value.

[0074] 3.2 General Relational Semantic Mining

[0075] By using common relation terms in RDF, the scope and depth of the search can be expanded and the semantic information of attribute terms can be automatically improved. For example, relation terms such as owl:sameAs and rdfs:subClassOf can be used.

[0076] The general attribute owl:sameAs indicates that it is the same resource as the current resource, but described on another website. Content related to the object resource can be used as a supplement to the data of the current resource.

[0077] The general attribute rdfs:subClassOf indicates the parent class of the current resource. Content related to the object resource can be used as a supplement to the current resource data.

[0078] 3.3 International Data Source Association Retrieval

[0079] RDF is flexible and supports data from different data sources. As long as the international open data nodes conform to the SPARQL specification, they can be quickly imported into queries.

[0080] For example, using the target resource ioz:Species_Stephania_japonica as the subject to retrieve data from international open data nodes such as DBpedia, LinkedGeoData, and OpenLink can expand the search scope and improve the search results.

[0081] 3.4 Supplementary Explanation of Ontology Information

[0082] When defining an RDF ontology, it typically includes detailed information about classes and properties. By using rdfs:comment to obtain this detailed information as external knowledge of the larger model, ambiguity is eliminated and the content becomes easier to understand.

[0083] Taking the definition of ioz:eatenBy as an example:

[0084] ioz:eatenBy a owl:ObjectProperty;

[0085] rdfs:label'eatenBy';

[0086] The comment 'eatenBy means: ....' is from rdfs:comment.

[0087] The detailed information about ioz:eatenBy is obtained through rdfs:comment, which is used to explain and describe the relationship between the two resources.

[0088] Based on the above four extensions, the overall process framework of SPARQL extended query is as follows: Figure 3 As shown.

[0089] 4. Generate semantic retrieval answers by aggregating large models.

[0090] The system integrates the user's question, the first external knowledge text matched from the RDF-converted vector database, and the second external knowledge text converted from the SPARQL query results to generate a complete, contextually logical prompt for solving the specified problem. This prompt is then submitted to the large model to generate the final answer. For example, if a user asks, "What are the medicinal parts and main components of wild cotton?", the current pre-trained large language model does not contain this information. First, the large model extracts key information such as "wild cotton," "medicinal parts," and "main components," which are then retrieved from the vector database and the SPARQL endpoint, respectively.

[0091] The following information was retrieved from the vector database.

[0092] "Uses Description: [Medicinal Part] The root is used medicinally. Chinese medicine name: Wild Cotton. [Harvesting and Processing] The root can be harvested year-round, washed, sliced, and sun-dried. [Main Components] Mainly contains ranunculin. [Properties and Functions] Bitter, pungent, toxic. Clears damp-heat, detoxifies and kills parasites, regulates qi and disperses blood stasis." "Morphological Characteristics (Chinese): Plant height 60-100 cm. Rhizome oblique, woody, 0.8-1.5 cm thick. Basal leaves 2-5, with long petioles; leaf blade cordate-ovate or cordate-broadly ovate, (5.2-)11-22 cm long, (6-)12-26 cm wide, apex acute with 3-5 shallow lobes, margin with small teeth, surface sparsely covered with short rough hairs, underside densely covered with white short tomentum; petiole (6.5-)25-60 cm long, with soft hairs..." and other information. The SPARQL endpoint retrieved "Previous level: Anemone genus" and "Distribution: Guangxi Zhuang Autonomous Region, Jiangxi Province, Hunan Province, Guangdong Province, Yunnan Province, Taiwan Province".

[0093] Based on the above information, the suggested keywords are as follows.

[0094] My question is: {User Question} Please answer my question based on the following information.

[0095] {Text matched by RDF vector similarity}

[0096] {Text of the SPARQL query results}

[0097] Please generate a summary, explaining each point below, and at the end of each point, indicate the corresponding index.

[0098] Your answer should be in the following format:

[0099] Summarize......:

[0100] 1. Explanation of this point. Reference [1]

[0101] 2. Explanation of this point. Reference [2]

[0102] 3. Explanation of this point. Reference [3]

[0103] Among them, the above references [1], [2] and [3] respectively represent the data pieces that make up the first external knowledge text or the second external knowledge text.

[0104] The integrated prompts are then fed into a larger model, which generates the final answer. The generated result is shown below.

[0105] The medicinal part of wild cotton is its root, which mainly contains ranunculin, and has the effects of clearing damp-heat, detoxifying and killing parasites, and regulating qi and dispersing blood stasis. Specific information is as follows:

[0106] 1. Other names for wild cotton include wild hemp, mountain hemp, creeping everywhere, five thunder fire, yellow maning, long-haired yellow sunflower, mountain hibiscus, field hibiscus, Xiao Fantian flower, and ground peach blossom.

[0107] 2. According to XTBG Data, the morphological characteristics of *Cotton Tree Blossom* (in Chinese) are as follows: plant height 60-100 cm, rhizome oblique, woody, 0.8-1.5 cm thick. Basal leaves 2-5, with long petioles; leaf blade cordate-ovate or cordate-broadly ovate, (5.2-)11-22 cm long, (6-)12-26 cm wide, apex acute with 3-5 shallow lobes, margin with small teeth, upper surface sparsely covered with short rough hairs, lower surface densely covered with white short tomentum; petiole (6.5-)25-60 cm long, pubescent.

[0108] 3. The type of wild cotton is: species.

[0109] 4. The use type of wild cotton is: medicinal plant.

[0110] 5. According to XTBG Data, the uses of wild cotton are described as follows:

Medicinal Part

Harvesting and Processing

Main Components

Properties and Functions

[0111] 6. The next level above wild cotton is the genus *Anemone*.

[0112] 7. The provinces where wild cotton is distributed are: Sichuan, Tibet Autonomous Region, and Yunnan.

[0113] As can be seen, the generated answer fully incorporates all the information found in RDF, and can deeply mine the semantic content in RDF data, realizing the complementary advantages of RDF data and large models.

[0114] In summary, this invention supports user input of natural language questions, deeply mines the semantic content in RDF data, and leverages the contextual understanding and language organization capabilities of large models to overcome the shortcomings of traditional RDF retrieval, such as lack of semantics and inability to effectively provide complex RDF retrieval results to large models.

[0115] The RDF data text similarity query of this invention converts RDF data into text while retaining semantic information, uses a text vector model to generate vectors, saves them to a vector database, and uses them to perform similarity retrieval with user questions to obtain a list of texts with the highest similarity as external knowledge of the large model.

[0116] The SPARQL extended query of this invention uses a large language model to extract entities from questions, uses SPARQL federated queries to distribute RDF data nodes, parses the semantics of related resources, and can also retrieve from international open data sources, and extend the retrieval through attributes such as owl:sameAs and rdfs:subClassOf, making the query results more comprehensive. RDF data and ontology information are used as external knowledge of the large model, as detailed below.

[0117] a) Related Resource Semantic Parsing: If the data type of the object value in the search results is a resource type and is not...

[0118] If owl:sameAs, then a one-step deep parsing is performed; if the data type of the object value is a resource type and owl:sameAs, then a two-step deep parsing is performed; if the data type of the object value is not a resource type, then the literal constant is directly obtained.

[0119] b) General relational semantic mining: Using common relation terms in RDF, the search scope and depth are expanded, and the semantic information of attribute terms is automatically improved. For example, relation terms such as owl:sameAs and rdfs:subClassOf are used.

[0120] c) International Data Source Linked Retrieval: Expand the retrieval scope by connecting to any international open data source conforming to the SPARQL specification and introducing multiple external data nodes. For example, data sources such as DBpedia, LinkedGeoData, and OpenLink can be connected.

[0121] d) Supplementary explanation of ontology information: Ontology information is also used as external knowledge of the large model. For the retrieved RDF data, the predicates are further parsed, and complete descriptive information is obtained through rdfs:comment, making the content easier to understand and unambiguous.

[0122] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Those skilled in the art can modify or make equivalent substitutions to the technical solutions of the present invention without departing from the scope of the present invention. The scope of protection of the present invention should be determined by the claims.

Claims

1. A distributed RDF data semantic retrieval method based on a large model, characterized in that, The method includes: Based on a large language model, entity keyword groups are extracted from user questions; A vector database is constructed based on RDF data resources, and the first external knowledge text of the large model is obtained by matching the similarity between user questions and vector data in the vector database. By using SPARQL federated queries to obtain RDF data information related to entity keyword groups, and by performing semantic parsing of associated resources, semantic mining of general relations, retrieval of international data sources, and supplementary explanations of ontology information on this RDF data information, a second external knowledge text of the large model is obtained. The user's question, the first external knowledge text, and the second external knowledge text are integrated, and the integrated result is submitted to the large model to obtain the answer to the user's question. The construction of a vector database based on RDF data resources includes: The RDF data resources are retrieved by parsing the distributed SPARQL endpoints. Transform RDF data resources into text without losing attribute semantics; Add appropriate auxiliary words to the text to make the converted document more readable. The converted text is vectorized and saved to a vector database; The step of converting RDF data resources into text without losing attribute semantics includes: Retrieve the name attribute and other attributes from an RDF data resource; Use the value of the name attribute as the subject; Use other attributes as predicates; Use the values ​​of other attributes as objects.

2. The method according to claim 1, characterized in that, The extension of the associated resource semantic parsing of the RDF data information includes: Retrieve the value range of attributes in RDF data; For attributes whose value range is resource type, further analysis is performed, and the semantic information of the RDF data information is improved based on the attribute values ​​of the resource obtained from the analysis.

3. The method according to claim 1, characterized in that, The extension of the general relational semantic mining of the RDF data information includes: Obtain the common relation terms from the RDF data information; wherein, the common relation terms include: the common attribute owl:sameAs and the common attribute rdfs:subClassOf; The search is performed using common relational terms, and the semantic information of the RDF data is improved based on the search results.

4. The method according to claim 1, characterized in that, The extension for performing international data source association retrieval on the RDF data information includes: Obtain international open data nodes that conform to the SPARQL specification; The semantic information of the RDF data is improved based on the information in the international open data nodes.

5. The method according to claim 1, characterized in that, The extension of the ontology information supplementation for the RDF data includes: Retrieve detailed information about the classes and attributes of RDF data using rdfs:comment; The semantic information of the RDF data is improved based on the detailed information of the class and attributes.

6. A distributed RDF data semantic retrieval system based on a large model, characterized in that, The system includes: The entity extraction module is used to extract entity keyword groups from user questions based on a large language model. The first knowledge generation module is used to build a vector database based on RDF data, and obtain the first external knowledge text of the large model by matching the similarity between user questions and vector data in the vector database. The second knowledge generation module is used to obtain RDF data information related to entity keyword groups through SPARQL federated queries, and to expand the RDF data information by performing semantic parsing of associated resources, semantic mining of general relations, retrieval of international data sources, and supplementary explanation of ontology information to obtain the second external knowledge text of the large model. The answer retrieval module is used to integrate the user's question, the first external knowledge text, and the second external knowledge text, and submit the integrated result to the large model to obtain the answer to the user's question; The construction of a vector database based on RDF data resources includes: The RDF data resources are retrieved by parsing the distributed SPARQL endpoints. Transform RDF data resources into text without losing attribute semantics; Add appropriate auxiliary words to the text to make the converted document more readable. The converted text is vectorized and saved to a vector database; The step of converting RDF data resources into text without losing attribute semantics includes: Retrieve the name attribute and other attributes from an RDF data resource; Use the value of the name attribute as the subject; Use other attributes as predicates; Use the values ​​of other attributes as objects.

7. A computer device, characterized in that, The computer device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the distributed RDF data semantic retrieval method based on a large model as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the distributed RDF data semantic retrieval method based on a large model as described in any one of claims 1-5.

Images