Method for generating large model agricultural intelligent questions and answers based on knowledge graph enhanced retrieval
Through low-rank adaptive algorithms and agricultural pest data knowledge graphs, the problem of insufficient accuracy and interpretability of content generated by large-language models in the agricultural field is solved, and more accurate and transparent answer results are achieved, helping agricultural personnel to quickly obtain Pest and disease query results.
Patent Information
- Application Number
- CN202410922279.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Large language models may introduce irrelevant or false information in natural language generation, leading to "illusion" phenomena, and the transparency and interpretability of the reasoning process are insufficient, especially in the field of agricultural pests and diseases, which is difficult to select appropriate knowledge from large heterogeneous data sources. , resulting in the accuracy and credibility of the generated content.
The large model is fine-tuned through low-rank adaptive algorithms, and the knowledge graph of agricultural pest and diseases data is used for enhanced search, and chain prompts are generated to improve the accuracy and interpretability of the answers, avoid "illusion" phenomena, and enhance the transparency of model reasoning.
It improves the accuracy and interpretability of the big model's answers in the field of agricultural pests and diseases, helping agricultural personnel quickly obtain relevant information, save time, and improve work efficiency.
Smart Images

Figure CN118939762A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and system for generating large-model agricultural intelligent question-answering based on knowledge graph enhanced retrieval, an electronic device, a storage medium and a computer program product. Background Art
[0002] In recent years, with the continuous development of deep learning, large models, also known as large language models (LLMs), have shown unprecedented capabilities in natural language understanding and text generation by pre-training on large-scale corpus datasets. LLMs not only have strong reasoning capabilities, but can also generate fluent and coherent texts. As a result, they are widely used in fields such as natural language processing and computer vision. Intelligent question and answer and generative retrieval have also been applied and developed. The demand for intelligent question and answer in various industries has become more and more urgent. Intelligent technology has greatly changed the way people live and work.
[0003] The use of large language models for natural language generation mainly relies on their reasoning ability and the quality and reliability of the generated text, but they may lack factual consistency, introduce irrelevant or false information, and cause "hallucination" phenomena. A common way to solve this problem is to add external knowledge sources that can provide relevant facts and context for the generation task, which is called retrieval-enhanced generation. However, it is very difficult to select the most appropriate knowledge from large heterogeneous data sources by calculating similarity retrieval through vector databases, and the lack of transparency and explainability of the large model reasoning process faces huge challenges. Therefore, this technical problem needs to be solved urgently. Summary of the invention
[0004] In view of the above problems, this application is proposed to provide a method and system for generating a large model of agricultural intelligent question answering based on knowledge graph enhanced retrieval, an electronic device, a storage medium and a computer program product that overcomes the above problems or at least partially solves the above problems. The user inputs a question, parses the user input question to generate a thinking prompt chain diagram, retrieves and filters related entities and corresponding agricultural knowledge fragments through the knowledge graph of agricultural pests and diseases data, and constructs a chain prompt input large model according to the query process and the retrieved knowledge, which can improve the accuracy and interpretability of the generated content. The technical solution is as follows:
[0005] In the first aspect, a method for generating large-model agricultural intelligent question answering based on knowledge graph enhanced retrieval is provided, comprising:
[0006] Through the low-rank adaptive algorithm, the agricultural pest knowledge question and answer data is used to perform fine-tuning pre-training learning on the large model to obtain a large model after fine-tuning pre-training;
[0007] Obtain user input questions related to agricultural pests and diseases, perform digital vector representation on the user input questions using a preset vector model, and obtain a represented question vector;
[0008] Based on the represented question vector, the pre-built agricultural pest and disease data knowledge graph is used for enhanced retrieval to generate contextual knowledge fragments related to the user input question;
[0009] The prompt chain is formed by integrating the user input question, enhanced retrieval steps and contextual knowledge fragments related to the user input question, and the prompt chain is provided to the fine-tuned pre-trained large model to obtain the intelligent answer result returned by the fine-tuned pre-trained large model.
[0010] In one possible implementation, a low-rank adaptive algorithm is used to perform fine-tuning pre-training learning on a large model using agricultural pest and disease knowledge question-answering data to obtain a large model after fine-tuning pre-training, including:
[0011] Preprocess the agricultural pest knowledge question-and-answer data and convert it into a data set containing instructions, context, and output content. The data set is divided into a training data set and a test data set, and the data set is saved as a json file.
[0012] Configure a single-card GPU operating environment on the server and select the preset large base model;
[0013] Statistics are collected to collect the maximum lengths of commands, contexts, and output contents, which are used to set the length of natural language commands, the maximum length of input text, and the maximum length of output text.
[0014] Through a low-rank adaptive algorithm, combined with the set length of the natural language instruction, the maximum length of the input text, and the maximum length of the output text, the preset base large model is fine-tuned and pre-trained using the training data set and the test data set to obtain a large model after fine-tuning and pre-training.
[0015] In one possible implementation, the agricultural pest data knowledge graph is constructed through the following steps:
[0016] Acquire agricultural professional knowledge files from a designated agricultural data source, and pre-process the agricultural professional knowledge files to obtain pre-processed documents;
[0017] The preprocessed document is segmented using a fixed-size block strategy to obtain segmented paragraphs;
[0018] Entities, attributes and relationships are extracted from the segmented paragraphs, and the extracted entities, attributes and relationships are vector-embedded and stored in the Neo4j graph database to obtain the knowledge graph of agricultural pests and diseases data. In the graph database, data is stored in the form of a graph, with nodes representing entities, relationships representing connections between entities, and attributes providing additional data descriptions.
[0019] In a possible implementation, the method further includes:
[0020] Vectorize the segmented paragraphs to obtain vectorized paragraphs;
[0021] Build an index for the vectorized paragraphs and generate an agricultural knowledge vector library.
[0022] In a possible implementation, based on the represented question vector, a pre-built agricultural pest and disease data knowledge graph is used for enhanced retrieval to generate contextual knowledge fragments related to the user input question, including:
[0023] Based on the represented question vector, similar nodes in the knowledge graph of agricultural pests and diseases data are identified; similar nodes jump around entities and relationships several times;
[0024] The attribute data of similar nodes are used as contextual knowledge fragments related to the user input question;
[0025] The enhanced retrieval step is generated based on the represented question vector, similar nodes and edge dependencies.
[0026] In a possible implementation, based on the represented question vector, similar nodes in the knowledge graph of agricultural pests and diseases data are identified, including:
[0027] Based on the represented question vector, graph node retrieval is performed in the knowledge graph of agricultural pest and disease data to realize subgraph traversal, search for local subgraphs within several jumps, and obtain similar entity nodes and knowledge texts by calculating the cosine similarity between nodes.
[0028] In the second aspect, a large-model agricultural intelligent question-answering system based on knowledge graph enhanced retrieval is provided, including:
[0029] A fine-tuning unit is used to perform fine-tuning pre-training learning of a large model using agricultural pest and disease knowledge question-answering data through a low-rank adaptive algorithm to obtain a large model after fine-tuning pre-training;
[0030] A vectorization unit is used to obtain user input questions related to agricultural pests and diseases, and to perform digital vector representation on the user input questions using a preset vector model to obtain a represented question vector;
[0031] The enhanced retrieval unit is used to perform enhanced retrieval based on the represented question vector using the pre-built agricultural pest and disease data knowledge graph to generate contextual knowledge fragments related to the user input question;
[0032] The answer generation unit is used to integrate the user input question, enhanced retrieval steps and contextual knowledge fragments related to the user input question to form a prompt chain, and provide the prompt chain to the fine-tuned pre-trained large model to obtain the intelligent answer result returned by the fine-tuned pre-trained large model.
[0033] In a possible implementation manner, the fine-tuning unit is further configured to:
[0034] Preprocess the agricultural pest knowledge question-and-answer data and convert it into a data set containing instructions, context, and output content. The data set is divided into a training data set and a test data set, and the data set is saved as a json file.
[0035] Configure a single-card GPU operating environment on the server and select the preset large base model;
[0036] Statistics are collected to collect the maximum lengths of commands, contexts, and output contents, which are used to set the length of natural language commands, the maximum length of input text, and the maximum length of output text.
[0037] Through a low-rank adaptive algorithm, combined with the set length of the natural language instruction, the maximum length of the input text, and the maximum length of the output text, the preset base large model is fine-tuned and pre-trained using the training data set and the test data set to obtain a large model after fine-tuning and pre-training.
[0038] In a possible implementation, the system further includes a construction unit, configured to:
[0039] Acquire agricultural professional knowledge files from a designated agricultural data source, and pre-process the agricultural professional knowledge files to obtain pre-processed documents;
[0040] The preprocessed document is segmented using a fixed-size block strategy to obtain segmented paragraphs;
[0041] Entities, attributes and relationships are extracted from the segmented paragraphs, and the extracted entities, attributes and relationships are vector-embedded and stored in the Neo4j graph database to obtain the knowledge graph of agricultural pests and diseases data. In the graph database, data is stored in the form of a graph, with nodes representing entities, relationships representing connections between entities, and attributes providing additional data descriptions.
[0042] In a possible implementation, the construction unit is further used to:
[0043] Vectorize the segmented paragraphs to obtain vectorized paragraphs;
[0044] Build an index for the vectorized paragraphs and generate an agricultural knowledge vector library.
[0045] In a possible implementation manner, the enhanced retrieval unit is further used to:
[0046] Based on the represented question vector, similar nodes in the knowledge graph of agricultural pests and diseases data are identified; similar nodes jump around entities and relationships several times;
[0047] The attribute data of similar nodes are used as contextual knowledge fragments related to the user input question;
[0048] The enhanced retrieval step is generated based on the represented question vector, similar nodes and edge dependencies.
[0049] In a possible implementation manner, the enhanced retrieval unit is further used to:
[0050] Based on the represented question vector, graph node retrieval is performed in the knowledge graph of agricultural pest and disease data to realize subgraph traversal, search for local subgraphs within several jumps, and obtain similar entity nodes and knowledge texts by calculating the cosine similarity between nodes.
[0051] According to a third aspect, an electronic device is provided, comprising a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the above-mentioned methods for generating large-model agricultural intelligent question-answering based on knowledge graph enhanced retrieval.
[0052] In a fourth aspect, a storage medium is provided, which stores a computer program, wherein the computer program is configured to execute any of the above-mentioned methods for generating large-model agricultural intelligent question-answering based on knowledge graph enhanced retrieval when running.
[0053] In a fifth aspect, a computer program product is provided, comprising a computer program, wherein the computer program is configured to execute any of the above-mentioned methods for generating large-model agricultural intelligent question-answering based on knowledge graph enhanced retrieval when running.
[0054] By means of the above-mentioned technical scheme, the embodiments of the present application provide a method and system for generating large-model agricultural intelligent question and answer based on knowledge graph enhanced retrieval, an electronic device, a storage medium and a computer program product. The method can use agricultural pest and disease knowledge question and answer data to perform large-model fine-tuning pre-training learning through a low-rank adaptive algorithm to obtain a large model after fine-tuning and pre-training; obtain user input questions related to agricultural pests and diseases, and use a preset vector model to perform digital vector representation on the user input questions to obtain the represented question vector; based on the represented question vector, use a pre-built agricultural pest and disease data knowledge graph to perform enhanced retrieval to generate contextual knowledge fragments related to the user input questions; integrate the user input questions, enhanced retrieval steps and contextual knowledge fragments related to the user input questions to form a prompt chain, and provide the prompt chain to the large model after fine-tuning and pre-training to obtain the intelligent answer results returned by the large model after fine-tuning and pre-training. It can be seen that this embodiment uses a pre-trained embedding large model fine-tuned based on a low-rank adaptive algorithm to obtain better user text vector representation; agricultural pest and disease data knowledge graph retrieval can obtain relevant entities and relationships, as well as corresponding knowledge fragments, which provides a basis for the large model to generate content and can avoid the "hallucination" phenomenon; by parsing user input questions to form chain prompts, the transparency and explainability of the large model reasoning are improved; this embodiment can be used to help agricultural personnel quickly obtain pest and disease query results, save time, and improve work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.
[0056] Figure 1 A flowchart of a method for generating a large-model agricultural intelligent question-answering method based on knowledge graph enhanced retrieval provided in an embodiment of the present application is shown;
[0057] Figure 2 A flowchart of a method for generating a large-model agricultural intelligent question-answering method based on knowledge graph enhanced retrieval provided by another embodiment of the present application is shown;
[0058] Figure 3 A class diagram of a fine-tuned agricultural large language model based on a low-rank adaptive algorithm (i.e., LoRA) provided in an embodiment of the present application is shown;
[0059] Figure 4 The agricultural knowledge graph construction diagram provided by the embodiment of the present application is shown;
[0060] Figure 5 The agricultural intelligent question-answering class diagram provided by the embodiment of the present application is shown;
[0061] Figure 6The structure diagram of the large-model agricultural intelligent question-answering system based on knowledge graph enhanced retrieval provided in the embodiment of the present application is shown;
[0062] Figure 7 A structural diagram of a large-model agricultural intelligent question-answering system based on knowledge graph enhanced retrieval provided by another embodiment of the present application is shown;
[0063] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0064] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.
[0065] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such use is interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "including" and its variants are to be interpreted as open-ended terms meaning "including but not limited to".
[0066] Before introducing this embodiment in detail, the following technical terms are introduced first.
[0067] A generative large model is a large model or a large language model that has the capabilities of natural language processing, image processing, or speech processing, and has the ability to generate content such as text, pictures, audio, and video.
[0068] The big model, also known as the big language model, is, first of all, large in scale, as the name implies, with network parameters reaching tens of billions, hundreds of billions or even more; second, universal, meaning it is not limited to specific problems or fields; third, emergent, that is, it generates unexpected new capabilities.
[0069] The prompt in the prompt project is a piece of text or question provided to the generated large model to guide the generated large model to generate a specific type of text or answer. The prompt can be a sentence, a question, an article or a topic, etc. It can guide the generated large model to generate information or answers related to the prompt.
[0070] For example, when given the question “What is artificial intelligence?”, the generative model can generate an article or answer about artificial intelligence.
[0071] Existing intelligent question-answering technologies use pre-trained large language generation models, and learn by designing prompt engineering or fine-tuning in specific fields. Prompt engineering can optimize model output to a certain extent, but its capabilities are limited. The model token upper limit and inference cost determine the effectiveness of prompt engineering. The token here refers to the smallest unit into which the text is segmented, which can be a word, phrase, symbol or subword, depending on the word segmentation method used.
[0072] Dependency hint engineering may not achieve the expected results when dealing with long texts and complex tasks. Although pre-trained large models are usually trained with general data, their performance in specific domains may not be satisfactory. By fine-tuning on datasets on specific domains, the model can master more domain-specific knowledge and improve its performance in these domains. Although large language models have made significant progress, the models generate wrong or non-existent information and contextual understanding, the models cannot understand the subtle relationship between different information, and the transparency and explainability of the reasoning process are insufficient.
[0073] In order to solve the above technical problems, the embodiment of the present application provides a method for generating large-model agricultural intelligent question answering based on knowledge graph enhanced retrieval, such as Figure 1 As shown, the method for generating a large-model agricultural intelligent question answering method based on knowledge graph enhanced retrieval may include the following steps S101 to S104:
[0074] Step S101, using a low-rank adaptive algorithm, uses agricultural pest and disease knowledge question and answer data to perform fine-tuning pre-training learning on a large model to obtain a large model after fine-tuning pre-training.
[0075] In this step, low-rank adaptation algorithms, or LoRA, are a class of techniques that aim to reduce the complexity of large models by approximating their high-dimensional structure with a low-dimensional structure. In the context of language models, this means creating a smaller, more manageable representation of the original model that still performs well for a specific task or domain.
[0076] LoRA fine-tuning is to add additional low-rank matrices to the specified parameters on the large language model, and only train the additional parameters during model training. The decomposed matrix is learned through back-propagation, and the new weight matrix adapted to the task is decomposed into a low-dimensional matrix without losing too much information.
[0077] Step S102, obtaining user input questions related to agricultural pests and diseases, and using a preset vector model to perform digital vector representation on the user input questions to obtain a represented question vector.
[0078] Step S103, based on the represented question vector, use the pre-built agricultural pest data knowledge graph to perform enhanced retrieval and generate contextual knowledge fragments related to the user input question.
[0079] Step S104, integrating the user input question, enhanced retrieval steps and contextual knowledge fragments related to the user input question to form a prompt chain, providing the prompt chain to the fine-tuned pre-trained large model, and obtaining the intelligent answer result returned by the fine-tuned pre-trained large model.
[0080] This embodiment uses a pre-trained embedding large model fine-tuned based on a low-rank adaptive algorithm to obtain better user text vector representation; agricultural pest and disease data knowledge graph retrieval can obtain relevant entities and relationships, as well as corresponding knowledge fragments, providing a basis for the large model to generate content and avoiding the "hallucination" phenomenon; by parsing user input questions to form chain prompts, the transparency and explainability of the large model reasoning are improved; this embodiment can be used to help agricultural personnel quickly obtain pest and disease query results, save time, and improve work efficiency.
[0081] The present application provides a possible implementation method. In the above step S101, a low-rank adaptive algorithm is used to perform large-model fine-tuning pre-training learning using agricultural pest knowledge question-and-answer data to obtain a large model after fine-tuning pre-training. Specifically, the following steps A1 to A4 may be included:
[0082] Step A1, preprocess the agricultural pest knowledge question and answer data, convert it into a data set containing instructions, context, and output content, and divide the data set into a training data set and a test data set, and save the data set as a json file.
[0083] In this step, JSON is a lightweight data exchange format that is easy for humans to read and write, and easy for machines to parse and generate.
[0084] Step A2: configure a single-card GPU operating environment on the server and select a preset large base model.
[0085] Step A3, statistically collects the maximum lengths of instructions, contexts, and output contents, and uses these to set the length of natural language instructions, the maximum length of input text, and the maximum length of output text.
[0086] Step A4, through a low-rank adaptive algorithm, combined with the set length of the natural language instruction, the maximum length of the input text, and the maximum length of the output text, uses the training data set and the test data set to fine-tune the preset base large model for pre-training learning to obtain a fine-tuned pre-trained large model.
[0087] This embodiment pre-processes the agricultural pest and disease knowledge question and answer data and converts it into a structured data set containing instructions, context, and output content, which helps the model to better understand and learn the information in the data; sets the input and output parameters of the model according to the maximum length of the instructions, context, and output content in the data set, which helps the model adapt to input and output requirements of different lengths; uses a low-rank adaptive algorithm to fine-tune the preset base large model for pre-training learning, which is an optimization algorithm that can improve the performance of the model on specific tasks; through the above steps, a fine-tuned pre-trained model is finally obtained, which is specially optimized for the agricultural pest and disease knowledge question and answer task, and is expected to provide more accurate and efficient question and answer services.
[0088] The present application provides a possible implementation method, which can construct the agricultural pest data knowledge graph mentioned in step S103 above by following the steps B1 to B3:
[0089] Step B1, obtaining an agricultural professional knowledge file from a designated agricultural data source, and preprocessing the agricultural professional knowledge file to obtain a preprocessed document;
[0090] Step B2, segmenting the preprocessed document using a fixed-size block strategy to obtain segmented paragraphs;
[0091] Step B3, extracting entities, attributes and relationships from the segmented paragraphs, embedding the extracted entities, attributes and relationships into vectors and storing them in the Neo4j graph database to obtain a knowledge graph of agricultural pests and diseases data; wherein, in the graph database, data is stored in the form of a graph, nodes represent entities, relationships represent connections between entities, and attributes provide additional data descriptions.
[0092] This embodiment first obtains professional knowledge files from the designated agricultural data source, and then preprocesses these documents to facilitate subsequent processing and analysis; the preprocessing may include steps such as removing irrelevant information and formatting text; the segmented paragraphs are deeply analyzed to extract entities, attributes and relationships, where entities may refer to the names of pests and diseases, types of crops, etc.; attributes may include the characteristics and impacts of pests and diseases; relationships describe the connections between entities, such as the relationship between a certain pest and disease and a specific crop; through the above steps, an agricultural pest and disease data knowledge graph is constructed. In a graph database, data is stored in the form of a graph, where nodes represent entities, edges represent connections between entities, and attributes provide additional descriptive information for nodes.
[0093] In the embodiment of the present application, a possible implementation method is provided. When constructing a knowledge graph of agricultural pest data, the following steps B4 and B5 may also be included:
[0094] Step B4, vectorizing the segmented paragraphs to obtain vectorized paragraphs;
[0095] Step B5: construct an index for the vectorized paragraphs to generate an agricultural knowledge vector library.
[0096] This embodiment performs vectorization processing on the segmented paragraphs, that is, converts the text data into numerical vectors so that the machine learning model can process it; an index is built on the vectorized paragraphs, which helps to quickly retrieve and access specific text information. The construction of the index can improve query efficiency and make it faster to find relevant information in large-scale data sets.
[0097] The embodiment of the present application provides a possible implementation method. In the above step S103, based on the characterized question vector, a pre-built agricultural pest data knowledge graph is used to perform enhanced retrieval to generate contextual knowledge fragments related to the user input question, which may specifically include the following steps C1 to C3:
[0098] Step C1, based on the represented question vector, identify similar nodes in the knowledge graph of agricultural pests and diseases data; wherein similar nodes jump around entities and relationships several times;
[0099] Step C2, using the attribute data of similar nodes as contextual knowledge fragments related to the user input question;
[0100] Step C3, generates enhanced retrieval steps based on the represented question vector, similar nodes and edge dependencies.
[0101] This embodiment can obtain relevant context data of user input question queries through the knowledge graph enhanced retrieval model, so that the responses generated by the large language model are not only more accurate, but also include explanations that can be traced back to specific data entities and related relationships, thereby enhancing their reliability and practicality in practical applications.
[0102] A possible implementation method is provided in an embodiment of the present application. The above step C1 identifies similar nodes in the agricultural pest and disease data knowledge graph based on the represented question vector. Specifically, it can be based on the represented question vector, perform graph node retrieval in the agricultural pest and disease data knowledge graph, implement subgraph traversal, search for local subgraphs within several jumps, and obtain similar entity nodes and knowledge texts by calculating the cosine similarity between nodes.
[0103] This embodiment achieves the ability to quickly and accurately retrieve and provide relevant knowledge information in the agricultural pest data knowledge graph through the characterized question vector, graph node retrieval, subgraph traversal and cosine similarity calculation.
[0104] The above introduces Figure 1There are multiple implementation methods for each link of the illustrated embodiment. The method for generating a large-model agricultural intelligent question-answering method based on knowledge graph enhanced retrieval according to the embodiment of the present application will be further explained below through specific embodiments.
[0105] like Figure 2 As shown, the user input questions are preprocessed, and the agricultural pest and disease data knowledge graph is used to enhance the retrieval, and the relevant entities and corresponding agricultural knowledge fragments are screened; the user input questions are integrated with the results of the enhanced retrieval to form a chain prompt input model to generate answers, which can improve the accuracy and interpretability of the generated content.
[0106] Based on LoRA fine-tuning agricultural data large language model see Figure 3 As shown, the steps include:
[0107] The agricultural pest and disease knowledge question and answer data is preprocessed and converted into json files in the form of instructions, context, and output, and the data set is divided into a training data set and a test data set.
[0108] The user input is tokenized through the word embedding model, and the LoRA layer is added to the pre-trained network model. The output of this layer is summed with the output of the multi-head attention layer and the feedforward network layer as the input of the normalization layer. The LoRA layer is amplified, input from the hidden layer, and the model parameters are trained to output h to the hidden layer.
[0109] Specifically, the weight matrix of the LoRA model can be expressed as w=w0+Δw, where w0 is the original weight of the pre-trained model, and Δw is the product of two matrices obtained by low-rank decomposition, which is expressed as Δw=A×B. The dimension of A is (m, r), m represents the size of the input dimension of the layer, the dimension of B is (r, n), n represents the size of the output dimension of the layer, and r is the rank of the matrix, which is generally 8.
[0110] The ranks of A and B are much smaller than w0, which reduces the number of parameters that need to be adjusted. The specific implementation is as follows:
[0111] (1) Select the weight matrix of the base pre-trained large model to initialize w0;
[0112] (2) Initialize the low-rank matrix A to zero and randomly initialize the matrix B;
[0113] (3) During fine-tuning, the original weights of the pre-trained model are frozen and only the parameters of the low-rank matrices A and B are adjusted.
[0114] Back-propagation updates network parameters using an adaptive gradient descent strategy. The learning rate is automatically updated and adjusted according to the step size of the previous state learning, so that the model converges in the direction of the fastest gradient descent, thereby quickly finding the global optimal solution.
[0115] like Figure 4 The figure shows the process of building a knowledge graph of agricultural pests and diseases data, including:
[0116] (1) Input agricultural professional knowledge files in formats such as pdf, word, and txt. Use a dedicated document loader to convert these knowledge sources into plain text data that can be understood by the large language model. Preprocess the text to remove junk characters such as *%@;
[0117] (2) Fixed-size block segmentation of text, by setting the block size to 512 and the block overlap ratio to 24, is a simple method with low computational cost and is easy to process by the model;
[0118] (3) Extract proper noun entities such as drug names, disease names, and place of origin based on named entity recognition technology, extract relationships and attributes based on grammatical semantic rules and neural networks, and store the extracted entities, node attributes, and relationships using Neo4j;
[0119] (4) Encode the attributes of diseases, drugs, and other entities in the knowledge graph into a vector space, and in a similar way transform the relationships into vectors that preserve the semantic content of the graph structure, and the corresponding text fragments are also saved in the form of vectors;
[0120] (5) In order to improve the speed and accuracy of queries, we build an index and perform a brute force full search based on cosine similarity.
[0121] The similarity is calculated using cosine similarity. The larger the value of the calculation result, the greater the similarity; otherwise, the smaller the similarity. Cosine similarity is to calculate the cosine value of the angle between the vector and the vector to measure the distance and similarity between the two vectors. X and Y represent the user query feature vector and the candidate node feature vector, respectively. The calculation method is as shown in formula (1). Since the value range of cosθ is -1 to 1, it is more desirable to normalize it to 0 to 1 in practice. Therefore, when calculating cosine similarity, the transformation formula is usually as shown in (2).
[0122]
[0123]
[0124] in, It means to calculate the dot product between two vectors. is the magnitude of the vector, are the elements of vector X, yi are the elements of vector Y.
[0125] A large-model agricultural intelligent question answering method based on knowledge graph enhanced retrieval, such as Figure 5 As shown, the method includes:
[0126] (1) Analyze and extract the user input questions to obtain entities, relations, and attributes, vectorize the user input questions through the embedding model, extract keyword information, and pre-process their capitalization, alias, and synonyms;
[0127] (2) Retrieve graph nodes based on keywords and vectors, implement subgraph traversal, search local subgraphs within a few hops, and obtain similar entity nodes and knowledge texts, that is, obtain the Topk related nodes from the knowledge graph. Topk refers to selecting the first k elements most relevant to the query in a certain set, where k is a positive integer, indicating the number of elements that the user or system wants to obtain.
[0128] (3) Based on the query traversal subgraph steps, a chain prompt project is formed, and the local subgraph data is formatted into text. It is used as the context and the question to form the Prompt Context, which is submitted to the LLM large model for processing, generates user questions and answers, and returns the user question and answer results.
[0129] It should be noted that the size of the sequence number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. In practical applications, all the above possible implementation methods can be combined in any way to form possible embodiments of the present application, which will not be described one by one here.
[0130] Based on the methods for generating large-model agricultural intelligent question and answer based on knowledge graph enhanced retrieval provided in the above embodiments, and based on the same inventive concept, the embodiments of the present application also provide a system for generating large-model agricultural intelligent question and answer based on knowledge graph enhanced retrieval.
[0131] Figure 6 : is a structural diagram of a large-model agricultural intelligent question-answering system based on knowledge graph enhanced retrieval provided in an embodiment of the present application. Figure 6 As shown, the large-model agricultural intelligent question-answering system based on knowledge graph enhanced retrieval can specifically include a fine-tuning unit 610, a vectorization unit 620, an enhanced retrieval unit 630 and an answer generation unit 640.
[0132] A fine-tuning unit 610 is used to perform fine-tuning pre-training learning of a large model using agricultural pest and disease knowledge question and answer data through a low-rank adaptive algorithm to obtain a large model after fine-tuning pre-training;
[0133] A vectorization unit 620 is used to obtain user input questions related to agricultural pests and diseases, and to perform digital vector representation on the user input questions using a preset vector model to obtain a represented question vector;
[0134] An enhanced retrieval unit 630 is used to perform enhanced retrieval based on the represented question vector using a pre-built agricultural pest data knowledge graph to generate contextual knowledge fragments related to the user input question;
[0135] The answer generation unit 640 is used to integrate the user input question, enhanced retrieval steps and contextual knowledge fragments related to the user input question to form a prompt chain, and provide the prompt chain to the fine-tuned pre-trained large model to obtain the intelligent answer result returned by the fine-tuned pre-trained large model.
[0136] A possible implementation method is provided in an embodiment of the present application, wherein the fine-tuning unit 610 is further configured to:
[0137] Preprocess the agricultural pest knowledge question-and-answer data and convert it into a data set containing instructions, context, and output content. The data set is divided into a training data set and a test data set, and the data set is saved as a json file.
[0138] Configure a single-card GPU operating environment on the server and select the preset large base model;
[0139] Statistics are collected to collect the maximum lengths of commands, contexts, and output contents, which are used to set the length of natural language commands, the maximum length of input text, and the maximum length of output text.
[0140] Through a low-rank adaptive algorithm, combined with the set length of the natural language instruction, the maximum length of the input text, and the maximum length of the output text, the preset base large model is fine-tuned and pre-trained using the training data set and the test data set to obtain a large model after fine-tuning and pre-training.
[0141] A possible implementation method is provided in the embodiment of the present application, such as Figure 7 As shown above Figure 6 The system shown may further include a construction unit 710 for:
[0142] Acquire agricultural professional knowledge files from a designated agricultural data source, and pre-process the agricultural professional knowledge files to obtain pre-processed documents;
[0143] The preprocessed document is segmented using a fixed-size block strategy to obtain segmented paragraphs;
[0144] Entities, attributes and relationships are extracted from the segmented paragraphs, and the extracted entities, attributes and relationships are vector-embedded and stored in the Neo4j graph database to obtain the knowledge graph of agricultural pests and diseases data. In the graph database, data is stored in the form of a graph, with nodes representing entities, relationships representing connections between entities, and attributes providing additional data descriptions.
[0145] A possible implementation method is provided in an embodiment of the present application, wherein the construction unit 710 is further configured to:
[0146] Vectorize the segmented paragraphs to obtain vectorized paragraphs;
[0147] Build an index for the vectorized paragraphs and generate an agricultural knowledge vector library.
[0148] A possible implementation method is provided in an embodiment of the present application, wherein the enhanced retrieval unit 630 is further configured to:
[0149] Based on the represented question vector, similar nodes in the knowledge graph of agricultural pests and diseases data are identified; similar nodes jump around entities and relationships several times;
[0150] The attribute data of similar nodes are used as contextual knowledge fragments related to the user input question;
[0151] The enhanced retrieval step is generated based on the represented question vector, similar nodes and edge dependencies.
[0152] A possible implementation method is provided in an embodiment of the present application, wherein the enhanced retrieval unit 630 is further configured to:
[0153] Based on the represented question vector, graph node retrieval is performed in the knowledge graph of agricultural pest and disease data to realize subgraph traversal, search for local subgraphs within several jumps, and obtain similar entity nodes and knowledge texts by calculating the cosine similarity between nodes.
[0154] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of the above-mentioned embodiments of the method for generating large-model agricultural intelligent question and answer based on knowledge graph enhanced retrieval.
[0155] In an exemplary embodiment, an electronic device is provided, such as Figure 8 As shown, Figure 8The electronic device 800 shown includes: a processor 801 and a memory 803. The processor 801 and the memory 803 are connected, such as through a bus 802. Optionally, the electronic device 800 may also include a transceiver 804. It should be noted that in actual applications, the transceiver 804 is not limited to one, and the structure of the electronic device 800 does not constitute a limitation on the embodiments of the present application.
[0156] Processor 801 may be a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 801 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0157] The bus 802 may include a path to transmit information between the above components. The bus 802 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 802 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0158] The memory 803 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.
[0159] The memory 803 is used to store the computer program code for executing the solution of the present application, and the execution is controlled by the processor 801. The processor 801 is used to execute the computer program code stored in the memory 803 to implement the contents shown in the above method embodiment.
[0160] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0161] Based on the same inventive concept, an embodiment of the present application also provides a storage medium, which stores a computer program, wherein the computer program is configured to execute any one of the above-mentioned embodiments of the method for generating large-model agricultural intelligent question and answer based on knowledge graph enhanced retrieval when running.
[0162] Based on the same inventive concept, an embodiment of the present application also provides a computer program product, including a computer program, which is configured to execute any one of the above-mentioned embodiments of the method for generating large-model agricultural intelligent question and answer based on knowledge graph enhanced retrieval when running.
[0163] Those skilled in the art can clearly understand that the specific working processes of the systems, devices, and modules described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they are not further described here.
[0164] Those skilled in the art can understand that the technical solution of the present application can be essentially or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, which includes a number of program instructions, so that an electronic device (such as a personal computer, a server, or a network device, etc.) executes all or part of the steps of the method described in each embodiment of the present application when running the program instructions. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0165] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as electronic devices such as personal computers, servers, or network devices), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of an electronic device, the electronic device executes all or part of the steps of the methods described in the embodiments of the present application.
[0166] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that within the spirit and principles of the present application, they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate from the protection scope of the present application.
Claims
1. A method for generating large-model agricultural intelligent question-answering based on knowledge graph enhanced retrieval, characterized in that: include: Through the low-rank adaptive algorithm, the agricultural pest knowledge question and answer data is used to perform fine-tuning pre-training learning on the large model to obtain a large model after fine-tuning pre-training; Obtain user input questions related to agricultural pests and diseases, perform digital vector representation on the user input questions using a preset vector model, and obtain a represented question vector; Based on the represented question vector, the pre-built agricultural pest and disease data knowledge graph is used for enhanced retrieval to generate contextual knowledge fragments related to the user input question; The prompt chain is formed by integrating the user input question, enhanced retrieval steps and contextual knowledge fragments related to the user input question, and the prompt chain is provided to the fine-tuned pre-trained large model to obtain the intelligent answer result returned by the fine-tuned pre-trained large model.
2. The method according to claim 1, characterized in that Through the low-rank adaptive algorithm, the agricultural pest knowledge question and answer data is used to perform large model fine-tuning pre-training learning, and the large model after fine-tuning pre-training is obtained, including: Preprocess the agricultural pest knowledge question-and-answer data and convert it into a data set containing instructions, context, and output content. The data set is divided into a training data set and a test data set, and the data set is saved as a json file. Configure a single-card GPU operating environment on the server and select the preset large base model; Statistics are collected to collect the maximum lengths of commands, contexts, and output contents, which are used to set the length of natural language commands, the maximum length of input text, and the maximum length of output text. Through a low-rank adaptive algorithm, combined with the set length of the natural language instruction, the maximum length of the input text, and the maximum length of the output text, the preset base large model is fine-tuned and pre-trained using the training data set and the test data set to obtain a large model after fine-tuning and pre-training.
3. The method according to claim 1, characterized in that The knowledge graph of agricultural pests and diseases data is constructed through the following steps: Acquire agricultural professional knowledge files from a designated agricultural data source, and pre-process the agricultural professional knowledge files to obtain pre-processed documents; The preprocessed document is segmented using a fixed-size block strategy to obtain segmented paragraphs; Entities, attributes and relationships are extracted from the segmented paragraphs, and the extracted entities, attributes and relationships are vector-embedded and stored in the Neo4j graph database to obtain the knowledge graph of agricultural pests and diseases data. In the graph database, data is stored in the form of a graph, with nodes representing entities, relationships representing connections between entities, and attributes providing additional data descriptions.
4. The method according to claim 3, characterized in that The method further comprises: Vectorize the segmented paragraphs to obtain vectorized paragraphs; Build an index for the vectorized paragraphs and generate an agricultural knowledge vector library.
5. The method according to claim 3, characterized in that: Based on the represented question vector, the pre-built agricultural pest and disease data knowledge graph is used for enhanced retrieval to generate contextual knowledge fragments related to the user input question, including: Based on the represented question vector, similar nodes in the knowledge graph of agricultural pests and diseases data are identified; similar nodes jump around entities and relationships several times; The attribute data of similar nodes are used as contextual knowledge fragments related to the user input question; The enhanced retrieval step is generated based on the represented question vector, similar nodes and edge dependencies.
6. The method according to claim 5, characterized in that Based on the represented question vector, similar nodes in the knowledge graph of agricultural pests and diseases data are identified, including: Based on the represented question vector, graph node retrieval is performed in the knowledge graph of agricultural pest and disease data to realize subgraph traversal, search for local subgraphs within several jumps, and obtain similar entity nodes and knowledge texts by calculating the cosine similarity between nodes.
7. A large-model agricultural intelligent question-answering system based on knowledge graph enhanced retrieval, characterized in that: include: A fine-tuning unit is used to perform fine-tuning pre-training learning of a large model using agricultural pest and disease knowledge question-answering data through a low-rank adaptive algorithm to obtain a large model after fine-tuning pre-training; A vectorization unit is used to obtain user input questions related to agricultural pests and diseases, and to perform digital vector representation on the user input questions using a preset vector model to obtain a represented question vector; The enhanced retrieval unit is used to perform enhanced retrieval based on the represented question vector using the pre-built agricultural pest and disease data knowledge graph to generate contextual knowledge fragments related to the user input question; The answer generation unit is used to integrate the user input question, enhanced retrieval steps and contextual knowledge fragments related to the user input question to form a prompt chain, and provide the prompt chain to the fine-tuned pre-trained large model to obtain the intelligent answer result returned by the fine-tuned pre-trained large model.
8. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the method for generating large-model agricultural intelligent question-answering based on knowledge graph enhanced retrieval as described in any one of claims 1 to 6.
9. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method for generating large-model agricultural intelligent question-answering based on knowledge graph enhanced retrieval according to any one of claims 1 to 6 when running.
10. A computer program product, comprising a computer program, characterized in that The computer program is configured to execute the method for generating large-model agricultural intelligent question-answering based on knowledge graph enhanced retrieval according to any one of claims 1 to 6 when running.
Citation Information
Patent Citations
Stylized legal consultation question and answer method and system, storage medium and equipment
CN116821297A
Knowledge graph generation type question answering method and system based on large language model
CN117033608A
Electric power defect image detection method based on image-text question-answer multi-modal model
CN117763107A
Model temperature self-adaptive fine tuning method for intelligent breeding questions and answers
CN117951276A
Intelligent session method and server based on table data retrieval
EP4339799A1
Cited By
Large model generation content information security enhancement-oriented system and method
CN119558403A
Method for finely adjusting TinyLma LLM
CN119718265A
Multi-hop agricultural question-answering system based on knowledge graph reasoning and large model and text fragment retrieval and answer generation method thereof
CN119782467A
Scientific and technological document question and answer method and device, storage medium and electronic equipment
CN120296137A
Whole-crop explainable disease and pest diagnosis method and system based on multi-modal large model
CN121303328A
