Intelligent decision-making method and system for herdsmen based on large language model and retrieval enhancement generation
By combining large language models with retrieval-enhanced generation technology, an intelligent decision-making system for herders was constructed, which solved the problem of incomplete information for herders' decision-making under climate change, provided personalized decision-making suggestions, and improved the intelligence level and production efficiency of herders' breeding.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTH CHINA AGRICULTURAL UNIVERSITY
- Filing Date
- 2025-08-14
- Publication Date
- 2026-05-26
AI Technical Summary
Herders lack comprehensive information on climate change, leading to decision-making errors. Existing large language models suffer from knowledge illusions and poor data real-time performance in professional applications, making it difficult to provide reliable intelligent decision support.
By combining large language models with retrieval-enhanced generation technology, a herder intelligent decision-making system is constructed. By collecting herder information and pastoral case studies, a database is built and vectorized. Hybrid retrieval-enhanced generation and graph neural network optimization are used to generate personalized intelligent decision-making data for herders and display it visually.
It enables herders to make intelligent decisions under extreme weather conditions, helping them optimize resource utilization and predict weather changes, ensure livestock health and safety, and improve production efficiency.
Smart Images
Figure CN121032259B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of big data intelligent analysis technology, and more specifically, to a method and system for intelligent decision-making by herders based on large language models and retrieval enhancement. Background Technology
[0002] The impact of climate change in pastoral areas on herders' decision-making has attracted widespread attention. In recent years, domestic scholars have conducted in-depth research on the impact of climate change on pastoral ecosystems, herders' strategies for adapting to climate change, and policy support. Although research has been conducted on herders' climate change adaptation behaviors, and it has been found that herders have gradually established strategies for coping with climate change risks based on family and community ties, including stockpiling more fodder, reducing livestock numbers, building warm sheds, selecting drought-resistant livestock breeds, nomadic rotational grazing, renting pastures, and relocating livestock, some herders still make decision-making errors due to limited cognitive abilities and incomplete information. In this case, the integration of natural language processing and artificial intelligence technologies can effectively assist herders' decision-making process and provide the possibility for efficient decision-making.
[0003] Large Language Models (LLMs), through deep learning techniques, can understand and generate natural language, capturing subtle differences between languages and maintaining contextual coherence in complex dialogues. However, LLMs suffer from problems such as knowledge illusions, poor data real-time performance, and a lack of domain-specific knowledge, making them unsuitable for highly specialized tasks and preventing direct application to specific scenarios. Retrieval-enhanced generation (RAG), one of the most advanced technologies in artificial intelligence, leverages reliable external knowledge combined with advanced natural language processing techniques and reinforcement learning algorithms to achieve deep understanding and accurate answers in specific domains. Combining RAG with LLMs enables large language models to acquire relevant domain information, effectively addressing issues such as knowledge illusions and a lack of domain-specific knowledge, and producing reliable output. Therefore, a key technical problem urgently needs to be solved: how to construct a herder intelligent decision-making model that integrates large language models and retrieval-enhanced generation, based on past cases and specific information about herders, to provide herders with intelligent decision-making under extreme weather conditions, helping them better adapt to climate change and optimize resource utilization. Summary of the Invention
[0004] To address the aforementioned technical issues, this invention proposes a method and system for intelligent decision-making by herders based on a large language model and retrieval enhancement, enabling intelligent management and efficient decision-making for herders' livestock breeding-related issues, improving the level of intelligence in herders' livestock breeding, and increasing breeding efficiency.
[0005] The first aspect of this invention provides a herder intelligent decision-making method based on a large language model and retrieval enhancement, comprising the following steps:
[0006] Collect and preprocess information on herders and pastoral cases, and use the preprocessed information on herders and pastoral cases to build a database of herder information and a database of herder cases, respectively.
[0007] The system obtains user questions and performs vector encoding. The encoded question vectors are then used for hybrid retrieval and enhancement in the herder information database and herder case database. Based on semantic adaptation, content that meets the requirements is obtained as the prompt context.
[0008] The prompt context is integrated with the user's original question to construct a prompt template, which is then imported into a preset large language model to obtain herders' intelligent decision-making data. The herders' intelligent decision-making data is then visualized.
[0009] In this scheme, information on herders and case studies of pastoral areas is collected and preprocessed. The preprocessed herder information and case study information are then used to construct a herder information database and a herder case database, respectively. Specifically:
[0010] Collect information on herders' family population structure, livestock types and numbers, grassland area, existing resources and historical decision-making records as herder information, and collect records of climate events in pastoral areas and historical decision-making records of responses as pastoral case information.
[0011] The herder information and pastoral case information are cleaned, missing values are removed and standardized. The preprocessed herder information and pastoral case information are segmented to obtain corresponding text blocks. Transform encoding is used to convert the herder text and pastoral case text into initial vectors.
[0012] The initial vector is deeply encoded by a pre-defined large language model to generate herder information embedding vectors and pastoral case embedding vectors. The herder information embedding vectors are associated with and stored with the original text to build an index and generate a herder information database. The pastoral case embedding vectors are associated with and stored with case descriptions to build an index and generate a pastoral case database.
[0013] In this solution, user questions are obtained and vectorized. The encoded question vectors are then used for enhanced generation through a hybrid retrieval process using the herder information database and the herder case database. Specifically:
[0014] The user-input questions are converted into dense vectors using a pre-trained RoBERTa language model. The dense vectors are then stored in a vector database in the form of high-dimensional floating-point numbers. The herder information database and herder case database are modeled as a knowledge graph.
[0015] A subgraph retrieval model is constructed using a graph neural network. The nodes and relationships in the knowledge graph are embedded and represented. The dense vector corresponding to the user's question is input into the subgraph retrieval model. The similarity between the dense vector and the node embedding vector in the knowledge graph is calculated. The most relevant entity nodes are selected based on a preset number.
[0016] The graph attention mechanism dynamically assigns association weights to the selected entity nodes, and retains entity nodes with association weights higher than a preset weight threshold and edges directly connected to the entity nodes to construct an initial subgraph.
[0017] The reciprocal of the attention score obtained from the graph attention mechanism is used as the edge weight in the initial subgraph. Starting from the core node in the initial subgraph that is most relevant to the problem, the path originates from the core node and the Dijkstra algorithm is used to obtain the shortest path with the minimum sum of edge weights from the path origin to other nodes.
[0018] Obtain a preset number of shortest paths, and optimize the initial subgraph by verifying the semantic coherence of the shortest paths through a pre-trained model. Then, take the original text corresponding to the optimized subgraph as candidates and return the top n most relevant text segments.
[0019] In this solution, user questions are obtained and vectorized. The encoded question vectors are then used for enhanced generation through a hybrid retrieval process using the herder information database and the herder case database. Specifically:
[0020] The user-input question is converted into a sparse vector using a pre-trained CodeBERT model. The sparse vector is then input into a Siamese neural network, and similarity calculation is performed between it and the summary vectors in the herder information database and the herder case database.
[0021] The similarity calculation retrieves a preset number of semantically closest summary vectors and their corresponding question-and-answer text blocks. Each question-and-answer text block is segmented and labeled with part-of-speech tags. Each word vector is used as a node. The grammatical relationship between words is determined based on grammatical dependency analysis. Directed edges are constructed. The question-and-answer text blocks are converted into a directed graph based on the nodes and directed edges.
[0022] The shortest path graph kernel function is used to calculate the syntactic structural similarity between the user question and the answer graph, and the top m text blocks with the best structural matching are returned. The n text segments retrieved from the subgraph and the m text segments retrieved from the sparse graph are merged, and duplicate content is removed.
[0023] In this solution, content that meets the requirements is obtained based on semantic adaptation and used as the prompt context, specifically:
[0024] The user question and the deduplicated text block are concatenated into a sequence and input into the embedding layer of the LLaMA2 model to generate a joint representation. The cross-attention score between the question and the text block is calculated through the MHA multi-head attention mechanism of LLaMA2. The semantic fit is obtained by averaging the cross-attention score.
[0025] The graph attention score of subgraph retrieval and the SPGK score of sparse retrieval are obtained as the original retrieval score. The semantic fit and the original retrieval score are dynamically weighted to obtain the final fit score. The final fit scores are sorted in descending order, and a preset number of text blocks are selected as the prompt context.
[0026] In this solution, the prompt context is integrated with the user's original question to construct a prompt template, which is then imported into a pre-set large language model to obtain intelligent decision-making data from herders. Specifically:
[0027] The user question is used as the core anchor point at the beginning of the prompt template. Text blocks arranged in descending order of the final fit score are inserted into the prompt template according to priority. Text blocks with a final fit score greater than the threshold are considered high fit texts, and the remaining text blocks are considered low fit texts. In the prompt template, the high fit texts are adjacent to the question, and the low fit texts are used as supplements.
[0028] Key tags are added to the domain entities and relationships in the text blocks to guide the model to focus on structured information. Weight coefficients are assigned to the text blocks based on the final score of fit, and the influence of the text blocks is dynamically adjusted.
[0029] A contrastive learning strategy is introduced, in which the combination of the text block with the highest final fit score and the user question is taken as a positive sample, and the combination of the user question with the user question is randomly sampled as a negative sample. Through contrastive learning, the semantic representation of the model output in the positive sample is brought closer, while the representation of the negative sample is pushed further away.
[0030] Using a small number of labeled samples of herders' decisions, we adjusted the generation strategy of the prompt template, trained only the dynamic weight parameters and tag embeddings in the prompt template, and froze the backbone parameters of the large language model.
[0031] Import the prompt template into the preset large language model, automatically identify high-frequency entities and numerical information, and generate intelligent decision-making data for herdsmen according to the structure of "problem → measure → expected effect".
[0032] In this solution, the intelligent decision-making data for herders is optimized based on user feedback, specifically as follows:
[0033] Obtain user feedback ratings for the herders' intelligent decision-making data. If the user feedback rating for the herders' intelligent decision-making data is lower than the threshold, record the ignored context text block in the prompt template and adjust the weight coefficient of the recorded context text block in subsequent searches.
[0034] In addition, the positive and negative sample database is dynamically updated by judging whether users accept or reject the generated answer based on the feedback rating of the herders' intelligent decision-making data.
[0035] The second aspect of this invention provides a herder intelligent decision-making system based on a large language model and retrieval enhancement, the system comprising: a data acquisition module, a herder intelligent decision-making module, and a data visualization module;
[0036] The data acquisition module collects herder information and pastoral case information and preprocesses them. The preprocessed herder information and pastoral case information are then used to construct a herder information database and a herder case database, respectively.
[0037] The herder intelligent decision-making module acquires the user's question and performs vectorization encoding. The encoded question vector is then enhanced by hybrid retrieval in the herder information database and herder case database. Based on semantic adaptation, content that meets the requirements is obtained as a prompt context. The prompt context is then fused with the user's original question to construct a prompt template, which is then imported into a preset large language model to obtain herder intelligent decision-making data.
[0038] The data visualization module returns the output intelligent decision-making data of herdsmen to the server and uses a visualization dashboard to visualize the intelligent decision-making data of herdsmen on the server.
[0039] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0040] This invention constructs a retrieval-enhanced generative model for intelligent decision-making by herders, providing them with personalized decisions and suggestions. Through an efficient and practical intelligent decision-making system for herders, the system realizes and displays the results of their intelligent decisions. This not only helps herders accurately manage grassland resources and optimize grazing strategies, but also predicts weather changes and provides early warnings of natural disasters, thereby ensuring livestock health and safety and improving production efficiency. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments or examples of the present invention, the drawings used in the embodiments or examples will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained according to these drawings without creative effort.
[0042] Figure 1 A flowchart of a herder intelligent decision-making method based on a large language model and retrieval enhancement is shown.
[0043] Figure 2The flowchart of dense vector retrieval and knowledge graph subgraph optimization retrieval is shown;
[0044] Figure 3 A flowchart of sparse retrieval based on semantic and syntactic structure matching is shown;
[0045] Figure 4 A block diagram of a herder intelligent decision-making system based on a large language model and retrieval enhancement is shown. Detailed Implementation
[0046] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.
[0047] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.
[0048] like Figure 1 As shown, this embodiment provides a herder intelligent decision-making method based on a large language model and retrieval enhancement, including:
[0049] S102, collect herder information and pastoral case information and preprocess them, and use the preprocessed herder information and pastoral case information to construct herder information database and herder case database respectively;
[0050] S104. Obtain the user's question and perform vector encoding. Then, perform hybrid retrieval enhancement on the herder information database and herder case database to generate the question vector. Obtain content that meets the requirements as the prompt context based on semantic adaptation.
[0051] S106, the prompt context is integrated with the user's original question to construct a prompt template, which is then imported into a preset large language model to obtain herder intelligent decision-making data, and the herder intelligent decision-making data is then visualized.
[0052] It should be noted that the herder information is collected through manual questionnaires and historical case data collection, including family population structure, livestock types and numbers, grassland area, existing resources (such as greenhouses and fodder reserves), and historical decision-making records (such as measures to cope with extreme weather). Pastoral case information is collected through records of climate events and historical decision-making records in pastoral areas. The herder information and pastoral case information undergo data cleaning to remove duplicate records, irrelevant information, or noisy data, and to correct obvious errors. Missing important fields are completed and standardized. The preprocessed herder information and pastoral case information are then segmented to obtain corresponding text blocks. In the herder information segmentation, text is segmented at the granularity of "{}" to ensure that the complete information of each herder is stored in one text block. In the pastoral case segmentation, text is segmented at the sentence level, with a maximum length limit (e.g., 600 characters) to ensure sentence integrity. Transform encoding is used to convert herder texts and pastoral case texts into initial vectors. These initial vectors are then deep-encoded using a pre-defined large language model (e.g., Zhipu Qingyan's Embedding-2 API) to generate herder information embedding vectors and pastoral case embedding vectors. The herder information embedding vectors are then associated with and stored in the original text, forming an index to generate a herder information database. Similarly, the pastoral case embedding vectors are associated with and stored in the case descriptions, forming an index to generate a pastoral case database. The preprocessed data is stored in a server database and updated periodically to include newly collected herder information or cases, ensuring database backup and security to prevent data loss or leakage.
[0053] Figure 2 The flowchart of dense vector retrieval and knowledge graph subgraph optimization retrieval is shown.
[0054] According to an embodiment of the present invention, the user's question is obtained and vectorized, and the retrieval is optimized based on dense vector retrieval and knowledge graph subgraph, specifically as follows:
[0055] S202, the user-input question is converted into a dense vector through a pre-trained RoBERTa language model, the dense vector is stored in a vector database in the form of a high-dimensional floating-point number, and the herder information database and herder case database are modeled as a knowledge graph;
[0056] S204, a subgraph retrieval model is constructed using a graph neural network to embed the nodes and relationships in the knowledge graph. The dense vector corresponding to the user's question is input into the subgraph retrieval model, and the similarity between the dense vector and the node embedding vector in the knowledge graph is calculated. The most relevant entity nodes are selected based on a preset number.
[0057] S206, Dynamically assign association weights to the selected entity nodes through a graph attention mechanism, and retain entity nodes with association weights higher than a preset weight threshold and edges directly connected to the entity nodes to construct an initial subgraph.
[0058] S208. The reciprocal of the attention score obtained by the graph attention mechanism is used as the edge weight in the initial subgraph. Starting from the core node most relevant to the problem in the initial subgraph as the path starting point, the Dijkstra algorithm is used to obtain the shortest path with the minimum sum of edge weights from the path starting point to other nodes.
[0059] S210: Obtain a preset number of shortest paths, and optimize the initial subgraph by verifying the semantic coherence of the shortest paths through a pre-trained model. Then, take the original text corresponding to the optimized subgraph as candidates and return the top n most relevant text segments.
[0060] It should be noted that user questions are encoded as dense vectors using a pre-trained RoBERTa language model to represent the semantic information of the questions. The herder information database and the pastoral case database are modeled as knowledge graphs, where nodes represent entities (such as herders, pastures, livestock, climate events, etc.), and edge structures represent relationships between entities (such as herder A owning pasture B, case X dealing with drought Y). The nodes and edges in the knowledge graph are pre-encoded as vectors using graph embedding methods. The question vector is compared with the cosine similarity of all node vectors in the knowledge graph to select the top-N most relevant entity nodes. An attention layer is introduced into the GNN to dynamically assign association weights between the question and nodes and edges, outputting a weighted subgraph candidate set. The weights of nodes and edges reflect their relevance to the question. Based on the attention weights of GAT, nodes with weights higher than a threshold and their directly connected edges are retained to form the initial subgraph. In critical path extraction, the edge weights in the subgraph are set to the reciprocal of the attention score. Starting from the core node in the subgraph most relevant to the problem (such as a drought event), the path originates. Dijkstra's algorithm is used to calculate the shortest path from this node to other high-weight nodes, ensuring that the sum of edge weights is minimized. The top K shortest paths are output. Low-weight nodes that do not appear in any shortest path are removed to reduce noise. The semantic coherence of the path is verified through a pre-trained model, for example, checking whether "fodder allocation" and "drought" are logically related. Subgraph pruning and semantic verification are used to optimize the initial subgraph.
[0061] Figure 3 A flowchart of sparse retrieval based on semantic and syntactic structure matching is shown.
[0062] According to an embodiment of the present invention, the user's question is obtained and vectorized, and sparse retrieval based on semantic and syntactic structure matching is performed, specifically as follows:
[0063] S302, the user-input question is converted into a sparse vector through a pre-trained CodeBERT model, and the sparse vector is input into a Siamese neural network to calculate the similarity with the summary vectors in the herder information database and the herder case database;
[0064] S304, retrieve a preset number of semantically closest summary vectors and their corresponding question-answer text blocks through similarity calculation, perform word segmentation and part-of-speech tagging on each question-answer text block, take each word vector as a node, determine the grammatical relationship between words based on grammatical dependency analysis, construct directed edges, and convert the question-answer text block into a directed graph based on the nodes and directed edges;
[0065] S306 uses the shortest path graph kernel function to calculate the syntactic structural similarity between the user question and the answer graph, returns the top m text blocks with the best structural matching, merges the n text segments retrieved from the subgraph and the m text segments retrieved from the sparse graph, and removes duplicate content.
[0066] It should be noted that user questions are simultaneously converted into sparse vector representations using a fine-tuned CodeBERT model, emphasizing keywords and syntactic features. These vectors are input into a Siamese neural network, and similarity calculations are performed with summary vectors in the database. The Siamese neural network can efficiently compare semantic similarity for different types of inputs. Each "question-answer" pair in the shepherd's case library generates a summary text, which is usually a condensation of the core content of the answer. The summary vector refers to the sparse vector representation of the semantic and syntactic features of the text extracted from the "question-answer" pairs in the shepherd's case library. The syntactic structural similarity between the user question and answer graphs is calculated using a shortest path graph kernel function. For both the user question graph and the candidate answer graph, the shortest path between all node pairs is calculated, the number of identical shortest paths in the two graphs is counted, and the matched paths are weighted by length (longer paths have higher weights). After normalization, the final similarity score is obtained. The k candidate answer graphs are sorted in descending order of the final similarity score, and the top m text blocks with the best syntactic structure matching are returned.
[0067] It should be noted that the process involves obtaining subgraph retrieval text and sparse retrieval text, removing completely duplicate or nearly duplicate content based on text fingerprints (such as MD5 hashes) or semantic fingerprints (embedded vector clustering), concatenating the user question with the deduplicated text blocks into a sequence, inputting it into the embedding layer of the LLaMA2 model, generating a joint representation, and using the MHA multi-head attention mechanism of LLaMA2, calculating an attention score matrix with the hidden state of the question part as the Query and the hidden state of the text block part as the Key. High attention scores indicate parts of the text block that are strongly related to the question. The semantic fit is obtained by averaging the cross-attention scores. The graph attention score of subgraph retrieval and the SPGK score of sparse retrieval are obtained as the original retrieval score. The semantic fit and the original retrieval score are dynamically weighted to obtain the final fit score. The text blocks are then sorted in descending order according to the final fit score, and a preset number of text blocks are selected as the prompt context.
[0068] It should be noted that the user question is used as the core anchor point at the beginning of the prompt template to clearly define the task objective. Text blocks arranged in descending order of their final fit score are inserted into the prompt template with priority. Text blocks with a final fit score higher than the threshold are considered high-fit texts, and the remaining text blocks are considered low-fit texts. In the prompt template, the high-fit texts are placed adjacent to the question, and the low-fit texts serve as supplementary information. Domain entities and relationships in the text blocks are marked as key markers to guide the model to focus on structured information. Weight coefficients are assigned to the text blocks based on their final fit score to dynamically adjust the influence of the text blocks.
[0069] A contrastive learning strategy, such as InfoNCE loss, is introduced. The combination of the text block with the highest final fit score and the user question is used as a positive sample, while combinations of randomly sampled irrelevant text blocks and the user question are used as negative samples. Contrastive learning brings the semantic representation of the model output closer to the positive samples and pushes it further away from the negative samples. Using 5-10 sets of labeled samples of herders' decisions, the generation strategy for the prompt template is adjusted. Only the dynamic weight parameters and tag embeddings in the prompt template are trained, while the backbone parameters of the large language model are frozen. The prompt template is imported into a preset large language model; preferably, the GLM-4-Flash large language generation model is used, which supports 128K contextual reasoning and multilingual processing capabilities. Temperature parameters are used to reduce randomness and ensure stable output. Maximum length limits are used to avoid redundancy. High-frequency entities and numerical information are automatically identified and prioritized in the generated answers. Herders' intelligent decision-making data is generated according to a causal chain structure of "problem → measure → expected effect".
[0070] For example: Input template: User question: How to deal with frostbite of livestock in winter?; High-weight context: Herdsman A reduced the frostbite rate to 5% (average winter temperature -25℃) by using a double-layered greenhouse. The greenhouse needs to be built facing south and the insulation layer should be thickened (suitable for winds of level 4 or above in pastoral areas); Instruction: Please list two feasible measures based on temperature and wind factors.
[0071] Model output: Based on highly correlated cases, the following measures are recommended to address winter frostbite: Construct a south-facing double-layered warm shed (refer to herder A, which reduced the frostbite rate to 5%), which must be completed before the end of November; group livestock according to their physical condition, prioritizing weaker animals for entry into the shed (refer to herder B, which reduced competitive heat loss); increase the calorie ratio of winter feed (add 10% corn daily) to enhance cold resistance.
[0072] The system obtains user feedback ratings for the herders' intelligent decision-making data. If a user's feedback rating for the herders' intelligent decision-making data is lower than a threshold, the ignored context text block is recorded in the prompt template, and the weight coefficient of the recorded context text block is adjusted in subsequent searches. In addition, the system dynamically updates the positive and negative sample database by judging whether the user accepts or rejects the generated answer based on the user's feedback rating for the herders' intelligent decision-making data.
[0073] Figure 4 A block diagram of a herder intelligent decision-making system based on a large language model and retrieval enhancement is shown.
[0074] The second aspect of this invention provides a herder intelligent decision-making system based on a large language model and retrieval enhancement, the system comprising: a data acquisition module, a herder intelligent decision-making module, and a data visualization module;
[0075] The data acquisition module collects herder information and pastoral case information and preprocesses them. The preprocessed herder information and pastoral case information are then used to construct a herder information database and a herder case database, respectively.
[0076] The system collects data through manual questionnaires and historical cases, and stores the data in the server database. The data preprocessing program retrieves the manual questionnaire data and historical case data from the server, performs preprocessing such as outlier removal, duplicate value removal, and missing value completion, and stores the processed dataset in the server database.
[0077] The herder intelligent decision-making module acquires the user's question and performs vectorization encoding. The encoded question vector is then enhanced by hybrid retrieval in the herder information database and herder case database. Based on semantic adaptation, content that meets the requirements is obtained as a prompt context. The prompt context is then fused with the user's original question to construct a prompt template, which is then imported into a preset large language model to obtain herder intelligent decision-making data.
[0078] The retrieved context fragments are fused with the original question posed by the user to construct a prompt template, which is then input into a large language model to generate the final answer. Preferably, the GLM-4-Flash large language generation model is used, which supports 128K context reasoning and multilingual processing capabilities.
[0079] The data visualization module returns the output intelligent decision-making data of herders to the server and uses a visualization dashboard to visualize the intelligent decision-making data of herders on the server, so as to present the results more intuitively.
[0080] A third aspect of the present invention provides a computer-readable storage medium comprising a herder intelligent decision-making method program based on a large language model and retrieval enhancement, wherein when the herder intelligent decision-making method program based on a large language model and retrieval enhancement is executed by a processor, it implements the steps of the herder intelligent decision-making method based on a large language model and retrieval enhancement.
[0081] In the several embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical, or other forms. Furthermore, in the various embodiments of the present invention, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0082] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0083] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A herder intelligent decision-making method based on large language models and retrieval enhancement, characterized in that, Includes the following steps: Collect and preprocess information on herders and pastoral cases, and use the preprocessed information on herders and pastoral cases to build a database of herder information and a database of herder cases, respectively. The system obtains user questions and performs vector encoding. The encoded question vectors are then used for hybrid retrieval and enhancement in the herder information database and herder case database. Based on semantic adaptation, content that meets the requirements is obtained as the prompt context. The prompt context is integrated with the user's original question to construct a prompt template, which is then imported into a preset large language model to obtain herders' intelligent decision-making data. The herders' intelligent decision-making data is then visualized. The user's questions are obtained and vectorized. The encoded question vectors are then used for enhanced retrieval from the herder information database and the herder case database. Specifically: The user-input questions are converted into dense vectors using a pre-trained RoBERTa language model. The dense vectors are then stored in a vector database in the form of high-dimensional floating-point numbers. The herder information database and herder case database are modeled as a knowledge graph. A subgraph retrieval model is constructed using a graph neural network. The nodes and relationships in the knowledge graph are embedded and represented. The dense vector corresponding to the user's question is input into the subgraph retrieval model. The similarity between the dense vector and the node embedding vector in the knowledge graph is calculated. The most relevant entity nodes are selected based on a preset number. The graph attention mechanism dynamically assigns association weights to the selected entity nodes, and retains entity nodes with association weights higher than a preset weight threshold and edges directly connected to the entity nodes to construct an initial subgraph. The reciprocal of the attention score obtained from the graph attention mechanism is used as the edge weight in the initial subgraph. Starting from the core node in the initial subgraph that is most relevant to the problem, the path originates from the core node and the Dijkstra algorithm is used to obtain the shortest path with the minimum sum of edge weights from the path origin to other nodes. Obtain a preset number of shortest paths, and optimize the initial subgraph by verifying the semantic coherence of the shortest paths through a pre-trained model. Then, take the original text corresponding to the optimized subgraph as candidates and return the top n most relevant text segments. The user's questions are obtained and vectorized. The encoded question vectors are then used for enhanced retrieval from the herder information database and the herder case database. Specifically: The user-input question is converted into a sparse vector using a pre-trained CodeBERT model. The sparse vector is then input into a Siamese neural network, and similarity calculation is performed between it and the summary vectors in the herder information database and the herder case database. The similarity calculation retrieves a preset number of semantically closest summary vectors and their corresponding question-and-answer text blocks. Each question-and-answer text block is segmented and labeled with part-of-speech tags. Each word vector is used as a node. The grammatical relationship between words is determined based on grammatical dependency analysis. Directed edges are constructed. The question-and-answer text blocks are converted into a directed graph based on the nodes and directed edges. The shortest path graph kernel function is used to calculate the syntactic structural similarity between the user question and the answer graph, and the top m text blocks with the best structural matching are returned. The n text segments retrieved from the subgraph and the m text segments retrieved from the sparse graph are merged and duplicate content is removed. Based on semantic adaptation, content that meets the requirements is obtained as the prompt context, specifically: The user question and the deduplicated text block are concatenated into a sequence and input into the embedding layer of the LLaMA2 model to generate a joint representation. The cross-attention score between the question and the text block is calculated through the MHA multi-head attention mechanism of LLaMA2. The semantic fit is obtained by averaging the cross-attention score. The graph attention score of subgraph retrieval and the SPGK score of sparse retrieval are obtained as the original retrieval score. The semantic fit and the original retrieval score are dynamically weighted to obtain the final fit score. The final fit scores are sorted in descending order, and a preset number of text blocks are selected as the prompt context.
2. The herder intelligent decision-making method based on large language model and retrieval enhancement generation according to claim 1, characterized in that, Information on herders and pastoral cases was collected and preprocessed. The preprocessed information was then used to construct a herder information database and a herder case database, respectively. Specifically: Collect information on herders' family population structure, livestock types and numbers, grassland area, existing resources and historical decision-making records as herder information, and collect records of climate events in pastoral areas and historical decision-making records of responses as pastoral case information. The herder information and pastoral case information are cleaned, missing values are removed and standardized. The preprocessed herder information and pastoral case information are segmented to obtain corresponding text blocks. Transform encoding is used to convert the herder text and pastoral case text into initial vectors. The initial vector is deeply encoded using a pre-defined large language model to generate herder information embedding vectors and pastoral case embedding vectors. The herder information embedding vectors are associated with and stored in the original text to build an index and generate a herder information database. The pastoral case embedding vectors are associated with and stored in the case descriptions to build an index and generate a pastoral case database.
3. The herder intelligent decision-making method based on large language model and retrieval enhancement generation according to claim 1, characterized in that, The system integrates the context of the prompts with the user's original question, constructs a prompt template, imports it into a pre-defined large language model, and obtains intelligent decision-making data for herders. Specifically: The user question is used as the core anchor point at the beginning of the prompt template. Text blocks arranged in descending order of the final fit score are inserted into the prompt template according to priority. Text blocks with a final fit score greater than the threshold are considered high fit texts, and the remaining text blocks are considered low fit texts. In the prompt template, the high fit texts are adjacent to the question, and the low fit texts are used as supplements. Key tags are added to the domain entities and relationships in the text blocks to guide the model to focus on structured information. Weight coefficients are assigned to the text blocks based on the final score of fit, and the influence of the text blocks is dynamically adjusted. A contrastive learning strategy is introduced, in which the combination of the text block with the highest final fit score and the user question is taken as a positive sample, and the combination of the user question with the user question is randomly sampled as a negative sample. Through contrastive learning, the semantic representation of the model output in the positive sample is brought closer, while the representation of the negative sample is pushed further away. Using a small number of labeled samples of herders' decisions, we adjusted the generation strategy of the prompt template, trained only the dynamic weight parameters and tag embeddings in the prompt template, and froze the backbone parameters of the large language model. Import the prompt template into the preset large language model, automatically identify high-frequency entities and numerical information, and generate intelligent decision-making data for herdsmen according to the "problem → measure → expected effect" structure.
4. The herder intelligent decision-making method based on large language model and retrieval enhancement generation according to claim 3, characterized in that, Based on user feedback, the intelligent decision-making data for herders has been optimized, specifically as follows: Obtain user feedback ratings for the herders' intelligent decision-making data. If the user feedback rating for the herders' intelligent decision-making data is lower than the threshold, record the ignored context text block in the prompt template and adjust the weight coefficient of the recorded context text block in subsequent searches. In addition, the positive and negative sample database is dynamically updated by judging whether users accept or reject the generated answer based on the feedback rating of the herders' intelligent decision-making data.
5. A herder intelligent decision-making system based on large language models and retrieval enhancement, characterized in that, To implement the herder intelligent decision-making method based on large language model and retrieval enhancement as described in any one of claims 1-4, the system includes: a data acquisition module, a herder intelligent decision-making module, and a data visualization module; The data acquisition module collects herder information and pastoral case information and preprocesses them. The preprocessed herder information and pastoral case information are then used to construct a herder information database and a herder case database, respectively. The herder intelligent decision-making module acquires the user's question and performs vectorization encoding. The encoded question vector is then enhanced by hybrid retrieval in the herder information database and herder case database. Based on semantic adaptation, content that meets the requirements is obtained as a prompt context. The prompt context is then fused with the user's original question to construct a prompt template, which is then imported into a preset large language model to obtain herder intelligent decision-making data. The data visualization module returns the output intelligent decision-making data of herdsmen to the server and uses a visualization dashboard to visualize the intelligent decision-making data of herdsmen on the server.