Knowledge reinforcement learning method and system based on multiple agents

Abstract

The invention relates to the technical field of AI computing, in particular to a knowledge reinforcement learning method and system based on multiple agents. Modifying parameters of the large language model by using a supervised fine-tuning training method for the pre-trained large language model; constructing a knowledge base of a large language model, and vectorizing, storing and retrieving texts in various file formats; a multi-agent workflow optimization information processing and decision-making process is developed, a real-time problem is decomposed, a logic chain for solving the problem is constructed, and internet texts are obtained, filtered and summarized. Compared with the prior art, the method has the advantages that the knowledge quantity of the pre-trained large language model in the specific field is increased, the knowledge base is high in flexibility, the obtained text is high in real-time performance, certain interpretability is achieved and the like, and then the ability of the large language model to solve problems in the specific field is improved.

Description

Technical Field

[0001] The present invention relates to the field of AI computing technology, and in particular to a multi-agent based knowledge enhancement learning method and system. Background Art

[0002] With the rapid development of artificial intelligence and natural language processing technology, generative pre-trained large language models (such as GPT-3, BERT, ERNIE, etc.) have demonstrated powerful capabilities in various natural language processing tasks. These models can generate high-quality texts through large-scale data training, and have achieved remarkable results in tasks such as question answering, translation, and summarization. At present, there are many technical developments related to the present invention, including but not limited to: generative pre-trained large language models: such as OpenAI's GPT-3, Baidu's ERNIE, etc., these models have powerful language generation and understanding capabilities through massive data training. Knowledge base question and answer system: a question and answer system based on a knowledge graph or database, which provides answers to users through retrieval and matching technology, such as IBM Watson, Baidu Knows, etc. Multi-agent system: In an agent system, each agent undertakes different tasks, and improves the overall performance of the system through collaboration and interaction, such as an intelligent customer service system, a multi-task robot system, etc.

[0003] Although the existing technology has made significant progress, there are still some shortcomings and challenges in practical applications, including: the existing generative pre-trained large language models often show insufficient reasoning ability when dealing with domain-specific reasoning tasks, and are unable to accurately understand and answer the corresponding questions. In addition, these models sometimes produce wrong information or inaccurate answers when generating answers, affecting the user experience. Many existing large language models lack real-time processing capabilities and cannot generate answers using real-time Internet information, and the generated text lacks timeliness. In addition, the generated answers often lack interpretability, and users find it difficult to understand the reasoning process of the model and the source of the answers, affecting trust and usability. The training of large language models usually requires a lot of computing resources and time, is costly, and has high requirements for hardware and software environments, limiting their widespread application. Taking OpenAI's GPT-3 and Baidu's ERNIE as examples, the parameter scale of these models has reached the hundreds of billions level, and the training and reasoning costs are very high. Existing knowledge base question-answering systems, especially those based on static knowledge graphs, have untimely knowledge updates and are difficult to provide the latest answers and information, affecting users' acquisition and decision-making of real-time information. A single large language model has obvious limitations when processing long text tasks. When the context is beyond the scope of the model or the constraints in the problem are too complex, the model cannot understand and process it at one time. This limits the effectiveness of large models in processing long texts and complex tasks. Summary of the invention

[0004] In view of the above problems existing in the prior art, the present invention is proposed.

[0005] Therefore, the present invention provides a multi-agent based knowledge enhancement learning method, which can solve the problems of lack of real-time and interpretability of traditional large models, untimely knowledge updating, and limited ability to process long text tasks.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a knowledge-enhanced learning method based on multi-agents, comprising: using a supervised fine-tuning training method for a pre-trained large language model to modify the parameters of the large language model; building a knowledge base of the large language model to vectorize, store and retrieve texts in various file formats; developing a multi-agent workflow to optimize information processing and decision-making processes, decompose real-time problems, build a logical chain for solving problems, obtain and filter, and summarize Internet texts.

[0007] As a preferred solution of the multi-agent based knowledge enhancement learning method described in the present invention, wherein: the supervised fine-tuning training method for the pre-trained large language model includes data preprocessing, question and answer text generation, post-processing method and large language model full parameter fine-tuning method;

[0008] The data preprocessing includes filtering of data sensitive information and text alignment methods;

[0009] The question-answer text generation includes using a number of prompt words input into a large language model with a scale of 100 billion or more, sorting out a plurality of progressive questions according to the preprocessed data, generating corresponding answers according to the questions, and obtaining a plurality of question-answer texts;

[0010] The post-processing includes methods for filling, text segmentation and formatting of the results returned by large models of a scale of 100 billion or more;

[0011] The full parameter fine-tuning includes the training parameter configuration and inference parameter configuration required for the full parameter fine-tuning of the model.

[0012] As a preferred solution of the multi-agent based knowledge enhancement learning method described in the present invention, wherein: the construction of a large language model knowledge base includes a text segmentation method and a similarity text distance algorithm;

[0013] The text segmentation method includes segmenting the original text into segments, controlling the upper and lower limits of the length of each segment, and ensuring that each sentence in each segment is complete, and using regular expressions to segment the original text according to Chinese and English punctuation marks to form a sentence sequence S.

[0014] S=split(text,regex(′[,.!;?,.!;?]+′))

[0015] Among them, split(·) is a string splitting function, regex(·) is a regular expression, and text is the original text;

[0016] After segmentation, trim each sentence to exclude blank or empty sentences.

[0017] S={s∈S:s≠”and s≠”}

[0018] Among them, s is the independent sentence after segmentation;

[0019] Iterates through the sentence sequence S, combines sentences into text blocks that do not exceed the preset maximum number of characters N, checks the length of the current text block C, and adds sentences to the current block or starts a new block as needed.

[0020] C=C+s if|C|+|s|+1≤N

[0021] For text blocks that exceed the preset maximum number of characters N, they are split into multiple blocks, each of which is no longer than N.

[0022] while|s|>N: add s[0:N] to the block list

[0023] s=s[N:]

[0024] Group the block text list by batch size B and get the embedding vector for each batch.

[0025] for each batchb in chunks:embeddings=get_embedding(b)

[0026] Among them, for each batchb in chunks is a loop iteration for each element in the set of chunks, embeddings is the storage variable of the embedding vector, get_embedding(·) is the function of generating text embedding vectors, and b is the variable of a single batch.

[0027] As a preferred solution of the multi-agent based knowledge enhancement learning method described in the present invention, the similarity text distance algorithm includes calculating the similarity value of the input text segment and each text segment in the knowledge base, and the most similar text segments are used as background information to assist the large language model in answering questions. It is confirmed that the length of the query text q does not exceed the preset maximum length L. If it exceeds, the text is truncated. If it does not exceed, it is not processed, which is expressed as,

[0028] q=q[:L]if|q|>L

[0029] Embed the query text q to get the vector representation q_embd of q.

[0030] q_embd = get_embedding(q)

[0031] Load the preprocessed and stored document collection Documents and extract the embedding vector doc_embds of each document doc from it.

[0032] doc_embds=[doc['embedding']for doc in Documents

[0033] The similarity between the query vector q_embd and all document embedding vectors doc_embds, the cosine similarity formula is expressed as,

[0034]

[0035] Among them, cos_sims is the cosine similarity, axis=1 is to calculate the norm of each document vector separately, np.dot(·) is the dot function of NumPy, and norm(·) is to calculate the norm of the vector;

[0036] According to the calculated similarity cos_sims, select the M documents with the highest similarity.

[0037] top_indices=np.argsort(-cos_sims)[:M]

[0038] Among them, top_indices is the index of the document with the highest similarity, and np.argsort(·) is the index sorting function;

[0039] Build and return a list of top_documents containing detailed information of the most relevant documents, including the document id, content content and the corresponding similarity score score.

[0040] top_documents=['id':Documents[i]['id'],

[0041] 'content':Documents[i]['content'],

[0042] 'score':cos_sims[i]for iin top_indices]

[0043] Here, i is the index of the most relevant document.

[0044] As a preferred solution of the multi-agent based knowledge enhancement learning method described in the present invention, the multi-agent workflow optimization information processing and decision-making process includes inputting prompt words for each agent and the calling relationship between agents. The prompt words of the agent specify the output and output format of the large model and specify the logical thinking of the large model to analyze the problem. The calling relationship between agents specifies the order of execution of the large model for the same problem and the flow order of output and output results.

[0045] As a preferred solution of the multi-agent based knowledge enhancement learning method described in the present invention, wherein: the Internet text includes search engine and news website webpage text;

[0046] The summarized Internet text includes a method for acquiring web page text and an optimization method for improving the speed of reading web page elements. The method for acquiring web page text includes a method for extracting search results and a method for extracting any search result text.

[0047] As a preferred solution of the multi-agent based knowledge enhancement learning method described in the present invention, the optimization method for improving the speed of reading web page elements includes converting the Python-based Selenium web page element search and acquisition operation into a JavaScript execution mode, using a JavaScript script to locate all relevant elements at one time through document.querySelectorAll, collect data, and complete the entire data extraction process on the client, reducing network delays and the overhead caused by data transmission, and reducing frequent access and modification to the DOM through one-time query and batch processing of data.

[0048] Another object of the present invention is to provide a multi-agent-based knowledge-enhanced learning system to enhance the ability of large language models in processing domain-specific reasoning tasks, especially to improve their real-time and interpretability, and to optimize the processing efficiency of long text tasks. The method of the present invention significantly improves the response speed and accuracy of the model by combining supervised fine-tuning training, diversified knowledge base construction, and multi-agent workflow design. The present invention effectively reduces training costs and time by updating the knowledge base in real time and optimizing model training, while enhancing the application capabilities of the model in specific fields, solving the problems of untimely knowledge updates and limited ability to process long text tasks in the prior art.

[0049] As a preferred solution of the multi-agent based knowledge enhancement learning system described in the present invention, it includes: a model fine-tuning module, a knowledge base construction module and a multi-agent workflow module;

[0050] The model fine-tuning module performs supervised fine-tuning on the pre-trained large language model, including data pre-processing, question-answer text generation, post-processing, and full parameter fine-tuning of the model;

[0051] The knowledge base building module creates a knowledge base for a large language model, which is used to vectorize, store and retrieve text in multiple file formats, helping the model to access and utilize stored knowledge more efficiently;

[0052] The multi-agent workflow module decomposes complex real-time problems through multi-agent workflow and constructs a logical chain to solve the problems. It also includes technology for acquiring, filtering and summarizing Internet texts, so that the model can use the latest information to improve the real-time and accuracy of its responses.

[0053] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of any one of the methods described in a multi-agent based knowledge enhancement learning method are implemented.

[0054] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of any one of the methods described in a multi-agent based knowledge enhancement learning method are implemented.

[0055] Beneficial effects of the present invention: The present invention adopts a model with a scale of 7B, which significantly reduces the parameters of large models with a scale of hundreds of billions of parameters such as OpenAI's GPT-3 and Baidu's ERNIE, and greatly reduces the training cost and reasoning cost, which significantly reduces the consumption of computing resources. This not only makes model training and reasoning more efficient, but also reduces the requirements of hardware and software environment, and is suitable for environments with limited resources. The model of the present invention is trained on a data set in a proprietary field, with special attention to knowledge and tasks in a specific field. This focus makes the model perform better in a specific field, and can better understand and handle professional terms and complex problems, thereby providing highly accurate and efficient knowledge retrieval and question-and-answer services. The present invention designs a multi-agent system, in which each agent assumes a different role or task in the system, and improves the overall performance through efficient collaboration and interaction between agents; for complex long text tasks, through the division of labor and collaboration of multiple agents, problems can be effectively decomposed, and the acquired text information can be filtered and summarized, thereby solving complex tasks and improving the processing power and flexibility of the system. By utilizing the division of labor and collaboration of the multi-agent system, long text tasks are decomposed into multiple small tasks, and each agent handles different types of tasks respectively; this method solves the problem that a single large language model is prone to exceed the context scope and has difficulty understanding complex constraints when processing long text tasks, making the system perform better when processing long texts and complex tasks. Combining knowledge base, vector database and real-time crawler technology, it provides online real-time retrieval capabilities; the system can provide the latest data and messages to ensure that users have access to the latest information, while improving the dynamic nature of information and the real-time nature of the system. In addition, through the collaboration and interaction of multiple agents, the interpretability of the model reasoning process is improved, and the user's trust in the system and usability are enhanced. The crawler-based function is integrated to automatically capture the latest information and entries from authoritative web pages; the knowledge base can be updated regularly to ensure that the answers and information provided by the system are always the latest, helping users to obtain real-time information in a timely manner, supporting rapid decision-making and dynamic response. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0057] Figure 1 A technical roadmap for a multi-agent based knowledge enhancement learning method is provided for one embodiment of the present invention.

[0058] Figure 2A data processing flow chart of a multi-agent based knowledge enhancement learning method provided for one embodiment of the present invention.

[0059] Figure 3 A flowchart of real-time web page retrieval based on a multi-agent knowledge enhancement learning method is provided for one embodiment of the present invention.

[0060] Figure 4 A question test diagram a of a multi-agent based knowledge enhancement learning method provided by an embodiment of the present invention.

[0061] Figure 5 A b-question test diagram of a multi-agent based knowledge enhancement learning method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0063] Example 1, reference Figure 1-Figure 5 , which is the first embodiment of the present invention, provides a knowledge enhancement learning method based on multi-agents, comprising:

[0064] S1: Modify the parameters of the large language model using the supervised fine-tuning training method for the pre-trained large language model.

[0065] It should be noted that if Figure 1 As shown, the method of the present invention first collects research report data open source on the Internet, and the research report data is preprocessed to obtain data that has been cleaned and screened. The preprocessed data is intelligently labeled using a large model to obtain question-answering dialogue data. For question-answering dialogue data, Huawei's 910B NPU and domestic AI architecture are used, and all parameters are fine-tuned based on Baichaun2-7B to obtain a more powerful generative pre-trained large language model. For vector data, the present invention builds a knowledge base, and allows users to access online websites to obtain the latest information, or upload private files themselves. Users can also access the provided knowledge base to assist large model question and answer.

[0066] The supervised fine-tuning training method for the pre-trained large language model includes data preprocessing, question-answer text generation, post-processing method and large language model full-parameter fine-tuning method.

[0067] The data preprocessing includes filtering of data sensitive information and text alignment methods to ensure the quality and consistency of input data.

[0068] The question and answer text generation includes using a number of prompt words input into a large language model of a scale of hundreds of billions or more, sorting out a plurality of progressive questions based on the preprocessed data, generating corresponding answers based on the questions, and obtaining a plurality of question and answer texts to form a high-quality training data set.

[0069] The post-processing includes padding, text segmentation and formatting methods for the results returned by large models of a scale of 100 billion or more, which are used to format the generated question-answer pairs to ensure that they meet the input requirements of the large language model.

[0070] The full parameter fine-tuning includes the training parameter configuration and inference parameter configuration required for the full parameter fine-tuning of the model. The core configuration of the training parameters is shown in Table 1, and the core configuration of the inference parameters is shown in Table 2.

[0071] Table 1. Training parameter core configuration table

[0072]

[0073] Table 2. Inference parameter core configuration table

[0074]

[0075] S2: Build a knowledge base for large language models to vectorize, store, and retrieve text in multiple file formats.

[0076] It should be noted that the construction of a large language model knowledge base includes a text segmentation method and a similarity text distance algorithm;

[0077] The text segmentation method includes segmenting the original text into segments, controlling the upper and lower limits of the length of each segment, and ensuring that each sentence in each segment is complete, and using regular expressions to segment the original text according to Chinese and English punctuation marks to form a sentence sequence S.

[0078] S=split(text,regex(′[,.!;?,.!;?]+′))

[0079] Among them, split(·) is a string splitting function, regex(·) is a regular expression, and text is the original text;

[0080] After segmentation, trim each sentence to exclude blank or empty sentences.

[0081] S={s∈S:s≠”and s≠”}

[0082] Among them, s is the independent sentence after segmentation;

[0083] Iterates through the sentence sequence S, combines sentences into text blocks that do not exceed the preset maximum number of characters N, checks the length of the current text block C, and adds sentences to the current block or starts a new block as needed.

[0084] C=C+s if|C|+|s|+1≤N

[0085] For text blocks that exceed the preset maximum number of characters N, they are split into multiple blocks, each of which is no longer than N.

[0086] while|s|>N: add s[0:N] to the block list

[0087] s=s[N:]

[0088] Group the block text list by batch size B and get the embedding vector for each batch.

[0089] for each batchb in chunks:embeddings=get_embedding(b)

[0090] Among them, for each batchb in chunks is a loop iteration for each element in the set of chunks, embeddings is the storage variable of the embedding vector, get_embedding(·) is the function of generating text embedding vectors, and b is the variable of a single batch.

[0091] It should be noted that the similarity text distance algorithm includes calculating the similarity value of the input text segment and each text segment in the knowledge base, and the most similar text segments are used as background information to assist the large language model in answering questions. It is confirmed that the length of the query text q does not exceed the preset maximum length L. If it exceeds, the text is truncated. If it does not exceed, it is not processed, which is expressed as,

[0092] q=q[:L]if|q|>L

[0093] Embed the query text q to get the vector representation q_embd of q.

[0094] q_embd = get_embedding(q)

[0095] Load the preprocessed and stored document collection Documents and extract the embedding vector doc_embds of each document doc from it.

[0096] doc_embds=[doc['embedding']for doc in Documents

[0097] The similarity between the query vector q_embd and all document embedding vectors doc_embds, the cosine similarity formula is expressed as,

[0098]

[0099] Among them, cos_sims is the cosine similarity, axis=1 is to calculate the norm of each document vector separately, np.dot(·) is the dot function of NumPy, and norm(·) is to calculate the norm of the vector;

[0100] According to the calculated similarity cos_sims, select the M documents with the highest similarity.

[0101] top_indices=np.argsort(-cos_sims)[:M]

[0102] Among them, top_indices is the index of the document with the highest similarity, and np.argsort(·) is the index sorting function;

[0103] Build and return a list of top_documents containing detailed information of the most relevant documents, including the document id, content content and the corresponding similarity score score.

[0104] top_documents=['id':Documents[i]['id'],

[0105] 'content':Documents[i]['content'],

[0106] 'score':cos_sims[i]for iin top_indices]

[0107] Here, i is the index of the most relevant document.

[0108] S3: Develop multi-agent workflows to optimize information processing and decision-making processes, decompose real-time problems, build logical chains to solve problems, and obtain, filter, and summarize Internet texts.

[0109] It should be noted that the method of constructing a multi-agent workflow includes setting prompt words for agents and calling relationships between agents. The prompt word setting clarifies the tasks, output content and format requirements of each agent, ensuring that the agents work efficiently in their respective fields. The calling relationship between agents specifies the order of problem processing, making the decomposition and solution of the problem logically rigorous, and ensuring that the outputs of each agent can be effectively connected and integrated. The agent prompt words are shown in Tables 3, 4 and 5.

[0110] Table 3. Agent 1 - Front-end master control agent - Prompt word list

[0111]

[0112]

[0113] Table 4. Agent 2 - Filtering Search Engine Results Agent - Prompt Word List

[0114]

[0115] Table 5. Agent 3 - Filter News List Agent - Prompt Word List

[0116]

[0117] It should be noted that the summary of Internet text includes a method for obtaining web page text and an optimization method for improving the speed of reading web page elements, and the method for obtaining web page text includes a method for extracting search results and a method for extracting any search result text. The implementation of obtaining Internet real-time text includes a method for obtaining Baidu search web page text, a method for obtaining news text from The Paper, and an optimization method for improving the speed of reading web page elements.

[0118] The steps for extracting Baidu search web pages are as follows:

[0119] Browser and driver configuration: Configure the Chrome browser to run in headless mode, where the browser runs in the background without a user interface. Disable sandbox mode to improve compatibility, avoid DevShm usage restrictions, and ensure that it works properly in resource-constrained environments.

[0120] Initialize WebDriver: Initializes the Chrome WebDriver, which is the interface to interact with the Chrome browser, using the configured options.

[0121] Navigate to the search page: Construct a Baidu search URL containing the search keyword and navigate to that page.

[0122] Wait for page to load: Set an implicit wait time to ensure that page elements are available.

[0123] Locate and fetch search results: Locate the main div element (id 'content_left') containing the search results and execute a script using JavaScript to extract text from multiple specified classes. Use JavaScript queries to select specific CSS classes to locate search results. For each located element, check and extract the element with the 'aria-label' attribute or directly get the text content of the element.

[0124] The steps for extracting the search results of The Paper are as follows:

[0125] URL validation: Use regular expressions to check if the incoming URL conforms to the expected format.

[0126] Browser configuration: Set the running options of the Chrome browser, configure it to headless mode to execute in the background, disable sandbox mode and reduce the use of DevShm to avoid environment restriction issues.

[0127] WebDriver Initialization: Creates and initializes a ChromeWebDriver using the preset configuration options, specifies the path to the chromedriver, and starts the WebDriver.

[0128] Navigate to the news page: Load the specified URL via WebDriver and wait for the page to fully load to ensure all content is accessible.

[0129] Content crawling: Try to locate the HTML elements containing news content, and then further extract the news title and the text of all paragraphs to synthesize the complete news content.

[0130] Find Title: Locates the title element and gets its text.

[0131] Find content: Iterate over all paragraph tags, extract and merge their text content.

[0132] The steps for extracting specific news pages from The Paper are as follows:

[0133] Configure browser options: Set Chrome to run in headless mode to ensure execution in the background. Configure to non-sandbox mode and disable the use of dev / shm to avoid issues due to resource limitations.

[0134] Initialize WebDriver: Initializes the Chrome WebDriver using the specified options and specifies the path to the ChromeDriver.

[0135] Navigate to the search page: Construct a search URL, including the search keywords, and load this URL through WebDriver.

[0136] Page content loading: Set an implicit wait to ensure that the page content is fully loaded so that all elements can be located and operated.

[0137] Extract news links and details: Locate all list items that contain search results. For each result item, try to find the link, title, and description. If found, save the link's href attribute, cleaned title, and description text to the news details list.

[0138] Extract and clean data: For each search result item: Get the link from the href attribute of the tag. <h2> Get the HTML content from the tag and use the cleaning function to remove the HTML tags. Get the text content directly from the tag.

[0139] It should be noted that the optimization method for improving the speed of reading web page elements includes converting the Python-based Selenium web page element search and acquisition operation into a JavaScript execution mode. In the original implementation, the Python Selenium library is used to locate and process web page elements one by one. Find a specific element in the DOM through Selenium's find_element or find_elements method. Call attributes or methods such as .text or .get_attribute() for each found element to obtain the required information. Repeat the above steps for each element until all relevant data is extracted.

[0140] The main disadvantages of the original method include that each interaction with WebDriver takes time, especially when there are a large number of elements, which can significantly increase the overall execution time. Frequent DOM operations increase the burden on CPU and memory, especially in resource-constrained environments. The processing of each element may involve network communication, increasing latency and instability.

[0141] The present invention transfers the data extraction logic to JavaScript execution, so that the entire process is completed at one time on the browser side. The specific optimization steps are as follows:

[0142] One-time script execution: Use a JavaScript script to locate all relevant elements at once through document.querySelectorAll and collect their data.

[0143] Reduce server interaction: The entire data extraction process is completed on the client side and does not need to be sent back to the server for processing, thereby reducing network latency and data transmission overhead.

[0144] Reduce the number of DOM operations: By querying once and processing data in batches, frequent access and modification to the DOM are reduced, improving performance.

[0145] Compared with the original method, the performance of the optimized method can be improved by about 30% to 70%, depending on the page complexity and the number of elements. The interaction between the server and the client is reduced, and the CPU and memory consumption are reduced. The simplified processing flow makes it more efficient and feasible to expand to larger-scale data crawling.

[0146] Furthermore, the interaction between agents depends on the defined function tools, which consist of functions and description prompts for the functions, as shown in Table 6.

[0147] Table 6. Function description word list

[0148]

[0149] Furthermore, the calling relationship between the agent and the webpage extraction tool is as follows:

[0150] (1) The front-end master control agent analyzes and believes that there is no need for Baidu search and news retrieval, and answers based on the text provided by the user.

[0151] (2) The front-end master control agent analyzes and believes that a Baidu search is needed. It summarizes the search keywords based on the text provided by the user, searches on the Baidu search engine, extracts the Baidu search webpage, and then filters the extracted text. The filtered results are returned to the front-end master control agent. The front-end master control agent answers based on the text provided by the user and the filtered results.

[0152] (3) The front-end master control agent analyzes and believes that it is necessary to search The Paper. It summarizes the search keywords based on the text provided by the user, searches on the The Paper search engine, extracts the search results page of The Paper, and then the filter news list agent selects the appropriate news, extracts the selected news page, and then the filter search engine result agent filters the extracted text and returns the filtered results to the front-end master control agent. The front-end master control agent answers based on the text provided by the user and the filtered results.

[0153] (4) Before summarizing the keywords, the front-end master control agent will call the date tool to obtain today's date for the context of the answer if necessary.

[0154] like Figure 2 As shown, this embodiment illustrates the data processing flow. First, sensitive information filtering is performed, the main purpose of which is to protect the privacy of individuals and institutions involved in the data, while ensuring the compliance of the data set. Use a predefined list of privacy words through a keyword list to identify and filter out content that may contain sensitive information. These keywords include personal information (such as name, contact information), professional qualifications (such as practice certificate number, registration number), etc. Regular expressions are used to filter out strings with special formats that may contain sensitive information, such as strings containing only numbers and date units.

[0155] Data merging can optimize data structure, improve information density and data utilization, and provide more coherent and information-rich text for model training. The filtered data is further processed to merge reports with the same title and short content until a suitable text length is reached. This ensures that relevant information is not fragmented, thereby improving data availability.

[0156] During the data merging stage, the script will check whether the titles of consecutive entries are consistent, and use the content length as the basis for merging, gradually merging short text content into longer text until the set length requirement is met.

[0157] In order to generate questions with professional depth, this embodiment designs a series of prompt word templates. These templates are intended to guide large language models (such as Wenxinyiyan) with more than 100 billion words to ask questions according to specific logic and depth. The template requires that the generated questions should not only cover the basic "what" and "how to do", but also include more complex "why" questions to ensure that there is a progressive and in-depth relationship between the questions.

[0158] This embodiment uses a large language model with hundreds of billions of parameters to automatically generate relevant questions based on the optimized data content and preset prompt templates. These questions are then used to generate corresponding answers to form a structured question-answer pair dataset.

[0159] Through this innovative method, the present invention not only effectively solves the problem of shortage of training data for dialogue systems, but also improves the performance of dialogue systems in handling complex consultations. In addition, the successful implementation of this method also demonstrates the potential and breadth of application of large-scale language models in professional field data generation, and provides a feasible technical path and practical experience for future related projects.

[0160] Figure 3 The workflow of the agent toolchain in processing user queries is shown. Specifically, the process includes the following main parts:

[0161] User Query: The query submitted by the user is first sent to the initial agent (front-end agent). This agent is responsible for parsing the query and determining the subsequent processing strategy.

[0162] Initial agent decision (front-end agent): The front-end agent selects appropriate tools or services to process the user's query content. These tools can be specialized agents for specific tasks, such as retrieving news, obtaining dates, performing calculations, etc.

[0163] Tool selection and invocation: Based on the user’s needs, the front-end agent selects an appropriate tool from a preset list of tools. For example, if the user asks for the latest news, the agent may select a tool specifically designed for crawling and parsing data from news websites.

[0164] Dedicated agent processing (filtering-agent): parses the data obtained from the Internet, extracts the information required by the user, organizes the text format, and then passes it to the front-end agent.

[0165] Result return: After processing, the process agent returns the processing results to the front-end agent. The front-end agent integrates these results and generates the final user response.

[0166] Response output: The final response is sent back to the user through the front-end agent, completing a complete cycle of query processing.

[0167] Figure 4 and Figure 5 It is a comparison of the question detection results of the model obtained by the present invention and the mainstream model. Figure 4 When asked "What is the stock price of Maotai today", the platform gave an answer, but when asked this question to ChatGPT 3.5, the big model said it could not answer. This shows that the big model platform has the ability to analyze problems. Figure 5 When the question "News about Shanghai's real estate policy" was asked, the platform could retrieve the news online and give a reply, while ChatGPT 3.5 could not give an answer, which shows that the model constructed by the present invention has the ability to obtain real-time information dialogue.

[0168] The above is a schematic scheme of a multi-agent-based knowledge enhancement learning method of this embodiment. It should be noted that the technical scheme of the multi-agent-based knowledge enhancement learning system and the technical scheme of the multi-agent-based knowledge enhancement learning method described above belong to the same concept. For details not described in detail in the technical scheme of the multi-agent-based knowledge enhancement learning system in this embodiment, please refer to the description of the technical scheme of the multi-agent-based knowledge enhancement learning method described above.

[0169] The multi-agent based knowledge enhancement learning system in this embodiment includes: a model fine-tuning module, a knowledge base construction module and a multi-agent workflow module;

[0170] The model fine-tuning module performs supervised fine-tuning on the pre-trained large language model, including data pre-processing, question-answer text generation, post-processing, and full parameter fine-tuning of the model;

[0171] The knowledge base building module creates a knowledge base for a large language model, which is used to vectorize, store and retrieve text in multiple file formats, helping the model to access and utilize stored knowledge more efficiently;

[0172] The multi-agent workflow module decomposes complex real-time problems through multi-agent workflow and constructs a logical chain to solve the problems. It also includes technology for acquiring, filtering and summarizing Internet texts, so that the model can use the latest information to improve the real-time and accuracy of its responses.

[0173] This embodiment further provides a computing device, which is applicable to the case of a multi-agent-based knowledge enhancement learning method, including:

[0174] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0175] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, apparatus or device and execute the instructions), or in conjunction with such instruction execution system, apparatus or device. For purposes of this specification, "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, apparatus or device, or in conjunction with such instruction execution system, apparatus or device.

[0176] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0177] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0178] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention. < / h2>

Claims

1. A knowledge-enhanced learning method based on multi-agent, characterized by: include, Modify the parameters of the large language model using the supervised fine-tuning training method for the pre-trained large language model; Build a knowledge base of large language models to vectorize, store, and retrieve text in multiple file formats; Develop multi-agent workflows to optimize information processing and decision-making processes, decompose real-time problems, build logical chains for problem solving, and acquire, filter, and summarize Internet texts.

2. A multi-agent based knowledge enhancement learning method as claimed in claim 1, characterized in that: The supervised fine-tuning training method for the pre-trained large language model includes data pre-processing, question-answer text generation, post-processing method and large language model full-parameter fine-tuning method; The data preprocessing includes filtering of data sensitive information and text alignment methods; The question-answer text generation includes using multiple prompt words input into a large language model with a scale of 100 billion or more, sorting out multiple progressive questions according to the preprocessed data, generating corresponding answers according to the questions, and obtaining multiple question-answer texts; The post-processing includes methods for filling, text segmentation and formatting of the results returned by large models of a scale of 100 billion or more; The full parameter fine-tuning includes the training parameter configuration and inference parameter configuration required for the full parameter fine-tuning of the model.

3. A multi-agent based knowledge enhancement learning method as claimed in claim 2, characterized in that: The construction of the large language model knowledge base includes a text segmentation method and a similarity text distance algorithm; The text segmentation method includes segmenting the original text into segments, controlling the upper and lower limits of the length of each segment, ensuring that each sentence in each segment is complete, and segmenting the original text using regular expressions according to Chinese and English punctuation marks to form a sentence sequence S. S=split(text,regex(′[,.!;?,.!;?]+′)) Among them, split(·) is a string splitting function, regex(·) is a regular expression, and text is the original text; After segmentation, trim each sentence to exclude blank or empty sentences. S={s∈S:s≠”and s≠”} Among them, s is the independent sentence after segmentation; Iterates through the sentence sequence S, combines sentences into text blocks that do not exceed the preset maximum number of characters N, checks the length of the current text block C, and adds sentences to the current block or starts a new block as needed. C=C+s if|C|+|s|+1≤N For text blocks that exceed the preset maximum number of characters N, they are split into multiple blocks, each of which is no longer than N. while|s|>N: add s[0:N] to the block list s=s[N:] Group the block text list by batch size B and get the embedding vector for each batch. for each batchbin chunks:embeddings=get_embedding(b) Among them, for each batchb in chunks is a loop iteration for each element in the set of chunks, embeddings is the storage variable of the embedding vector, get_embedding(·) is the function of generating text embedding vectors, and b is the variable of a single batch.

4. A multi-agent based knowledge enhancement learning method as claimed in claim 3, characterized in that: The similarity text distance algorithm includes calculating the similarity value of the input text segment and each text segment in the knowledge base, using the most similar text segments as background information to assist the large language model in answering questions, and confirming that the length of the query text q does not exceed the preset maximum length L. If it exceeds, the text is truncated, and if it does not exceed, no processing is performed, which is expressed as: q=q[:L]if|q|>L Embed the query text q to get the vector representation q_embd of q. q_embd = get_embedding(q) Load the preprocessed and stored document collection Documents and extract the embedding vector doc_embds of each document doc from it. doc_embds=[doc['embedding']for docin Documents The similarity between the query vector q_embd and all document embedding vectors doc_embds, the cosine similarity formula is expressed as, Among them, cos_sims is the cosine similarity, axis=1 is to calculate the norm for each document vector separately, np.dot(·) is the dot function of NumPy, and norm(·) is used to calculate the norm of a vector; According to the calculated similarity cos_sims, select the M documents with the highest similarity. top_indices=np.argsort(-cos_sims)[:M] Among them, top_indices is the index of the document with the highest similarity, and np.argsort(·) is the index sorting function; Build and return a list top_documents containing detailed information of the most relevant documents, including the document id, content content and the corresponding similarity score score, top_documents=['id':Documents[i]['id'], 'content':Documents[i]['content'], 'score':cos_sims[i]foriin top_indices] Here, i is the index of the most relevant document.

5. A multi-agent based knowledge enhancement learning method as claimed in claim 4, characterized in that: The multi-agent workflow optimization information processing and decision-making process includes inputting prompt words for each agent and the calling relationship between agents. The prompt words of the agent specify the output and output format of the large model and specify the logical thinking of the large model to analyze the problem. The calling relationship between agents specifies the order of execution of the large model for the same problem and the flow order of output and output results.

6. A multi-agent based knowledge enhancement learning method as claimed in claim 5, characterized in that: The Internet texts include search engine and news website webpage texts; The summarized Internet text includes a method for acquiring web page text and an optimization method for improving the speed of reading web page elements. The method for acquiring web page text includes a method for extracting search results and a method for extracting any search result text.

7. A multi-agent based knowledge enhancement learning method as claimed in claim 6, characterized in that: The optimization method for improving the speed of reading web page elements includes converting the Python-based Selenium web page element search and acquisition operation into a JavaScript execution mode, using a JavaScript script to locate all relevant elements at one time through document.querySelectorAll, and collecting data. The entire data extraction process is completed on the client side, reducing the overhead caused by network delay and data transmission, and reducing frequent access and modification to the DOM through one-time query and batch processing of data.

8. A system based on a multi-agent knowledge enhancement learning method according to any one of claims 1 to 7, characterized in that: Includes model fine-tuning module, knowledge base construction module and multi-agent workflow module; The model fine-tuning module performs supervised fine-tuning on the pre-trained large language model, including data pre-processing, question-answer text generation, post-processing, and full parameter fine-tuning of the model; The knowledge base building module creates a knowledge base for a large language model, which is used to vectorize, store and retrieve text in multiple file formats, helping the model to access and utilize stored knowledge more efficiently; The multi-agent workflow module decomposes complex real-time problems through multi-agent workflow and constructs a logical chain to solve the problems. It also includes technology for acquiring, filtering and summarizing Internet texts, so that the model can use the latest information to improve the real-time and accuracy of its responses.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images