Scientific research assisting method based on large language model

Abstract

The invention discloses a scientific research auxiliary method based on a large language model, and belongs to the field of artificial intelligence. The method specifically comprises the following steps: firstly, building a scientific research auxiliary system comprising a core control module, a tool calling module and a multi-mode input module; setting initial parameters for user login; then, the core control module receives an input instruction YI and starts task execution of the first round to obtain a task plan YT1; and a task execution record Y * is further obtained, a decision action YA1 is generated in combination with historical information YH, and after the decision action YA1 is transmitted to the tool calling module, an action execution result YO1 is obtained and returned. Finally, judging whether the first round of task execution of the input instruction YI is completed, if so, generating a final output YF, and waiting for new user input; otherwise, calling the large language model F theta to generate the second round of task planning, and starting the next round of task execution circulation until the n round is finished. According to the method, more convenient support is provided for researchers to read and search literatures, obtain scientific research trends and the like.

Description

Technical Field

[0001] The invention belongs to the field of artificial intelligence and relates to a scientific research auxiliary method based on a large language model. Background Art

[0002] The Large Language Model (LLM) is a large-scale artificial intelligence computing model that has become popular in recent years. It can understand and generate language by learning large amounts of text data, and perform various language-related tasks such as text generation, language understanding, translation, and summary writing.

[0003] Artificial Intelligence Agent (AI Agent) is a software or hardware decision-making system that can automatically perform tasks or achieve goals in a specific environment. Its architecture mainly includes three modules: brain module, perception module and execution module. AI Agent makes decisions based on the perceived environmental information and takes actions to achieve predetermined goals or tasks.

[0004] The artificial intelligence agent based on the large language model uses the large language model as its brain, and utilizes the natural language understanding ability of the large language model to process natural language and generate tool call instructions. It can automatically make decisions in complex environments and use tools to perform tasks.

[0005] Traditional scientific research assistance systems generally include two main functions: literature retrieval and literature reading assistance. For literature retrieval, traditional methods rely on manually setting keywords and search conditions, and require researchers to filter out useful information that meets their needs from noisy noise information. For literature reading assistance, traditional methods can often only provide basic functions, such as word translation, and cannot achieve advanced functions such as content analysis, summarizing the main points, or answering questions in papers.

[0006] In short, traditional scientific research support systems require deep participation of scientific researchers and will consume a lot of their time and energy. Summary of the invention

[0007] The present invention discloses a scientific research assistance method based on a large language model, which aims to solve the problem of difficulty in literature research in the work process of existing scientific researchers. By constructing an artificial intelligence agent based on a large language model, more convenient support is provided for scientific researchers to read literature, search literature and obtain scientific research dynamics.

[0008] The scientific research assistance method based on the large language model has the following specific steps:

[0009] Step 1: Build a scientific research auxiliary system including a core control module, a tool calling module and a multimodal input module;

[0010] The core control module is the processing center, with a large language model as the main body. Its main function is to understand user input and generate tool call instructions.

[0011] The tool calling module is responsible for executing specific tasks, including three tools: paper analysis, paper retrieval, and search engine retrieval. Among them, the paper analysis tool can use the large language model to answer user questions based on the content of the paper; the paper retrieval tool can use the API provided by the paper collection platform to obtain paper metadata and automatically download the paper to the local computer while returning the search results; the search engine retrieval tool calls an external search engine to answer user questions based on the search results.

[0012] In addition to traditional text input, the multimodal input module can also realize voice and image input.

[0013] Step 2: After the user logs in to the system page, initially set the parameters of the large language model and the tool calling module;

[0014] The parameters of the large language model include model ID, model API-KEY, model BaseURL, and model temperature;

[0015] The parameters of the tool module include the available tools and the initialization parameters of each tool.

[0016] Among them, ∑ is the set of all initialization parameters of the tool, σ i Represents different initialization parameters, N ∑ is the total number of initialization parameters required by the tool.

[0017] For the paper parsing tool, the initialization parameters include: the path of the paper to be parsed Path (string type), the length of the text segment ChunkLength (integer type), and whether to use the retrieval enhancement technology Retrieval (Boolean type);

[0018] For the paper search tool, the initialization parameters include: the paper collection platform Platform (string type) and whether to download the retrieved papers Download (Boolean type);

[0019] For the search engine retrieval tool, the initialization parameters include: the name of the search engine Name (string type), and the number of search results finally returned SearchNum (integer type).

[0020] Step 3: The core control module receives the user's input command Y I , start the first round of task execution and get the task plan Y T1 ;

[0021] 1) When the input command is a natural language, the core control module calls the large language model F θ , based on historical information Y H and user input command Y I Generate the current round 1 task plan Y T1 =F θ (Y I , Y H ).

[0022] Historical Information H Including the input command Y I All previous input instructions execute tasks, generate actions and loop results after action execution, and historical information Y before the initial input instruction H Is an empty set.

[0023] The specific method is: given an input sequence text Y in , output text sequence Y out satisfy:

[0024]

[0025] The input sequence text Y in Y I , Y H ), output text sequence Y out Plan Y for the task T1 , Y is a possible output text sequence, y i is the i-th word in the output text sequence Y, p(Y|Y in ) is given by Y in The probability that the model generates the text sequence Y under the condition .

[0026] 2) When the input command is voice, the core control module calls the API to convert it into text and obtains the input sequence text Y in , and then use the case where the input instruction in this step is a natural language to process. Output text sequence Y out satisfy:

[0027]

[0028] 3) When the input command is an image, the core control module calls the large language model F θ Generate mission plan Y T1 ; The specific method is: given an image input and text input sequence Vision encoder and connector P φ Image Aligned with the text modality, its output text sequence Y outsatisfy:

[0029]

[0030] The user inputs Y I For images Historical Information H For text

[0031] Step 4: The core control module executes the task according to the execution record Y of the first round of tasks. * , combined with historical information Y H , generate the decision action Y of the first round A1 , passing it to the specific tool calling module.

[0032] The core control module calls the large language model F θ Generate the most likely decision action Y A1 :

[0033] Y A1 =F θ (Y * , Y H )

[0034] Execution Record Y * =(Y I , Y T1 ).

[0035] Step 5: The tool calls the module to execute decision action Y A1 , get the action execution result Y O1 Passed back to the core control module.

[0036] The tool call module of the task execution cycle contains initialization parameters and operating parameters Λ n Represents the set of all running parameters of the tool. Represents different operating parameters, Represents the total number of operating parameters required by the tool, n represents the total number of task execution rounds, and the initial value is 1.

[0037] The process of the tool calling module generating the tool calling result is as follows: An From decision action Y A1 Extract the operating parameter Λ n , generate tool call result Y O1 , that is, Y O1 =T An (∑ n , Λ n ) and returns to the core control module.

[0038] Specifically for the paper parsing tool, its workflow is as follows:

[0039] 1. Convert the documents uploaded by the user into plain text, or slice the obtained paper segments into a set T0. If Retrieval is set to true during initialization, the enhanced segment set T is obtained using retrieval enhancement technology, otherwise T=T0.

[0040] 2. The paper parsing module follows instruction Y A1 Extract the running parameters question and method.

[0041] 3. Embed the text set T into the set prompt words, use the large language model to generate answers, and use them as the result Y of this round of action execution O1 , return to the core control module.

[0042] Specifically for the paper retrieval tool, its workflow is as follows:

[0043] 1. The paper retrieval module follows instruction Y A1 Extract the running parameters query, sort_by, and sort_order.

[0044] 2. Call the API to search for papers and return the searched paper metadata in JSON format.

[0045] 3. The paper retrieval module extracts key information such as paper title, paper author, paper abstract, paper publication time, etc. from the paper metadata in JSON format and generates a structured action execution result Y O1 , and returns to the core control module.

[0046] Specifically for search engine retrieval tools, its workflow is:

[0047] 1. Search engine retrieval module based on Y A1 Generates the running parameter query.

[0048] 2. Call the API to search from the Internet, cut the retrieved web page results into the form of content + link, use the search enhancement technology on all the cut results to obtain the filtered search results. The filtered search results are spliced ​​as the action execution result Y O1 Return to the core control module.

[0049] Step 6: Execute the action result Y O1 Determine input command Y I Whether the current round 1 task execution is completed, if so, generate the final output Y F , wait for new user input and return to step 3; otherwise, call the large language model F θGenerate the next round of task planning and return to step 4 to start the next round of task execution cycle.

[0050] The next round of task planning is Y T2 =F θ (Y * , Y H ); and so on, when the n+1th round of task planning Y is generated T(n+1) Before, the execution record of the task Y * Contains all the contents of the first n rounds of task execution cycles, that is,

[0051] Y * =(Y I , Y T1 , Y A1 , Y O1 , Y T2 , Y A2 , Y O2 , ..., Y Tn , Y An , Y On )

[0052] Let the task planning of the nth round be Y Tn , the decision action Y in round n An This is the tool call instruction. The result of the nth round of action execution is Y On ;

[0053] When the task has been completed, according to the current execution record Y * and historical information Y H Generate the final output Y F , that is, Y F =F θ (Y * , Y H ), and then sort it out as the execution result and feed it back to the user; at the same time, the historical information is updated using the execution record of that time, i.e. Y H ←(Y H , Y * ).

[0054] The advantages of the present invention are:

[0055] (1) The present invention provides a scientific research assistance method based on a large language model. The core control module uses the large language model to analyze semantics and generates tool call instructions with a specific format in combination with designed prompt words. With the large language model as the decision-making center, relatively vague natural language input can be converted into specific and clear tool call instructions, so that the system can handle the user's vague needs and reduce the system's need for user participation.

[0056] (2) The present invention is a scientific research assistance method based on a large language model. For scientific research assistance tasks, a special tool calling module is designed to execute tool calling instructions. Each tool focuses on a specific function, and the content and format of the input and output are relatively fixed. After the processing center defines the processing target, it can achieve the target in a relatively clear way, ensuring the feasibility of achieving the target and the reliability of the processing result.

[0057] (3) The present invention provides a scientific research assistance method based on a large language model, which realizes a multimodal input function and can accept inputs in multiple different modes such as text, voice, and image. Different processing methods are used for different modules, which not only enriches the functions of the system but also can match the diverse needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The module structure and workflow diagram of the scientific research auxiliary system (SciAgent) based on the large language model of the present invention.

[0059] Figure 2 A flowchart of a scientific research assistance method based on a large language model of the present invention; DETAILED DESCRIPTION

[0060] The present invention describes the system design and workflow of the scientific research auxiliary system SciAgent as a computer program.

[0061] The present invention proposes a research assistance method based on a large language model, such as Figure 1 As shown in the figure, with the large language model as the decision-making center, the core control module can process relatively vague natural language instructions and filter out noise information in search results; in addition, the system can provide advanced functions such as literature analysis and literature summarization based on the large language model, and use tool call modules to perform paper parsing, paper retrieval and search engine retrieval; it can also process multimodal input including voice, images and text. The system can reduce the time and energy spent by researchers on literature research, which is of great significance to improving the research efficiency of researchers.

[0062] The scientific research assistance method based on the large language model, such as Figure 2 As shown, the specific steps are as follows:

[0063] Step 1: Build a scientific research auxiliary system including a core control module, a tool calling module and a multimodal input module;

[0064] The core control module is the processing center, with a large language model as the main body. Its main function is to understand user input and generate tool call instructions.

[0065] The tool calling module is responsible for executing specific tasks, including three tools: paper analysis, paper retrieval, and search engine retrieval. Among them, the paper analysis tool can use the large language model to answer user questions based on the content of the paper; the paper retrieval tool can use the API provided by the paper collection platform to obtain paper metadata and automatically download the paper to the local computer while returning the search results; the search engine retrieval tool calls an external search engine to answer user questions based on the search results.

[0066] In addition to traditional text input, the multimodal input module can also realize voice and image input.

[0067] Step 2: After the user logs in to the system page, initially set the parameters of the large language model and the tool calling module;

[0068] The parameters of the large language model include model ID, model API-KEY, model BaseURL, and model temperature;

[0069] The parameters of the tool module include the available tools and the initialization parameters of each tool.

[0070] Among them, ∑ is the set of all initialization parameters of the tool, σ i Represents different initialization parameters, N ∑ is the total number of initialization parameters required by the tool.

[0071] For the paper parsing tool, the initialization parameters include: the path of the paper to be parsed Path (string type), the length of the text segment ChunkLength (integer type), and whether to use the retrieval enhancement technology Retrieval (Boolean type);

[0072] For the paper search tool, the initialization parameters include: the paper collection platform Platform (string type) and whether to download the retrieved papers Download (Boolean type);

[0073] For the search engine retrieval tool, the initialization parameters include: the name of the search engine Name (string type), and the number of search results finally returned SearchNum (integer type).

[0074] Step 3: The core control module receives the user's input command Y I , start the first round of task execution and get the task plan Y T1 ;

[0075] 1) When the input command is a natural language, the core control module calls the large language model F θ , based on historical information Y H and user input command YI Generate mission plan Y T1 =F θ (Y I , Y H ).

[0076] Historical Information H Including the input command Y I All previous input instructions execute tasks, generate actions and loop results after action execution, and historical information Y before the initial input instruction H Is an empty set.

[0077] The specific method is: given an input sequence text Y in , output text sequence Y out satisfy:

[0078]

[0079] The input sequence text Y in Y I , Y H ), output text sequence Y out Plan Y for the task Tn , Y is a possible output text sequence, y i is the i-th word in the output text sequence Y. p(Y|Y in ) is given by Y in The probability that the model generates text Y under the condition .

[0080] 2) When the input command is voice, the core control module calls the API to convert it into text and obtains the input sequence text Y in , output text sequence Y out satisfy:

[0081]

[0082] 3) When the input command is an image, the core control module calls the large language model F θ Generate mission plan Y T1 ; The specific method is: given an image input and text input sequence Vision encoder and connector P φ Image Aligned with the text modality, its output text sequence Y out satisfy:

[0083]

[0084] The user inputs Y I For images Historical Information H For text

[0085] Step 4: The core control module executes the record Y according to the task * , and combined with historical information Y H , generate the decision action Y of the first round A1 , passing it to the specific tool calling module.

[0086] Initial execution record Y * The value is Y * =(Y I , Y T1 ), according to the core control module, the large language model F is called θ Generate the most likely decision action Y A1 :

[0087] Y A1 =F θ (Y * , Y H )

[0088] Step 5: The tool call module executes the tool call instruction Y A1 , get the action execution result Y O1 Passed back to the core control module.

[0089] Tool call module T for task execution cycle An Contains initialization parameters and operating parameters

[0090] Λ n Represents the set of all running parameters of the tool. Represents different operating parameters, Represents the total number of operating parameters required by the tool, n represents the total number of task execution rounds, and the initial value is 1.

[0091] The process of the tool calling module generating the tool calling result is as follows: A1 From decision action Y A1 Extract the running parameter Λ1 from the tool and generate the tool call result Y O1 , that is, Y O1 =T A1 (∑1, Λ1) and returns to the core control module.

[0092] Specifically for the paper parsing tool, its workflow is as follows:

[0093] 1. Convert the documents uploaded by the user into plain text, or slice the obtained paper segments into a set T0. If Retrieval is set to true during initialization, the enhanced segment set T is obtained using retrieval enhancement technology, otherwise T=T0.

[0094] 2. The paper parsing module follows instruction Y A1 Extract the running parameters question and method.

[0095] 3. Embed the text set T into the set prompt words, use the large language model to generate answers, and use them as the result Y of this round of action execution O1 , return to the core control module.

[0096] Specifically for the paper retrieval tool, its workflow is as follows:

[0097] 1. The paper retrieval module follows instruction Y A1 Extract the running parameters query, sort_by, and sort_order.

[0098] 2. Call the API to search for papers and return the searched paper metadata in JSON format.

[0099] 3. The paper retrieval module extracts key information such as paper title, paper author, paper abstract, paper publication time, etc. from the paper metadata in JSON format and generates a structured action execution result Y O1 , and returns to the core control module.

[0100] Specifically for search engine retrieval tools, its workflow is:

[0101] 1. Search engine retrieval module based on Y A1 Generates the running parameter query.

[0102] 2. Call the API to search from the Internet, cut the retrieved web page results into the form of content + link, use the search enhancement technology on all the cut results to obtain the filtered search results. The filtered search results are spliced ​​as the action execution result Y O1 Return to the core control module.

[0103] Step 6: Execute the action result Y O1 Determine input command Y I Whether the current round 1 task execution is completed, if so, generate the final output Y F , wait for new user input and return to step 3; otherwise, call the large language model F θ Generate the next round of task planning and return to step 4 to start the next round of task execution cycle.

[0104] The next round of task planning is Y T2 =F θ (Y * , Y H ); and so on, the core control module evaluates the execution record Y of the n+1th round of tasks * Contains all the contents of the first n rounds of task execution cycles, that is,

[0105] Y * =(Y I , Y T1 , Y A1 , Y O1 , Y T2 , Y A2 , Y O2 , ..., Y Tn , Y An , Y On )

[0106] The task planning for round n is Y Tn , the result of the action execution in round n is Y On , decision action Y An That is the tool call instruction;

[0107] When the task has been completed, according to the current execution record Y * and historical information Y H Generate the final output Y F , that is, Y F =F θ (Y * , Y H ), and then sort it out as the execution result and feed it back to the user; at the same time, the historical information is updated using the execution record of that time, i.e. Y H ←(Y H , Y * ).

[0108] Example:

[0109] The system workflow of SciAgent of the present invention specifically includes its system design and system implementation; it is divided into the following steps:

[0110] Step 1: After the user enters the system page, initialize an artificial intelligence agent instance according to the user settings.

[0111] The parameters that need to be set include those of the large language model and the tool module.

[0112] Step 2: Receive the user's natural language or other modal input instructions such as voice, image, etc. The core control module generates a tool call instruction based on the user input and passes it to the specific tool call module.

[0113] For natural language input Y I The core control module needs to first evaluate the task execution record Y * , call the large language model F θ Generate mission plan Y T :If the task has just started, only Y I , that is, Y * =(Y I ); if several cycles have been completed, the execution record will also contain records of each cycle, that is:

[0114] Y * =(Y I , Y A1 , Y O1 , Y T1 , Y A2 , Y O2 , Y T2 , ..., Y An , Y On ).

[0115] Afterwards, the core control module combines the current execution record Y * and historical information Y H Calling the large language model F θ Generate the most likely decision action Y A (i.e. tool call instruction):

[0116] Y A The generation probability can be calculated based on Calculate, where N A is the instruction length.

[0117] After getting the tool call instruction Y A After that, it is passed to the specific tool calling module T A .

[0118] Tool call module T A Contains initialization parameters and operating parameters For the input Y from the core control module A , the tool call module needs to extract the operating parameters Λ from it, and then generate the tool output Y O , that is, Y O =T A (∑,Λ).

[0119] If the input contains speech, you need to call the API to convert it into text;

[0120] If the input contains an image X, it needs to go through a visual encoder and connector P φ Align it with the text modal. A The probability of generation is based on The rest of the calculation is the same as above.

[0121] Since different tools have different computer program implementation methods under different initialization parameters ∑, the operating parameters Λ required to be provided by the upper-level module during the actual operation process are also different. Therefore, different tool instances need to provide different operating parameter descriptions.

[0122] In addition to relying on user input, the process of generating tool call instructions by the core control module also needs to refer to the operating parameter descriptions of different tool instances. In the actual operation process, the tool description needs to be organized into a specific format as part of the prompt words of the large language model.

[0123] For the paper parsing tool, there are two operating parameters: question and method. See Table 1 for detailed explanation.

[0124] For the paper retrieval tool, there are three operating parameters: query, sort_by, and sort_order. See Table 2 for specific explanations.

[0125] For search engine retrieval tools, the only operating parameter is query. See Table 3 for detailed explanation.

[0126] Table 1

[0127]

[0128] Table 2

[0129]

[0130] Table 3

[0131]

[0132]

[0133] Step 3: The tool calling module executes the tool and passes the tool calling result back to the core control module;

[0134] The tool calling module includes paper parsing tools, paper retrieval tools and search engine retrieval tools;

[0135] The workflow of the paper parsing tool is:

[0136] (1) Convert the documents uploaded by the user into plain text. If the text is too long, it needs to be sliced ​​to obtain the paper segments. The paper text or the original paper segments constitute the set T0. If Retrieval is set to true during initialization, it is necessary to use the retrieval enhancement technology to obtain the enhanced segment set T, otherwise set T = T0.

[0137] (2) The core control module generates the paper parsing tool call instruction Y A , the paper analysis module is based on Y A Extract the running parameters question and method.

[0138] (3) Embed the text set T into the set prompt words and use the big model to generate answers. The generated answers are used as tool output Y O , return to the core control module.

[0139] The workflow of the paper search tool is:

[0140] (1) The core control module generates the paper retrieval tool call instruction Y A , the paper retrieval module is based on Y A Extract the running parameters query, sort_by, and sort_order.

[0141] (2) Call the API to search for papers and return the searched paper metadata in JSON format.

[0142] (3) The paper retrieval module extracts key information such as paper title, paper author, paper abstract, and paper publication time from the paper metadata in JSON format and generates a structured tool output Y O , and returns to the core control module.

[0143] The workflow of the search engine retrieval tool is:

[0144] (1) The core control module generates a search engine retrieval tool call instruction Y A , the search engine retrieval module is based on Y A Generates the running parameter query.

[0145] (2) Call the API to search from the Internet, cut the retrieved web page results into the form of content + link, use the search enhancement technology on all the cut results to obtain the filtered search results. The filtered search results are spliced ​​and used as the tool output Y O Return to the core control module.

[0146] In the calling process of the above modules, the search enhancement technology is used to complete the relevant tasks. The search enhancement technology selects one or more texts with the highest relevance to the input text from a number of candidate texts, which can filter out noise texts and reduce the calling cost of the large language model. The details of the search enhancement technology are as follows:

[0147] The original text is segmented into an original text set c, which is vectorized through the embedding model to form a vector set V = (V1, V2, ..., V n ), each paragraph corresponds to a set of vectors. The user input is also vectorized to obtain vector I0, which is traversed and compared in the vector database to obtain the relevance score R(V i , I0), the calculation method is cosine similarity, that is:

[0148]

[0149] The smaller the correlation score of two vectors, the higher the correlation between the texts corresponding to the two vectors. top And the correlation threshold R s , filter out the values ​​that satisfy the relevance score R≤R S And the k with the smallest correlation top vectors, the set they form is V', and the corresponding set of paragraphs is C'. Return C'.

[0150] Step 4: The core control module calls the large language model F θ , based on historical information Y H And the current execution record Y * =(Y I , Y A1 , Y O1 , Y T1 , Y A2 , Y O2 , Y T2 , ..., Y An , Y On ) Evaluate the current state and generate a task plan Y for the current state Tn ,Right now If the evaluation result is that the task is not completed, go to step 2 and perform the task according to the current execution record Y * and historical information Y H Generate the next decision action Y A(n+1) If the evaluation result is that the task is completed, go to step 5 to generate the final output Y F .

[0151] Step 5: The core control module executes the record Y according to the current execution * and historical information Y HGenerate the final output Y F , and then sort it out as the execution result and feed it back to the user. Use the current execution record to update the historical information, that is, Y H ←(Y H , Y * ).

[0152] The scientific research assistance system based on the large language model described in this specification can be implemented with tangible concrete implementation computer software or firmware, with computer hardware (including the structure disclosed in this specification and its structural equivalents), or with a combination of one or more of them. The scientific research assistance system based on the large language model described in this specification can be implemented as one or more computer programs. A computer program (which may also be referred to or described as a program, software, software application, module, software module, script or code) can be written in any form of programming language, including compiled or interpreted languages ​​or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

Claims

1. A scientific research assistance method based on a large language model, characterized in that: The specific steps are as follows: Step 1: Build a scientific research auxiliary system including a core control module, a tool calling module and a multimodal input module; Step 2: After the user logs in to the system page, initially set the parameters of the large language model and the tool calling module; The parameters of the large language model include model ID, model API-KEY, model BaseURL, and model temperature; The parameters of the tool module include the available tools and the initialization parameters of each tool. Among them, Σ is the set of all initialization parameters of the tool, σ i Represents different initialization parameters, N Σ is the total number of initialization parameters required by the tool; Step 3: The core control module receives the user's input command Y I , start the current round 1 task execution and get the task plan Y T1 ; 1) When the input command is a natural language, the core control module calls the large language model F θ , based on historical information Y H and user input command Y I Generate the current round 1 task plan Y T1 =F θ (Y I ,Y H ); The specific method is: given an input sequence text Y in , output text sequence Y out satisfy: The input sequence text Y in Y I ,Y H ), output text sequence Y out Plan Y for the task Tn , Y is a possible output text sequence, y i is the i-th word in the output text sequence Y; p(Y|Y in ) is given by Y in The probability of the model generating the text sequence Y under the condition ; 2) When the input command is voice, the core control module calls the API to convert it into text and obtains the input sequence text Y in , that is, the output text sequence Y using natural language computing out ; 3) When the input command is an image, the core control module calls the large language model F θ Generate mission plan Y T1 ; The specific method is: given an image input and text input sequence Vision encoder and connector P φ Image Aligned with the text modality, its output text sequence Y out satisfy: The user inputs Y I For images Historical InformationY H For text Step 4: The core control module uses the task planning Y T1 Get task execution record Y * , and combined with historical information Y H , generate the decision action Y for the first round A1 , passing it to the specific tool calling module; The core control module calls the large language model F θ Generate the most likely decision action Y A1 : AND A1 =F θ (AND * ,AND H ) Initial execution record Y * The value is Y * =(Y I ,Y T1 ); Step 5: The tool call module executes the tool call instruction Y A1 , get the action execution result Y O1 Passed back to the core control module; Tool call module T for the first task execution cycle An Contains initialization parameters and operating parameters Λ n Represents the set of all running parameters of the tool. Represents different operating parameters, Represents the total number of parameters required to run the tool; The process of the tool calling module generating the tool calling result is as follows: An From decision action Y A1 Extract the operating parameter Λ n , generate tool call result Y O1 , that is, Y O1 =T An (Σ n ,Λ n ), and return to the core control module; Step 6: Execute the action result Y O1 Determine input command Y I Whether the current round 1 task execution is completed, if so, generate the final output Y F , wait for new user input and return to step 3; otherwise, call the large language model F θ Generate the second round of task planning, and return to step 4 to start the next round of task execution cycle; The next round of task planning is Y T2 =F θ (Y * ,Y H ); and so on, when the execution record Y of the n+1th round of tasks is generated * Contains all the contents of the first n rounds of task execution cycles, that is, AND * (And I ,AND T1 ,AND A1 ,AND O1 ,AND T2 ,AND A2 ,AND O2 ,…,AND Tn ,AND An ,AND On ) The task planning for round n is Y Tn , the result of the action execution in round n is Y On , decision action Y An That is the tool call instruction; When the task has been completed, according to the current execution record Y * and historical information Y H Generate the final output Y F , that is, Y F =F θ (Y * ,Y H ), and then sort it out as the execution result and feed it back to the user; at the same time, the historical information is updated using the execution record of that time, i.e. Y H ←(Y H ,Y * ).

2. A scientific research assistance method based on a large language model as claimed in claim 1, characterized in that: In step one, the core control module is the processing center, with a large language model as the main body, and its main function is to understand user input and generate tool call instructions; the tool call module is responsible for executing specific tasks, including three tools: paper analysis, paper retrieval and search engine retrieval; in addition to traditional text input, the multimodal input module can also realize voice and image input.

3. A scientific research assistance method based on a large language model as described in claim 2, characterized in that: The paper parsing tool uses a large language model to answer user questions based on the paper content; the paper retrieval tool can use the API provided by the paper collection platform to obtain paper metadata, and automatically download the paper to the local computer while returning the search results; the search engine retrieval tool calls an external search engine to answer user questions based on the search results.

4. A scientific research assistance method based on a large language model as claimed in claim 2, characterized in that: The paper parsing tool, the initialization parameters include: the paper path Path to be parsed, the length of the text segment ChunkLength and whether to use the retrieval enhancement technology Retrieval; For the paper search tool, the initialization parameters include: the paper collection platform Platform and whether to download the retrieved papers Download; For the search engine retrieval tool, the initialization parameters include: the name of the search engine Name, and the number of search results finally returned SearchNum.

5. A scientific research assistance method based on a large language model as claimed in claim 1, characterized in that: In step 3, historical information Y H Including the input command Y I All previous input instructions execute tasks, generate actions and loop results after action execution, and historical information Y before the initial input instruction H Is an empty set.

6. A scientific research assistance method based on a large language model as claimed in claim 1, characterized in that: In step 5, the paper parsing tool obtains the action execution result Y O1 The workflow is: 1) Convert the documents uploaded by the user into plain text, or slice the obtained paper segments into a set T0. If Retrieval is set to true in the initialization, the enhanced segment set T is obtained using the retrieval enhancement technology, otherwise T = T0; 2) The paper parsing module follows instruction Y A1 Extract the running parameters question and method; 3) Embed the text set T into the set prompt words, use the large language model to generate answers, and use them as the result Y of this round of action execution O1 , return to the core control module.

7. A scientific research assistance method based on a large language model as claimed in claim 1, characterized in that: In step 5, the paper retrieval tool obtains the action execution result Y O1 The workflow is: 1) The paper retrieval module follows instruction Y A1 Extract the running parameters query, sort_by and sort_order; 2) Call the API to search for papers and return the searched paper metadata in JSON format; 3) The paper retrieval module extracts the paper title, paper author, paper abstract and paper publication time information from the paper metadata in JSON format and generates a structured action execution result Y O1 , and returns to the core control module.

8. A scientific research assistance method based on a large language model as claimed in claim 1, characterized in that: In step 5, the search engine retrieval tool obtains the action execution result Y O1 The workflow is: 1) Search engine retrieval module based on Y A1 Generate running parameter query; 2) Calling API to search from the Internet, cutting the retrieved web page results into the form of content + link, and applying search enhancement technology to all the cut results to obtain filtered search results; The filtered search results are concatenated as the action execution result Y O1 Return to the core control module.

Images