Intelligent flow-tuned question-answering method fusing different-granularity knowledge prompts
By integrating different granularity knowledge prompts and fine-tuning of large language models, combined with LoRA technology, accurate questions are generated, and the problems of human dependence and single templates in the Epidemic Investigation Q&A are solved, efficient and accurate information collection is achieved, and the Epidemic Investigation Q&A is improved effect.
Patent Information
- Application Number
- CN202411113730.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-12-10
AI Technical Summary
The existing flow investigation question and answer methods rely on a large amount of manpower, with a single question and answer template, low effective information coverage, low work efficiency, and it is difficult to generate sufficient quantity and quality problems, which affects the comprehensiveness and depth of the survey.
An intelligent flow investigation question-and-answer method is adopted that integrates knowledge prompts of different granularity. Through fine-tuning of flow investigation knowledge graphs and large language models, combined with LoRA technology, dialogue history and knowledge prompts are used to generate accurate questions, and the coverage and accuracy of flow investigation question-and-answer questions are improved.
Rapidly improve the effectiveness of epidemiological investigation and Q&A, solve the problem of relying on a large amount of manpower, single question-and-answer templates, and low efficiency, and achieve more efficient information collection and more accurate epidemiological investigation.
Smart Images
Figure CN119106164A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and specifically to an intelligent flow survey question-answering method that integrates knowledge prompts of different granularities. Background Art
[0002] In the process of fighting against infectious diseases, epidemiological investigation (epidemiological survey) is a key step for the epidemic prevention and control departments to control the spread of the epidemic. Its main purpose is to quickly and comprehensively grasp the possible transmission path of the epidemic by investigating individuals who may be infected with the virus, their activity trajectories, contact with people and other information, and then take corresponding prevention and control measures in a targeted manner. Therefore, a high-quality epidemiological survey method should, on the one hand, be able to assist in identifying, tracking and managing potential risk groups, so as to block the spread of pathogens in a targeted manner. On the other hand, it should also be able to trace the root cause of the epidemic based on the information collected by the epidemiological survey, investigate the source of infection, exposure links and transmission routes, so as to effectively control new infection events in the early stages of the epidemic. In order to achieve the above goals, various information needs to be collected during the epidemiological survey, including basic information of the epidemiological survey subjects, health status, recent travel records, and any important information that is helpful for disease prevention and control.
[0003] Designing a questionnaire is a key step in epidemiological surveys. The questionnaire must be clear and the questions must be specific and avoid ambiguity to ensure that respondents understand and can answer correctly. However, manually designing questions requires a lot of time and effort, and it is difficult to design a sufficient number and quality of questions within a limited time, which may limit the comprehensiveness and depth of the survey. Existing models for automatically designing epidemiological survey questions have problems such as the generated questions may not be smooth enough, not well connected with the previous questions, and not comprehensive enough, which affects the overall quality of the survey, or the quality of the questions generated in epidemiological surveys in specific areas is not high enough, and not as accurate and targeted as the questions designed by professionals.
[0004] In the current field of artificial intelligence, large pre-trained language models such as the GPT series have become the mainstream model architecture. The ChatGPT model trained on this basis has demonstrated strong language generation capabilities, contextual learning capabilities, thought chain capabilities, and extensive world knowledge. It has achieved good results in various natural language processing tasks such as machine translation, sentiment analysis, question-answering, and dialogue systems, and has triggered people's thinking about general artificial intelligence.
[0005] Existing technologies for epidemic investigation rely on a large amount of manpower, have a single question and answer template, low coverage of effective information, and low work efficiency. Summary of the invention
[0006] The purpose of the present invention is to provide an intelligent epidemic investigation question and answer method that integrates knowledge prompts of different granularities in response to the deficiencies of the prior art. The method adopts a method that integrates knowledge prompt strategies of different granularities, and automatically generates accurate questions for epidemic investigation based on the dialogue history and knowledge prompts of different granularities through dialogue with the epidemic investigation object. The method is based on the question and answer fine-tuned by the instructions of a large language model, and uses the knowledge prompts as a thinking chain to stimulate the rich world knowledge accumulated during the language model training process. It also uses the context learning ability, thinking chain ability, and multi-round dialogue ability of the large model to realize intelligent and automated epidemiological related information collection through a dialogue guided by knowledge prompts, thereby improving the effect of epidemic investigation question and answer. The coverage and accuracy of epidemic investigation question and answer are improved through fine-tuning of the large model, which better solves the problems of relying on a large amount of manpower, a single question and answer template, and low efficiency in the process of epidemic investigation information collection, and quickly improves the effect of epidemic investigation question and answer. The method is simple and has good application prospects and commercial development value.
[0007] The specific technical solution to achieve the purpose of the present invention is: an intelligent flow survey question-answering method integrating knowledge prompts of different granularities, specifically comprising the following steps:
[0008] Step 1: Knowledge Extraction
[0009] 1-1: Construction of Epidemic Investigation Knowledge Graph
[0010] The epidemiological survey knowledge graph covers the elements of epidemiological surveys in various scenarios and is organized in a top-down hierarchical classification method. It includes five main modules: basic personal information, place of residence, place of work, medical treatment and medication, and activity trajectory survey. Each module contains corresponding epidemiological survey question elements, which are interrelated and mainly contain relationships. In the first four modules, namely basic personal information, place of residence, place of work, medical treatment and medication, the question elements are usually relatively stable and easy to summarize manually. However, the situation of activity trajectory surveys will change with the activities of different epidemiological survey objects. The question elements related to it are varied and complex, involving detailed information of specific activities. For example, in a restaurant dining scenario, the relevant question elements include dining methods (such as table meals or buffets), service staff, seat distribution, payment methods, etc., which poses a huge challenge to intelligent epidemiological surveys. Therefore, the present invention focuses on solving activity trajectory surveys in different scenarios.
[0011] 1-2: Knowledge extraction at different granularities
[0012] Scene recognition: Based on knowledge graphs and large language models, coarse-grained scene knowledge is obtained. In actual flow investigation operations, it is often difficult to directly know the specific scenes involved. NTherefore, in order to make the model more suitable for practical application needs, the present invention uses the epidemic survey question and answer data of the marked scenes to perform supervised fine-tuning on the large language model to establish a scene recognition model. The model is based on the conversation history and converts the latest reply V of the epidemic survey object into N Associated to the most relevant scene S in the knowledge graph N .
[0013] Question element recognition: After scene recognition, fine-grained question element knowledge is further extracted. N , combined with the knowledge graph, extract the scene S from the knowledge graph N Corresponding question elements Q N .
[0014] Step 2: Knowledge prompts of different granularities
[0015] Fusion of coarse-grained scene hints: Based on the acquired scene S N The present invention designs a coarse-grained scenario prompt S = {the scene of the flow survey object answering: [S N ], the next question should involve one of the following scenarios: ['General', S N ]}, Since the questions represented by “general” are universal in any scenario, the label “general” is specially added to the next question scenario prompt to ensure that the subsequent questions of the model can more closely reflect the actual situation.
[0016] Integrate prompts with fine-grained question elements: Based on the scene prompts, more fine-grained question element prompts are integrated. Based on the question elements extracted from the knowledge graph, the question element prompt content Q = {You can choose one of the following topics to ask: Among them, Q C and Represents general and scenario S respectively N A collection of question elements.
[0017] Afterwards, combining the instruction I, the conversation history H, and the knowledge prompts S and Q of different granularities, the present invention designs a format for expressing the input data as shown in the following formula (a):
[0018]
[0019] Among them, X represents the input data, Indicates a connector. The specific command input format is: command + conversation history + scenario prompt + question element prompt. The output format is: the next question asked by the epidemiologist under the current conversation history. Instruction I defines the task description, and the content is fixed. I = {You need to play the role of an epidemiological investigator and have a conversation with the epidemiological investigation subject to understand the epidemiological investigation subject's recent activity trajectory and the people he has contacted. The conversation is in the form of one question and one answer}.
[0020] Step 3: Fine-tune the large model
[0021] Large models usually have a huge number of parameters, and fine-tuning these models requires a lot of computing resources and storage space. This may require the use of high-performance computing equipment and expensive hardware, increasing the cost and complexity of fine-tuning. Therefore, the present invention uses LoRA technology to fine-tune the large model, adding a bypass next to the original large model, which are the dimension reduction matrix A and the dimension increase matrix B, respectively. Matrix A is initialized with a random Gaussian distribution, and B is initialized with a 0 matrix. The input and output dimensions of the original large model remain unchanged. When outputting, the parameters of matrix B and matrix A are superimposed with the parameters of the original large model. During training, the parameters θ of the original model are fixed, and only the parameters of the newly added low-rank matrices A and B are updated. The fine-tuning update of the specific model is expressed by the following formula (b):
[0022] W+ΔW=W+BA,W∈R d*k , B∈R d*r , A∈R r*k (b).
[0023] Among them, W is the parameter of the initial model; ΔW is the training parameter of the LoRA model; B is a 0 matrix; A is a matrix initialized by a random Gaussian distribution; d is the output dimension of the previous layer of the model; k is the input dimension of the next layer of the model.
[0024] After the input data X is input into the model encoding, the input vector feature representation H is obtained. Then, the model parameters θ, the dimension reduction matrix A and the dimension increase matrix B of the original model are used. Using formula (b), B, A and the original model parameters θ are added to obtain the target prediction response Y with a length of T represented by the following formula (c). The decoder generates the next question P to be asked in the current dialogue history one by one, represented by the following formula (d):
[0025] Y=[y1,y2,...,y T ](c).
[0026] Among them, y1,y2,...,y T is the predicted response for each target.
[0027] P(y t |y <t ,H)=Dec(E y<t,H,θ;A,B)(d).
[0028] Among them, E y<t is the output generated at the current time t; H is the feature representation of the input vector; y <t represents the sequence of all output tokens that have been generated before time t; y t The output generated at the current time t; P is the conditional probability, which represents the tag sequence y generated before the current input vector H <t Based on this, generate the next word y t probability.
[0029] The loss function is expressed by the following formula (e): Make the questions generated by the model as consistent as possible with the real questions:
[0030]
[0031] Among them, P is the conditional probability in formula (d), T represents the length of the entire generated sequence, and L is the loss function, which represents the gap between the output generated by the model and the true output.
[0032] Compared with the prior art, the present invention has the following beneficial technical effects and significant technical progress:
[0033] 1) By building a knowledge graph for epidemic investigation and designing knowledge prompts of different granularities based on the knowledge graph, we can stimulate the powerful language generation capabilities of large language models;
[0034] 2) Fine-tune the large model through LoRA technology to quickly improve the model's adaptability in epidemic investigation and question-answering tasks with minimal computational cost;
[0035] 3) It can effectively solve the problems of existing epidemic investigation, such as reliance on a large amount of manpower, single question and answer template, low coverage of effective information and low efficiency, and quickly improve the effect of epidemic investigation question and answer. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a flow chart of the present invention;
[0037] Figure 2 This is an example diagram of the knowledge graph. DETAILED DESCRIPTION
[0038] See also Figure 1 , the present invention specifically comprises the following steps:
[0039] Step 1: Knowledge Extraction
[0040] 1-1: Building a knowledge graph for epidemic investigation
[0041] See also Figure 2, the epidemiological survey knowledge graph covers the elements of epidemiological surveys in various scenarios and is organized in a top-down hierarchical classification method. It includes five main modules: basic personal information, place of residence, place of work, onset of medical treatment and medication, and activity trajectory survey. Each module contains corresponding epidemiological survey question elements, which are interrelated and mainly contain relationships. In the first four modules, namely basic personal information, place of residence, place of work, onset of medical treatment and medication, the question elements are usually relatively stable and easy to summarize manually. However, the situation of activity trajectory surveys will change with the activities of different epidemiological survey objects. The related question elements are varied and complex, involving detailed information of specific activities. For example, in the restaurant dining scenario, the relevant question elements include dining methods (such as table meals or buffets), service staff, seat distribution, payment methods, and so on.
[0042] 1-2: Extracting knowledge of different granularities
[0043] 1-2-1: Based on the knowledge graph and large language model, coarse-grained scene knowledge is obtained. The large language model is fine-tuned in a supervised manner using the epidemic survey question and answer data of the labeled scenes to establish a scene recognition model. The model is based on the conversation history and converts the latest reply V of the epidemic survey object into N The most relevant scenario S associated with the constructed knowledge graph N .
[0044] 1-2-2: Extract scene S from the knowledge graph N Corresponding question elements Q N .
[0045] Step 2: Knowledge prompts of different granularities
[0046] 2-1: Based on the acquired scene S N , coarse-grained scenario prompt S = {the scene of the flow survey object answer: [S N ], the next question should involve one of the following scenarios: ['General', S N ]}.
[0047] 2-2: Based on the question elements extracted from the knowledge graph, the question element prompt Q is formed = {You can choose one of the following topics to ask: Among them, Q C and Represents general and scenario S respectively N A collection of question elements.
[0048] 2-3: The format of input data X is expressed by the following formula (a):
[0049]
[0050] Where X is the input data; is a connector; I is an instruction; H is a conversation history; S is a scene prompt; Q is a question element prompt.
[0051] The instruction I = {You need to play the role of an epidemiological investigator and have a conversation with the subject of the investigation to understand the subject's recent activity trajectory and the people he has contacted. The conversation is in the form of question and answer}. The input format of instruction I is: instruction I + conversation history H + scene prompt S + question element prompt Q; the output format of instruction I is: the next question asked by the investigator under the current conversation history.
[0052] Step 3: Fine-tune the large model
[0053] 3-1: Use LoRA technology to fine-tune the large model. Add a bypass next to the original large model, which are the dimension reduction matrix A and the dimension increase matrix B. Initialize matrix A with random Gaussian distribution and initialize B with 0 matrix. The input and output dimensions of the original large model remain unchanged. When outputting, the parameters of the dimension reduction matrix A and the dimension increase matrix B are superimposed with the parameters of the original large model. During training, the parameters θ of the original model are fixed, and the parameters of the dimension reduction matrix A and the dimension increase matrix B are updated. The fine-tuning update of the specific model is expressed by the following formula (b):
[0054] W+ΔW=W+BA,W∈R d*k , B∈R d*r , A∈R r*k (b).
[0055] Among them, W is the parameter of the initial model; ΔW is the training parameter of the LoRA model; B is a 0 matrix; A is a matrix initialized by a random Gaussian distribution; d is the output dimension of the previous layer of the model; k is the input dimension of the next layer of the model.
[0056] 3-2: Input the input data X into the fine-tuned large model, and after encoding, obtain the input vector feature representation H. Then, using the model parameters θ, the dimension reduction matrix A and the dimension increase matrix B of the original model, using formula (b), add BA to the original model parameters θ to obtain the target prediction response Y with a length of T represented by the following formula (c):
[0057] Y=[y1,y2,...,y T ](c).
[0058] Among them, y1,y2,...,y T is the predicted response for each target.
[0059] 3-3: The decoder generates the next question to be asked in the current conversation history, represented by the following formula (d):
[0060] P(y t |y<t ,H)=Dec(E y<t ,H,θ;A,B) (d);
[0061] Among them, E y<t is the output generated at the current time t; H is the feature representation of the input vector; y <t represents the sequence of all output tokens that have been generated before time t; y t The output generated at the current time t; P is the conditional probability, which represents the tag sequence y generated before the current input vector H <t Based on this, generate the next word y t probability.
[0062] 3-3: The loss function expressed by the following formula (e) Make the questions generated by the model as consistent as possible with the real questions;
[0063]
[0064] Where P is the conditional probability in formula (d), T represents the length of the entire generated sequence, and L is the loss function, which represents the gap between the output generated by the model and the true output.
[0065] In order to make the purpose, technical solution and advantages of the present invention clearer, the following will further describe the implementation mode of the present invention in detail by using the example of the flow survey data set in combination with the accompanying drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are common knowledge and common common sense in the field, and the present invention does not specifically limit the contents.
[0066] Example 1
[0067] The conversation history content is described as follows:
[0068] Epidemic investigator: Hello, were you in the hotel room on the morning of May 26?
[0069] Epidemiological investigation subject: Yes, I was in the hotel room the whole morning.
[0070] Epidemic investigator: Did you go out this afternoon?
[0071] Epidemic investigation subject: Yes, I received a task and went to the construction site of Block C1 in Financial Street opposite the gas station at No. 970 Zhonghua New Road to carry out disinfection work with two colleagues.
[0072] Epidemic investigator: Did you wear isolation protective clothing before entering the construction site?
[0073] Epidemiological investigation subject: Yes, the three of us put on isolation protective clothing before entering the construction site.
[0074] Based on the above conversation history information, the epidemiologist can learn that the subject of the investigation entered the construction site on the afternoon of May 26 to carry out disinfection work, and has been confirmed to be wearing isolation protective clothing. Therefore, the epidemiologist may continue to ask the subject of the investigation: Have you been wearing a mask since entering the construction site?
[0075] Based on this example, the specific process of intelligent flow survey question and answer is as follows:
[0076] (1) Knowledge extraction at different granularities
[0077] Scene recognition: Based on the latest response of the survey subject, "Yes, the three of us all put on isolation protective clothing before entering the construction site.", the scene recognition model can identify the scene involved as "construction site". On the other hand, the questions represented by the "general" scene are universal in any scene. Therefore, in the next step of the question scene prompt, this method specially adds the label "general" to ensure that the model's subsequent questions can more accurately reflect the actual situation, that is, the coarse-grained scene knowledge S N =“Construction site”+“General”.
[0078] Question element extraction: Combined with the epidemic survey knowledge graph, the question element knowledge in the "construction site" and "general" scenarios is extracted: Q N =["Work content", "Protection status", "Contact status", "Scanning status", ..., "Wearing a mask", "Others", "Travel mode", "Traveling personnel", "Activity time"]
[0079] (2) Knowledge prompts of different granularities
[0080] Integrate the prompts of coarse-grained scenarios: Based on the acquired scenarios, the coarse-grained scenario prompt S = {the scenario answered by the flow survey subject: [“construction site”], the next question needs to involve one of the following scenarios: [“general”, “construction site”]}.
[0081] Prompts that integrate fine-grained question elements: Based on the scenario prompts, the fine-grained question element prompts Q = {You can choose one of the following topics to ask questions: ["work content", "protection situation", "contact situation", "code scanning situation", ..., "wearing a mask", "others", "travel mode", "companions", "activity time"]}.
[0082] According to the data input format (command + dialogue history + scene prompt + question element prompt), the input data can be obtained: "You need to play the role of an epidemiological investigator and have a dialogue with the subject of the epidemiological investigation to understand the subject's recent activity trajectory and the people he has contacted. The dialogue is in the form of one question and one answer." + "Epidemiological investigator: Hello, were you in the hotel room on the morning of May 26?\nEpidemiological investigation subject: Yes, I was in the hotel room all morning.\nEpidemiological investigator: Did you go out in the afternoon?\nEpidemiological investigation subject: Yes, I received a task to go to the construction site of the C1 plot of Financial Street opposite the gas station at No. 970 Zhonghua New Road to carry out disinfection work with two colleagues.\nEpidemiological investigator: Did you wear isolation protective clothing before entering the construction site? \nPublic survey subject: Yes, the three of us all wore isolation protective clothing before entering the construction site. \n"+"Public survey officer: \n"+"Public survey subject answers scenario: ['Construction site'], the next question needs to involve one of the following scenarios: ['Construction site', 'General']\n"+"You can choose one of the following topics to ask: ["Work content", "Protection situation", "Contact situation", "Scanning situation",..., "Wearing a mask", "Other", "Travel mode", "Fellow personnel", "Activity time"]", input the command into the large model and hope to get the output: Have you been wearing a mask after entering the construction site?
[0083] (3) Fine-tuning of large models
[0084] Large models usually have a huge number of parameters, and fine-tuning these models requires a lot of computing resources and storage space. This may require the use of high-performance computing equipment and expensive hardware, increasing the cost and complexity of fine-tuning. Therefore, this method uses LoRA technology to fine-tune the large model, adding a bypass next to the original large model, which are the dimension reduction matrix A and the dimension increase matrix B, respectively. Matrix A is initialized with a random Gaussian distribution, and B is initialized with a 0 matrix. The input and output dimensions of the original large model remain unchanged. When outputting, the parameters of matrix B and matrix A are superimposed with the parameters of the original large model. During training, the parameters θ of the original model are fixed, and only the parameters of the newly added low-rank matrices A and B are updated. The fine-tuning update of the specific model can be expressed by the following formula (b):
[0085] W+ΔW=W+BA,W∈R d*k , B∈R d*r , A∈R r*k (b).
[0086] Among them, W is the parameter of the initial model; ΔW is the training parameter of the LoRA model; B is a 0 matrix; A is a matrix initialized by a random Gaussian distribution; d is the output dimension of the previous layer of the model; k is the input dimension of the next layer of the model.
[0087] After the input data X is input into the model encoding, the input vector feature representation H is obtained, and then combined with the model parameters θ, A, B to obtain the target prediction response Y = [y1, y2, ..., y T ], and are generated one by one by the decoding end, and the following formula (d) represents the next question to be asked in the current conversation history:
[0088] P(y t |y <t ,H)=Dec(E y<t ,H,θ;A,B)(d).
[0089] Among them, E y<t is the output generated at the current time t; H is the feature representation of the input vector; y <t represents the sequence of all output tokens that have been generated before time t; y t The output generated at the current time t; P is the conditional probability, which represents the tag sequence y generated before the current input vector H <t Based on this, generate the next word y t probability.
[0090] The loss function is expressed by the following formula (e): Make the questions generated by the model as consistent as possible with the real questions;
[0091]
[0092] Where P is the conditional probability in formula (d), T represents the length of the entire generated sequence; L is the loss function, which represents the gap between the output generated by the model and the true output.
[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. An intelligent epidemic investigation question-answering method integrating knowledge prompts of different granularities, characterized in that: The method specifically includes: Step 1: Knowledge Extraction 1-1: Building a knowledge graph for epidemic investigation The epidemiological survey elements are organized in a top-down hierarchical classification method to construct an epidemiological survey knowledge graph consisting of five functional modules covering various scenarios. The five functional modules correspond to basic personal information, residence, work place, onset of illness, medical treatment and medication, and activity trajectory survey. Each module contains corresponding epidemiological survey question elements, and each module is interrelated. 1-2: Extracting knowledge of different granularities 1-2-1: Based on the knowledge graph and large language model, coarse-grained scene knowledge is obtained, and the large language model is fine-tuned in a supervised manner using the epidemic survey question and answer data of the labeled scenes to establish a scene recognition model. This model uses the latest reply V of the epidemic survey subject based on the conversation history. N The most relevant scenario S associated with the constructed knowledge graph N ; 1-2-2: Extract scene S from the knowledge graph N Corresponding question elements Q N ; Step 2: Knowledge prompts of different granularities 2-1: Based on the acquired scene S N , coarse-grained scenario prompt S = {the scene of the flow survey object answer: [S N ], the next question should involve one of the following scenarios: ['General', S N ]}; 2-2: Based on the question elements extracted from the knowledge graph, the question element prompt Q is formed = {You can choose one of the following topics to ask: Among them, Q C and Represents general and scenario S respectively N A collection of question elements; 2-3: The format of input data X is expressed by the following formula (a): Where X is the input data; is a connector; I is an instruction; H is a dialogue history; S is a scene prompt; Q is a question element prompt; The instruction I = {You need to play the role of an epidemiological investigator and have a conversation with the subject of the epidemiological investigation to understand the subject's recent activity trajectory and the people he has contacted. The conversation is in the form of a question and an answer}. The input format of the instruction I is: instruction I + conversation history H + scene prompt S + question element prompt Q; the output format of the instruction I is: the next question of the epidemiological investigator under the current conversation history; Step 3: Fine-tune the large model 3-1: Use LoRA technology to fine-tune the large model. Add a bypass next to the original large model, which are the dimension reduction matrix A and the dimension increase matrix B. Initialize matrix A with random Gaussian distribution and initialize B with 0 matrix. The input and output dimensions of the original large model remain unchanged. When outputting, the parameters of the dimension reduction matrix A and the dimension increase matrix B are superimposed with the parameters of the original large model. During training, the parameters θ of the original model are fixed, and the parameters of the dimension reduction matrix A and the dimension increase matrix B are updated. The fine-tuning update of the specific model is expressed by the following formula (b): W+ΔW=W+BA,W∈R d*k ,B∈R d*r ,A∈R r*k (b); Among them, W is the parameter of the initial model; ΔW is the training parameter of the LoRA model; B is a 0 matrix; A is a matrix initialized by random Gaussian distribution; d is the output dimension of the previous layer of the model; k is the input dimension of the next layer of the model; 3-2: Input the input data X into the fine-tuned large model, and after encoding, obtain the input vector feature representation H. Then, use the above formula (b) to add the dimension reduction matrix A, the dimension increase matrix B and the original model parameter θ to obtain the target prediction response Y with a length of T represented by the following formula (c): Y=[y1,y2,...,y T ] (c); Among them, y1,y2,...,y T The predicted response for each target; 3-3: The decoder generates the next question to be asked in the current conversation history, represented by the following formula (d): P(y t |y <t ,H)=Dec(E y<t ,H,θ;A,B) (d); Among them, E y<t is the output generated at the current time t; H is the feature representation of the input vector; y <t is the sequence of all output tags generated before time t; y t is the output generated at the current time t; P is the conditional probability, which represents the tag sequence y generated before the current input vector H <t Based on this, generate the next word y t probability; 3-3: The loss function expressed by the following formula (e) Make the questions generated by the model as consistent as possible with the real questions; Among them, P is the conditional probability; T is the length of the entire generated sequence; L is the loss function, which represents the gap between the output generated by the model and the true output.