Large-model-based multi-round question and answer method and system for operation and maintenance industry

Abstract

The invention discloses a multi-round question-answering method and system based on a large model in the operation and maintenance industry, and belongs to the technical field of large model fine tuning, the method is implemented by comprising an input module, a multi-round question-answering module, an NL2SQL module and an output module, the input module is responsible for obtaining text content of a question asked by a user, and the text content comprises voice input and character input; the multi-round question and answer module is responsible for complementing the text content of the input module according to context information and then outputting the text content to the NL2SQL module; the NL2SQL module is responsible for converting the complemented text into an SQL statement, executing the SQL statement in a database, and returning an execution result to the output module; and the output module is responsible for displaying the SQL execution result to a user. The problem that when a user uses an NL2SQL module, each question has to contain complete information is solved, so that the whole system has the ability of multi-round question and answer.

Description

Technical Field

[0001] The present invention relates to the technical field of large model fine-tuning, and specifically to a multi-round question-answering method and system based on a large model in the operation and maintenance industry. Background Art

[0002] NL2SQL (Natural Language to SQL) is a technology that converts natural language into structured query language (SQL). This technology allows users to interact with databases through natural language without having to deeply understand SQL syntax. Since NL2SQL itself does not have the ability of multi-round conversation, when using the NL2SQL module, each input from the user must contain complete context information to ensure accurate result output. For example, when the user asks the NL2SQL module in the first round: Query which alarms have occurred today. After the NL2SQL module gives an answer, if the user still wants to know how many alarms occurred yesterday, in the second round, the user must completely input: Query which alarms occurred yesterday. Instead of directly inputting on the basis of the first-round context: What about yesterday? This is not user-friendly. Summary of the Invention

[0003] The technical task of the present invention is to address the above deficiencies and provide a multi-round question-answering method and system based on a large model in the operation and maintenance industry, which solves the problem that users must include complete information in each question when using the NL2SQL module, enabling the entire system to have the ability of multi-round question-answering.

[0004] The technical solution adopted by the present invention to solve its technical problems is as follows:

[0005] A multi-round question-answering method based on a large model in the operation and maintenance industry, the implementation of this method includes an input module, a multi-round question-answering module, an NL2SQL module, and an output module.

[0006] The input module is responsible for obtaining the text content of the user's question, including voice input, text input, etc.

[0007] The multi-round question-answering module is responsible for complementing the text content of the input module according to the context information and then outputting it to the NL2SQL module.

[0008] The NL2SQL module is responsible for converting the complemented text into an SQL statement, then executing it in the database, and returning the execution result to the output module.

[0009] The output module is responsible for presenting the SQL execution result to the user, and the specific presentation methods include charts, HTML pages, etc.

[0010] This method constructs a multi-round Q&A dataset for operation and maintenance personnel users and large models, fine-tunes the base model, and uses the model after fine-tuning to automatically complete the questions asked by users according to the context information, so that each round of questions contains complete context information, which is convenient to use the completed questions as the input of the NL2SQL module, and finally returns accurate query results to users.

[0011] Further, the implementation of the multi-round Q&A module specifically includes the following steps:

[0012] Step 1: Construct a multi-round Q&A seed dataset that can cover common user scenarios;

[0013] Step 2: Expand the constructed seed dataset;

[0014] Step 3: Use the expanded dataset to fine-tune the base model;

[0015] Step 4: Use the fine-tuned model to conduct an effect test on the seed dataset to obtain the accuracy rate;

[0016] Step 5: If the accuracy rate meets the standard, continue to execute Step 6; otherwise, adjust the expanded dataset and re-execute Step 3 and Step 4;

[0017] Step 6: Use the fine-tuned model as the multi-round Q&A module to complete the context of the questions asked by users.

[0018] Further, Step 1 specifically includes:

[0019] Step 1.1: Define the format of the required multi-round dialogue seed dataset, and each seed sample contains at least three rounds of Q&A content; the specific attributes included in each round of Q&A are: sample serial number, round number, user's original question, question after completing the context information, and the answer given by the NL2SQL module;

[0020] Among them, the round number represents which round of Q&A this round belongs to in this sample; except for the first round, the user's original question does not have to contain complete context information in the following rounds; the answer given by the NL2SQL module is the answer corresponding to the question after completing the context information, and this answer does not have to guarantee accuracy, and is only used to train the model to extract context information from the answer;

[0021] Step 1.2: Prepare at least 50 or more samples in the format described in Step 1.1 and write them into an Excel table or database for storage.

[0022] Further, Step 2 specifically includes:

[0023] Step 2.1: For each sample in the seed dataset output in Step 1, expand it with corresponding samples in multiple similar scenarios; the specific expansion method is implemented according to Steps 2.2 - 2.4;

[0024] Step 2.2: Determine whether there are replaceable fields in each round of question - answer content in the seed sample; the replaceable fields include personal names, place names, time, etc.; for example, for the question "Query which alarms were generated today", the "today" can be replaced with "yesterday", "this week", "last month", etc.; the time fields in the corresponding answers also need to be adjusted accordingly;

[0025] Step 2.3: Construct questions and answers semantically similar to each round of question - answer content in the sample; for example, for the question "Query which alarms were generated today", semantically similar questions can be constructed such as "Check the list of alarms generated today", "Find out which alarms were generated today", etc.;

[0026] Step 2.4: Split the samples expanded in Steps 2.2 and 2.3 by rounds to generate multiple samples; the specific method is: for a sample containing three rounds of question - answer, take the first round, the first two rounds, and the first three rounds respectively to generate three samples, and so on for samples with other numbers of rounds.

[0027] Further, Step 3 specifically includes:

[0028] Step 3.1: Convert the dataset expanded in Step 2 into a training set required for model fine - tuning. Traverse this dataset. For each sample, when converting it into a training sample, use the first few rounds except the last round as the historical conversation, the question in the last round as the input to the model, and the answer in the last round as the expected output of the model; it should be noted that if the total number of rounds of the sample is 1, set the historical conversation of this training sample to be empty;

[0029] Step 3.2: Traverse the training set processed in Step 3.1. For each sample in it, take out the question asked by the user and determine its round. If it is the question in the first round, append the following prompt: "Your task is to complete the user's question based on the known context information to query the corresponding result; if the user's question already contains complete information or is irrelevant to the context, return it as it is; the user's question is:

{first_question}

[0030] If the question taken from the sample is not the first-round question, then splice the following prompt: "Given that for the question:

{known_question}

{follow_question}

[0031] Step 3.3: Select a base model, set the fine-tuning parameters, and start fine-tuning using the training set processed in Step 3.2; Taking the open-source Baichuan2-13B-Chat as an example, the main fine-tuning parameters to be set are: epoch is 3.0, learning rate is 1e-4, and the rest can be kept default. The fine-tuning framework selects the open-source deepspeed;

[0032] Step 3.4: Merge the model after fine-tuning output with the base model, and start the model inference service using vllm.

[0033] Further, Step 4 specifically includes:

[0034] Step 4.1: Construct a test set for model inference. Use the seed data set described in Step 1, and adopt the same idea as in Steps 3.1 - 3.2 for constructing the training set to generate the test set;

[0035] Step 4.2: For each sample in the test set, call the model inference interface to output the answer. The prompt used during inference needs to be consistent with the prompt described in Step 3.2;

[0036] Step 4.3: Compare the answer output by the model with the question in the seed data set after complementing the context information. If no key information is missed, it is considered that the model complementation is successful; otherwise, it is considered that the model complementation fails. Calculate the accuracy rate of the model based on this idea.

[0037] Further, for Step 5, generally when the accuracy rate reaches 90%, it can be considered up to standard, and it can be adjusted according to the actual situation during implementation; If the accuracy rate does not meet the standard, the data set to be expanded and adjusted is required. The specific steps include:

[0038] Step 5.1: Analyze the wrong answers output by the model counted in Step 4.3, and make classification marks for the reasons for the errors;

[0039] Step 5.2: For the relevant seed samples classified according to each error reason, expand the data set correspondingly. The expansion idea is the same as in Steps 2.2 - 2.4, and after expansion, merge it with the original data set.

[0040] The present invention also claims protection for a multi-round question-answering system based on a large model in the operation and maintenance industry, including:

[0041] An input module for obtaining the text content of the user's question, including voice input, text input, etc.;

[0042] A multi-round question-answering module for completing the text content of the input module according to the context information and then outputting it to the NL2SQL module;

[0043] An NL2SQL module for converting the completed text into an SQL statement, then executing it in the database, and returning the execution result to the output module;

[0044] An output module for presenting the SQL execution result to the user, and the specific presentation methods include charts, HTML pages, etc.;

[0045] This system realizes multi-round question-answering based on a large model through the above method.

[0046] The present invention also claims protection for an apparatus for implementing multi-round question-answering based on a large model in the operation and maintenance industry, including: at least one memory and at least one processor;

[0047] The at least one memory is used for storing machine-readable programs;

[0048] The at least one processor is used for calling the machine-readable program to implement the above method.

[0049] The present invention also claims protection for a computer-readable medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the processor is caused to execute the above method.

[0050] Compared with the prior art, the multi-round question-answering method and system based on a large model in the operation and maintenance industry of the present invention have the following beneficial effects:

[0051] 1. For users, there is no need to input information that has already been included in the context every time they interact with this question-answering system, improving the user experience.

[0052] 2. The present invention can not only be used as a pre-module of the NL2SQL module, but also be applied to the direction of knowledge base retrieval, complementing the context information during each knowledge base retrieval, making the retrieved content more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a schematic diagram of the overall structure of the multi-round question-answering method based on a large model in the operation and maintenance industry provided by an embodiment of the present invention;

[0054] Figure 2 This is a flow diagram showing the implementation of a multi-round Q&A method based on large models in the operation and maintenance industry provided by an embodiment of the present invention. Detailed implementation manners

[0055] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0056] An embodiment of the present invention provides a multi-round Q&A method based on large models in the operation and maintenance industry. The implementation of this method includes an input module, a multi-round Q&A module, an NL2SQL module, and an output module.

[0057] The input module is responsible for obtaining the text content of the user's question, including voice input, text input, etc.

[0058] The multi-round Q&A module is responsible for complementing the text content of the input module according to the context information and then outputting it to the NL2SQL module.

[0059] The NL2SQL module is responsible for converting the complemented text into an SQL statement, then executing it in the database, and returning the execution result to the output module.

[0060] The output module is responsible for presenting the SQL execution result to the user. The specific presentation methods include charts, HTML pages, etc.

[0061] Furthermore, the implementation of the multi-round Q&A module specifically includes the following steps:

[0062] Step 1: Construct a multi-round Q&A seed data set that can cover common user scenarios;

[0063] Step 2: Expand the constructed seed data set;

[0064] Step 3: Use the expanded data set to fine-tune the base model;

[0065] Step 4: Use the fine-tuned model to conduct an effect test on the seed data set to obtain the accuracy rate;

[0066] Step 5: If the accuracy rate meets the standard, continue to execute Step 6; otherwise, adjust the expanded data set and re-execute Step 3 and Step 4;

[0067] Step 6: Use the fine-tuned model as the multi-round Q&A module to complement the context of the question asked by the user.

[0068] The specific content of Step 1 includes:

[0069] Step 1.1, define the format of the required multi-round dialogue seed data set. Each seed sample contains at least three rounds of question and answer content. The specific attributes of each round of question and answer include: sample number, round, user's original question, question after completing the context information, and answer given by the NL2SQL module. Among them, the round represents the round of question and answer that this round of question and answer belongs to this sample; except for the first round, the user's original question needs to contain complete context information, and the subsequent rounds do not need to contain complete information; the answer given by the NL2SQL module is based on the answer to the question after completing the context information. The answer does not need to guarantee accuracy and is only used to train the model to extract context information from the answer.

[0070] Step 1.2: Prepare at least 50 samples according to the format described in step 1.1 and save them in an Excel spreadsheet or database.

[0071] The step 2 specifically includes:

[0072] Step 2.1: For each sample in the seed data set outputted in step 1, expand the samples corresponding to multiple similar scenarios. The specific expansion method is implemented according to steps 2.2-2.4.

[0073] Step 2.2: Determine whether there are replaceable fields in each round of question and answer content in the seed sample. The replaceable fields are names of people, places, time, etc. For example, for the question "Query which alarms were generated today", "today" can be replaced with "yesterday", "this week", "last month", etc. The time field in the corresponding answer also needs to be adjusted accordingly.

[0074] Step 2.3: Construct questions and answers that are semantically similar to the content of each round of questions and answers in the sample. For example, for the question "Query which alarms were generated today", you can construct semantically similar questions such as "Check the list of alarms generated today", "Check which alarms were generated today", etc.

[0075] Step 2.4: Split the sample expanded by steps 2.2 and 2.3 into rounds to generate multiple samples. Specifically, for a sample containing three rounds of questions and answers, take the first round, the first two rounds, and the first three rounds. Generate three samples respectively, and the same goes for samples of other rounds.

[0076] The step 3 specifically includes:

[0077] Step 3.1. Convert the expanded dataset in Step 2 into a training set required for model fine-tuning. Traverse this dataset. For each sample in it, when converting it into a training sample, the previous several rounds except the last round should be used as the historical conversation, the question in the last round should be used as the input to the model, and the answer in the last round should be used as the expected output of the model. It should be specifically noted that if the total number of rounds of the sample is 1, the historical conversation of this training sample should be set to empty.

[0078] Step 3.2. Traverse the training set processed in Step 3.1. For each sample in it, extract the question asked by the user and determine its round. If it is the question in the first round, concatenate the following prompt: "Your task is to complete the user's question based on the known context information so as to call the interface to query the corresponding result. If the user's question already contains complete information or is irrelevant to the context, return it as it is. The user's question is:

{first_question}

[0079] If the question extracted from the sample is not the question in the first round, concatenate the following prompt: "It is known that for the question:

{known_question}

{follow_question}

[0080] Step 3.3. Select a base model, set the fine-tuning parameters, and start fine-tuning using the training set processed in Step 3.2. Taking the open-source Baichuan2-13B-Chat as an example, the main fine-tuning parameters to be set are: epoch is 3.0, learning rate is 1e-4, and the rest can be kept default. The fine-tuning framework selects the open-source deepspeed.

[0081] Step 3.4. Merge the model after fine-tuning output with the base model, and start the model inference service using vllm.

[0082] The specific content of Step 4 includes:

[0083] Step 4.1. Construct a test set that can be used for model inference. Use the seed dataset described in Step 1 and generate the test set using the same idea as when constructing the training set in Steps 3.1 - 3.2.

[0084] Step 4.2: For each sample in the test set, call the model inference interface to output an answer. The prompt used during inference needs to be consistent with the prompt described in Step 3.2.

[0085] Step 4.3: Compare the answer output by the model with the question in the seed dataset after completing the context information. If no key information is missed, it is considered that the model has successfully completed the complementation. Otherwise, it is considered that the model complementation has failed. Calculate the accuracy rate of the model based on this idea.

[0086] In Step 5, generally, when the accuracy rate reaches 90%, it can be considered qualified, and it can be adjusted according to the actual situation during implementation. If the accuracy rate does not meet the standard, the dataset to be expanded and supplemented needs to be adjusted. The specific steps include:

[0087] Step 5.1: Analyze the wrong answers output by the model statistically in Step 4.3 and make classification marks for the reasons for the errors.

[0088] Step 5.2: For the relevant seed samples classified according to each error reason, expand the dataset accordingly. The expansion idea is the same as that in Steps 2.2 - 2.4. After the expansion is completed, merge it with the original dataset.

[0089] This method builds a multi-round Q&A dataset for operation and maintenance personnel users and large models, fine-tunes the base model, and uses the model after fine-tuning output to automatically complete the questions asked by users according to the context information, so that each round of questions contains complete context information, which is convenient to use the completed questions as the input of the NL2SQL module, and finally returns accurate query results to the users.

[0090] The embodiment of the present invention also provides a multi-round Q&A system for the operation and maintenance industry based on a large model. This system realizes multi-round Q&A based on a large model through the multi-round Q&A method for the operation and maintenance industry based on a large model described in the above embodiment.

[0091] The system includes:

[0092] An input module, used to obtain the text content of the questions asked by users, including voice input, text input, etc.

[0093] A multi-round Q&A module, used to complete the text content of the input module according to the context information and then output it to the NL2SQL module.

[0094] An NL2SQL module, used to convert the completed text into an SQL statement, then execute it in the database, and return the execution result to the output module.

[0095] An output module, used to display the SQL execution result to the user. The specific display methods include charts, HTML pages, etc.

[0096] Among them, the specific implementation of the multi-round Q&A module includes the following steps:

[0097] Step 1: Construct a multi-round Q&A seed data set that can cover common user scenarios;

[0098] Step 2: Expand the constructed seed data set;

[0099] Step 3: Use the expanded data set to fine-tune the base model;

[0100] Step 4: Use the fine-tuned model to conduct an effect test on the seed data set to obtain the accuracy rate;

[0101] Step 5: If the accuracy rate meets the standard, continue to execute Step 6; otherwise, adjust the expanded data set and re-execute Steps 3 and 4;

[0102] Step 6: Use the fine-tuned model as the multi-round Q&A module to complete the context for the questions asked by the user.

[0103] The specific content of Step 1 includes:

[0104] Step 1.1: Define the format of the required multi-round dialogue seed data set. Each seed sample contains at least three rounds of Q&A content. The specific attributes included in each round of Q&A are: sample serial number, round number, the original question asked by the user, the question after completing the context information, and the answer given by the NL2SQL module. Among them, the round number represents which round of Q&A in this sample; except for the first round, the original question asked by the user does not need to include complete context information in the following rounds; the answer given by the NL2SQL module is the answer corresponding to the question after completing the context information, and this answer does not need to guarantee accuracy, only for training the model to extract context information from the answer.

[0105] Step 1.2: Prepare at least 50 samples according to the format described in Step 1.1 and write them into an Excel table or database for storage.

[0106] The specific content of Step 2 includes:

[0107] Step 2.1: For each sample in the seed data set output in Step 1, expand the corresponding samples in multiple similar scenarios. The specific expansion method is implemented according to Steps 2.2 - 2.4.

[0108] Step 2.2: Determine whether there are replaceable fields in each round of Q&A content in this seed sample. The replaceable fields are, namely, names of people, places, times, etc. For example, for the question "Query which alarms have occurred today", the "today" can be replaced with: "yesterday", "this week", "last month", etc. The corresponding time fields in the answer also need to be adjusted accordingly.

[0109] Step 2.3: Construct questions and answers that are semantically similar to the Q&A content in each round of the sample. For example, for the question: "Query which alarms were generated today", semantically similar questions can be constructed as: "Check the list of alarms generated today", "Find out which alarms were generated today", etc.

[0110] Step 2.4: Split the sample expanded in Step 2.2 and Step 2.3 into multiple samples by rounds. Specifically, for a sample containing three rounds of Q&A, take the first round, the first two rounds, and the first three rounds respectively to generate three samples. Samples with other numbers of rounds are processed analogously.

[0111] The specific steps of Step 3 are as follows:

[0112] Step 3.1: Convert the dataset expanded in Step 2 into a training set required for model fine-tuning. Traverse the dataset. For each sample in it, when converting it into a training sample, use the first few rounds except the last round as the historical conversation, the question in the last round as the input to the model, and the answer in the last round as the expected output of the model. It should be specifically noted that if the total number of rounds of the sample is 1, set the historical conversation of this training sample to be empty.

[0113] Step 3.2: Traverse the training set processed in Step 3.1. For each sample in it, take out the question asked by the user and determine its round. If it is the question in the first round, concatenate the following prompt: "Your task is to complete the user's question based on the known context information so as to call the interface to query the corresponding result. If the user's question already contains complete information or is irrelevant to the context, return it as it is. The user's question is:

{first_question}

[0114] If the question taken out from the sample is not the question in the first round, concatenate the following prompt: "It is known that for the question:

{known_question}

{follow_question}

[0115] Step 3.3: Select the base model, set the fine-tuning parameters, and start fine-tuning using the training set processed in Step 3.2. Taking the open-source Baichuan2-13B-Chat as an example, the main fine-tuning parameters to be set are: epoch is 3.0, learning rate is 1e-4, and the rest can be kept default. The open-source deepspeed is selected as the fine-tuning framework.

[0116] Step 3.4: Merge the model after fine-tuning output with the base model, and start the model inference service using vllm.

[0117] The specific steps of Step 4 include:

[0118] Step 4.1: Construct a test set for model inference. Using the seed data set described in Step 1, generate the test set with the same idea as when constructing the training set in Steps 3.1 - 3.2.

[0119] Step 4.2: For each sample in the test set, call the model inference interface to output the answer. The prompt used during inference needs to be consistent with the prompt described in Step 3.2.

[0120] Step 4.3: Compare the answer output by the model with the question after supplementing the context information in the seed data set. If no key information is missed, it is considered that the model supplementation is successful. Otherwise, it is considered that the model supplementation fails. Calculate the accuracy rate of the model based on this idea.

[0121] In Step 5, generally, when the accuracy rate reaches 90%, it can be considered qualified, and it can be adjusted according to the actual situation during implementation. If the accuracy rate does not meet the standard, the expanded data set needs to be adjusted. The specific steps include:

[0122] Step 5.1: Analyze the wrong answers output by the model counted in Step 4.3, and make classification marks for the reasons of the errors.

[0123] Step 5.2: For the relevant seed samples classified according to each error reason, expand the data set correspondingly. The expansion idea is the same as that in Steps 2.2 - 2.4. After expansion, merge it with the original data set.

[0124] The embodiment of the present invention also provides a multi-round question-answering implementation device for the operation and maintenance industry based on a large model, including: at least one memory and at least one processor;

[0125] The at least one memory is used to store machine-readable programs;

[0126] The at least one processor is used to call the machine-readable program to implement the multi-round question-answering method for the operation and maintenance industry based on a large model described in the above embodiment.

[0127] An embodiment of the present invention also provides a computer-readable medium, on which computer instructions are stored. When the computer instructions are executed by a processor, the processor is caused to execute the multi-round question-and-answer method based on a large model in the above embodiment. Specifically, a system or device equipped with a storage medium can be provided, on which software program code for implementing the functions of any one of the above embodiments is stored, and the computer (or CPU or MPU) of the system or device is caused to read and execute the program code stored in the storage medium.

[0128] In this case, the program code read from the storage medium itself can implement the functions of any one of the above embodiments. Therefore, the program code and the storage medium storing the program code constitute a part of the present invention.

[0129] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0130] Furthermore, it should be clear that not only can the functions of any one of the above embodiments be implemented by executing the program code read by the computer, but also by causing an operating system or the like operating on the computer based on the instructions of the program code to complete part or all of the actual operations.

[0131] In addition, it can be understood that the program code read from the storage medium is written into a memory provided in an expansion board inserted into the computer or into a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, the CPU or the like installed on the expansion board or the expansion unit is caused to execute part and all of the actual operations, thereby implementing the functions of any one of the above embodiments.

[0132] The present invention has been described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above-mentioned multiple embodiments, those skilled in the art can know that the code review means in different above embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A multi-round question-answering method based on a large model in the operation and maintenance industry, characterized in that: The implementation of this method includes an input module, a multi-round question-answering module, an NL2SQL module, and an output module. The input module is responsible for obtaining the text content of the user's questions, including voice input and text input; The multi-round question-answering module is responsible for completing the text content of the input module according to the context information and then outputting it to the NL2SQL module; The NL2SQL module is responsible for converting the completed text into SQL statements, executing them in the database, and returning the execution results to the output module; The output module is responsible for displaying the SQL execution results to the user.

2. According to claim 1, a multi-round question-answering method based on a large model in the operation and maintenance industry is characterized in that: The implementation of the multi-round question-answering module specifically includes the following steps: Step 1: Build a multi-round question-answering seed dataset that covers common user scenarios; Step 2: Expand the constructed seed data set; Step 3: Use the expanded dataset to fine-tune the base model; Step 4: Use the fine-tuned model to test the effect on the seed data set and obtain the accuracy; Step 5: If the accuracy meets the requirement, proceed to step 6; otherwise, adjust the expanded data set and re-execute steps 3 and 4; Step 6: Use the fine-tuned model as a multi-round question-answering module to complete the context of the questions asked by the user.

3. According to claim 2, a multi-round question-answering method based on a large model in the operation and maintenance industry is characterized in that: The step 1 specifically includes: Step 1.1: Define the format of the required multi-round dialogue seed dataset. Each seed sample contains at least three rounds of Q&A content. The specific attributes of each round of Q&A include: sample number, round, user's original question, question after completing the context information, and answer given by the NL2SQL module. Among them, round number indicates the round number of questions and answers that this round of question and answer belongs to for this sample; except for the first round, the original questions asked by the user do not need to contain complete context information, and the answers given by the NL2SQL module are based on the answers corresponding to the questions after the context information is completed. The answers do not need to be accurate and are only used to train the model to extract context information from the answers; Step 1.2: Prepare at least 50 samples according to the format described in step 1.1 and save them in an Excel spreadsheet or database.

4. According to claim 2, a multi-round question-answering method based on a large model in the operation and maintenance industry is characterized in that: The step 2 specifically includes: Step 2.1: For each sample in the seed data set outputted in step 1, expand the samples corresponding to multiple similar scenarios; the specific expansion method is implemented according to steps 2.2-2.4; Step 2.2: determine whether there are replaceable fields in each round of question and answer content in the seed sample; the replaceable fields include names of people, places, and time; Step 2.3: Construct questions and answers that are semantically similar to the content of each round of questions and answers in the sample; Step 2.4: Split the sample expanded by steps 2.2 and 2.3 into rounds to generate multiple samples. The specific method is as follows: for a sample containing three rounds of questions and answers, take the first round, the first two rounds, and the first three rounds to generate three samples respectively, and the same applies to samples of other rounds.

5. According to claim 2, a multi-round question-answering method based on a large model in the operation and maintenance industry is characterized in that: The step 3 specifically includes: Step 3.1: Convert the dataset expanded in step 2 into the training set required for model fine-tuning. Traverse the dataset. For each sample in the dataset, when converting it into a training sample, take the previous rounds except the last round as the historical dialogue, the questions in the last round as the input to the model, and the answers in the last round as the expected output of the model. If the total number of rounds of the sample is 1, set the historical dialogue of the training sample to empty. Step 3.2, traverse the training set processed by step 3.1, for each sample, take out the question asked by the user, and determine its round. If it is the first round of questions, splice the following prompt: "Your task is to complete the user's question based on the known context information, so as to call the interface to query the corresponding results; if the user's question already contains complete information or is irrelevant to the context, return it directly as is; the user's question is: [{first_question}]"; where {first_question} is the user's first round of questions; If the sample is not the first-round question, the following prompt is concatenated: "It is known that the result of calling the interface query for question: [{known_question}] is: ```{known_answer}```; the user's question is: [{follow_question}]"; where {known_question} is the previous round question, {known_answer} is the answer to the previous round question, and {follow_question} is the current round question; Step 3.3, select the base model, set the fine-tuning parameters, and start fine-tuning using the training set processed in step 3.2; Step 3.4: Merge the fine-tuned output model with the base model and start the model inference service using vllm.

6. A multi-round question-answering method based on a large model in the operation and maintenance industry according to claim 5, characterized in that: The step 4 specifically includes: Step 4.1: Build a test set that can be used for model reasoning. Use the seed dataset described in step 1 and the same idea as in steps 3.1-3.2 to build the training set to generate a test set. Step 4.2: For each sample in the test set, call the model inference interface and output the answer. The prompt used during inference must be consistent with the prompt described in step 3.

2. In step 4.3, the answers output by the model are compared with the questions in the seed dataset after the context information is completed. If no key information is missed, the model completion is considered successful; otherwise, the model completion is considered failed.

7. A multi-round question-answering method based on a large model in the operation and maintenance industry according to claim 6, characterized in that: In step 5, if the accuracy rate does not meet the standard, it is necessary to adjust the expanded data set. The specific steps include: Step 5.1: Analyze the wrong answers output by the model counted in step 4.3, and classify and mark the causes of the errors; Step 5.2: For the relevant seed samples classified according to each error cause, the data set is expanded accordingly, and after the expansion is completed, it is merged with the original data set.

8. A multi-round question-answering system based on a large model in the operation and maintenance industry, characterized in that: include: The input module is used to obtain the text content of the user's question, including voice input and text input; The multi-round question-answering module is used to complete the text content of the input module according to the context information and then output it to the NL2SQL module; NL2SQL module, which converts the completed text into SQL statements, executes them in the database, and returns the execution results to the output module; Output module, used to display SQL execution results to users; The system implements multi-round question and answer based on a large model through the method described in any one of claims 1 to 7.

9. A multi-round question-answering implementation device based on a large model in the operation and maintenance industry, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, which, when executed by a processor, cause the processor to execute the method according to any one of claims 1 to 7.

Images