A consulting question optimization method and system based on large language model prompt engineering

Abstract

The present application discloses a consulting question optimization method and system based on a large language model prompt engineering, the consulting question optimization method comprising constructing a prompt template for model input reconstruction; obtaining a legal consulting question data set, reconstructing the original questions in the legal consulting question data set using the prompt template to obtain corresponding input data; for each input data, using an existing large model to generate optimization questions, and performing expert judgment and re-optimization to obtain a final optimization question data set; using the generated optimization question data set as fine-tuning data to fine-tune a small base model to obtain a question optimization model; when a user consults on a legal question, optimizing the user's question based on the question optimization model, the present application can utilize the prompt engineering to generate data and construct a question optimization model in a data-scarce scenario, thereby improving the professionalism and accuracy of the user's expression of legal consulting questions.

Description

Technical Field

[0001] The present application belongs to the field of intelligent legal data interaction, and in particular, relates to a consulting question optimization method and system based on a large language model prompting project. Background Art

[0002] As society continues to develop, people's demand for legal consultation is increasing. Traditional legal consultation usually relies on professional lawyers to answer questions offline. Although this method can provide accurate legal advice, due to the limited lawyer resources and huge demand for consultation, there are problems such as imbalance between service supply and demand and delayed service response. In response to this demand, online legal consultation platforms have emerged, enabling people to obtain legal consultation in a timely and rapid manner and expanding legal services to a wider user group, but the limitations of manual services still exist. With the advancement of deep learning technology, intelligent legal consultation that does not rely on manual work has become possible to help improve the efficiency of legal consultation. However, the accuracy of the answers of existing model-based intelligent legal consultation methods is largely affected by the quality of user questions. Users lack professional legal knowledge and often find it difficult to express their legal problems clearly and accurately. This lack of quality in expression, such as incomplete information, vague problem descriptions, and incorrect wording, will lead to deviations in the model generation results and affect the final consultation effect. Therefore, in order to provide users with more accurate and comprehensive legal answers, it is necessary to effectively optimize user questions to stimulate model capabilities.

[0003] At present, in the field of intelligent legal consultation, thanks to the powerful semantic understanding and generation capabilities of large language models, large language models can have better performance than traditional deep learning methods. However, there are still certain limitations in the optimization of legal consultation issues. First, the existing legal consultation data sets are mostly question-answer pairs containing "problem-analysis or solution to the problem", but lack "problem-optimization of the problem" data sets, which greatly restricts the training of the model for specific tasks; secondly, the current methods of constructing data sets are mostly manual construction methods, with high construction costs and low efficiency; in addition, user questions are diverse and complex, and how to quickly improve the model's adaptability to specific tasks is also an issue that needs to be considered. It can be seen that in the current intelligent legal consultation, there is a problem that users lack professionalism, resulting in poor question quality and low model accuracy. Summary of the invention

[0004] The present application provides a consultation question optimization method and system based on a large language model prompting project, which is used to solve the problem that users lack professionalism in current intelligent legal consultation, resulting in poor question quality and low model accuracy.

[0005] In order to solve the above technical problems, the technical solution proposed in this application is:

[0006] In a first aspect, the present application provides a method for optimizing consulting questions based on a large language model prompting project, comprising:

[0007] S1: Obtain a legal consultation question dataset, where the legal consultation question dataset contains original legal consultation questions;

[0008] S2: construct prompt template;

[0009] S3: reconstructing the original legal consultation question based on the prompt template to obtain a plurality of input data;

[0010] S4: using a large language model to optimize each piece of input data to generate an optimization problem, and performing expert judgment and re-optimization on the optimization problem to form a set format of problem pairs, thereby obtaining a final optimization problem data set;

[0011] S5: fine-tune the base model using the optimization problem data set as fine-tuning data to obtain a problem optimization model, and perform real-time optimization on the user problem based on the problem optimization model.

[0012] In the second aspect, the present application also provides a consulting question optimization system based on a large language model prompt project, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0013] This application has the following beneficial effects:

[0014] The consulting question optimization method based on the large language model prompt engineering provided in this application reconstructs the user questions in the legal consulting data set according to the scenario requirements by constructing a prompt template; calls the API of the existing general large model with a large number of parameters to generate optimization problems, and reasonably judges and improves the optimization problems through expert judgment and re-optimization methods, so as to realize the construction of an optimization problem data set containing "original problem-optimization problem". This solution uses the prompt template to reasonably transform the input data to stimulate the ability of the large model, and uses the powerful generation ability of the general large model with a large number of parameters to generate effective data. It can generate data according to scenario requirements, solves the problem of lack of specific legal optimization problem data sets, and realizes efficient and high-quality data set construction.

[0015] In the further solution, the large model fine-tuning technology is used to design a prompt template for input data in a targeted manner, and the base model is fine-tuned using a specific optimization problem data set to build a problem optimization model. The problem optimization model is constructed through fine-tuning technology, which efficiently adapts the model to specific legal consulting scenarios and reduces the resource requirements for model construction. The powerful semantic understanding and generation capabilities of the large model are used to optimize the user's legal consulting questions, thereby improving the professionalism and comprehensiveness of the questions, so that users can obtain more appropriate answers.

[0016] In a further solution, by clipping the gradients of model parameters to keep the model parameters within a predetermined range, the possible gradient explosion problem can be prevented.

[0017] In the further calculation scheme, the first-order moment estimate of the gradient in the i-th round is first calculated, which can smooth the gradient and stabilize the parameter update process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A flowchart of a method for optimizing consulting questions based on a large language model prompting project provided for this application;

[0020] Figure 2 A recommended framework diagram for optimizing legal issues in the embodiments of this application;

[0021] Figure 3 This is a diagram of the results of the specific implementation of this application. DETAILED DESCRIPTION

[0022] To facilitate the understanding of the present application, the present application will be described in more comprehensive and detailed manner below in conjunction with the accompanying drawings and preferred embodiments of the specification, but the protection scope of the present application is not limited to the following specific embodiments.

[0023] Unless otherwise defined, all professional terms used below have the same meaning as those generally understood by those skilled in the art. The professional terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of this application.

[0024] It should be understood that in current intelligent legal consultation, there is a problem that users lack professionalism, resulting in poor question quality and low model accuracy. Based on this, this application provides a consultation question optimization method based on a large language model prompt project. It is worth noting that the consultation question optimization method based on a large language model prompt project provided by this application can also be applied to similar questioning scenarios, not limited to legal questioning scenarios.

[0025] Embodiment 1:

[0026] See also Figure 1-Figure 2 In this embodiment, a method for optimizing consulting questions based on a large language model prompting project is disclosed, comprising the following steps:

[0027] S1: Obtain a legal consultation question dataset, where the legal consultation question dataset contains original legal consultation questions;

[0028] S2: construct prompt template;

[0029] S3: reconstructing the original legal consultation question based on the prompt template to obtain a plurality of input data;

[0030] S4: using a large language model to optimize each piece of input data to generate an optimization problem, and performing expert judgment and re-optimization on the optimization problem to form a set format of problem pairs, thereby obtaining a final optimization problem data set;

[0031] S5: fine-tune the base model using the optimization problem data set as fine-tuning data to obtain a problem optimization model, and perform real-time optimization on the user problem based on the problem optimization model.

[0032] In this embodiment, the setting format refers to the "original problem - optimization problem" format. Among them, the large language model includes but is not limited to versions of GPT-3, PaLM, LLaMA, etc. with more than 1 billion parameters.

[0033] Among them, a problem optimization model is constructed, and the generated optimized problem dataset is used as fine-tuning data. The input is adjusted using prompt templates, and the model is fine-tuned so that it can fully learn the optimized expression characteristics of legal consultation questions, thereby optimizing the professionalism and comprehensiveness of the expression of legal consultation questions input by users.

[0034] This application can use prompt templates to reconstruct the large model input to adapt it to the needs of legal consultation problem optimization, use large models and expert judgment and re-optimization methods to efficiently generate and construct optimized problem data sets, and use prompt templates and generated optimized problem data sets to fine-tune the problem optimization model to optimize the expression of user-input legal consultation questions, which can improve the quality of users' questions and enable the AI ​​assistant in intelligent legal consultation to provide users with more accurate answers.

[0035] Embodiment 2:

[0036] Embodiment 2 is a preferred embodiment of Embodiment 1. The difference between Embodiment 2 and Embodiment 1 is that the specific steps of the consultation question optimization method based on the large language model prompt engineering are expanded:

[0037] In this embodiment, the prompt template satisfies the following structure:

[0038] ;

[0039] in, is the first prompt word that appears; The input text to be filled; This is the second prompt word that appears; is a special identifier; is the mask key to be predicted, and the mask key is used to indicate the position mark for which the model is required to generate an answer.

[0040] Optionally, the S3 includes:

[0041] Read the legal consultation question dataset Original legal consultation questions, get the original question text data , , Indicates the Original legal advice questions;

[0042] Ask questions for each original legal advice mentioned , using the prompt template to reconstruct and obtain the input data corresponding to each original legal consultation question , ,in, Indicates the data sequence number. is the first prompt word that appears; This is the second prompt word that appears; is a special identifier; is the mask key to be predicted, and the mask key is used to indicate the position mark for which the model is required to generate an answer.

[0043] Optionally, the S4 includes:

[0044] For the i-th input data , construct it into json format input data, call the API of the large language model to generate the return result corresponding to the input data, and parse the initial optimization problem from the return result ;

[0045] For each of the initial optimization problems , and conduct expert manual judgment to obtain the score , ,when , indicating that the initial optimization problem Unqualified; when , indicating that the initial optimization problem qualified;

[0046] For the initial optimization problem where the expert judgement is unqualified , using the expert manual re-optimization method, the optimization problem is finally obtained ; For the initial optimization problem that is judged by experts to be qualified , then directly As an optimization problem ;

[0047] For each of the original legal advice questions And the corresponding optimization problem , which is composed into a set of formatted questions , combine all problem pairs to generate an optimization problem dataset , .

[0048] Optionally, the S5 includes:

[0049] Load the base model and the corresponding word segmenter, and modify the corresponding configuration parameters;

[0050] Using the Optimization Problem Dataset As fine-tuning data, The optimization problem , using the prompt template Ask questions about original legal advice Reconstruct and get input for fine-tuning ,in, Indicates the data sequence number. is the first prompt word that appears; This is the second prompt word that appears; is a special identifier; is the mask key to be predicted, and the mask key is used to indicate the position mark for which the model is required to generate an answer;

[0051] Use the word segmenter to Input Processing word sequences , For the first input word, For the second input word segmentation, is the mth input word segmentation; use the word segmentation device to segment the mth Optimization Problem Processing word sequences , For the first target word, For the second target word segmentation, Segment the nth target word;

[0052] The current input word sequence The input parameters are The model is used to obtain the prediction results through forward propagation and calculate the loss function;

[0053] Construct the loss function to satisfy the following relationship:

[0054] ;

[0055] in, For the The target word segmentation expected to be generated in time steps is Indicated in All input word sequences before time steps, The model parameters are At the time, Time steps have As conditional context, the model predicts Participle The probability when n represents the total number of target words;

[0056] Calculate the model parameters in the eth round by backpropagation Gradient :

[0057] ;

[0058] in, To find the sign of the partial derivative;

[0059] Use the optimizer to adjust the model parameters based on the updated gradients;

[0060] Compute clipped gradients In the The first moment estimate in the rounds :

[0061] ;

[0062] in is the attenuation coefficient of the first-order moment, It is The first moment estimate in rounds;

[0063] Correct the bias in the first moment estimate:

[0064] ;

[0065] In the formula, express to the power of e;

[0066] Compute clipped gradients In the The second moment estimates in the rounds :

[0067] ;

[0068] in, is the attenuation coefficient of the second-order moment, It is The second moment estimates in the rounds;

[0069] Correct the bias in the second moment estimate:

[0070] ;

[0071] In the formula, express to the power of e;

[0072] Update the model parameters:

[0073] ;

[0074] in, It is Model parameters that were not updated in rounds, is the learning rate, is a constant to prevent division by zero errors. is the weight decay term for L2 regularization, are the updated model parameters;

[0075] Through the fine-tuning steps, the problem optimization model is obtained.

[0076] Optionally, before using the optimizer to adjust the model parameters according to the updated gradient, the method further includes:

[0077] For model parameters Gradient Clipping is performed to keep the model parameters within the predetermined range, which is achieved through the following formula:

[0078] ;

[0079] in is the clipped gradient, is the gradient The L2 norm of , which represents the square root of the sum of the squares of all parameter gradients, is the threshold for gradient clipping.

[0080] Specifically, in this embodiment, a consulting question optimization method based on a large language model prompt engineering is disclosed. Through prompt engineering, large model generation and fine-tuning and other technologies, an optimization problem dataset containing "original problem-optimization problem" is efficiently constructed, and the optimized expression characteristics of legal consulting questions are learned to improve the quality of user questions.

[0081] In this embodiment, the consultation question optimization method based on the large language model prompt engineering is applied to the actual legal consultation scenario to optimize the questions, which specifically includes the following steps:

[0082] Step 1: Build a prompt template

[0083] The scenario for this embodiment is the intelligent legal consultation scenario. Since the accuracy of the answers to the model-based intelligent legal consultation method is largely affected by the quality of the user's questions, the goal of this application is to optimize the user's legal consultation questions and optimize the expressions of incomplete user information, vague descriptions, and inaccurate wording into comprehensive, accurate, and professional question expressions to improve the accuracy of the intelligent answering assistant. In order to guide the large model to generate content that meets the requirements, this embodiment constructs a prompt template optimized for legal consultation questions. The prompt template constructed by this application uses a discrete prompt structure, which mainly includes four parts: prompt words, input text to be filled, mask keys to be predicted, and special identifiers.

[0084] Prompt words refer to phrases or keywords used to guide the large language model to understand and generate, including task instructions and expected outputs. In this embodiment, the prompt word part is expressed in text, such as setting the expected output description , set task instructions .

[0085] The input text to be filled is the user's original question text, using Represents, for example, a sample of a user's original question text .

[0086] The mask key to be predicted is the position marker where the model needs to fill in or generate content, usually expressed as Indicates that the large model will be in the prompt template Location generated content.

[0087] Special identifiers are symbols used to mark specific positions in the input sequence or to indicate the beginning or end of a sentence. In this example, the identifier indicating the end of the sequence is used. .

[0088] The prompt template in this embodiment is constructed as follows: .in, is the first prompt word that appears; The input text to be filled; This is the second prompt word that appears; is a special identifier; is the mask key to be predicted, and the mask key is used to indicate the position mark for which the model is required to generate an answer.

[0089] Step 2: Build an optimization problem dataset

[0090] This application is aimed at intelligent legal consulting scenarios. Since most existing legal consulting datasets contain question-answer pairs of the "problem - analysis or solution to the problem" type, but lack a "problem - optimization of the problem" type dataset, it is necessary to construct an optimized problem dataset for use in this scenario. The optimized problem dataset contains the original problem and the optimized problem.

[0091] In the data preprocessing part, the public data set "Legal Knowledge Question and Answer" data set is obtained, which contains the fields "answer", "candidate_answer", "cause", and "question". The "answer" field is the answer that best matches the original question in the "question" field, the "candidate_answer" field is the candidate answer set, the "cause" field is the type of question, and the "question" field is the user's original question about legal consultation. Extract the "question" field data to obtain the original question text data , is the number of samples.

[0092] Enter the refactoring part again, using the prompt template Reconstruct the original question text, and the reconstructed input Example:

[0093] ={"Make the question more detailed and professional, add details / correct professional errors. The original question is:", "Do I get paid during my probation period as a waiter in a restaurant?", "Optimize the expression of legal consultation questions:", " <eos>", "[mask]"};

[0094] In the data generation part, for each reconstructed input , according to the API call requirements of the existing large-scale general model with large parameters, convert it into JSON format, set the request key API_KEY and the URL of the request interface, call the text generation capability of the existing large-scale general model with large parameters through the requests API interface, generate the initial optimization problem, and use the parsing function to parse the initial optimization problem from the returned results . Data generation through large models greatly reduces the human resource consumption of building data sets and improves data generation efficiency.

[0095] Because the content generated by the large language model is somewhat random and may not meet the target requirements, it needs to be judged. In the expert judgment and re-optimization part, each initial optimization problem generated by the large model is , and conduct expert manual evaluation to obtain the score , ,when , indicating that the initial optimization problem Not meeting the standards; , indicating that the initial optimization problem Meet the standard and directly As an optimization problem ; For initial optimization problems that experts judged as not meeting the standards , using the expert manual re-optimization method to modify the parts with incomplete information, vague descriptions or inaccurate wording, and finally get the optimization problem ;

[0096] For each of the original questions And the corresponding optimization problem , forming a problem pair in the format of "original problem - optimization problem" , combine all problem pairs to generate the optimization problem dataset .

[0097] Example of a sample optimization problem dataset:

[0098] {"question": "Do I get paid during the probation period as a waiter in a hotel?", "response": ["Do I get paid during the probation period as a waiter in a hotel? What are the relevant laws and regulations regarding wages for workers during the probation period?"]}

[0099] The "question" field is the original question, and the "response" field is the optimized question.

[0100] Step 3: Fine-tune the optimization model based on the optimization problem dataset

[0101] The model base used in this embodiment is Chinese-LLaMA-Alpaca-2, which uses a large-scale Chinese data set for incremental pre-training, has certain Chinese basic semantics and instruction understanding capabilities, conforms to the Chinese legal consulting scenario targeted by this application, and can adapt to specific tasks after fine-tuning. The tokenizer used in this embodiment is LlamaTokenizer, which can vectorize the input text string and convert it into a token sequence.

[0102] Each sample in the optimization problem dataset will use the prompt template to reconstruct the original problem and obtain the input for fine-tuning. , and then use the word segmenter to process it into word sequences .

[0103] When fine-tuning the model, we use autoregressive generation, that is, the model relies on historically generated sequences to predict the distribution of the next word. For example, let the model parameters be , the word segmentation sequence of the model input is , then the model is When predicting the next word, we use the context to make a probability prediction. , then the context is ;if , then the context is model first All the words that have been predicted in the step, that is, The model is based on the context in Step to the next word To make a prediction, the model generates the probability distribution of the next word The calculation method is shown in formula (1):

[0104] ;

[0105] in The model is in front The sequence of all words that have been predicted. The model is The logits vector generated in the step is used to represent the original score value of the model for each word. The function is a function that transforms the logits vector into a probability distribution.

[0106] After obtaining the probability distribution, the model uses the kernel sampling method to generate the next word. In kernel sampling, the model sorts the probabilities from high to low and selects a set of words with a cumulative probability of The words are used as candidate words, and then the next word is randomly sampled from them as the generated content. For example, if , the vocabulary is sorted from high to low probability. , and their probability distribution is , then only one word is selected from the minimum word set with a cumulative probability of 0.6, without considering other words with a cumulative probability less than 0.6, so the candidate set is , then, the model randomly selects a word from the candidate set as the next generated word. This method can avoid sampling some inappropriate or irrelevant words, while also retaining the diversity and creativity of the generated content.

[0107] After obtaining the prediction results of each step, the loss value of each word is calculated through the loss function, and the average value is taken to get the final loss value, as shown in formula (2):

[0108] ;

[0109] in, For the The target word segmentation expected to be generated in time steps is Indicated in All input word sequences before time steps, The model parameters are At the time, Time steps have As conditional context, the model predicts Participle The probability when n represents the total number of target words;

[0110] Using the loss value during the backpropagation of the model Calculate the gradient of the model and use the optimizer to adjust the model parameters according to the updated gradient so that the loss minimize.

[0111] Through the fine-tuning steps based on the optimization problem dataset, the problem optimization model is obtained.

[0112] Step 4: Optimize user questions based on the question optimization model

[0113] Load the fine-tuned question optimization model and the corresponding word segmenter, and set the model generation parameters according to specific needs. For example, set the maximum model generation length to 512, the minimum model generation length to 5, and the kernel sampling parameter to 0.6.

[0114] When users ask legal advice questions, the question optimization model is called to optimize the generation of questions.

[0115] See also Figure 3 , in one example:

[0116] User asked: "Hello, I am from another place and I am working in X. Can I apply for agricultural insurance in Shaodong?"

[0117] Model generation optimization question: "Hello, I am a non-local resident and currently working in X. I would like to know the relevant information about applying for agricultural insurance in Shaodong. Does this comply with relevant laws and regulations? Is it necessary for non-local resident to apply for agricultural insurance? "

[0118] This embodiment also provides a consulting question optimization system based on a large language model prompting project, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program. The consulting question optimization system based on a large language model prompting project can implement the various embodiments of the consulting question optimization method based on a large language model prompting project described above, and can achieve the same beneficial effects, which will not be described in detail here.< / eos>

Claims

1. A consulting question optimization method based on a large language model prompting project, characterized in that: include: S1: Obtain a legal consultation question dataset, where the legal consultation question dataset contains original legal consultation questions; S2: construct prompt template; S3: reconstructing the original legal consultation question based on the prompt template to obtain a plurality of input data; S4: using a large language model to optimize each piece of input data to generate an optimization problem, and performing expert judgment and re-optimization on the optimization problem to form a set format of problem pairs, thereby obtaining a final optimization problem data set; S5: fine-tuning the base model using the optimization problem data set as fine-tuning data to obtain a problem optimization model, and performing real-time optimization on the user problem based on the problem optimization model; The S5 includes: Load the base model and the corresponding word segmenter, and modify the corresponding configuration parameters; Using the Optimization Problem Dataset As fine-tuning data, The optimization problem , using the prompt template Ask questions about original legal advice Reconstruct and get input for fine-tuning ,in, Indicates the data sequence number. is the first prompt word that appears; This is the second prompt word that appears; is a special identifier; is the mask key to be predicted, and the mask key is used to indicate the position mark for which the model is required to generate an answer; Use the word segmenter to Input Processing word sequences , For the first input word, For the second input word segmentation, is the mth input word segmentation; use the word segmentation device to segment the mth Optimization Problem Processing word sequences , For the first target word, For the second target word segmentation, Segment the nth target word; The current input word sequence The input parameters are The model is used to obtain the prediction results through forward propagation and calculate the loss function; Construct the loss function to satisfy the following relationship: ； in, For the The target word segmentation expected to be generated in time steps is Indicated in All input word sequences before time steps, The model parameters are At the time, Time steps have As conditional context, the model predicts Participle The probability when n represents the total number of target words; Calculate the model parameters in the eth round by backpropagation Gradient : ； in, To find the sign of the partial derivative; Use the optimizer to adjust the model parameters based on the updated gradients; Compute clipped gradients In the The first moment estimate in the rounds : ； in is the attenuation coefficient of the first-order moment, It is The first moment estimate in rounds; Correct the bias in the first moment estimate: ； In the formula, express to the power of e; Compute clipped gradients In the The second moment estimates in rounds : ； in, is the attenuation coefficient of the second-order moment, It is The second moment estimates in the rounds; Correct the bias in the second moment estimate: ； In the formula, express to the power of e; Update the model parameters: ； in, It is Model parameters that were not updated in rounds, is the learning rate, is a constant to prevent division by zero errors. is the weight decay term for L2 regularization, are the updated model parameters; Through the fine-tuning steps, the problem optimization model is obtained.

2. The consulting question optimization method based on large language model prompt engineering according to claim 1 is characterized in that: The structure of the prompt template is a discrete prompt structure, including a prompt word, an input text to be filled, a mask key to be predicted, and a special identifier; the prompt template satisfies the following structure: ； in, is the first prompt word that appears; The input text to be filled; This is the second prompt word that appears; is a special identifier; is the mask key to be predicted, and the mask key is used to indicate the position mark for which the model is required to generate an answer.

3. The consulting question optimization method based on large language model prompt engineering according to claim 1 is characterized in that: The S3 includes: Read the legal consultation question dataset Original legal consultation questions, get the original question text data , , Indicates the Original legal advice questions; Questions for each original legal advice mentioned , using the prompt template to reconstruct and obtain the input data corresponding to each original legal consultation question , ,in, Indicates the data sequence number. is the first prompt word that appears; This is the second prompt word that appears; is a special identifier; is the mask key to be predicted, and the mask key is used to indicate the position mark for which the model is required to generate an answer.

4. The consulting question optimization method based on large language model prompt engineering according to claim 1 is characterized in that: The S4 includes: For the i-th input data , construct it into json format input data, call the API of the large language model to generate the return result corresponding to the input data, and parse the initial optimization problem from the return result ; For each of the initial optimization problems , and conduct expert manual evaluation to obtain the score , ,when , indicating that the initial optimization problem Unqualified; when , indicating that the initial optimization problem qualified; For the initial optimization problem where the expert judgement is unqualified , using the expert manual re-optimization method, the optimization problem is finally obtained ; For the initial optimization problem that is judged as qualified by experts , directly As an optimization problem ; For each of the original legal advice questions And the corresponding optimization problem , which is composed into a set of formatted questions , combine all problem pairs to generate an optimization problem dataset , .

5. The consulting question optimization method based on large language model prompt engineering according to claim 1 is characterized in that: Before using the optimizer to adjust the model parameters according to the updated gradient, the method further includes: For model parameters Gradient Clipping is performed to keep the model parameters within the predetermined range, which is achieved through the following formula: ； in is the clipped gradient, is the gradient The L2 norm of , which represents the square root of the sum of the squares of all parameter gradients, is the threshold for gradient clipping.

6. A consulting question optimization system based on a large language model prompting project, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 5 are implemented.

Images