Large language model optimization generation method based on optimal cue word selection

Abstract

The invention provides a large language model optimization generation method based on optimal cue word selection, and belongs to the technical field of large language model optimization generation. The technical problem to be solved is to provide the improvement of the large language model optimization generation method based on optimal cue word selection. According to the technical scheme, a selected data set is preprocessed, a sequence is formed by splicing questions and answers, and a spectral clustering algorithm is adopted to divide the sequence into multiple groups of question and answer pairs with similar characteristics; sampling representative samples from each cluster, and based on professional knowledge, language logic and semantic association, constructing high-quality cue words which can guide and generate a large model aiming at a specific problem; a large-parameter online model is used, a diversified and targeted cue word set is generated for each cluster with reference to the created high-quality cue words, generated questions, candidate cue words and answers of the candidate cue words are integrated, and a data set used for fine tuning of the cue evaluation model is constructed; the method is used for generating the language model.

Description

Technical Field

[0001] The invention proposes a large language model optimization generation method based on optimal prompt word selection, belonging to the technical field of large language model optimization generation. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have become an important tool in the field of natural language processing. Their powerful language understanding and generation capabilities can meet the needs of multiple scenarios such as text creation, question-answering systems, and machine translation. However, despite the wide versatility of large models, their generation effect and adaptability still have certain defects when dealing with specific fields and specific types of problems. It is necessary to guide large language models to focus on the context of specific problems, accurately capture task requirements, and then generate highly relevant and high-quality texts to improve the accuracy and efficiency of large model applications.

[0003] In exploring the optimization generation performance of large language models, previous studies have achieved a series of important results, especially in using prompt words to guide and enhance the effectiveness of model output. For example, the research of Brown et al. emphasized the key role of carefully planned prompt words in guiding language models to complete specific tasks. Through such prompt words, the model can significantly improve the performance on a variety of tasks by relying on only a few examples (Few-Shot Learning) without gradient updates. However, this method that relies on manually designed prompt words may face the problem of insufficient versatility and is difficult to cope with all task types or complex scenarios. The dynamic question expansion method proposed by Betz et al. uses the model itself to generate additional context related to the question to improve the accuracy of zero-shot reasoning tasks. Although this method enhances the model's understanding of the problem, its dynamic generation process may be affected by the initial state of the model, resulting in fluctuations in the quality of the generated prompt words or lack of pertinence. Lester et al. use the prompt tuning technology to learn a small number of adjustable parameters to optimize the prompt words, so that the model can adapt to specific tasks without large-scale parameter updates. However, this method is limited by the number of adjustable parameters and may not be able to achieve sufficient prompt word refinement for complex or highly customized tasks. Gao et al. studied how to improve the generalization ability of the model with a small number of examples, by including examples related to new tasks in the model input to assist the model in understanding and execution. Although this method helps the model cope with new tasks, it lacks direct guidance on the selection and optimization of prompt words, which may lead to the model's understanding of certain tasks being limited. Liao et al. proposed the Zero-Label Prompt Selection method, which improves the performance of the model in the zero-shot setting by self-predicting and evaluating the model and selecting the best prompt word combination. Although this method achieves automatic prompt word selection under unlabeled data conditions, it relies on the model's own prediction ability, and there may be model bias or uncertainty that affects the accuracy of the selection. Pang et al. proposed a shared latent space prompt fine-tuning method, which overcomes the difficulty and cost of identifying the optimal source task in traditional soft prompt adjustment methods by learning a shared latent space to capture the basic skill set from multiple source tasks. Although this method greatly reduces the computing resource requirements by fine-tuning only a small number of parameters compared to full fine-tuning, it cannot adapt to large language models with limited access to training data.

[0004] Currently, large language models usually receive prompts from users to clarify the context and goals of their generation tasks. This method can better drive large models to meet specific task requirements without adjusting model parameters. The selection and construction of prompts directly affect the accuracy and effectiveness of model output. Appropriate prompts can effectively stimulate the deep knowledge of the model and make the content generated by the model more in line with expectations, while poor prompts may cause the model output to deviate from the topic or lack depth. However, the prompts designed in practice often rely on manual experience. When performing different tasks, a lot of tests are required to come up with better prompts that can activate model parameters. There is a lack of systematic theoretical guidance and automated methods, making it difficult to ensure that the best prompts are always selected when facing complex and changing problem sets.

[0005] In view of the above defects, there is an urgent need to propose an automatic prompt word selection strategy suitable for general large models to cope with challenges such as the diversity and complexity of problems, while reducing dependence on manual labeling. Summary of the invention

[0006] In order to overcome the deficiencies in the prior art, the present invention aims to solve the technical problem of providing an improved method for optimizing and generating a large language model based on optimal prompt word selection.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: a large language model optimization generation method based on optimal prompt word selection, comprising the following generation steps: Step 1: Preprocess the selected data set by concatenating questions and answers to form a sequence, and use the spectral clustering algorithm to divide it into multiple groups of question-answer pairs with similar features; Step 2: Select representative samples from each cluster and construct high-quality prompt words that can guide the generation of large models for specific problems based on professional knowledge, language logic and semantic associations; Step 3: Use a large-parameter online model to generate a diverse and targeted set of prompt words for each cluster, refer to the created high-quality prompt words, integrate the generated questions, candidate prompt words and their answers, and build a dataset for fine-tuning the prompt evaluation model; Step 4: Use the LoRA fine-tuning model to adjust the prompt evaluation model parameters so that it can accurately measure the relevance and effectiveness of each prompt for a specific question; Step 5: Select the highest-scoring candidate prompt word as the best prompt word for a specific question, combine it with the question, input it into the downstream large model for text generation, and finally evaluate the text correctness.

[0008] The specific method of preprocessing the data set and dividing the question-answer pairs using the spectral clustering algorithm in step 1 is: The C-Eval dataset was selected for training and validation, questions and answers were concatenated into a single sequence, and the spectral clustering algorithm was used to group these sequences; Use the text embedding method to convert the text into a vector. Specifically, the input layer receives the one-hot encoding of the current word and its context words, and then these encodings are converted through the weight matrix W1 to obtain the vector representation of each word. The obtained vectors are sent to the projection layer for weighted summation and addition to obtain a sum representing the relationship between the current word and the context words; The sum is input to the hierarchical Softmax layer, which is a binary tree with each leaf node corresponding to a word. The probability of each word is obtained by calculating the weight on the path from the root node to the leaf node. The probability represents the possibility of the current word appearing in a given context. The spectral clustering algorithm is used to calculate the similarity matrix of the question vector. The calculation formula is: ; Among them, v i and v j They represent two vectors in the embedded vector space, ||·|| represents the Euclidean distance of the vectors, and σ is a parameter that controls the similarity decay rate, which can be automatically determined according to the distance from each point to its k nearest neighbors.

[0009] The specific method for constructing high-quality prompt words in step 2 is: After data preprocessing and cluster analysis, a certain number of clusters are obtained. Problem samples that can fully reflect the semantic characteristics of the cluster are extracted from each cluster. For the selected samples, excellent prompt words with strong pertinence and significant guiding effect are constructed based on domain knowledge, language logic and potential semantic associations. The template of excellent prompt words is constructed. The expression for constructing the template is: Prompt=Instruct+Overall / Detailed Description+Question; Among them, Instruct contains clear instructions for the tasks that the model is expected to perform; Overall Description represents the theme that the large model should follow when answering a certain type of question; Detailed Description represents a detailed description of the specific task steps in a point-by-point manner; Question is the specific question raised by the user.

[0010] The specific method of generating the candidate prompt word set and constructing the fine-tuning dataset in step 3 is: Based on the divided clusters and the corresponding excellent prompt words, the word library and fine-tuning dataset are constructed. Referring to the constructed excellent prompt word library, the Qwen model is used to generate targeted excellent prompt words for various pre-divided question clusters, covering the unique attributes of various input questions in the cluster, and then the generated questions, candidate prompt words and corresponding answers are integrated to finally construct a candidate prompt word library; Then, we use a combination of online large models and manual annotation to build a dedicated dataset for fine-tuning the prompt evaluation model, including: Expert reviewers evaluate and score each question-prompt word combination based on the closeness of the connection between the question and the prompt word, whether the prompt word can effectively stimulate the model to generate the expected answer, and the comprehensive performance of the prompt word in terms of semantics, grammar, and domain expertise. A few-shot learning strategy is adopted, and the Qwen model is used to simulate manual scoring. For the prompt word evaluation task, the K-way, N-shot form is adopted, where K represents the number of question categories and N represents the number of samples in each category used to train the model, that is, the manually labeled scoring data.

[0011] The specific method of using the LoRA fine-tuning model to adjust the prompt evaluation model parameters in step 4 is: Add a pair of low-rank matrices U and V to each weight matrix in the selected layer, corresponding to the row and column directions of the original weight matrix respectively; During the training process, only the low-rank matrices U and V are updated, and the original weight matrix W is not directly modified. The calculation formula for the actual weight update h of the model is: h=W+U·V T ; By adjusting the low-rank matrices U and V and using the Adam optimization algorithm and early stopping strategy, we can ensure that the model achieves optimal performance within a limited number of training steps. After each round of training, the performance indicators of the model are calculated, and the model parameters and learning rate hyperparameters are adjusted according to the performance indicators to achieve the optimal model performance.

[0012] The specific method of selecting the best prompt word and evaluation in step 5 is: Classify the questions according to the cluster they belong to, and send the questions and the candidate prompt word set in the cluster to the evaluation model; Based on the evaluation model scores, the system automatically selects the prompt word with the highest score in the candidate prompt word set as the optimal prompt word for a specific question; The program logically merges the optimal prompt word with the corresponding question text to form a question statement containing clear prompt information, which is passed to the downstream large model as a unified input to drive the downstream large model to generate answers based on this; After the model is generated, the answers generated by the model are compared with the standard answers to the corresponding questions in the C-Eval test set to evaluate the generation performance.

[0013] The beneficial effects of the present invention compared with the prior art are as follows: the present invention integrates technical means such as cluster analysis, automatic prompt word generation, and fine-tuning of special evaluation models to form an adaptive prompt word optimization framework suitable for large language models. The present invention selects the prompt word with the highest score as the optimal prompt through the evaluation model, merges it with the corresponding question, and inputs it into the downstream large model for final generation. This method systematically optimizes the prompt word selection, ensures that the large language model can obtain the most suitable guidance information when facing different questions, and can improve the generation quality in the question-answering system scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention will be further described below in conjunction with the accompanying drawings: Figure 1 A flow chart of the steps of the model optimization generation method of the present invention; Figure 2 A flowchart of the steps of text embedding for the present invention; Figure 3 It is a structural schematic diagram of the excellent prompt word template of the present invention; Figure 4 A flowchart of the steps of generating and fine-tuning candidate prompt words of the present invention; Figure 5 This is a flow chart of the steps of LoRA fine-tuning of the present invention; Figure 6 It is a schematic diagram of generating evaluation results using zero-sample prompts of the Qwen series model in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In view of the limitations of large language model optimization generation pointed out in the background technology, the present invention proposes a large language model optimization generation method based on optimal prompt word selection, aiming to systematically optimize the prompt word generation and selection process, overcome the problem that the manually designed prompt words are not universal enough and the model itself cannot handle complex tasks well, and overcome the problem that the prompt words cannot be adjusted in real time and adapted to the large model with limited access by building a candidate prompt word library in advance, and adopt the LoRA fine-tuning method to train a dedicated prompt evaluation model, overcoming the selection bias and uncertainty that may be caused by the model's self-prediction. The present invention proposes a large language model optimization generation method based on optimal prompt selection (OPS), which can realize the optimization of prompt word selection of the large language model in a systematic way. The main method steps include: introducing spectral clustering technology to automatically classify question and answer pairs; formulating high-standard prompt templates based on typical samples; generating multiple groups of candidate prompt words to enrich the response strategy, and building a fine-tuning data set of the prompt evaluation model based on this; finally, by training the prompt evaluation model, the prompt word with the highest score is screened as the optimal prompt, and after merging with the corresponding question, it is input into the downstream large model for final generation; The present invention aims to improve the generation effect and adaptability of a large language model when processing complex tasks by optimizing the prompt word selection strategy of the large language model, thereby promoting the accuracy and effectiveness of the application of large language models. Specifically, the spectral clustering technology is used to cluster the question and answer pairs in the training data set to generate different clusters, and excellent prompt words are constructed as reference standards for representative samples. An online model is used to generate multiple groups of candidate prompt words for each cluster based on the excellent prompt words. A fine-tuning data set is constructed by combining questions and candidate prompt words. A low-rank fine-tuning (LoRA) method is used to train a prompt evaluation model, which can quantitatively score the relevance and effectiveness of each candidate prompt word to a specific input question.

[0016] like Figure 1 As shown, the large language model optimization generation method based on optimal prompt word selection proposed in the present invention mainly includes the following large language model optimization generation steps: Step 1: Data preprocessing and spectral clustering. Preprocess the selected data set, form a sequence by splicing questions and answers, and use spectral clustering technology to divide it into multiple groups of question-answer pairs with similar features.

[0017] Step 2: Construction of excellent prompt words. Representative samples are sampled from each cluster, and based on professional knowledge, language logic and semantic associations, high-quality prompt words are designed to guide the large model to efficiently generate specific problems.

[0018] Step 3: Candidate prompt word generation and fine-tuning dataset construction. Using a large-parameter online model, refer to the created high-quality prompt words to generate a diverse and targeted prompt word set for each cluster, and then integrate the generated questions, candidate prompt words and their answers to build a dataset for fine-tuning the prompt evaluation model.

[0019] Step 4: Fine-tune the prompt evaluation model. Use LoRA fine-tuning technology to adjust the prompt evaluation model parameters so that it can accurately measure the relevance and effectiveness of each prompt for a specific problem.

[0020] Step 5: Select and evaluate the best prompt words. The highest-scoring prompt words are selected from the candidate prompt words as the best prompt words for a specific question. After combining them with the question, they are input into the downstream large model for text generation and final evaluation of text correctness.

[0021] Furthermore, the specific method for performing data preprocessing and spectral clustering calculation in step 1 is: The present invention selects some representative large-scale question and answer pairs in the C-Eval dataset as the original training dataset. C-Eval is a comprehensive Chinese basic model evaluation suite, which contains 13,948 multiple-choice questions covering 52 different subjects and four difficulty levels. The present invention is trained and validated based on the C-Eval dataset to ensure that the data covers a variety of topics and question-answering patterns, then splices the questions and answers into a single sequence, and uses the spectral clustering algorithm to group these sequences.

[0022] The spectral clustering algorithm is an unsupervised learning method based on graph theory and matrix spectral analysis, which is mainly used to cluster complex data sets. Its core idea is to transform the similarity relationship between data points into the structure of a graph, and to perform spectral analysis on the Laplace matrix of the graph to achieve efficient clustering of data points. By adjusting the clustering parameters, multiple clusters are obtained, each of which contains pairs of questions and answers with similar characteristics.

[0023] The present invention specifically uses a text embedding method to convert text into a vector. The specific text embedding process is as follows: Figure 2 shown in Figure 2 In the example, the input layer receives the one-hot encoding of the current word and its context words, and then these encodings are transformed through the weight matrix W1 to obtain the vector representation of each word. Then these vectors are sent to the projection layer for weighted summation and addition, and finally a sum representing the relationship between the current word and the context words is obtained. Finally, the sum is input to the hierarchical Softmax layer. This hierarchical structure is a binary tree, and each leaf node corresponds to a word. By calculating the weight on the path from the root node to the leaf node, the probability of each word can be obtained. This probability represents the possibility of the current word appearing in a given context.

[0024] After the text embedding process is completed, the similarity matrix of the question vector is calculated. The expression of this calculation process is: ; (1) Among them, v i and v jRespectively represent two vectors in the embedding vector space, ||·|| represents the Euclidean distance of the vectors, and σ is a parameter that controls the similarity decay rate, which can be automatically determined based on the distance from each point to its k nearest neighbors. In the spectral clustering algorithm, the parameter n_clusters specifies the number of clusters, and the k in the k nearest neighbor method is also set to the same value. σ needs to be selected according to the specific situation, and can usually be adjusted to make the clustering results meet expectations.

[0025] Compared with directly using the K-Means algorithm, the spectral clustering algorithm can better handle data with complex distribution, high dimension and sensitivity to initial state, thus bringing better clustering effect, higher stability and deeper revelation of the intrinsic structure of the data.

[0026] Furthermore, the specific method for constructing excellent prompt words in step 2 is: After data preprocessing and cluster analysis, a certain number of clusters are obtained. In order to improve the universality of excellent prompt words, question samples that can fully reflect the semantic characteristics of the cluster are extracted from each cluster. For the selected samples, excellent prompt words with strong pertinence and significant guiding effect are constructed based on domain knowledge, language logic and potential semantic associations. These prompt words are designed to accurately stimulate the optimal generation ability of the large model under the corresponding question type.

[0027] The present invention constructs an excellent prompt word template, and the calculation expression of the template is: Prompt=Instruct+Overall / Detailed Description+Question; (2) Among them, Instruct contains clear instructions for the tasks that the model is expected to perform; Overall Description represents the theme that the large model should follow when answering a certain type of question; Detailed Description represents a detailed description of the specific task steps in a point-by-point manner; Question is the specific question raised by the user. The specific relationship between the parameters in formula (2) is as follows: Figure 3 shown.

[0028] Furthermore, the specific method for generating candidate prompt words and constructing a fine-tuning dataset in step 3 is as follows: In this step, a vocabulary and fine-tuning dataset are constructed based on the divided clusters and the corresponding excellent prompt words. The steps for generating and fine-tuning candidate prompt words are as follows: Figure 4 shown.

[0029] The present invention specifically adopts the Qwen series model of Tongyi Qianwen for research. This series of models is trained and produced by the Alibaba team based on the LLaMA base model. It mainly refers to the constructed excellent prompt word library. For various pre-divided question clusters, the present invention designs and uses the Qwen model to generate targeted excellent prompt words, covering the unique attributes of various input questions in the cluster, and then integrates the generated questions, candidate prompt words and corresponding answers to finally construct a candidate prompt word library; compared with manually writing prompt words, the method adopted by the present invention can generate a variety of candidate prompt words, providing high-quality prompt words for subsequent downstream large models to generate question answers.

[0030] After constructing the prompt word library, the present invention uses a method combining an online large model with manual annotation to construct a dedicated dataset for fine-tuning the prompt evaluation model. The process of constructing this dataset is intended to ensure that the model can accurately evaluate and optimize the guiding effect of prompt words in actual question-answering situations. The specific operation steps are as follows: For each question and prompt word pair in the prompt word library, manual scoring is first performed: Expert reviewers carefully evaluate each pair of question-prompt word combinations and assign corresponding scores based on the closeness of the connection between the question and the prompt word, whether the prompt word can effectively stimulate the model's ability to generate expected answers, and the comprehensive performance of the prompt word in terms of semantics, grammar, and domain expertise. The manual scoring work at this stage aims to establish high-quality standards and provide an accurate reference benchmark for subsequent model learning.

[0031] Next, a few-shot learning strategy is adopted, using the Qwen series model to simulate manual scoring; few-shot learning is a machine learning method designed specifically for dealing with small sample problems, especially when the training data is extremely limited; in this context, for the prompt word evaluation task, the present invention adopts the form of K-way, N-shot, where K represents the number of problem categories, and N represents the number of samples in each category used to train the model, that is, the manually labeled scoring data. Although the number of samples under each problem category is extremely limited, few-shot learning aims to enable the model to have the ability to quickly learn and generalize under such sparse data conditions.

[0032] The above method not only greatly reduces the workload of manual labeling and improves the efficiency of dataset construction, but also ensures that the model can maintain a high evaluation accuracy when faced with a small number of samples, providing a high-quality, widely applicable dedicated dataset for subsequent fine-tuning of the prompt evaluation model.

[0033] Furthermore, the specific method for fine-tuning the prompt evaluation model in step 4 is: Based on the evaluation dataset constructed above, the present invention further uses the LoRA fine-tuning method to adjust the parameters of the prompt evaluation model, aiming to improve the model's ability to accurately evaluate the relevance and effectiveness of each prompt for a specific input problem. As a lightweight fine-tuning strategy, LoRA is particularly suitable for adaptive adjustment of large pre-trained models due to its high efficiency and resource saving. The core idea of ​​this method is to keep most of the parameters of the original pre-trained model unchanged, and only locally adjust the key layers of the model by introducing a low-rank matrix. The model structure of the LoRA fine-tuning method is as follows: Figure 5 shown.

[0034] Compared with traditional fine-tuning methods, LoRA only updates some low-rank matrix parameters, significantly reducing the number of parameters that need to be adjusted, thereby reducing computing costs and storage requirements, allowing the model to quickly adapt to new tasks with limited resources. In this process, since most of the pre-trained model parameters are frozen, the rich language knowledge accumulated during pre-training can be effectively retained, avoiding excessive interference with the general language understanding ability of the original model during the fine-tuning process. Coupled with the reduction of parameter space and good initialization of initial weights, the LoRA fine-tuned model can often achieve better performance in a shorter training time, accelerating the entire fine-tuning process. The specific steps for evaluating the fine-tuning of the model are: First, we determine which layers need to use LoRA technology, and then add a pair of low-rank matrices U and V to each weight matrix in the selected layer, corresponding to the row and column directions of the original weight matrix respectively. The dimensions of these low-rank matrices are much smaller than the original weight matrix.

[0035] Since only the low-rank matrices U and V are updated during training without directly modifying the original weight matrix W, the actual weight update h of the model is calculated as: h=W+U·V T ; (3) By adjusting the low-rank matrices U and V, and using the Adam optimization algorithm and early stopping strategy, we can ensure that the model achieves optimal performance within a limited number of training steps. After each round of training, we calculate the performance indicators of the model, and adjust the model parameters, learning rate and other hyperparameters based on these indicators to pursue the optimal model performance. This method can make the model output the prompt effectiveness score as close to the manual annotation standard as possible when given a prompt and question pair.

[0036] Furthermore, the specific method for selecting and evaluating the optimal prompt words in step 5 is as follows: Specifically, the dedicated prompt evaluation model fine-tuned above is used to select candidate prompt words: First, the questions are classified according to the cluster they belong to, and then the questions and the candidate prompt word set in the cluster are sent to the evaluation model. Based on the evaluation model's score, the system automatically selects the prompt word with the highest score in the candidate prompt word set as the optimal prompt word for a specific question. This process ensures that the selected prompt has the highest level of semantic relevance, guidance, and fit with the question context.

[0037] Once the optimal prompt word is determined, the program will automatically logically merge it with the corresponding question text to form a question statement containing clear prompt information and pass it to the downstream large model as a unified input, ultimately driving the downstream large model to generate answers based on this.

[0038] After the model is generated, the answers generated by the model are compared with the standard answers to the corresponding questions in the C-Eval test set to evaluate the model's generation performance for complex questions based on the optimal prompt word selection method. If the model answer performs well on multiple indicators, it means that it has achieved a high standard of accuracy when generating text answers using the optimal prompt words. On the contrary, if some indicators score low, it may indicate that the model's generation ability in some aspects needs to be improved, or that the matching degree between the prompt word and the question still needs to be optimized.

[0039] The model optimization method proposed in the present invention can improve the accuracy of the language model for most question-answering data sets, especially arithmetic problems and logical deduction problems. Their reasoning process is very complicated. Unlike typical retrieval question-answering scenarios, the original local large language model is difficult to be directly used to solve arithmetic and logical deduction problems. In an embodiment of the present invention, the present invention focuses on the Civil Servant, Middle School Questions, and High School Questions corpora in the C-Eval data set, wherein the Civil Servant corpus contains administrative professional ability tests, mainly examining common sense judgment, verbal comprehension and expression, quantitative relations, judgment reasoning, and data analysis. The Middle School Questions corpus contains seven parts of biology, chemistry, geography, history, mathematics, physics, and politics in the junior high school stage, and the High School Questions corpus contains eight parts of biology, chemistry, Chinese, geography, history, mathematics, physics, and politics in the high school stage. The present invention divides the above-mentioned data set corpus into a training set and a test set according to a certain ratio to evaluate the pros and cons of the method. In order to eliminate the influence of unbalanced sampling of different corpus classifications in the data set, the present invention first performs spectral clustering analysis on the data set to obtain clusters representing different subject classifications. The data set division and subject classification data are shown in Table 1 below: Table 1 Dataset division and subject classification The training and test data sets were divided into approximately 8:2, of which 2808 training data were used to construct a candidate prompt word library and a fine-tuning data set for the prompt evaluation model. In order to better fine-tune the prompt evaluation model, expert reviewers carefully scored the question-prompt word combination based on the candidate prompt word library, the relevance between the question and the prompt word, the effectiveness of the prompt word in guiding the model to produce the expected answer, and its comprehensive performance in terms of semantics, grammar, professionalism, etc., in order to establish a high-standard reference benchmark to facilitate model learning. Subsequently, based on this reference standard, the Qwen series model was used to simulate the manual scoring process.

[0040] The scoring examples used in the present invention are as follows: Question: 1, 10, 37, 82, 145, ____. A.170 B.197 C.224 D.226; Prompt word; When faced with mathematical and logical reasoning problems, artificial intelligence should follow these steps to solve them: 1. Problem analysis: Read the question carefully and clarify the mathematical concepts and logical relationships involved; 2. Pattern recognition: Find the recursive relationship, general formula or special properties of the series, and use mathematical induction, inductive reasoning and other methods to reveal the hidden patterns; 3. Formula application: use relevant mathematical formulas, theorems or laws to perform calculations or reasoning; 4. Logical construction: Build a logical framework based on the problem description, which may include logical reasoning skills such as conditional judgment, hypothetical reasoning, and proof by contradiction; 5. Data processing: perform accurate calculations on given values, paying attention to details such as unit conversion and simplification; 6. Answer verification: Substitute the obtained answer into the original question to ensure that the logic is self-consistent, meets all conditions, and is consistent with the actual situation; 7. Clear presentation: A concise and clear explanation of the problem-solving process, including explanations of key steps and reasoning; The score is 0.65.

[0041] The parameters for the above experiments were set as follows: The experimental environment configuration is: GPU, NVIDIA GeForce RTX 4090*4; video memory, 24*4GB; Python version, 3.10.13; CUDA version, 11.8. When using spectral clustering analysis to divide the question and answer pairs contained in the data set, by manually observing the possible classifications in the data set, the number of clusters n_clusters is set to 9, and the affinity parameter is set to nearest_neighbors, that is, the similarity calculation method based on the K nearest neighbor (KNN) algorithm is used. The LoRA low-rank fine-tuning training prompt evaluation model is used. The specific instructions for fine-tuning the hyperparameters are shown in Table 2 below: Table 2 suggests hyperparameter settings for fine-tuning the evaluation model Among the above hyperparameters, the model_name parameter specifies the name of the pre-trained model to be fine-tuned. The present invention specifies that the model is a Qwen series quantized model with 180 million parameters. The training_stage parameter defines the training process as supervised fine-tuning, which means that the model will be trained on a labeled data set to learn the characteristics of a specific task. The learning_rate specifies the rate at which weights are updated during model training. The max_grad_norm is used to specify gradient clipping to prevent the gradient explosion problem. When the norm of the gradient update exceeds 1.0, it will be automatically scaled. The compute_type parameter specifies that the numerical precision type used in the training process is bf16, which is a floating point format that provides a dynamic range similar to that of single-precision floating point numbers while using only the number of digits of half-precision floating point numbers, which helps to reduce memory usage and accelerate calculations while maintaining accuracy. The cutoff_len parameter defines the maximum length of the sequence processed by the model. Sequences exceeding this length will be truncated. The present invention does not involve long texts, so the value is set to 1024. The batch_size parameter specifies the number of samples used to update the model weights in each training iteration. Based on the consideration of available hardware resources and memory capacity, the parameter is set to 6. The gradient_accumulation_steps parameter is used to specify how many batches of gradients are accumulated before a parameter update is performed, which helps maintain the stability of the prompt evaluation model training.

[0042] Perform model selection The present invention aims to build a rigorous and unified experimental framework, aiming to reveal the specific text processing rules, and mainly uses the Qwen series models. The text embedding module adopts the text2vec-base-chinese text embedding model, which is designed for Chinese text and can effectively capture the semantic association and contextual dependency between words, and generate high-quality low-dimensional vector representations. The prompt word generation model selects the Qwen-max online API service and specifies Temperature as 0.7 to better generate candidate excellent prompt words for the problem. The prompt evaluation model uses Qwen1.5-1.8B-Chat-int4 fine-tuned based on the candidate prompt word library. The model is optimized for the prompt word evaluation task and can accurately depict the intrinsic connection between the prompt word and the task performance. The downstream large model uses Qwen-7B-Chat-Int4, Qwen-14B-Chat- int4, and Qwen-turbo models, intending to maximize the use of excellent prompt words on models of different sizes to verify the effectiveness and universality of the research method of the present invention.

[0043] Define the evaluation indicators: This study uses the administrative professional competency test questions and subject test questions for junior and senior high schools in the C-Eval dataset, aiming to evaluate the model performance through the key indicator of accuracy. Accuracy is defined as the ratio of the number of correct answers to the total number of questions. This indicator is directly related to the model's ability to handle specific problem categories. The higher the accuracy, the stronger the model's ability to solve this type of problem. The accuracy calculation formula is: ; (4) Experimental results and analysis: In order to verify the effectiveness of the optimization generation method based on the selection of the optimal prompt word, the present invention evaluates the question answers generated by the downstream large model on the test set. The prompt in the zero-sample environment consists of only one instruction and one question.

[0044] The evaluation results of the Qwen series model zero-sample prompt generation are as follows Figure 6 As shown, according to Figure 6The evaluation results of this test are displayed. Only several specific subjects in the C-Eval dataset were selected for this test. STEM refers to accuracy in science, technology, engineering, and mathematics. Social Science corresponds to accuracy in the field of social sciences, covering subjects such as economics, political science, sociology, geography, and psychology. Humanities refers to accuracy in the field of humanities, including language, literature, history, philosophy, art history, and cultural studies. In the Civil Servant field, this test only targeted civil service exam questions. Hard refers to subjects that are challenging for language models. This test includes three parts: high school mathematics, high school physics, and high school chemistry. Partial AVERAGE represents the score of the model in this specific test set. AVERAGE represents the score of the model in the C-Eval test set.

[0045] Depend on Figure 6 It can be seen that most of the subjects selected by the present invention show high suitability for language models. Therefore, even if the method of the present invention is not used, the average performance of the Qwen series models still exceeds the results of the evaluation using the entire C-Eval data set, thanks to the superior performance of the large language model in text processing and knowledge-based tasks. However, in areas involving rigorous logical reasoning and complex computing capabilities, its performance still has room for improvement, especially in areas where the evaluation model generally has low results in the Hard and Civil Servant fields, with an average score of only 59.13 points. Such complex subjects often require the model to not only have a deep language comprehension ability, but also to demonstrate accurate logical reasoning, complex mathematical modeling or rigorous scientific argumentation capabilities. The method provided by the present invention was born based on this background.

[0046] As shown in the data in Table 3 below, the method of the present invention has a certain degree of improvement for this type of problem, with the average improvement of 8.13% in the Hard and Civil Servant fields reaching 64.4 points.

[0047] Table 3: Scores of zero-shot generation and generation based on optimal prompt word selection According to the data shown in Table 3, when the optimal prompt word selection optimization generation method is used, the Qwen-14B-Chat- int4 and Qwen-turbo models can better use the optimal prompt words to correct the original wrong answers, while the Qwen-7B-Chat-Int4 model may have a small number of parameters or the prompt words are still not clear enough. When using this method, the model is only slightly improved compared with before, and even slightly declined in some aspects.

[0048] The following example is generated by optimizing the selection of the optimal prompt word: Question: One day, a farmer's donkey fell into a dry well. The poor donkey screamed miserably in the well for several hours. The farmer was so anxious that he finally decided that the donkey was old and the well should be filled up. It was not worth the effort to save the donkey. He asked all the neighbors to help him fill the well. Everyone grabbed a shovel and started to fill the well with soil. The donkey soon realized what was happening. At first, it cried in panic, but soon it calmed down. People couldn't help but look into the well, and a miracle happened. Every time a shovel of soil hit the donkey's back, it did something unexpected: it quickly shook off the dust and then stepped on it hard. In this way, not long after, the donkey actually lifted itself up, and everyone present was amazed. The truth of this passage is ____.

[0049] Options: A. Every difficulty in life is a stepping stone to success; B. Changing your way of thinking can often produce unexpected results; C. Calm thinking is the first condition for overcoming difficulties; D. It is better to rely on yourself than on others. In many cases, you are your own biggest savior. Correct answer: A.

[0050] Zero-Shot: This passage tells us that it is better to rely on yourself than on others. In many cases, you are your own greatest savior. Answer: B OPS: 1. Understand the plot: A farmer's donkey accidentally fell into a dry well, and the farmer decided to give up the rescue and fill the well. During the filling process, the donkey panicked at first, but then quickly adjusted its response, shook off the soil on its body and stepped on it, gradually raised its position, and finally successfully jumped out of the dry well.

[0051] 2. Distill the theme of the story: When facing a desperate situation, you can turn adversity into opportunity and achieve self-salvation by changing your mindset and actively saving yourself.

[0052] 3. Relate to life experience: In real life, people often encounter seemingly unsolvable problems or get into trouble, such as unemployment, illness, interpersonal relationship problems, etc. By changing perspectives, innovating strategies or improving self-ability, you can find new ways out and even grow in adversity.

[0053] 4. Contrast the connotations of the options: Every difficulty in life is a stepping stone to success: This option emphasizes the positive role of difficulties in success, which is consistent with the theme of the story. The donkey used the "difficulties" of filling the well to turn it into an opportunity to escape. Changing your mindset can often produce unexpected results: This option emphasizes the importance of changing your mindset, which is consistent with the process of the donkey in the story from panic to calm response and innovative self-rescue methods. Calm thinking is the first condition for overcoming difficulties: Although the donkey in the story did change from panic to calm response, the main emphasis is on its innovative self-rescue behavior rather than just calm thinking, so this option is less relevant to the theme of the story. It is better to rely on yourself than on others. In many cases, you are your own biggest savior: This option directly corresponds to the plot where the donkey successfully escaped from danger by relying on its own strength when no one rescued him.

[0054] 5. Choose the best revelation: Among the options that are closely related to the theme of the story, both option A and option B can better reflect the moral of the story, but option A has more extensive guiding value. It not only includes how the donkey uses the predicament (the soil filling the well) as a stepping stone, but also implies the universal truth that any difficulty can be transformed into a boost to success. In contrast, although option B accurately describes the effect of the donkey's change of thinking in the story, its coverage is narrower and only targets the change of thinking in a specific situation. Therefore, the best revelation is: every difficulty in life is a stepping stone to success.

[0055] The answer is: A.

[0056] The above case responses are composed of two methods, where the Zero-Shot method represents the text generated by the downstream large model when only one instruction and question are used as prompt words, and the OPS method represents the text generated by the downstream large model when the optimal prompt words are selected. It can be seen that the model did not answer the question correctly under the Zero-Shot method, but after using the OPS method, the model gave the correct answer and question analysis.

[0057] In view of the problem that large language models cannot handle complex logical texts well, the present invention proposes a large language model optimization generation method based on optimal prompt word selection. Through the systematic optimization of the prompt word selection strategy, a set of adaptive frameworks with spectral clustering, excellent prompt word construction, diversified candidate prompt word generation and special prompt evaluation model as the core are developed, which provides a new optimization direction for large models in handling various problems. The above-mentioned embodiment experiments show that compared with zero-sample generation, the method provided by the present invention has a certain degree of improvement in the performance of arithmetic and complex tasks.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large language model optimization generation method based on optimal prompt word selection, characterized in that: The generation steps include the following: Step 1: Preprocess the selected data set by concatenating questions and answers to form a sequence, and use the spectral clustering algorithm to divide it into multiple groups of question-answer pairs with similar features; Step 2: Select representative samples from each cluster and construct high-quality prompt words that can guide the generation of large models for specific problems based on professional knowledge, language logic and semantic associations; Step 3: Use a large-parameter online model to generate a diverse and targeted set of prompt words for each cluster, refer to the created high-quality prompt words, integrate the generated questions, candidate prompt words and their answers, and build a dataset for fine-tuning the prompt evaluation model; Step 4: Use the LoRA fine-tuning model to adjust the prompt evaluation model parameters so that it can accurately measure the relevance and effectiveness of each prompt for a specific question; Step 5: Select the highest-scoring candidate prompt word as the best prompt word for a specific question, combine it with the question, input it into the downstream large model for text generation, and finally evaluate the text correctness.

2. The large language model optimization generation method based on optimal prompt word selection according to claim 1, characterized in that: The specific method of preprocessing the data set and dividing the question-answer pairs using the spectral clustering algorithm in step 1 is: The C-Eval dataset was selected for training and validation, questions and answers were concatenated into a single sequence, and the spectral clustering algorithm was used to group these sequences; Use the text embedding method to convert the text into a vector. Specifically, the input layer receives the one-hot encoding of the current word and its context words, and then these encodings are converted through the weight matrix W1 to obtain the vector representation of each word. The obtained vectors are sent to the projection layer for weighted summation and addition to obtain a sum representing the relationship between the current word and the context words; The sum is input to the hierarchical Softmax layer, which is a binary tree with each leaf node corresponding to a word. The probability of each word is obtained by calculating the weight on the path from the root node to the leaf node. The probability represents the possibility of the current word appearing in a given context. The spectral clustering algorithm is used to calculate the similarity matrix of the question vector. The calculation formula is: ； Among them, v i and v j They represent two vectors in the embedded vector space, ||·|| represents the Euclidean distance of the vectors, and σ is a parameter that controls the similarity decay rate, which can be automatically determined according to the distance from each point to its k nearest neighbors.

3. The large language model optimization generation method based on optimal prompt word selection according to claim 1, characterized in that: The specific method for constructing high-quality prompt words in step 2 is: After data preprocessing and cluster analysis, a certain number of clusters are obtained. Problem samples that can fully reflect the semantic characteristics of the cluster are extracted from each cluster. For the selected samples, excellent prompt words with strong pertinence and significant guiding effect are constructed based on domain knowledge, language logic and potential semantic associations. The template of excellent prompt words is constructed. The expression for constructing the template is: Prompt=Instruct+Overall / Detailed Description+Question; Among them, Instruct contains clear instructions for the tasks that the model is expected to perform; Overall Description represents the theme that the large model should follow when answering a certain type of question; Detailed Description represents a detailed description of the specific task steps in a point-by-point manner; Question is the specific question raised by the user.

4. The large language model optimization generation method based on optimal prompt word selection according to claim 1, characterized in that: The specific method of generating the candidate prompt word set and constructing the fine-tuning dataset in step 3 is: Based on the divided clusters and the corresponding excellent prompt words, the word library and fine-tuning dataset are constructed. Referring to the constructed excellent prompt word library, the Qwen model is used to generate targeted excellent prompt words for various pre-divided question clusters, covering the unique attributes of various input questions in the cluster, and then the generated questions, candidate prompt words and corresponding answers are integrated to finally construct a candidate prompt word library; Then, we use a combination of online large models and manual annotation to build a dedicated dataset for fine-tuning the prompt evaluation model, including: Expert reviewers evaluate and score each question-prompt word combination based on the closeness of the connection between the question and the prompt word, whether the prompt word can effectively stimulate the model to generate the expected answer, and the comprehensive performance of the prompt word in terms of semantics, grammar, and domain expertise. A few-shot learning strategy is adopted, and the Qwen model is used to simulate manual scoring. For the prompt word evaluation task, the K-way, N-shot form is adopted, where K represents the number of question categories and N represents the number of samples in each category used to train the model, that is, the manually labeled scoring data.

5. The large language model optimization generation method based on optimal prompt word selection according to claim 1, characterized in that: The specific method of using the LoRA fine-tuning model to adjust the prompt evaluation model parameters in step 4 is: Add a pair of low-rank matrices U and V to each weight matrix in the selected layer, corresponding to the row and column directions of the original weight matrix respectively; During the training process, only the low-rank matrices U and V are updated, and the original weight matrix W is not directly modified. The calculation formula for the actual weight update h of the model is: h=W+U·V T ; By adjusting the low-rank matrices U and V and using the Adam optimization algorithm and early stopping strategy, we can ensure that the model achieves optimal performance within a limited number of training steps. After each round of training, the performance indicators of the model are calculated, and the model parameters and learning rate hyperparameters are adjusted according to the performance indicators to achieve the optimal model performance.

6. The large language model optimization generation method based on optimal prompt word selection according to claim 1, characterized in that: The specific method of selecting the best prompt word and evaluation in step 5 is: Classify the questions according to the cluster they belong to, and send the questions and the candidate prompt word set in the cluster to the evaluation model; Based on the evaluation model scores, the system automatically selects the prompt word with the highest score in the candidate prompt word set as the optimal prompt word for a specific question; The program logically merges the optimal prompt word with the corresponding question text to form a question statement containing clear prompt information, which is passed to the downstream large model as a unified input to drive the downstream large model to generate answers based on this; After the model is generated, the answers generated by the model are compared with the standard answers to the corresponding questions in the C-Eval test set to evaluate the generation performance.

Images