A bayesian automatic prompt word optimization method based on meta prompt words

Abstract

The application relates to the technical field of prompt optimization, and discloses a Bayesian automatic prompt optimization method based on meta prompts, which comprises the following steps: constructing a challenging data set pool for the cognitive weaknesses of a target large language model, obtaining an initial prompt, and combining the data set pool to construct a hierarchical optimization target system; based on the hierarchical optimization target system and the data set pool, constructing an optimization target, a task example and a historical learning module of the meta prompt, and assembling the meta prompt into a complete meta prompt in a predetermined format; using the meta prompt to drive an optimizer large language model to generate a new prompt candidate; after performance evaluation of the data set pool, Bayesian optimization strategy iteration and improvement to convergence, the optimal prompt is output; the application realizes accurate and automatic optimization of the prompt of the target large language model, and guarantees the adaptability of the prompt to the model requirements and the stability of the optimization effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of prompt optimization, more particularly, to a Bayesian automatic prompt optimization method based on meta-prompt. BACKGROUND

[0002] With the in-depth application of large language models in various fields, as the core medium of human-computer interaction, the quality of prompts directly determines the accuracy and applicability of model output, so prompt optimization is a key link to improve the performance of large language models. In the prior art, optimization methods based on Bayesian theory have been gradually applied to the field of prompt-related technologies due to their efficient optimization capabilities, but they have focused on specific functional scenarios and have not yet formed a systematic prompt optimization scheme targeting the cognitive weaknesses of large language models.

[0003] In existing patent technologies, a Chinese patent with authorization announcement number CN120017420B discloses a prompt detection resource optimization method, system, optimization terminal and medium, which receives user prompts and identifies prompt parameters, constructs a multi-stage dynamic Bayesian game model to obtain detection probability, filters safe prompts based on detection probability and inputs them into a large language model for processing, the core being to optimize prompt detection resource consumption, improve model output security and reduce benign user service delay; a Chinese patent application with publication number CN118709782A discloses a reasoning method, system, storage medium and terminal based on Bayesian network and large model collaborative optimization, which constructs a knowledge graph, Bayesian network and knowledge base, combines prompt engineering to drive large model reasoning, and dynamically adjusts the output weights of Bayesian network and large model to realize heterogeneous model collaborative optimization to improve the quality of reasoning results.

[0004] However, the above-mentioned existing technologies still have significant limitations: CN120017420B only focuses on the safety detection and resource optimization of prompts and does not involve prompt optimization design targeting the inherent cognitive weaknesses of large language models, which cannot solve the output deviation problem caused by the mismatch between prompts and model cognitive defects; CN118709782A combines prompt engineering and Bayesian networks, but does not construct a special test carrier targeting model cognitive weaknesses, nor does it establish a hierarchical optimization target system and a modular architecture of meta-prompt, resulting in a vague direction of prompt optimization and a lack of automatic iterative convergence mechanism based on Bayesian strategy, which requires manual intervention and adjustment, not only is the optimization efficiency low, but also the optimized prompts may still not effectively compensate for the performance short board of the model in specific tasks, ultimately affecting the application reliability of large language models in professional scenarios. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the present application provides a Bayesian automatic prompt word optimization method based on meta prompt words, which realizes the precise and automatic optimization of prompt words by constructing a challenging data set pool and a hierarchical optimization target system for the cognitive weaknesses of the target large language model, combining the modular design of the meta prompt word module and the Bayesian optimization iteration, effectively solving the problem that the prior art cannot anchor the model cognitive weaknesses. Relying on the meta prompt word driven optimizer large language model to generate candidate prompt words, and through the challenging data set pool evaluation and Bayesian strategy convergence, the adaptability of the optimized prompt words and the model requirements is significantly improved, and the stability of the output performance of the target large language model is ensured.

[0006] In order to achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0007] A Bayesian automatic prompt word optimization method based on meta prompt words, comprising:

[0008] A challenging data set pool for the cognitive weaknesses of the target large language model is constructed, the initial prompt words of the target large language model are obtained, and a hierarchical optimization target system is constructed according to the initial prompt words and the challenging data set pool;

[0009] According to the hierarchical optimization target system and the challenging data set pool, an optimization target module, a task example module and a historical learning module of the meta prompt word are constructed; the optimization target module, the task example module and the historical learning module are assembled according to a predetermined format to generate a complete meta prompt word;

[0010] The meta prompt word is used to drive the optimizer large language model to generate new prompt word candidates, the performance of the new prompt word candidates is evaluated on the challenging data set pool and the meta prompt word is updated, and the Bayesian optimization strategy is iteratively improved until convergence, and the optimal prompt word is output.

[0011] Further, the construction method of the challenging data set pool comprises: receiving a plurality of candidate data sets and calculating the challenging index of the target large language model on each candidate data set, and according to the challenging index, the candidate data sets are screened to form the challenging data set pool.

[0012] Further, the method for calculating the challenging index of the target large language model on each candidate data set comprises:

[0013] The task type features of each candidate data set are extracted, and the cognitive weakness categories of the target large language model are identified according to the task type features;

[0014] Based on the cognitive weakness categories, the target large language model is used to perform a benchmark test on each candidate data set, and the challenging index composed of the error rate, the response time and the reasoning complexity is calculated.

[0015] Further, the method of screening the candidate dataset to form the challenging dataset pool comprises: normalizing the error rate, response time and inference complexity to obtain a normalized error rate, a normalized response time and a normalized inference complexity; and weighting and summing the normalized error rate, the normalized response time and the normalized inference complexity to obtain a challenging comprehensive score.

[0016] All candidate datasets are sorted according to the challenging comprehensive score from high to low, and the top K% of candidate datasets are included in the challenging dataset pool, where K is a percentage threshold parameter.

[0017] Further, the method of constructing the hierarchical optimization target system comprises:

[0018] The initial prompt word is used to test the target large language model on the challenging dataset pool to obtain a multi-dimensional performance indicator; the multi-dimensional performance indicator is subjected to correlation analysis to identify a performance bottleneck dimension and an improvement potential dimension, and an initial performance benchmark report is generated.

[0019] According to the initial performance benchmark report and the multi-dimensional performance indicator, a hierarchical optimization target system is generated.

[0020] Further, the method of obtaining a multi-dimensional performance indicator comprises:

[0021] The initial prompt word is combined with each data sample in the challenging dataset pool to form a complete test input sequence;

[0022] The test input sequence is input into the target large language model, the output response of the target large language model is collected, and the multi-dimensional performance indicator is calculated; the multi-dimensional performance indicator includes accuracy, recall, F1 score and inference step integrity.

[0023] Further, the method of identifying the performance bottleneck dimension is that the indicator value of the multi-dimensional performance indicator is significantly lower than the ideal value, and is strongly correlated with the accuracy.

[0024] Further, the determination standard that the indicator value of the multi-dimensional performance indicator is significantly lower than the ideal value is that the indicator value of the i-th indicator in the multi-dimensional performance indicator is C i , the ideal value of the i-th indicator is C 0,i , and if is greater than a preset proportion threshold, it is determined that the i-th indicator is significantly lower than the ideal value.

[0025] Further, the determination method that the i-th indicator is strongly correlated with the accuracy is that the correlation coefficient of the i-th indicator and the accuracy is calculated, and if the absolute value of the correlation coefficient is greater than or equal to a preset strong correlation threshold, it is determined that the i-th indicator is strongly correlated with the accuracy.

[0026] Further, the generation method of the hierarchical optimization target system comprises:

[0027] Based on the performance bottleneck dimension in the initial performance benchmark report, set the main optimization target and the corresponding target improvement range;

[0028] According to the improvement potential dimension, determine the secondary optimization target and the constraint condition;

[0029] Organize the main optimization target, the secondary optimization target and the constraint condition according to the priority, and construct the hierarchical optimization target system containing the target weight distribution.

[0030] Compared with the prior art, the beneficial effects of the present application are:

[0031] The present application constructs a challenging data set pool focusing on the cognitive weaknesses of the target large language model, so that the prompt word optimization can accurately anchor the core defects of the model, avoiding the direction deviation caused by the lack of targeted test carriers in traditional optimization; the hierarchical optimization target system constructed based on the initial prompt word provides clear and orderly target guidance for the optimization process, solving the problem of fuzzy targets and no clear priority in traditional optimization; through the construction and structural assembly of the three modules of the meta-prompt word, a standard generation basis is provided for the optimizer large language model, ensuring that the new prompt word candidate always meets the optimization requirements; then relying on the Bayesian optimization strategy to realize the automatic iteration and convergence of the prompt word, greatly reducing the inefficiency and blindness of manual trial and error, and finally the optimal prompt word suitable for the target large language model can be stably output, significantly improving the accuracy, automation level and effect reliability of prompt word optimization, effectively solving the problems of traditional prompt word optimization, such as relying on manual experience, low efficiency and unstable effect. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0033] Figure 1 The method flowchart of the present application is provided for a Bayesian automatic prompt word optimization method based on meta-prompt word;

[0034] Figure 2 The method flowchart of the present application is provided for a Bayesian automatic prompt word optimization method based on meta-prompt word;

[0035] Figure 3 The priority order schematic diagram in the hierarchical optimization target system provided by the present application is provided;

[0036] Figure 4The principle diagram of the generation method of new prompt word candidates provided by the embodiment of the present application is shown in the figure;

[0037] Figure 5 The functional module diagram of a Bayesian automatic prompt word optimization system based on meta prompt words in the present application is shown in the figure. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0039] Embodiment 1:

[0040] Please refer to Figure 1 The present embodiment provides a Bayesian automatic prompt word optimization method based on meta prompt words, which comprises:

[0041] Step S10, constructing a challenging data set pool for the cognitive weaknesses of the target large language model, obtaining the initial prompt word of the target large language model, and constructing a hierarchical optimization target system according to the initial prompt word and the challenging data set pool;

[0042] Step S10 aims to construct a challenging data set pool for the cognitive weaknesses of the target large language model, and generate a hierarchical optimization target system in combination with the test results of the initial prompt word, which solves the technical problems that the prior art lacks a standardized evaluation system for specific weaknesses of the target large language model and cannot accurately identify performance bottlenecks, and provides a precise test carrier and clear optimization direction for subsequent meta prompt word construction and prompt word iterative optimization.

[0043] Further, step S10 comprises:

[0044] Step S11, receiving a plurality of candidate data sets and calculating the challenging index of the target large language model on each candidate data set, and screening to form a challenging data set pool;

[0045] Please refer to Figure 2 Further, step S11 comprises:

[0046] Step S111, receiving a plurality of candidate data sets and extracting the task type features of each candidate data set, and identifying the cognitive weakness categories of the target large language model according to the task type features;

[0047] Step S112, based on the cognitive weakness categories, performing a benchmark test on each candidate data set using the target large language model, and calculating the challenging index composed of the error rate, response time and reasoning complexity;

[0048] Step S113: According to the challenge index, the candidate data set is screened to form a challenge data set pool.

[0049] Step S11 aims to form a challenge data set pool focusing on the cognitive weaknesses of the target large language model by screening the candidate data set and calculating the challenge index. The core solves the technical problem that the candidate data set in the prior art lacks pertinence and only covers general tasks, which cannot accurately expose the model defects. Through the technical link of "feature extraction-multi-dimensional evaluation-comprehensive screening", high-quality and high-pertinence test carriers are provided for the initial performance test of step S12 and the task example module construction of step S22, ensuring that the entire optimization scheme is anchored to the model cognitive weak link from the source.

[0050] The candidate data set covers various task type samples related to the potential cognitive defects of the target large language model, such as samples containing "SVG path analysis", "geometric shape recognition", "graph closure judgment", and other spatial reasoning tasks, samples of "text spatial relationship analysis", "multi-object relative position reasoning", and other tasks, and samples of "abstract visual description interpretation" tasks, which can correspond to the model's "spatial reasoning" and "abstract visual description" cognitive weaknesses, respectively.

[0051] The implementation of step S111 needs to extract the task type features of the candidate dataset and identify the cognitive weakness category. First, a text structuring analysis tool is used to deeply process each received candidate dataset to extract core features that reflect the nature of the task: for candidate datasets containing graphics-related samples, the sample input format and output requirements are analyzed to extract task type features such as "SVG path analysis", "geometric shape recognition", and "graph closure judgment"; the input format, for example, whether it contains SVG path elements, whether the path elements contain straight line commands "L", curve commands "Q", and closure commands "Z"; the output requirements, for example, whether to judge the geometric shape category or describe the graph generation logic. For candidate datasets containing text spatial descriptions, the spatial relationship expressions and reasoning requirements in the samples are analyzed to extract task type features such as "text spatial relationship analysis" and "multiple object relative position reasoning"; the spatial relationship expressions, for example, "object A is directly above object B, and object C is to the left of object B"; the reasoning requirements, for example, "judge the relative position of object A and object C". Based on these features, a mapping relationship between "task type features-cognitive weakness categories" is established: candidate datasets containing "SVG path analysis", "geometric shape recognition", and "spatial relationship reasoning" features are classified into the "spatial reasoning" cognitive weakness; candidate datasets containing "abstract visual description interpretation" features are classified into the "abstract visual description" cognitive weakness. Among them, the "spatial reasoning" cognitive weakness is manifested as the model's difficulty in analyzing input containing spatial information and performing logical reasoning to obtain results, for example, given a complete SVG path element, the model cannot accurately judge the geometric shape generated by executing the path; the "abstract visual description" cognitive weakness is manifested as the model's difficulty in restoring the corresponding visual structure or interpreting the visual logic implied by the abstract visual information in the form of text.

[0052] For example, if a candidate dataset sample is "given SVG path <pathd="M50,50L150,50L100,150Z" / >, judge the corresponding geometric shape", the "SVG path analysis" and "triangle recognition" features are extracted, and it is classified into the "spatial reasoning" cognitive weakness. This process solves the problem in the prior art that candidate dataset selection relies on subjective experience and is disconnected from the model's cognitive weaknesses. Through structured feature extraction and directional classification, it ensures that each candidate dataset can directly point to the inherent defects of the model, laying a foundation for subsequent accurate evaluation.

[0053] Step S112 needs to perform benchmarking based on the cognitive weakness category, calculate the challenge index composed of error rate, response time, reasoning complexity. First, match the corresponding initial test prompt word for each cognitive weakness category, the initial test prompt word refers to the general prompt word adapted to the task type, such as the initial test prompt word of the spatial reasoning class is "analyze the following input content and output the corresponding result according to the requirements", combine each sample in the candidate data set with the initial test prompt word according to the unified structure of "initial test prompt word + sample detailed information + output format requirement" to form a standard input sequence. Input the standard input sequence into the target large language model one by one, record the output response, and calculate the challenge index by dimension: error rate calculation needs to exclude invalid output first, such as model timeout without response, output content irrelevant to task requirements, count the number of samples inconsistent with the ground truth (annotated correct answer) in the data set in the valid output, error rate = (inconsistent sample number / total number of valid samples) x 100%, this calculation method ensures that only the cognitive ability defects of the model for the task are reflected, and not the errors caused by input format or system failure; response time calculation needs to record the timestamp from the submission of the standard input sequence to the output of the complete response of the model, and take the average value after repeating the test N times for each sample, N is determined according to the size of the data set to ensure statistical significance; at the same time, the system hardware basic delay time is obtained by inputting an empty string into the model, and the test average time is subtracted by the basic delay time to obtain the actual response time of the model processing the sample, avoiding the interference of hardware performance difference on the index; reasoning complexity calculation needs to predefine the key reasoning link of the corresponding cognitive weakness category, for example, the key reasoning link of the spatial reasoning class is "analyze the spatial information in the input → extract the core features (such as SVG path commands, object position relationship) → perform logical reasoning based on the features → output the result", count the proportion of samples containing all key reasoning links in the output response, if the sample output directly gives the answer without reasoning process, or the reasoning link has logical discontinuity, such as directly judging the geometric shape without analyzing the SVG path command, then mark the sample reasoning complexity as "high", contain 2 / 3 and above key links marked as "medium", contain all key links marked as "low", the reasoning complexity index is quantified as a numerical value through "high / medium / low" levels, such as high=3, medium=2, low=1, to facilitate subsequent comprehensive calculation. This multi-dimensional index system solves the limitation of existing technology that only relies on error rate evaluation, fully describes the challenge of the data set to the model from the "correctness" "efficiency" "reasoning quality" three dimensions, and ensures that the selected data set can fully expose the multiple defects of the model.

[0054] Step S113 needs to calculate the challenge comprehensive score based on the above three challenging indexes, sort all candidate datasets according to the challenge comprehensive score from high to low, and include the top K% candidate datasets into the challenge dataset pool, K being a percentage threshold parameter determined according to optimization requirements, for example, K=60. First, normalize the error rate, response time and inference complexity to obtain the normalized error rate, normalized response time and normalized inference complexity, calculate the challenge comprehensive score according to the normalized error rate, normalized response time and normalized inference complexity, and the formula design follows the principle of "correctness first, efficiency and inference quality second". Since the error rate directly reflects the core ability of the model to complete the task, it is given the highest weight, and the response time and inference complexity are given the second highest weight. The formula expression is:

[0055]

[0056] Wherein:

[0057] S is the challenge comprehensive score, and the higher the score indicates the stronger the challenge of the dataset;

[0058] E is the normalized error rate, T is the normalized response time, and C is the normalized inference complexity;

[0059] W1, W2 and W3 are the weights of the error rate, response time and inference complexity, respectively, and the values need to satisfy W1+W2+W3=1 and W1>W2≥W3. For example, W1=0.5, W2=0.3 and W3=0.2. The weight determination method is to invite 3-5 technical personnel in the field to compare the importance of the three indexes two by two, construct a judgment matrix and pass the consistency test (consistency ratio CR<0.1), to ensure the rationality of the weight allocation.

[0060] Step S11 solves the problems of lack of pertinence of candidate data sets, single evaluation index and fuzzy screening logic in the prior art as a whole. The feature extraction and cognitive weakness classification of step S111 provide a directional task range for the benchmark test of step S112, ensuring that the test focuses on model defects. The multi-dimensional index calculation of step S112 provides a quantitative basis for the comprehensive screening of step S113, avoiding subjectivity in screening. The data set pool of step S113 directly provides a carrier for the initial performance test of step S12, and at the same time provides a representative sample source for the task example module of step S22. The cooperation of steps S111 and S112 enables the index calculation to accurately correspond to model weaknesses, for example, the reasoning complexity calculated for spatial reasoning type data sets can directly reflect the defects of the model in spatial logic, and the subsequent optimization can be designed to provide prompt words in a targeted manner. The cooperation of steps S112 and S113 makes the screened data sets not only have a high error rate, but also have a response time and reasoning complexity that can reflect the challenge, so that the optimization process can achieve "one optimization with multiple benefits", for example, improving the integrity of the reasoning step while reducing the error rate and response time.

[0061] The existing evaluation often only focuses on the error rate, which easily ignores the model processing efficiency problem, resulting in some data sets that have little difference in error rate from regular tasks, but their hidden efficiency defects are not discovered. By introducing the response time index, such data sets with significantly longer response time can be accurately identified, which can directly expose the efficiency shortcomings of the model when processing complex tasks. Subsequently, by guiding the model to simplify the invalid reasoning steps through prompt words, the accuracy and processing efficiency can be improved at the same time. Traditional evaluation does not correlate and analyze the error rate and reasoning complexity, making it difficult to locate the core defects of the reasoning logic level. After introducing the reasoning complexity index, it can effectively identify the strongly correlated data sets with "high error rate-high reasoning complexity". Such data sets can guide the optimization process to focus on the reasoning logic problem, thereby significantly reducing the error rate and achieving the chain effect of "reasoning optimization driving accuracy improvement". If step S11 is missing, the test of subsequent step S12 will be based on regular data sets, which cannot expose the cognitive weaknesses of the model, resulting in blind optimization direction. The task example module of step S22 will also lack high-quality samples, and the meta-prompt words generated cannot accurately guide the prompt word optimization, breaking the logical chain of the entire technical solution from the source. Therefore, step S11 is a basic prerequisite to ensure the accuracy and effectiveness of subsequent optimization.

[0062] Step S12, obtaining an initial prompt word, testing the target large language model on the challenge data set pool using the initial prompt word, and obtaining multi-dimensional performance indexes; performing correlation analysis on the multi-dimensional performance indexes to generate an initial performance benchmark report;

[0063] Further, step S12 includes:

[0064] Step S121, obtain an initial prompt, combine the initial prompt with each data sample in the challenging data set pool to form a complete test input sequence;

[0065] Step S122, input the test input sequence into the target large language model, collect the output response of the target large language model, and calculate the multi-dimensional performance index;

[0066] Step S123, perform correlation analysis on the multi-dimensional performance index, identify the performance bottleneck dimension and the improvement potential dimension, and generate an initial performance benchmark report.

[0067] Step S12 aims to obtain the multi-dimensional performance index of the target large language model and generate an initial performance benchmark report through the test of the initial prompt on the challenging data set pool. The core solves the technical problems in the prior art of lacking a standardized performance evaluation system for the target large language model on cognitive weak point tasks, and being unable to accurately identify optimization bottlenecks and potential. It provides quantitative data support for the subsequent step S13 of constructing a hierarchical optimization target system and the step S23 of initializing a historical learning module, realizes a technical closed loop of "testing-analysis-positioning", and ensures that the optimization direction is anchored to the core defects of the model.

[0068] The implementation of step S121 requires combining the initial prompt with the samples of the challenging data set pool into a complete test input sequence. First, the acquisition method of the initial prompt needs to be determined: the initial prompt is a general prompt that adapts to the task type of the challenging data set pool. The acquisition approaches include a domain common prompt library, such as a natural language processing task prompt library, a basic prompt in public research of the same type of task, or a simple instruction generated based on task description. The initial prompt needs to avoid containing any specific optimization strategies, such as "analyzing SVG path commands" and "counting the number of graph edges", to ensure that it can reflect the original performance of the model without optimization guidance. Then, the initial prompt is combined with the sample detailed information and the output format requirement into a complete test input sequence according to the unified structure of "initial prompt + sample detailed information + output format requirement". The sample detailed information needs to completely preserve the core content of the challenging data set pool sample, and the output format requirement needs to be clear and unified to avoid evaluation errors caused by output format differences. This combination process solves the problem of chaotic test input format in the prior art and cannot exclude format interference. Through the unified structure, it ensures that the test environment of each sample is consistent, so that the subsequent performance index can truly reflect the cognitive ability of the model to the task, rather than the adaptability to the input format.

[0069] The implementation of step S122 needs to calculate multi-dimensional performance indicators composed of accuracy, recall rate, F1 score, and reasoning step integrity. The calculation of each indicator needs to strictly correspond to the task characteristics of the challenging data set pool and the ground truth to ensure the objectivity and traceability of the quantitative results. The accuracy calculation needs to count the proportion of the number of samples whose output results of the target large language model are consistent with the ground truth in the data set to the total number of test samples. Before calculation, invalid outputs need to be excluded, and only valid outputs are counted. The formula uses the well-known "accuracy = correct sample number / total effective sample number" in the art. This indicator reflects the correctness of the model's overall task completion, but needs to be combined with other indicators to avoid one-sidedness of "only good at simple samples". The recall rate calculation needs to be calculated separately according to the task categories of the challenging data set pool. For each category, recall rate = the number of samples correctly identified in this category / the total number of samples in this category. This indicator solves the problem that accuracy cannot reflect the model's coverage ability for small category samples. For example, a data set contains 100 easily identifiable "triangle" samples and 20 difficult to identify "fan" samples. The model correctly identifies 90 "triangles" and 5 "fans". The accuracy is 95 / 120 ≈ 79.2%, but the recall rate of the "fan" category is only 25%. Through the recall rate, it can be found that the model has defects in identifying small category samples. F1 score is the harmonic mean of accuracy and recall rate, F1 = 2 × (accuracy × recall rate) / (accuracy + recall rate). F1 score is used to balance the deviation of accuracy and recall rate, objectively reflects the comprehensive performance of the model, and avoids misjudgment caused by a single indicator. The reasoning step integrity calculation needs to refer to the key reasoning links defined in step S112, count the proportion of the number of samples whose output responses contain all key reasoning links to the total number of valid samples, and if the sample output contains part of the key links, it needs to be converted according to the inclusion ratio. For example, if it contains 2 key links, a total of 3 links, then the integrity is 66.7%. The reasoning step integrity directly reflects the quality of the model's reasoning process, solving the problem that the prior art only focuses on the correctness of the result and ignores the reasoning logic. For example, the model correctly judges the geometric shape, but does not analyze the SVG path command, and its reasoning step integrity is 0. Subsequent optimization can guide the model to supplement the reasoning links through prompt words.

[0070] The implementation of step S123 requires correlation analysis of the multi-dimensional performance indicators to identify performance bottleneck dimensions and improvement potential dimensions and generate an initial performance benchmark report. First, a correlation analysis tool such as the Pearson correlation coefficient calculation method is used to analyze the correlation between indicators, for example, the correlation coefficient between reasoning step integrity and accuracy is calculated. If the absolute value of the correlation coefficient is ≥ 0.7 and is positive, it indicates that the more complete the reasoning step, the higher the accuracy, and the two are positively correlated. The correlation coefficient between recall rate and accuracy is calculated. If the absolute value of the correlation coefficient is ≤ 0.3 and is negative, it indicates that improving the recall rate has little negative impact on accuracy. Based on the correlation analysis results, combined with the gap between the absolute value and the ideal value of each indicator, the performance bottleneck dimensions and improvement potential dimensions are identified, where the ideal value is the excellent performance of the target large language model on the same type of regular task. For example, the ideal value of the accuracy of the regular task is 80%, and the ideal value of the reasoning step integrity is 75%. The performance bottleneck dimension is the dimension where the indicator value of the multi-dimensional performance indicator is significantly lower than the ideal value and is strongly correlated with accuracy. The criterion for determining whether the indicator value is significantly lower than the ideal value is: let the indicator value of the ith indicator in the multi-dimensional performance indicator be C i , the ideal value of the ith indicator be C 0,i , and the difference between the two be is greater than the preset proportion threshold, then the ith indicator is determined to be significantly lower than the ideal value. The proportion threshold is set based on the excellent performance of the target large language model on the same type of regular task, i.e. the ideal value, combined with the optimization feasibility, to determine whether the indicator is significantly lower than the ideal value. The proportion threshold needs to cover the performance gap of "non-accidental fluctuations" to avoid misjudging slight fluctuations as "significant defects". If the gap is too large, it indicates that the defect in this dimension has seriously restricted the core performance, and it needs to be prioritized as a bottleneck to ensure that optimization resources are focused on "high-impact, improvable" core issues. For example, the proportion threshold is set to 0.4.

[0071] The determination method for strong correlation with accuracy is: calculate the correlation coefficient between the ith indicator and accuracy. If the absolute value of the correlation coefficient is greater than or equal to the preset strong correlation threshold, it is determined that the ith indicator is strongly correlated with accuracy. Otherwise, if the absolute value of the correlation coefficient is less than the strong correlation threshold, it is determined that the ith indicator is not strongly correlated with accuracy. The strong correlation threshold is used to filter dimensions that have a significant impact on core indicators to avoid wasting resources by optimizing non-critical dimensions. Based on the Pearson correlation coefficient, a correlation coefficient absolute value ≥ 0.7 is generally considered to be strongly correlated.

[0072] For example, the current value of the reasoning step integrity is 35%, the ideal value is 75%, the proportion threshold is 0.4, the strong correlation threshold is 0.7, = 0.53, which is greater than the proportion threshold 0.4, and the correlation coefficient with the core indicator, i.e. accuracy, is 0.8, which is greater than the strong correlation threshold 0.7, and is strongly correlated. Therefore, the reasoning step integrity is determined to be a performance bottleneck dimension, and the defect in the performance bottleneck dimension directly restricts the improvement of the core indicator.

[0073] The improvement potential dimension is a dimension where the index value deviates from the ideal value, but is not strongly correlated with the accuracy,

[0074] The method for determining whether the index value deviates from the ideal value is: is less than a preset difference threshold. If the difference threshold is too high, it will lead to "minor gaps being listed as potential dimensions", increasing the optimization cost; if it is too low, it will miss "dimensions with moderate gaps and easy optimization"; for recall rate and other indicators, the difference threshold needs to ensure that "minor category recognition defects" can be captured while avoiding conflicts with core performance. Illustratively, the difference threshold is set to 0.8.

[0075] Illustratively, for example, the current value of the recall rate is 50%, the ideal value is 70%, the difference threshold is 0.8, and 50% / 70%=0.71. The correlation coefficient with the accuracy is -0.2, which is not strongly correlated, so the recall rate is determined to be an improvement potential dimension. Improving the improvement potential dimension does not need to sacrifice the core performance, and optimization can be achieved by prompting the model to pay attention to the minor category samples.

[0076] The initial performance benchmark report needs to be presented in a structured form, including the following core content: specific values of each dimension performance indicator; correlation coefficients between indicators; specific performance of bottleneck dimensions, for example, the main reason for the low inference step integrity is "80% of the samples do not analyze the closed command 'Z' in the SVG path"; the improvement direction of the improvement potential dimension, for example, the low recall rate is mainly due to "the identification rate of fan-shaped and kite-shaped samples is less than 30%, the model needs to pay attention to the features of such samples".

[0077] Step S121 provides a consistent test environment for the index calculation of step S122, ensuring that the indexes are comparable; the multi-dimensional indexes of step S122 provide quantitative data for the bottleneck and potential identification of step S123, avoiding subjective analysis; the benchmark report of step S123 directly provides the basis for the hierarchical optimization target setting of step S13, and provides initial data for the history learning module of step S23. The cooperation of steps S121 and S122 enables the index calculation to truly reflect the original performance of the model, for example, the unified test input sequence avoids the accuracy fluctuations caused by format differences, ensuring that the accuracy objectively reflects the original level of the model in spatial reasoning tasks; the cooperation of steps S122 and S123 makes the bottleneck identification more accurate, for example, through correlation analysis, it is found that the inference step integrity is strongly correlated with the accuracy, and subsequent optimization focusing on this dimension can simultaneously improve the accuracy, achieving "targeted optimization".

[0078] Step S12 solves the problems of single performance evaluation index, ambiguous bottleneck positioning and lack of standardized benchmark report in the prior art as a whole: the inconsistency of test environment is solved by the unified test input sequence of S121, the one-sidedness of evaluation is solved by the multi-dimensional index of S122, and the ambiguity of bottleneck positioning is solved by the statistical analysis and report of S123. The multi-dimensional index system avoids the misleading of single accuracy evaluation; the correlation analysis reveals the linkage between indexes, providing a "point-to-area" idea for subsequent optimization, for example, the reasoning step integrity is strongly correlated with accuracy, optimizing this dimension can improve accuracy and reduce optimization resource waste; the initial performance benchmark report provides structured and traceable evaluation results, providing a unified standard for performance comparison at different iteration stages, avoiding the subjectivity of optimization effect evaluation. Through the analysis of the reasoning step integrity index, the regularity of missing key reasoning steps in the model can be found, such as the common missing of "SVG path closure command analysis", and subsequent optimization only needs to add clear instructions for this step in the prompt words, for example, supplementing the instruction "first analyze the closure command in the SVG path, judge whether the figure is a closed structure, and then count the number of edges based on other path commands" in the prompt words, which can improve the reasoning step integrity and accuracy, achieving "small change and big improvement" optimization efficiency. If step S12 is missing, the hierarchical optimization target of step S13 will lack quantitative basis, and may set "too high to achieve" or "too low to be meaningful" targets; the historical learning module of step S23 will also lack initial data support, and cannot form an optimization trajectory, resulting in the whole optimization scheme losing "starting point" and "direction".

[0079] Step S13, generating a hierarchical optimization target system according to the initial performance benchmark report and the multi-dimensional performance index.

[0080] Further, step S13 includes:

[0081] Step S131, setting a main optimization target and a corresponding target improvement amplitude based on the performance bottleneck dimension in the initial performance benchmark report;

[0082] Step S132, determining a secondary optimization target and a constraint condition according to the improvement potential dimension;

[0083] Step S133, organizing the main optimization target, the secondary optimization target and the constraint condition according to priority to build a hierarchical optimization target system containing target weight distribution.

[0084] Step S13 aims to determine the core direction of optimization, target priority and unbreakable constraint boundary according to the multi-dimensional performance indicators in the initial performance benchmark report, and build a hierarchical optimization target system. The technical problems of the prior art, such as ambiguous optimization target of prompt words, no priority ranking between targets, and lack of constraint conditions, are solved. The ambiguous target refers to the lack of quantitative standard, and is only expressed as a general requirement such as "improve performance". The lack of priority ranking between targets easily leads to mismatch of optimization resources to non-core targets. The lack of constraint conditions may cause core performance deterioration for improving a target. Through the technical link of "bottleneck positioning-target quantification-priority allocation", accurate and executable structured content is provided for the construction of the prompt word optimization target module in step S21, which ensures that the prompt word candidates generated by the optimizer large language model can focus on the core cognitive defects of the target large language model and balance multiple target requirements, avoiding the blindness of the optimization process.

[0085] When setting the main optimization target according to the performance bottleneck dimension in the initial performance benchmark report, the core information of the performance bottleneck dimension needs to be extracted from the initial performance benchmark report, including the current performance value of the dimension and the ideal value of the target large language model for the same type of regular task. The improvement requirement of the performance bottleneck dimension is converted into an explicit quantitative target. The determination of the target improvement amplitude needs to follow the "balance between challenge and feasibility" logic. The specific calculation method is: main target improvement amplitude = current value + (ideal value - current value) x K', where K' is the main target improvement coefficient. The value of K' needs to be dynamically adjusted according to the difference between the current value and the ideal value. When the difference is ≥ 50%, it means that the defect is serious, and K' is taken as 0.6-0.7 to reduce the difficulty of the target and ensure its realizability. When the difference is between 30% and 50%, it means that the defect is moderate, and K' is taken as 0.7-0.8. When the difference is < 30%, it means that the defect is slight, and K' is taken as 0.8-0.9 to increase the challenge of the target. Following the above example, if the reasoning step integrity current value is 35%, the ideal value is 75%, the difference is 50%, and K' is taken as 0.6, then the target improvement amplitude = 35% + (75%-35%) x 0.6 = 60%, i.e. the main optimization target is set as "improve the reasoning step integrity to more than 60%". The quantitative logic of "difference-coefficient matching" is used to set the improvement amplitude, which solves the problem of no standard in the optimization target in the prior art and avoids subjectivity. At the same time, the strong correlation condition ensures that the main optimization target can directly drive the core performance improvement, so that the optimization resources focus on the core link with high investment and high return.

[0086] When determining the secondary optimization target, the key information of the improvement potential dimension, including the current value and ideal value of the improvement potential dimension, is extracted from the initial performance benchmark report. The improvement requirement of the improvement potential dimension is converted into the secondary target improvement amplitude by referring to the quantitative logic of the main optimization target. Since the secondary target priority is lower than the main target, it is necessary to avoid occupying too many optimization resources, so the secondary target improvement amplitude needs to be appropriately reduced by the improvement coefficient, that is, the secondary target improvement amplitude = current value + (ideal value-current value) x K'', where K''=K'-0.1, K' is the main target improvement coefficient. In the above example, K'=0.6, then K''=0.5, and the secondary optimization target is set to "increase the recall rate to 50%+(80%-50%)x0.5=65% or more". The determination of the constraint condition needs to be based on the stable fluctuation range of the core indicator, that is, the accuracy rate. The stable fluctuation range is obtained by calculating the standard deviation of the accuracy rate in the test process in step S122. The standard deviation is denoted as σ, and the lower limit of the stable fluctuation range is the initial value of the accuracy rate-2σ. Then calculate the lower limit based on the fluctuation coefficient = accuracy rate initial value x (1-L'), where L' is the fluctuation coefficient, and the value of L' needs to ensure that the core performance does not deteriorate significantly. The L' of the accuracy rate is usually set to 0.1, which means that the accuracy rate cannot decrease by more than 10% during the optimization process. When the index fluctuation exceeds 10% of the initial value, the stability of the target large language model in actual application cannot be guaranteed. The initial value of the accuracy rate is the accuracy rate value measured in step S122. By comparing the lower limit of the stable fluctuation range and the lower limit based on the fluctuation coefficient, the higher value of the two is taken as the constraint threshold, and the constraint condition is to maintain the accuracy rate to be greater than or equal to the constraint threshold.

[0087] For example, if the step S122 test shows that the initial value of the accuracy rate is 60%, and the standard deviation σ calculated by the statistical sample accuracy rate data is 2%, then the lower limit of the stable fluctuation range is 60%-2x2%=56%; the lower limit based on the fluctuation coefficient is 60% x (1-0.1)=54%; the higher value of the two is taken as the constraint threshold, that is, the constraint condition is set to "maintain the accuracy rate to be greater than or equal to 56%".

[0088] By "standard deviation calculation stable fluctuation range", the boundary of natural performance fluctuation of the model is accurately captured, and the problem that the traditional constraint setting ignores the inherent fluctuation of the model and is easy to misjudge the optimization effect is solved; by "double lower limit taking high", "standard deviation-fluctuation coefficient matching" is realized, which not only avoids excessive constraint, that is, does not limit the slight fluctuation caused by normal optimization, but also prevents significant deterioration of core performance, ensuring that the constraint condition is both scientific and practical. At the same time, this step cooperates with the multi-dimensional performance index statistics of step S122 to form a stable fluctuation range based on the original test data of S122, ensuring consistent data sources and reliable results, avoiding the disconnection between the constraint condition and the previous test; the quantitative setting of the constraint threshold also provides a clear boundary for the construction of the optimization target module in the subsequent step S21, so that the optimizer large language model can clearly grasp the "optimizable space" and "bottom line that cannot be broken" when generating prompt words, avoiding the generation of invalid prompt words that may lead to deterioration of core performance.

[0089] Please refer to Figure 3 As shown in FIG. 13, the implementation of step S133 needs to organize the target system in the order of "main optimization target-secondary optimization target-constraint condition" priority, and allocate target weights through the analytic hierarchy process (AHP), to ensure that the priority is converted into a quantifiable resource allocation basis. First, the priority rules are defined: the main optimization target is the first priority, which needs to be satisfied first, and the optimization resource allocation proportion is the highest; the secondary optimization target is the second priority, which is realized on the premise of satisfying the first priority, and the resource proportion is second; the constraint condition is the third priority, which cannot be broken, although the resource proportion is the lowest, but it needs to be strictly followed. The determination of the target weight needs to be completed through the analytic hierarchy process: 3-5 field technical personnel are invited to compare the importance of the three priority levels with each other, a judgment matrix is constructed, for example, the importance ratio of the first priority level to the second priority level is 2:1, the importance ratio of the first priority level to the third priority level is 3:1, and the importance ratio of the second priority level to the third priority level is 2:1; then the characteristic vector of the judgment matrix is calculated, which is the weight value of each level, and needs to pass the consistency test (consistency ratio CR<0.1), to ensure that the weight allocation logic is self-consistent. For example, through calculation, the main optimization target weight m1=0.6, the secondary optimization target weight m2=0.3, and the constraint condition weight m3=0.1, and the weight sum is 1. When constructing the hierarchical optimization target system, it needs to be presented in a structured form of "priority-target content-quantitative standard-weight". Step S133 realizes the scientific allocation of weights through the analytic hierarchy process, solving the problem of fuzzy target priority and easy misallocation of optimization resources in the prior art; the structured system organization ensures that the optimizer large language model can clearly identify the importance of each target when the subsequent step S21 is converted into natural language description, avoiding the generation of prompt word candidates deviating from the core demand.

[0090] Step S131 relies on the initial performance benchmark report of step S123 to ensure that the positioning of the performance bottleneck dimension has quantitative data support and avoids subjective judgment; the correlation analysis of step S132 needs to be based on the multi-dimensional performance index original data of step S122 to ensure the accuracy of the judgment of the correlation between the secondary target and the core index; the weight allocation result of step S133 directly provides the weight labeling content for the optimization target module of step S21, so that the optimization target module can clearly guide the optimizer large language model to preferentially meet the high-weight target, and the consistency of the optimization direction and resource allocation is ensured by the cooperation of the two. The "strong correlation screening" of step S131 enables the main optimization target to accurately hit the core defect, realizing "one optimization and multiple benefits".

[0091] The challenging data set pool of S11 in step S10 provides a precise carrier for S12 testing, and the multi-dimensional performance index of S12 provides a basis for S13 target setting. The three form a "test carrier-performance baseline-optimization target" closed loop, and at the same time provide structured content for the optimization target module, task example module and historical learning module of the subsequent S20 meta prompt word, provide a clear direction for the Bayesian iterative optimization of S30, avoid the entire optimization process from the source to fall into blindness, and ensure that the subsequent prompt word optimization is accurate, efficient and executable.

[0092] In step S20, according to the hierarchical optimization target system and the challenging data set pool, the optimization target module, the task example module and the historical learning module of the meta prompt word are constructed; the optimization target module, the task example module and the historical learning module are assembled according to the predetermined format to generate a complete meta prompt word.

[0093] Step S20 aims to solve the technical problem of the lack of a systematic guidance framework in traditional prompt word design. In traditional design, prompt word generation relies heavily on human experience and lacks fixed structure support, resulting in generated prompt words that either do not focus on the core defects of the model or cannot balance multiple optimization targets, and even cause core performance deterioration due to the lack of constraints. At the same time, it is difficult to adapt to the optimization needs of different iteration stages. This step provides structured and dynamically updated generation guidance for the optimizer large language model through the technical link of "module disassembly-directional construction-sequential assembly", ensuring that the new prompt word candidate generated in subsequent step S31 can accurately anchor the cognitive weaknesses of the target large language model, and has the characteristics of traceability and iteration.

[0094] Further, step S20 includes:

[0095] In step S21, the main optimization target, secondary optimization target, constraint condition and target weight distribution in the hierarchical optimization target system are converted into structured natural language descriptions to form the optimization target module of the meta prompt word.

[0096] The implementation of step S21 needs to clarify the definition and boundary of the core elements in the hierarchical optimization target system: the main optimization target refers to the quantitative target set for the performance bottleneck dimension, which directly determines the core performance improvement; the secondary optimization target refers to the quantitative target set for the improvement potential dimension, which can be optimized without affecting the core performance; the constraint condition refers to the unbreakable threshold set to avoid the deterioration of the core indicators in the optimization process; and the target weight allocation refers to the weight value assigned according to the influence degree of each optimization target on the core performance. On this basis, the above elements are converted into structured natural language description, and the conversion process needs to follow the principles of "target quantification, constraint clarification, and weight clarity": for the main optimization target, it needs to include the complete information of "dimension name + current value + target value + improvement amplitude"; for the secondary optimization target, it needs to supplement the "association relationship with the core indicator", for example, "secondary optimization target: increase the recall rate to the current value + (ideal value - current value) x K'' or more, and the improvement process needs to meet the constraint threshold of accuracy rate not less than"; for the constraint condition, it needs to clarify "indicator name + constraint threshold"; and for the target weight allocation, it needs to explain "weight value + weight determination method". The structured natural language description needs to use itemized expression, each element is independent and clear in category, such as "1. Main optimization target:..." "2. Secondary optimization target:..." "3. Constraint condition:..." "4. Target weight allocation:...", to ensure that the optimizer large language model can directly analyze the priority and quantification requirements of each element.

[0097] Step S13 forms a hierarchical optimization target system through "bottleneck positioning-target quantification-priority allocation", which provides structured description raw materials for step S21. If the quantification target and weight allocation of step S13 are missing, the structured description of step S21 will lose precision and may have the problem of "only expressing the completeness of the improvement reasoning step without specific target value"; and step S21 converts the abstract data of the hierarchical optimization target system into natural language that can be understood by the optimizer large language model, so that the core optimization direction set by step S13 can be effectively executed, avoiding the deviation of optimization due to improper target expression form. In the prior art, the prompt word optimization target often exists in general expressions such as "improve model performance", and the optimizer cannot identify the core direction. Step S21 converts the abstract target into specific and measurable instructions through structured description, solves the problem of fuzzy optimization target, and clearly allocates weights and constraints to avoid the situation of resource mismatching to secondary targets or core performance deterioration due to no constraints in the optimization process, for example, when the main optimization target is "improve the completeness of the reasoning step" and the weight is the highest, the optimizer will prefer to generate prompt word candidates containing reasoning steps instructions, rather than putting resources into secondary targets with less impact on core performance.

[0098] Step S22, perform clustering analysis on the challenge dataset pool, select representative samples from each cluster, and organize the representative samples in order of increasing difficulty as the task example module of the meta prompt word;

[0099] The implementation of step S22 requires performing clustering analysis on the challenge dataset pool first. The clustering method selects an unsupervised clustering algorithm based on task type features and challenge indicators, such as the K-means clustering algorithm. The clustering dimensions include "task type features", "challenge comprehensive score", and "cognitive weakness category". The number of clusters needs to be determined according to the size of the challenge dataset pool and the number of cognitive weakness categories, ensuring that each cluster represents the model's cognitive weaknesses in a certain sub-task scenario. After clustering, representative samples are selected from each cluster. The selection criteria need to meet "covering the main task features in the cluster", "challenge comprehensive score at the upper-middle level in the cluster", and "sample format consistent with the test input sequence format in step S121", to avoid selecting extremely simple or extremely complex samples that lose the reference value of task examples. For example, if a cluster corresponds to the "spatial reasoning class-SVG path closed figure recognition" task, and the cluster contains 100 samples with challenge comprehensive scores distributed between 0.6 and 0.9, 5-8 samples with scores between 0.7 and 0.8 and covering different closed figures are selected as representative samples. Then, all representative samples are organized in order of increasing difficulty. The difficulty sorting basis is the challenge comprehensive score calculated in step S113. The lower the score, the lower the difficulty, and the higher the score, the higher the difficulty. After sorting, it needs to ensure that the score difference between adjacent samples is within a preset range, to avoid too large difficulty span that causes the optimizer unable to learn the task rules gradually.

[0100] The challenge data set pool constructed in step S11 focuses on the cognitive weaknesses of the target large language model, provides a high-quality clustering data source for step S22, and if a conventional data set is used, the representative samples after clustering may not be able to expose the weaknesses of the model; and step S22 converts the dispersed samples of the challenge data set pool into structured task examples through clustering and sorting, so that the data set constructed in step S11 can more efficiently provide reference for the optimizer and avoid the problem that the optimizer cannot identify the task mode due to unordered samples. In the prior art, the task examples of the prompt words are often randomly selected samples, which lack pertinence and order, and the optimizer cannot learn the rules for solving the weaknesses of the model from the examples. Step S22 ensures that the examples cover various sub-weakness scenarios through clustering, and through difficulty increasing sorting, the optimizer can gradually master the processing logic of complex tasks, for example, from simple "triangle SVG path recognition" examples to complex "polygon SVG path recognition" examples, the optimizer can gradually learn the inference logic of SVG path parsing, and then generate prompt word candidates that are more suitable for the weaknesses of the model. In addition, the task examples sorted by difficulty can also guide the optimizer to generate prompt words with "ladder guidance", that is, the prompt words contain simple to complex inference step instructions, so that the target large language model can gradually improve the ability to handle complex tasks, and through the difficulty gradient of the examples, the guiding level of the prompt words for the target large language model is indirectly improved, rather than just serving as an example reference.

[0101] Step S23, initialize the optimization trajectory record structure, take the initial prompt word as the first historical record item, associate the multi-dimensional performance indicators of the initial prompt word, design a dynamic update mechanism including historical prompt words, multi-dimensional performance indicators corresponding to the historical prompt words, and improvement differences, and construct a historical learning module of meta prompt words;

[0102] The implementation of step S23 requires the initialization of an optimization trajectory record structure, which is in tabular form and includes four core fields: "historical prompt ID", "historical prompt content", "associated multi-dimensional performance indicators", and "improvement difference". Each field is defined as follows: "historical prompt ID" is a unique identifier, numbered in the order of prompt generation. For example, if the initial prompt ID is P0, the first new prompt candidate ID is P1, and so on. "Historical prompt content" needs to record the prompt text completely, including the initial prompt and the new prompt candidates generated in step S31, to ensure traceability. "Associated multi-dimensional performance indicators" need to record the accuracy, recall rate, F1 score, and reasoning step integrity of the corresponding prompt on the challenging data set pool. The initial record is the test data of the initial prompt in step S122, and the test is dynamically supplemented with new prompt candidates in the future. "Improvement difference" is the quantitative difference in multi-dimensional performance indicators between the current prompt and the previous version of the prompt. The calculation method is "current indicator value - previous version indicator value". If it is the initial prompt, there is no improvement difference. For example, if the accuracy of the new prompt candidate P1 is 63%, and the accuracy of the initial prompt P0 is 60%, the accuracy item in the improvement difference of P1 is "+3%". The design of the dynamic update mechanism needs to be linked with the performance evaluation of step S32: when step S32 obtains the multi-dimensional performance score of the new prompt candidate, it automatically triggers the update of the historical learning module, fills in the ID, content, and performance indicators of the new prompt candidate into the optimization trajectory record structure, and calculates the improvement difference with the previous version. The update process needs to ensure the real-time and completeness of the data, and avoid missing any record of optimization iteration.

[0103] Step S23 cooperates with steps S12 and S32. The initial prompt word provided by step S12 and the multi-dimensional performance index are the first data items for optimizing the trajectory record structure, ensuring that the historical learning module has a clear optimization starting point. The performance evaluation data of step S32 provides a data source for the dynamic updating of the historical learning module, allowing the optimization trajectory to continuously accumulate during the iteration process and form a complete optimization history. In existing technologies, prompt word optimization lacks a history recording and analysis mechanism, and each optimization is an independent process that cannot draw on previous optimization experience, which can easily result in repeated attempts to ineffective strategies. Step S23 solves this deficiency by using the optimization trajectory record structure and dynamic updating mechanism. The optimizer large language model can identify effective optimization directions by analyzing the history of prompt words and improvement differences, avoiding the repeated generation of ineffective prompt words. In addition, the complete optimization trajectory also provides a basis for subsequent optimization effect review. By analyzing the trend of improvement differences, it can be determined whether the optimization has entered a bottleneck period. For example, when the improvement differences of three consecutive iterations are all less than a pre-set threshold (such as an absolute value of each index difference being less than 1%), it can be determined that the optimization is close to convergence and does not need to continue to invest resources. Through trend analysis of the historical trajectory, a quantitative basis is provided for the termination of optimization iteration, avoiding resource waste caused by ineffective iteration. Table 1 shows the functions and input and output of the three core modules of the meta prompt word.

[0104] Table 1 Functions and input and output of the three core modules of the meta prompt word

[0105]

[0106] Step S24 assembles the optimization target module, task example module, and historical learning module according to a predetermined format to generate a complete meta prompt word.

[0107] The implementation of step S24 requires assembling the optimization target module, task example module, and historical learning module according to a predetermined format. The design of the predetermined format needs to follow the principle of "logical progression and emphasis". The assembly order is "optimization target module → task example module → historical learning module". Clear module identifiers need to be added between each module, and the internal structure of each module needs to be maintained as determined in steps S21, S22, and S23, to ensure that the optimizer large language model can parse the core information in order: first, determine "what to optimize" through the optimization target module, then understand "what the task is like" through the task example module, and finally grasp "how to optimize previously" through the historical learning module, forming a complete information transmission link. During the assembly process, it is necessary to ensure that the content of each module is consistent, such as ensuring that the sample format of the task example module is consistent with the task type mentioned in the optimization target module, and the multi-dimensional performance index of the historical learning module is consistent with the index name in the optimization target module, to avoid confusion in the optimizer parsing due to inconsistent information between modules.

[0108] The three modules constructed in steps S21, S22 and S23 are the basis for assembly in step S24. If any module is missing, the complete meta prompt word will lose key information, the optimizer will have no direction if the optimization target module is missing, the optimizer will have no reference if the task example module is missing, and the optimizer will have no experience if the historical learning module is missing. Step S24 integrates the three modules into an organic whole through a predetermined format, so that the functions of each module are complementary. For example, the optimization target module clearly states “improve the integrity of the reasoning steps”, the task example module provides samples containing SVG path analysis, and the historical learning module shows “adding path analysis guidelines can improve reasoning integrity”. The three modules work together to enable the optimizer to generate a prompt word candidate that “contains SVG path analysis guidelines, fits task examples, and draws on historical experience”, which is better than the guiding effect of a single module. In the prior art, the components of the prompt word are often randomly stacked, and the optimizer cannot establish the association between the components. Step S24 solves the problem of disordered module assembly through a predetermined format, ensuring the efficiency and logic of information transmission. At the same time, the structured assembly form makes the meta prompt word maintainable. When the optimization target needs to be adjusted or the task examples need to be updated, only the corresponding module content needs to be modified, without the need to restructure the entire meta prompt word, reducing the subsequent maintenance cost.

[0109] Step S20 solves the technical problem of lack of systematic framework in traditional prompt word design by constructing the optimization target module, task example module and historical learning module and assembling them in order. Specifically, for the problem of vague optimization targets, the target is quantified, weighted and constrained through the structured description in step S21. For the problem of task examples lacking specificity and order, precise and hierarchical examples are provided through the clustering and difficulty sorting in step S22. For the problem of lack of optimization history tracking, experience is accumulated through the trajectory recording and dynamic updating in step S23. For the problem of disordered module assembly, the modules are integrated into an organic whole through the predetermined format in step S24.

[0110] The generated meta prompt word has reusability. When a target large language model is replaced, only a challenging data set pool and a hierarchical optimization target system need to be reconstructed according to the cognitive weaknesses of the new model, and the contents of the three modules are updated in step S20 to generate a meta prompt word adapted to the new model, without the need to redesign the overall framework of the meta prompt word, thereby greatly reducing the cost of cross-model optimization. The meta prompt word has explainability. The structured design of the three modules makes the generation logic of each prompt word candidate traceable. The optimization direction comes from the optimization target module, the task adaptation comes from the task example module, and the improvement idea comes from the historical learning module, thereby avoiding the “black box” problem of traditional prompt word generation and facilitating subsequent analysis of the causes of optimization effect. The step S23 historical learning module records a complete optimization trajectory, and the step S33 can construct a Gaussian process model of the Bayesian optimization framework based on the trajectory. If the historical learning module constructed in step S20 is missing, the Bayesian optimization will lack historical data support and cannot achieve efficient iterative convergence. Therefore, step S20 not only solves the problem of missing prompt word design framework, but also provides key data basis for the subsequent step S30 Bayesian optimization, ensuring the closed-loop nature and efficiency of the entire optimization scheme.

[0111] In step S30, the meta prompt word is used to drive the optimizer large language model to generate new prompt word candidates, the performance of the new prompt word candidates is evaluated on the challenging data set pool, and the meta prompt word is updated. Through the Bayesian optimization strategy, iterative improvement is performed until convergence, and the optimal prompt word is output.

[0112] Step S30 aims to use the complete meta prompt word generated in step S20 to drive the optimizer large language model to generate new prompt word candidates adapted to the hierarchical optimization target system. Performance evaluation is performed on the challenging data set pool, and automatic iterative improvement of the prompt word is realized through the Bayesian optimization strategy until the optimal prompt word is output. The core solves two technical defects of traditional manual optimization methods: one is low efficiency, manual design of prompt words requires repeated trial and error and relies on experience, and cannot adapt to the rapid optimization needs of complex cognitive weaknesses of large language models; the second is easy to fall into local optimum, manual optimization can only be based on limited experience adjustment, and it is difficult to explore the global optimal space of prompt word design, which may cause the optimization effect to stagnate due to neglecting key reasoning links. Through the technical closed loop of “instruction generation-performance verification-model iteration”, the advantages of OPRO algorithm and Bayesian optimization are combined, so that the prompt word optimization process has automatic, precise and global optimization capabilities, while ensuring that the optimization trajectory is traceable and the optimization effect is quantifiable.

[0113] Further, step S30 includes:

[0114] In step S31, the meta prompt word is input into the optimizer large language model to generate new prompt word candidates that meet the requirements of the hierarchical optimization target system;

[0115] Referring to Figure 4 As shown, further, step S31 comprises:

[0116] Step S311, parse the target weight distribution in the optimization target module, and generate prioritized target instructions;

[0117] Step S312, analyze the representative samples in the task example module, identify the task mode and difficulty characteristics, and generate a task understanding summary;

[0118] Step S313, extract all historical prompt words, multi-dimensional performance indicators corresponding to the historical prompt words, and improvement differences from the historical learning module, determine the effective optimization direction through comparative analysis;

[0119] Step S314, integrate the prioritized target instructions, task understanding summary, and effective optimization direction into generated instructions, input the generated instructions into the optimizer large language model, and generate new prompt word candidates that meet the requirements of the hierarchical optimization target system.

[0120] The implementation of step S31 requires the meta prompt word assembled in step S20 as input, through parsing the core information of each module of the meta prompt word and integrating it into generated instructions, guiding the optimizer large language model to generate new prompt word candidates that meet the requirements of the hierarchical optimization target system. Among them, "meet the requirements of the hierarchical optimization target system" means that the new prompt word candidate needs to meet: it has clear guiding strategies for the main optimization target, contains adaptive improvement measures for the secondary optimization target, and strictly follows the constraint conditions, and needs to match the task mode of the task example module and the effective optimization direction of the historical learning module.

[0121] The implementation of step S311 needs to analyze the target weight distribution in the optimization target module first. The analysis tool selects a natural language processing model with structured information extraction capability, such as a BERT-based text extraction model. The extracted dimensions include the main optimization target weight m1, the secondary optimization target weight m2, and the constraint condition weight m3. The extraction result is consistent with the weight value determined by the analytic hierarchy process in step S133. The analysis process needs to locate the entry expression of "target weight distribution" in the optimization target module first, and then extract the corresponding values through keyword matching, such as "main optimization target weight", "secondary optimization target weight", and "constraint condition weight". Subsequently, the weight size is converted into a prioritized target instruction based on the weight size. The generation of the prioritized target instruction needs to follow the principle of "the higher the weight, the higher the priority", which clearly defines the execution order and resource allocation proportion of each target. For example, "prioritized target instruction: 1. Allocate 60% of the optimization resources to achieve the main optimization target; 2. Allocate 30% of the optimization resources to achieve the secondary optimization target; 3. The remaining 10% of the resources is used to ensure that the constraint condition is met, and in any case, the constraint condition must not be broken." In the prior art, the prompt word generation often has no priority indication, and the optimizer may mismatch the resources to the secondary target which has less impact on the core performance, resulting in the failure to achieve the main optimization target. Step S311 solves this resource mismatch problem through weight analysis and priority instruction generation. The structured natural language description of step S21 provides a clear text carrier for weight analysis, avoiding analysis errors caused by chaotic target expression. Step S311 converts the weight data into an executable priority instruction for the optimizer, so that the target weight distribution set in step S21 can be truly implemented, and the optimization resources can be tilted towards the core target.

[0122] The implementation of step S312 requires in-depth analysis of representative samples in the task example module, and the analysis dimensions include task mode and difficulty characteristics. Task mode recognition needs to parse the "input-output" logic of representative samples through text structuring tools to extract the core operation process required to complete the task, for example, for the representative sample of "spatial reasoning class-SVG path recognition", the task mode is parsed as "parse SVG path command → judge the closure of the figure → count the number of path edges → identify geometric shapes"; difficulty characteristic recognition needs to associate the challenge comprehensive score calculated in step S113 to extract the difficulty correlation factors of each sample, for example, "a sample with a challenge comprehensive score S=0.7 contains only straight line commands and closed commands, and the difficulty characteristic is'single path command + simple figure'; a sample with S=0.9 contains curve commands and multiple closed paths, and the difficulty characteristic is 'complex path command + nested figure'. Based on the above analysis, a task understanding summary is generated, which needs to include "task core process + difficulty gradient characteristics + adaptation suggestions", for example, the task understanding summary is expressed as "task mode: for the SVG path recognition task, the process of 'parsing path commands → judging closure → counting edges → identifying shapes' needs to be executed; difficulty characteristics: the difficulty of the sample increases with the challenge comprehensive score from S low to S high, and the low difficulty sample contains single command, and the high difficulty sample contains complex command; adaptation suggestions: the prompt words need to contain step-by-step instructions, and the high difficulty sample is supplemented with curve command parsing rules". In the prior art, task examples are only used as simple references, and the optimizer cannot extract task rules from the examples, resulting in generated prompt words that cannot adapt to the unified needs of tasks of different difficulties. Step S312 analyzes the task mode and difficulty characteristics to make the optimizer accurately grasp the nature of the task and generate prompt word candidates with universality. Step S22 sorts the representative samples by difficulty to provide an ordered data source for difficulty characteristic recognition, avoiding the difficulty of extracting difficulty rules due to unordered samples; and the task understanding summary of step S312 converts the sample resources of step S22 into task logic that the optimizer can understand, ensuring that new prompt word candidates can cover task needs from low to high difficulty interval, rather than only adapting to a single difficulty sample.

[0123] The implementation of step S313 needs to extract complete optimization trajectory data from the historical learning module, and the extraction content includes all historical prompt words, multi-dimensional performance indicators corresponding to each historical prompt word, and improvement differences between adjacent prompt words. After extraction, comparative analysis is performed, and the analysis method includes two types: one is the correlation analysis of text features and performance indicators. By calculating the Pearson correlation coefficient of the prompt word text features and the multi-dimensional performance indicators, if the absolute value of the correlation coefficient of a certain text feature and the core indicator is ≥0.7 and is a positive value, it is determined that the feature is an effective optimization feature. The prompt word text features are, for example, whether to contain keywords such as “analyze SVG closed command” and “focus on small category samples”. The second is the trend analysis of the improvement difference. The change direction of the improvement difference of each indicator in continuous iteration is counted. If the improvement difference corresponding to a certain optimization direction is continuously positive, it is determined that the direction is an effective optimization direction. For example, the extracted historical data is shown in the following table. Through analysis, it is known that the correlation coefficient of the text feature “containing SVG closed command analysis instruction” and the reasoning step completeness is 0.85, and the improvement difference is continuously positive, so it is determined that “enhancing the explicit instruction of SVG path closed command analysis” is an effective optimization direction. The prompt word optimization in the prior art lacks historical data support, and each optimization is an independent attempt, which is easy to repeat invalid strategies, such as repeatedly adjusting expressions unrelated to performance. Step S313 compares and analyzes the historical trajectory, so that the optimizer can determine the effective direction based on the existing experience, and avoid invalid trial and error. Step S23 constructs an optimization trajectory record structure to provide a standardized format for data extraction, ensuring that the extracted historical prompt words, performance indicators and improvement differences have consistency and comparability; and the analysis result of step S313 provides a clear basis for the generation of subsequent iteration prompt words, so that the “learning” function of the historical learning module is truly implemented, rather than just serving as a data storage carrier.

[0124] The implementation of step S314 needs to integrate the prioritized target instruction, the task understanding summary and the effective optimization direction into a unified generated instruction. The integration logic follows the progressive relationship of "target-task-experience": first, the optimization priority is determined according to the prioritized target instruction; second, the task boundary and rule are defined according to the task understanding summary; and finally, the reusable improvement strategy is provided through the effective optimization direction, ensuring that the generated instruction is logically coherent and information complete. The integrated generated instruction needs to be in a natural language format that can be parsed by the optimizer large language model. After inputting the generated instruction into the optimizer large language model, the optimizer large language model can generate new prompt candidates based on the instruction. In the prior art, the instructions for generating prompts are often scattered and disordered, and the optimizer cannot establish the association between the target, task and experience, resulting in prompt candidates that only meet a single requirement. Step S314 integrates the structure to ensure that the generated instruction covers the priority, task rule and historical experience, ensuring that the new prompt candidate can adapt to the optimization requirements in multiple dimensions. The prioritized target instruction ensures that the candidate does not deviate from the core target, the task understanding summary ensures that the candidate adapts to the nature of the task, and the effective optimization direction ensures that the candidate has experience support. The three work together to avoid the "trade-off" problem in traditional optimization and achieve the unity of "target-task-experience".

[0125] Step S32, input the new prompt candidate into the target large language model, perform testing on the challenging data set pool, and obtain the multi-dimensional performance score of the new prompt candidate;

[0126] The implementation of step S32 needs to follow the test input sequence format established in step S121 to combine the new prompt candidate with each sample in the challenging data set pool into a new standard test sequence of "new prompt candidate + sample detailed information + output format requirement", ensuring that the test environment is consistent with the initial performance test in step S12 and avoiding performance score distortion due to input format differences. Then, the new standard test sequence is input into the target large language model one by one, the output response is recorded, and the multi-dimensional performance score of the new prompt candidate is calculated according to the multi-dimensional performance index calculation method defined in step S122. The performance index of the historical prompt needs to use the same quantization standard to ensure the accuracy of the subsequent improvement difference calculation. The challenging data set pool ensures that the test can accurately expose the model's cognitive weaknesses and avoid performance score distortion due to simple test carriers; the multi-dimensional performance index system ensures comprehensive evaluation of the candidate from four dimensions of "correctness-coverage-comprehensive performance-reasoning quality", avoiding optimization effect misjudgment caused by single index, for example, a candidate may have improved accuracy but decreased recall rate. Through multi-dimensional scoring, this problem can be discovered in time to avoid it being selected as an optimization item in subsequent iterations.

[0127] Step S33, updating the new prompt word candidate and the multi-dimensional performance score of the new prompt word candidate to the historical learning module, constructing a Gaussian process model under a Bayesian optimization framework according to the updated historical learning module, and obtaining the optimal prompt word according to the Gaussian process model.

[0128] The implementation of step S33 needs to be carried out in three stages: the first stage is the historical learning module update, the new prompt word candidate and its multi-dimensional performance score are supplemented to the historical learning module according to the optimization trajectory record structure defined in step S23, and the improvement difference between the new prompt word candidate and the previous version of the prompt word candidate is calculated to ensure the continuity and integrity of the historical trajectory; the second stage is the construction of the Gaussian process model under the Bayesian optimization framework, the input of the Gaussian process model is the text feature vector of the historical prompt word and the corresponding multi-dimensional performance score, the text feature vector is a high-dimensional vector converted from the prompt word by the word embedding model, the core function of the Gaussian process model is to fit the mapping relationship between the prompt word features and the performance scores based on the historical data, to predict the performance potential of the unexplored prompt word space by calculating the posterior probability distribution, and to realize the balance between exploration and utilization; the third stage is convergence judgment and optimal prompt word output, the convergence condition is set as "the multi-dimensional performance score improvement difference of continuous n times iteration is less than the preset convergence threshold T1", wherein the value of n is determined according to the optimization demand and the size of the data set, and the statistical significance needs to be ensured, and an exemplary value of n is 3; the setting of the convergence threshold T1 needs to refer to the performance fluctuation range of the target large language model on the same type of task to avoid judging the normal fluctuation as convergence, and an exemplary value of T is set as the absolute value of each index improvement difference is less than 1%. If the convergence condition is met, the candidate with the optimal multi-dimensional performance score is selected from the historical prompt word as the optimal prompt word; if the convergence condition is not met, the optimal feature combination predicted by the Gaussian process model is used as a guide to return to step S31 to generate a new prompt word candidate, and enter the next iteration.

[0129] The construction of the Gaussian process model needs to follow the core logic of Bayesian optimization: first, define a kernel function (such as a square exponential kernel function) to measure the similarity of different prompt feature vectors, the higher the similarity, the lower the uncertainty of the model's prediction of the performance of the new prompt; second, calculate the prior distribution based on historical data, and then update it to the posterior distribution combined with the performance score of the new candidate, and determine the exploration direction of the next iteration through the mean and variance of the posterior distribution, the area with high mean and low variance is preferred to be "used", the area with high mean and high variance is preferred to be "explored", and local optimization is avoided. For example, the model predicts that the prompt "contains SVG curve command Q analysis guide" has a high performance mean but a large variance, so the step S31 generates a candidate containing the guide to explore the new feature space. The traditional optimization method in the prior art, such as random search and grid search, can only blindly explore and cannot balance exploration and utilization based on historical data, which is easy to stagnate in a local optimal area. Step S33 solves the local optimal problem through the probability prediction ability of the Gaussian process model. The historical learning module provides sufficient training data for the Gaussian process model to ensure the fitting accuracy of the model; the prediction result of the Gaussian process model provides a clear feature improvement direction for step S31, avoiding the generation of candidates without basis in step S31, forming an iterative closed loop of "data accumulation-model prediction-candidate generation".

[0130] The step S30 solves the core problems of low efficiency and local optimum of traditional manual optimization through the technical chain of "generating instruction construction-performance verification-Bayesian iteration". For low efficiency, the manual trial and error is replaced by the automatic instruction generation of step S31 and the standardized performance evaluation of step S32, greatly shortening the optimization period. For local optimum, the global exploration and utilization balance is realized through the Gaussian process model of step S33, ensuring that the global optimal prompt word covering the core optimization target is found. The posterior variance of the Gaussian process model can predict the optimization potential in advance. If the performance mean of all unexplored regions predicted by the model after a certain iteration is lower than the current optimal candidate, the iteration can be terminated in advance without waiting for continuous n times of convergence, reducing the consumption of invalid resources. Since the entire optimization process is based on the general framework of "meta prompt-Bayesian optimization", when changing tasks, only the challenging dataset pool of step S11 and the hierarchical optimization target system of step S13 need to be updated, and the iteration logic of step S30 does not need to be adjusted, greatly reducing the adaptation cost of cross-task optimization. If step S30 is missing, the meta prompt generated by step S20 cannot be converted into actual optimization results, and the prompt word design remains in the "framework stage". At the same time, traditional manual optimization cannot meet the optimization needs of the complex cognitive weaknesses of large language models, resulting in the loss of the core execution link of the entire prompt optimization scheme. Step S31 ensures the accuracy of new prompt candidates, step S32 ensures the quantifiability of optimization effects, and step S33 ensures global optimization and convergence of the optimization process. The three work together to make the "test carrier-optimization target-meta prompt" system constructed in steps S10-S20 truly convert into optimal prompt words that can be landed, realizing the closed loop of the entire technical scheme.

[0131] Embodiment 2:

[0132] This embodiment is based on embodiment 1 and provides a Bayesian automatic prompt word optimization system based on meta prompt words, as shown in Figure 5 , which includes

[0133] A target system construction module is used to construct a challenging dataset pool for the cognitive weaknesses of a large language model, obtain an initial prompt word of the large language model, and construct a hierarchical optimization target system according to the initial prompt word and the challenging dataset pool.

[0134] A meta prompt word generation module is used to construct an optimization target module, a task example module, and a historical learning module of the meta prompt word according to the hierarchical optimization target system and the challenging dataset pool. The optimization target module, the task example module, and the historical learning module are assembled in a predetermined format to generate a complete meta prompt word.

[0135] The prompt word optimization module generates new prompt word candidates using the meta prompt word driven optimizer, evaluates the performance of the new prompt word candidates on the challenging dataset pool, and updates the meta prompt word, iteratively improves until convergence through a Bayesian optimization strategy, and outputs the optimal prompt word.

[0136] Further, in the target system construction module, the method of constructing the challenging dataset pool for the cognitive weaknesses of the target large language model comprises:

[0137] Step S111, receiving a plurality of candidate datasets and extracting the task type features of each candidate dataset, and identifying the cognitive weakness categories of the target large language model according to the task type features;

[0138] Step S112, based on the cognitive weakness categories, performing a benchmark test on each candidate dataset using the target large language model, and calculating the challenging indicators composed of error rate, response time and reasoning complexity;

[0139] Step S113, according to the challenging indicators, screening the candidate datasets to form the challenging dataset pool.

[0140] Further, in the target system construction module, the method of constructing the hierarchical optimization target system according to the initial prompt word and the challenging dataset pool comprises: obtaining the initial prompt word, testing the target large language model on the challenging dataset pool using the initial prompt word, and obtaining the multi-dimensional performance indicators; performing correlation analysis on the multi-dimensional performance indicators to generate an initial performance benchmark report; and generating a hierarchical optimization target system according to the initial performance benchmark report and the multi-dimensional performance indicators.

[0141] Further, the method of generating the hierarchical optimization target system comprises:

[0142] Step S131, based on the performance bottleneck dimension in the initial performance benchmark report, setting the main optimization target and the corresponding target improvement range;

[0143] Step S132, according to the improvement potential dimension, determining the secondary optimization target and the constraint condition;

[0144] Step S133, organizing the main optimization target, the secondary optimization target and the constraint condition according to the priority, and constructing a hierarchical optimization target system containing target weight allocation.

[0145] The methods and systems of the present application can be implemented in many ways. For example, the methods and systems of the present application can be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustration only and the steps of the methods of the present application are not limited to the above specifically described order unless otherwise specifically stated.

[0146] In addition, parts of the above technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of corresponding technical solutions in the prior art are not described in detail to avoid excessive repetition.

[0147] The specific embodiments described above are further explained in connection with the purposes, technical solutions, and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for Bayesian automatic prompt optimization based on meta-prompt, characterized in that, The method comprises: constructing a challenging dataset pool for the cognitive weaknesses of the target large language model, obtaining an initial prompt word of the target large language model, testing the target large language model on the challenging dataset pool using the initial prompt word, obtaining multi-dimensional performance indicators; performing correlation analysis on the multi-dimensional performance indicators to identify performance bottleneck dimensions and improvement potential dimensions, and generating an initial performance benchmark report; generating a hierarchical optimization target system according to the initial performance benchmark report and the multi-dimensional performance indicators; The method for constructing the challenging dataset pool comprises: receiving a plurality of candidate datasets, extracting the task type features of each candidate dataset, and identifying the cognitive weakness categories of the target large language model according to the task type features; performing benchmark testing on each candidate dataset using the target large language model based on the cognitive weakness categories, calculating the error rate, response time and reasoning complexity to obtain the challenging indicators; performing normalization processing on the error rate, response time and reasoning complexity to obtain the normalized error rate, normalized response time and normalized reasoning complexity; and performing weighted summation on the normalized error rate, normalized response time and normalized reasoning complexity to obtain the challenging comprehensive score; sorting all candidate datasets according to the challenging comprehensive score from high to low, and selecting the top K% of candidate datasets into the challenging dataset pool, wherein K is a percentage threshold parameter; According to the hierarchical optimization target system and the challenging dataset pool, an optimization target module, a task example module and a historical learning module of the meta prompt word are constructed; the optimization target module, the task example module and the historical learning module are assembled in a predetermined format according to the order of the optimization target module, the task example module and the historical learning module, and explicit module identifiers are added between each module to generate a complete meta prompt word; the hierarchical optimization target system comprises main optimization targets, secondary optimization targets, constraint conditions and target weights; The method for constructing the optimization target module, the task example module and the historical learning module comprises: converting the main optimization targets, the secondary optimization targets, the constraint conditions and the target weights in the hierarchical optimization target system into structured natural language descriptions to form the optimization target module of the meta prompt word; performing clustering analysis on the challenging dataset pool, selecting representative samples from each cluster, and organizing the representative samples into the task example module of the meta prompt word in ascending order of difficulty; initializing an optimization trajectory record structure, taking the initial prompt word as the first historical record item, associating the multi-dimensional performance indicators of the initial prompt word, designing a dynamic update mechanism including a historical prompt word, a multi-dimensional performance indicator corresponding to the historical prompt word and an improvement difference, and constructing the historical learning module of the meta prompt word; The meta prompt word is used to drive the optimizer large language model to generate a new prompt word candidate, the performance of the new prompt word candidate is evaluated on the challenging dataset pool, and the meta prompt word is updated, and the Bayesian optimization strategy is iteratively improved until convergence, and the optimal prompt word is output.

2. The Bayesian automatic prefix optimization method based on meta-prefixed according to claim 1, characterized in that, The method for obtaining the multi-dimensional performance indicators comprises: combining the initial prompt word with each data sample in the challenging dataset pool to form a complete test input sequence; The test input sequence is input into the target large language model, the output response of the target large language model is collected, and the multi-dimensional performance index is calculated; the multi-dimensional performance index includes accuracy, recall rate, F1 score and reasoning step integrity.

3. The Bayesian automatic prefix optimization method based on meta-prefixed according to claim 2, characterized in that, The performance bottleneck dimension identification method is that the index value of the multi-dimensional performance index is significantly lower than the ideal value, and is strongly correlated with the accuracy.

4. The Bayesian automatic prefix optimization method based on meta-prefixed according to claim 3, characterized in that, The determination criterion that the index value of the multi-dimension performance index is significantly lower than the ideal value is: assuming that the index value of the i-th index in the multi-dimension performance index is C i , the ideal value of the i-th index is C 0,i , if is greater than a preset proportion threshold value, it is determined that the i-th index is significantly lower than the ideal value.

5. The Bayesian meta-prompt word optimization method based on meta-prompt words according to claim 4, characterized in that, The strong correlation determination method with the accuracy is that the correlation coefficient of the i-th index and the accuracy is calculated, and if the absolute value of the correlation coefficient is greater than or equal to a preset strong correlation threshold, it is determined that the i-th index is strongly correlated with the accuracy.

6. The Bayesian metatoken-based automatic metatoken optimization method of claim 5, wherein, The generation method of the hierarchical optimization target system includes: Based on the performance bottleneck dimension in the initial performance benchmark report, set the main optimization target and the corresponding target improvement range; According to the improvement potential dimension, determine the secondary optimization target and the constraint condition; The main optimization target, the secondary optimization target and the constraint condition are organized according to the priority, and a hierarchical optimization target system containing target weight distribution is constructed.

Images