Automatic optimization method of large model prompt words based on distributed intelligent agent

Abstract

The present invention relates to the field of artificial intelligence technology, and in particular to a large-model prompt word automatic optimization method based on a distributed intelligent agent. The method obtains an initial score value sequence for each candidate prompt word in a candidate prompt word set; obtains a score consensus index and an evaluation influence value index of any candidate prompt word based on the difference in the initial score value sequence of any candidate prompt word; divides the candidate prompt word set into a high-consensus prompt word set and a low-consensus prompt word set based on the score consensus index and the evaluation influence value index of each candidate prompt word; performs adaptive scoring processing on the high-consensus prompt word set and the low-consensus prompt word set to obtain a final score for each candidate prompt word, so as to screen the optimal candidate prompt word in the candidate prompt word set and obtain the optimal candidate prompt word set. Through the adaptive evaluation and screening of the candidate prompt words, the resource utilization efficiency of the evaluation process is significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large model prompt word automatic optimization method based on distributed intelligent agents. Background Art

[0002] Automated optimization of prompt words to improve the performance of large language models on specific tasks is a current research hotspot. A representative method for automated prompt word optimization is the Automatic PromptEngineer (APE). The APE method treats the process of finding the optimal prompt word as a black-box optimization problem. Specifically, the APE method automatically generates and selects prompt words using a large language model (LLM) to optimize model performance. Its core process includes:

[0003] (1) Generate candidate prompt words: Use a large language model (as a "proposal model") to generate a set of candidate prompt words based on the task description and a small number of input and output examples.

[0004] (2) Candidate cue word scoring: For each candidate cue word, its effectiveness in guiding the large language model to complete the task needs to be evaluated. This step usually involves: a. Using the candidate cue word and a portion of the input from the training or validation data to drive the large language model to generate output; b. Calculating a scoring function to measure the quality of the output; c. Taking the average score of the candidate cue word on the evaluation data as its quality evaluation result.

[0005] (3) Candidate prompt word selection: Based on the calculated quality evaluation results, the candidate prompt word with the highest quality evaluation result is selected from the candidate prompt word set as the final output.

[0006] However, during the candidate cue word scoring stage, APE needs to calculate the quality assessment results of a large number of candidate cue words. This stage involves invoking a large language model and calculating scores, which consumes a large amount of computing resources. To improve efficiency, existing APE methods use a multi-stage filtering strategy based on preliminary scores on a small subset to reduce the number of candidate cue words that require comprehensive evaluation. However, while this multi-stage filtering strategy reduces the total number of candidate cue words that need to be evaluated, the core process of scoring individual candidate cue words still suffers from inefficient allocation of computing resources. For example, when evaluating the quality score of a candidate cue word, the strategy for determining the amount of evaluation resources to invest is fixed and fails to adaptively adjust based on information revealed during the evaluation process. If the scores on a small number of samples are highly consistent (regardless of the score), this indicates that the quality judgment has stabilized, and the need for further computing resources for repeated verification is reduced. Conversely, if the preliminary scores fluctuate significantly and exhibit significant uncertainty, it indicates that more resources are needed to obtain a reliable score. Due to the lack of a mechanism to dynamically adjust computing resource investment based on the evaluation status, the scoring process of the APE method easily leads to a waste of computing resources. As a result, for those candidate prompt words whose scores are sufficiently stable, unnecessary evaluation calculations are still performed on more samples. It is also impossible to prioritize computing resources for those candidate prompt words whose preliminary evaluation results show high uncertainty and are most in need of further precise evaluation. The existing scoring resource allocation strategy, due to its inherent non-adaptive nature, limits the efficiency of the overall optimization process and its scalability under resource-constrained conditions.

[0007] Therefore, how to optimize the candidate cue word scoring stage in the APE method to improve the adaptability of the scoring resource allocation strategy has become an urgent problem to be solved. Summary of the Invention

[0008] In view of this, an embodiment of the present invention provides a large model prompt word automatic optimization method based on distributed intelligent agents to solve the problem of how to optimize the candidate prompt word scoring stage in the APE method to improve the adaptability of the scoring resource allocation strategy.

[0009] An embodiment of the present invention provides a method for automatically optimizing large model prompt words based on a distributed intelligent agent, the method comprising the following steps:

[0010] In the candidate prompt word scoring process of APE, a candidate prompt word set generated based on the large model is obtained, and an initial score value sequence of each candidate prompt word in the candidate prompt word set is obtained based on a preset basic evaluation source set;

[0011] For any candidate prompt word in the candidate prompt word set, obtaining a score consensus index for the candidate prompt word based on the difference in initial score values in the initial score value sequence of the candidate prompt word; obtaining an evaluation influence value index for the candidate prompt word based on the mean and sample skewness of the initial score value sequence of the candidate prompt word, combined with the score consensus index;

[0012] dividing the candidate prompt word set into a high-consensus prompt word set and a low-consensus prompt word set based on the score consensus index and the evaluation influence value index of each candidate prompt word in the candidate prompt word set; performing adaptive scoring processing on the high-consensus prompt word set and the low-consensus prompt word set to obtain a final score for each candidate prompt word;

[0013] According to the final score of each candidate prompt word, the optimal candidate prompt word is screened in the candidate prompt word set to obtain the optimal candidate prompt word set.

[0014] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0015] The present invention uses a low-evaluation source to perform a preliminary evaluation of each candidate prompt word, obtaining a corresponding initial score value sequence. A score consensus index, calculated based on the score value differences of the initial score value sequence, is introduced to quantify the consistency of the candidate prompt word evaluation results. The higher the consistency, the more reliable the candidate prompt word's score, and the fewer evaluation resources should be invested. To structure the high-computing resource evaluation process for low-consensus candidate prompt words, an evaluation impact value index is introduced to characterize the potential impact of the candidate prompt word's accurate evaluation results on the final decision. The greater the impact potential, the more accurate the assessment of its true quality. Therefore, by combining the score consensus index and evaluation impact value index of each candidate prompt word, early decision-making is made under high consensus conditions, and necessary high-computing resource assessments under low consensus conditions are prioritized and filtered based on accurate scores. This optimizes the number of candidate prompt words that require comprehensive evaluation in the APE method, significantly improving the resource utilization efficiency of the candidate prompt word scoring stage in the APE method while ensuring the reliability of the evaluation results and the quality of the final output prompt word. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1This is a method flow chart of a method for automatically optimizing large model prompt words based on a distributed intelligent agent provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0018] The embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present disclosure, but should not be understood as limiting the present disclosure.

[0019] It should be noted that the terms "first," "second," and the like in the specification of the present disclosure and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure.

[0020] In order to illustrate the technical solution of the present invention, specific embodiments are provided below.

[0021] See also Figure 1 , is a flow chart of a method for automatically optimizing large model prompt words based on distributed intelligent agents provided in the first embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0022] Step S101 , in the candidate prompt word scoring process of APE, a candidate prompt word set generated based on a large model is obtained, and an initial scoring value sequence of each candidate prompt word in the candidate prompt word set is obtained according to a preset basic evaluation source set.

[0023] The APE algorithm uses an iterative Monte Carlo search strategy to search for and optimize candidate instructions, providing a more convenient and efficient interaction method for human users. In the APE algorithm, a large model (LLM) is used to generate candidate instructions and evaluate their potential through reasoning and semantic analysis. Researchers transform the instruction generation problem into a black-box optimization problem and introduce a scoring function to measure the fit between the dataset and the model-generated data. In this way, APE can effectively search and optimize candidate instructions, improving the efficiency and processing power of AI tasks.

[0024] The development of large-scale models has evolved from basic machine learning models to deep learning models, gradually acquiring powerful capabilities for understanding and generating natural language. As parameter size continues to increase, large models can more accurately capture subtle differences in language, enabling more intelligent tasks such as conversation, text generation, and translation. Specifically, these models relate to artificial intelligence technologies such as deep learning, natural language recognition, generative models, and large language models. These models are not only capable of processing large amounts of complex semantic information but also demonstrate superior performance in multi-task learning and knowledge reasoning, leading to their widespread application in various industries, such as finance and medicine. The core process of the APE algorithm includes generating candidate prompts, scoring candidate prompts, and selecting candidate prompts. Candidate prompts serve as a bridge for interaction between users and large models. During interaction with the large model, the design of candidate prompts directly affects the quality of the model output. A good candidate prompt can guide the large model to generate more accurate and predictable results, while an inappropriate candidate prompt can cause the generated content to deviate from the topic or produce misleading information. Therefore, the purpose of the present invention is to optimize the scoring of each candidate prompt during the candidate prompt scoring phase to achieve automatic optimization of prompts.

[0025] In the prior art, in order to improve efficiency, APE adopts a strategy of multi-stage filtering on a small subset based on preliminary scores to reduce the number of candidate prompt words that need to be fully evaluated. However, this method has insufficient efficiency in calculating evaluation resources. Therefore, the embodiment of the present invention dynamically adjusts the investment mechanism of evaluation resources by entering the scoring state to improve the adaptability of the scoring resource allocation strategy and reduce the waste of evaluation resources. Specifically, in the candidate prompt word scoring process of APE, a candidate prompt word set generated based on a large model is first obtained, wherein the candidate prompt word set is generated by the large model based on the task description given by any user. The form of the task description can be in any form, including but not limited to the following: direct task overview, task specification details with special requirements, and only input and output samples. It belongs to the prior art and will not be described in detail here. Then, a preset basic evaluation source set is used to perform a preliminary evaluation on each candidate prompt word in the candidate prompt word set to obtain a preliminary score summary of all task inputs of each candidate prompt word on the basic evaluation source set, that is, the initial score value sequence of each candidate prompt word in the candidate prompt word set, and the initial score value sequence of the i-th candidate prompt word is recorded as ,in, represents the initial score of the first basic evaluation source for the i-th candidate prompt word, represents the initial score of the first basic evaluation source for the i-th candidate prompt word, represents the initial score of the second basic evaluation source for the i-th candidate prompt word, represents the initial score of the third basic evaluation source for the i-th candidate prompt word, Represents the initial score value of the Nth basic evaluation source for the Nth candidate prompt word.

[0026] At this point, the initial score value sequence S of each candidate prompt word in the candidate prompt word set is obtained.

[0027] Step S102 : For any candidate prompt word in the candidate prompt word set, a rating consensus index of any candidate prompt word is obtained based on the difference in initial rating values in the initial rating value sequence of any candidate prompt word; and an evaluation impact value index of any candidate prompt word is obtained based on the mean and sample skewness of the initial rating value sequence of any candidate prompt word, combined with the rating consensus index.

[0028] Since the APE algorithm fails to dynamically adjust the subsequent evaluation source input according to the stability of the initial evaluation results during the candidate prompt word scoring process, the embodiment of the present invention analyzes the discrete degree of the initial score value sequence of each candidate prompt word in the candidate prompt word set to quantify the stability of the initial evaluation result of each candidate prompt word, and then introduces the scoring consensus index. , used to distinguish whether the evaluation status of different candidate cue words is stable or highly uncertain. This refers to the level of consensus on the quality judgment of each candidate cue word in the initial evaluation. The higher the consensus level, the more stable the candidate cue word evaluation status. When the consensus level is high, further investing a large number of evaluation resources in a more refined or extensive evaluation is expected to bring less information gain (i.e., the degree of change in the final judgment). Therefore, identifying this high consensus level is a key basis for subsequent decisions to reduce or cease further evaluation of the candidate cue word, thereby conserving evaluation resources. Conversely, a low consensus level indicates significant inconsistency or uncertainty in the initial rating sequence. This may be due to significant variability in the performance of the candidate cue word in different scenarios, or due to the inherent instability of the initial evaluation method. Therefore, in this low consensus situation, the average score of the initial rating sequence cannot reliably reflect the true quality of the candidate cue word, and decisions based on this score are risky. Identifying this low consensus state is also crucial, as it clearly indicates which candidate cue word quality judgments are unclear and require more computational resources to obtain a more reliable score.

[0029] Based on the above analysis, in this embodiment of the present invention, the i-th candidate prompt word in the candidate prompt word set is used as any candidate prompt word, and then the score consensus index of the i-th candidate prompt word is obtained based on the difference in the initial score value sequence of the i-th candidate prompt word. The specific acquisition is as follows:

[0030] First, to measure the consistency of the ratings, it is necessary to quantify the degree of dispersion of the initial rating value sequence of the i-th candidate prompt word. The standard deviation is a standard measure used to describe the dispersion between data points and their average value. Therefore, the standard deviation of the initial rating value sequence of the i-th candidate prompt word is calculated. The size of the standard deviation directly reflects the absolute degree of dispersion of the initial rating values in the initial rating value sequence. The larger the standard deviation, the more dispersed the elements of the initial rating value sequence and the worse the consistency.

[0031] Since the scores of different task evaluation indicators may have different scale ranges, directly using the absolute standard deviation for comparison across prompt words or tasks will produce deviations. In order to eliminate this scale effect and obtain a standardized and comparable discrete degree index, the standard deviation needs to be normalized. The embodiment of the present invention uses the standard deviation to be normalized relative to the theoretical maximum value R of the score, and sets the theoretical maximum value R of the score value to 100. There is no restriction here, and it can also be 1 and 10, etc. The maximum value is selected according to the actual scoring scale. The theoretical maximum value R of the score value is then used to normalize the standard deviation. The corresponding normalization calculation formula is:

[0032]

[0033] in, Indicates the relative discreteness of the i-th candidate prompt word, represents the standard deviation of the initial score value sequence of the i-th candidate prompt word, Indicates the theoretical maximum value of the rating, Indicates the preset parameter, used to prevent the denominator from being zero, preferably set .

[0034] The relative discreteness provides a discreteness measure around [0, 1] that is independent of the original rating scale. The smaller the value, the more concentrated the initial rating values of the initial rating value sequence of the i-th candidate prompt word are, and the higher the consistency. After obtaining the relative discreteness of the i-th candidate prompt word, the relative discreteness is converted into an intuitive evaluation consensus index. The evaluation consensus index should be inversely proportional to the relative discreteness, that is, the smaller the relative discreteness, the higher the evaluation consensus index. Due to the exponential decay function It has the characteristic that when the independent variable x is 0, the value is 1, and it decays smoothly to 0 as x increases. Therefore, in the embodiment of the present invention, the calculation formula of the evaluation consensus index of the i-th candidate prompt word is defined as:

[0035]

[0036] in, represents the evaluation consensus index of the i-th candidate prompt word, represents a preset positive scalar tuning parameter that controls the sensitivity of the consensus index to the dispersion of the ratings. Indicates the relative discreteness of the i-th candidate prompt word.

[0037] Preferably, in the embodiment of the present invention, The larger the value is, the faster the evaluation consensus index decreases as the dispersion of ratings increases, that is, the heavier the "penalty" for inconsistent ratings.

[0038] It should be noted that the calculation formula of the evaluation consensus index of the i-th candidate prompt word ensures that the initial score values in the initial score value sequence of the i-th candidate prompt word are completely consistent ( )hour, (highest consensus); as the relative dispersion increases ( Increase), The consensus index decreases smoothly and approaches 0 (low consensus). The consensus index provides a key basis for quantifying the reliability of preliminary evaluation results, directly solving the problem of the lack of such a judgment mechanism in existing technologies and forming the basis for implementing subsequent adaptive evaluation resource allocation strategies.

[0039] By using the above-mentioned method for obtaining the evaluation consensus index, the scoring consensus index of each candidate prompt word in the candidate prompt word set can be obtained, thereby identifying candidate prompt words with inconsistent preliminary evaluation results, that is, candidate prompt words with low scoring consensus indexes, so as to further evaluate the candidate prompt words with low scoring consensus indexes to obtain reliable scores. Therefore, the embodiment of the present invention needs to determine a priority order for processing candidate prompt words with low scoring consensus indexes, and then introduce the evaluation impact value index. , which is used to identify candidate prompt words that have a significant impact on the final sorting or screening decision. The candidate prompt words with the highest decision-making criticality refer to those whose preliminary evaluation results show high uncertainty (low scoring consensus index), but the mean of the initial scoring value sequence happens to be in the key area that affects the final screening decision. Accurately evaluating these candidate prompt words has the greatest potential influence on the correct final classification or sorting.

[0040] Therefore, this embodiment of the present invention quantifies the decision-making criticality of accurate evaluation for each candidate prompt word, that is, the potential impact of the accurate evaluation result on the final screening decision, providing a basis for processing samples in order of decreasing information value. For the i-th candidate prompt word, the evaluation impact value index of the i-th candidate prompt word is obtained based on the mean and sample skewness of the initial score value sequence of the i-th candidate prompt word, combined with the score consensus index. The specific calculation is as follows:

[0041] The decision-making criticality of a candidate prompt word is directly related to the uncertainty of its initial evaluation state. The more inconsistent the initial evaluation results are (the lower the score consensus index is), the more ambiguous the judgment of its true quality is, and the stronger the need for accurate evaluation to clarify this ambiguous state is. Therefore, it is necessary to convert the score consensus index of the i-th candidate prompt word into an indicator to measure uncertainty. The specific measurement method is: according to the difference between the constant 1 and the score consensus index, the uncertainty index of the i-th candidate prompt word is obtained, that is, the uncertainty index of the i-th candidate prompt word , the smaller the scoring consensus index, the greater the degree of uncertainty.

[0042] In order to more precisely reflect the reality and increase the ability to express uncertainty, the embodiment of the present invention uses the uncertainty index of the i-th candidate prompt word to Apply an exponent greater than 1 Adjustments are made to emphasize the contribution of extremely high uncertainty to the final screening decision more significantly. The adjustment method is to use the uncertainty index is the base, the default value is the exponential function value corresponding to the index, recorded as the uncertainty contribution degree of the i-th candidate prompt word, that is, the uncertainty contribution degree of the i-th candidate prompt word , preferably, set to ensure that situations of extreme uncertainty receive higher attention.

[0043] However, high uncertainty alone is not enough to constitute the highest priority. It is also necessary to consider whether the initial score result of the candidate prompt word falls within a key area that is sensitive to the final screening decision. Therefore, in this embodiment of the present invention, the mean of the initial score value sequence of the i-th candidate prompt word is obtained. , used to represent the initial scoring result of the i-th candidate prompt word, because the final screening decision depends on a key decision threshold for distinguishing between good and bad , preferred settings ,when Very close Even if Only a small difference may lead to a change in the final screening decision. This state close to the critical decision threshold is a high decision sensitivity. Therefore, the critical decision threshold is used to , for the mean Quantify the sensitivity so that The sensitivity is the greatest when keep away And decrease, since the properties of the one-dimensional Gaussian function meet this requirement, the basic decision sensitivity is set ,in, Indicates the width of the basic sensitive area, set , the smaller the value, the more sensitive it is, indicating that only scores that are very close to the critical decision threshold are considered to be highly sensitive.

[0044] But when the initial assessment results are extremely unreliable ( High), the fixed basic sensitive area width It may be too narrow, causing some candidate prompt words that are actually extremely uncertain but slightly deviate from the critical decision threshold to be prematurely downgraded. Therefore, it is necessary to dynamically increase the width of the sensitive area with the uncertainty index of the candidate prompt word. The formula for dynamically adjusting the width of the basic sensitive area according to the uncertainty index of the i-th candidate prompt word is:

[0045]

[0046] in, Indicates the width of the sensitive area of the i-th candidate prompt word, Indicates the width of the basic sensitive area, 1 indicates a constant, A non-negative parameter representing the width of the sensitive region affected by control uncertainty, Represents the uncertainty indicator of the i-th candidate prompt word.

[0047] It should be noted that Used to dynamically adjust the scope of decision-sensitive areas according to uncertainty, setting , Indicates that no dynamic adjustment is performed.

[0048] After dynamically adjusting the width of the basic sensitive area using the uncertainty index of the i-th candidate cue word, the decision sensitivity of the i-th candidate cue word is recalculated so that the decision sensitivity evaluation can adapt to different uncertainty levels. The calculation formula of the decision sensitivity of the i-th candidate cue word is:

[0049]

[0050] in, represents the decision sensitivity of the i-th candidate prompt word, represents an exponential function with a natural constant as base, represents the mean of the initial score value sequence of the i-th candidate prompt word, Indicates the preset critical decision threshold.

[0051] Considering that the decision sensitivity of the i-th candidate prompt word only considers the mean and key decision thresholds The distance between them is not comprehensive enough. The distribution skewness of the initial score value sequence also contains information. If the score distribution is highly asymmetric (sample skewness The absolute value of near , which also means that most of the scores are actually biased towards one side. At this time, the actual risk of crossing the key decision threshold due to randomness is reduced. Therefore, an adjustment factor that decreases as the absolute value of the sample skewness increases is introduced to correct the decision sensitivity of the i-th candidate prompt word. Specifically: the sample skewness of the initial score value sequence of the i-th candidate prompt word is obtained. Sample skewness belongs to the prior art and will not be described in detail here. The absolute value of the sample skewness is used as the independent variable of an exponential function with a natural constant as the base to obtain the corresponding function value. The reciprocal of the sum of the function value and the constant 1 is used as the adjustment factor. The product of the adjustment factor and the decision sensitivity is used as the sensitivity contribution degree of the i-th candidate prompt word.

[0052] The calculation formula of the adjustment factor is: , represents the adjustment factor, 1 represents a constant, represents an exponential function with a natural constant as base, It represents a positive parameter that controls the sensitivity of sample skewness and is set to 1. represents the sample skewness of the initial score value sequence of the i-th candidate prompt word, and || represents the absolute value symbol.

[0053] Among them, the calculation formula for the sensitivity contribution of the i-th candidate prompt word is: , Indicates the sensitivity contribution of the i-th candidate prompt word.

[0054] Finally, the i-th candidate cue word's uncertainty contribution and sensitivity contribution are combined to obtain its impact value index. Taking into account the different scales and extreme value effects of the two, a weighted geometric mean is taken of the uncertainty contribution and sensitivity contribution of the i-th candidate cue word, and the resulting weighted geometric mean is used as the impact value index of the i-th candidate cue word.

[0055] The calculation formula for weighted geometric mean processing is:

[0056]

[0057] in, represents the evaluation impact value index of the i-th candidate prompt word, Indicates the sensitivity contribution of the i-th candidate prompt word, Indicates the uncertainty contribution of the i-th candidate prompt word, represents the sensitivity weight, represents the uncertainty weight.

[0058] It should be noted that setting , which is not limited here. The evaluation impact value index of the i-th candidate prompt word more accurately and comprehensively quantifies the relative priority of the high evaluation source for each candidate prompt word, providing a reliable basis for subsequent orderly evaluation. The higher the value of the candidate cue word, the greater the potential for its accurate evaluation result to influence the final screening decision, ensuring that the evaluation process is carried out in a structured manner that reflects the inherent screening decision importance of each candidate cue word.

[0059] At this point, the score consensus index and evaluation influence value index of the i-th candidate prompt word in the candidate prompt word set can be obtained. Similarly, the score consensus index and evaluation influence value index of each candidate prompt word in the candidate prompt word set can be obtained.

[0060] In step S103, the candidate prompt word set is divided into a high-consensus prompt word set and a low-consensus prompt word set based on the score consensus index and the evaluation influence value index of each candidate prompt word in the candidate prompt word set; the high-consensus prompt word set and the low-consensus prompt word set are adaptively scored to obtain a final score for each candidate prompt word.

[0061] After obtaining the score consensus index and evaluation impact value index of each candidate prompt word in the candidate prompt word set, the score consensus index of each candidate prompt word is used to classify the candidate prompt words to distinguish which candidate prompt words have sufficiently reliable preliminary evaluation results and which candidate prompt words need subsequent high computing resource evaluation. , obtain the preset high consensus threshold ,like , then the preliminary evaluation result of the i-th candidate prompt word is determined to have high credibility, and the i-th candidate prompt word can be classified into the high consensus prompt word set Therefore, the candidate prompt words in the candidate prompt word set whose score consensus index is greater than or equal to the high consensus threshold are combined into a high consensus prompt word set; at the same time, for the candidate prompt words whose score consensus index is less than the high consensus threshold, they are recorded as remaining prompt words, indicating that the consistency of their preliminary evaluation results is insufficient and the reliability of their quality evaluation is poor, and they need to be further re-evaluated by a high evaluation source. Therefore, according to the evaluation impact value index of each remaining prompt word, all the remaining prompt words are sorted in descending order to obtain a low consensus prompt word set. , and the sorting order reflects the criticality of the remaining prompt words to the screening decision.

[0062] Furthermore, after the candidate prompt word set is divided into a high-consensus prompt word set and a low-consensus prompt word set, the high-consensus prompt word set and the low-consensus prompt word set can be adaptively scored to obtain the final score of each candidate prompt word. The specific method is as follows: for any candidate prompt word in the high-consensus prompt word set, since the high-consensus prompt word set belongs to the candidate prompt word with a preliminary evaluation result of being sufficiently credible, the mean of the initial score value sequence of any candidate prompt word can be obtained. As the final score; and for any candidate prompt word in the low-consensus prompt word set, it is necessary to perform a high-computing-resource evaluation to obtain a final score, so as to obtain a score that is more accurate and comprehensive than the initial evaluation result. In this embodiment of the present invention, the score of any candidate prompt word is obtained based on all evaluation sources of the large model and recorded as the final score.

[0063] It should be noted that the scoring of each candidate cue word in the low-consensus cue word set, obtained from all evaluation sources based on the large model, is a high-computing resource evaluation performed strictly according to the ranking results. This high-computing resource evaluation is consistent with the evaluation process based on the complete training dataset used in the APE algorithm to ultimately select the optimal cue word. Specifically, for any candidate cue word P in the low-consensus cue word set, the same large model used in the initial evaluation in step S101 is used, but this time the evaluation is performed on the complete training dataset. For each task input and expected output pair in the complete training dataset, the candidate cue word P is combined with the task input and then provided to the large model to obtain the generated output. A high-quality scoring function relevant to the final optimization objective is then used (this embodiment uses the execution accuracy commonly used in APE, which compares whether the model output is completely consistent with the expected output) to calculate the score of the candidate cue word P for each task input-output pair. Finally, the scores obtained for the candidate cue word P across all these task input-output pairs are aggregated (this embodiment uses the arithmetic average commonly used in APE) to obtain the average score, which is recorded as the final score. It should be noted that the above process belongs to the existing technology and will not be described in detail here.

[0064] In other implementations, a large model with low computing resources may be used to perform an initial quality assessment on all candidate prompt words, and a large model with high computing resources may be used to perform a quality assessment on each candidate prompt word in the low-consensus prompt word set.

[0065] At this point, the final score of each candidate prompt word in the candidate prompt word set is obtained.

[0066] Step S104 : Screening the optimal candidate prompt words in the candidate prompt word set according to the final score of each candidate prompt word to obtain the optimal candidate prompt word set.

[0067] Because the final score of each candidate prompt word in the candidate prompt word set is the most reliable quality metric, in this embodiment of the present invention, optimal candidate prompt words are screened from the candidate prompt word set based on the final score of each candidate prompt word. Specifically, a preset low-score threshold is obtained for any candidate prompt word in the candidate prompt word set. If the final score of any candidate prompt word is greater than the low-score threshold, the candidate prompt word is marked as the optimal candidate prompt word. All optimal candidate prompt words in the candidate prompt word set are obtained to form the optimal candidate prompt word set.

[0068] In one embodiment, a preset low score threshold is obtained. and high score threshold ,Right now , for high consensus prompt word set If the final score of any candidate prompt word is greater than or equal to 0.9, the candidate prompt word is marked as "high quality retention", and for candidate prompt words with a final score between 0.7 and 0.9, they are also marked as "high quality retention". , it is necessary to determine whether each candidate prompt word meets the minimum performance requirements, so each candidate prompt word is compared with the low score threshold. If the final score of any candidate prompt word is less than or equal to 0.7, it is determined that the performance of the candidate prompt word is still obviously too low even after accurate evaluation, and it is marked as "low-quality elimination". For candidate prompt words with a final score greater than 0.7, they meet the minimum performance requirements and should be retained. Therefore, they are marked as "high-quality retention". Then, the candidate prompt words marked as "high-quality retention" in the high-consensus prompt word set and the low-consensus prompt word set are taken as the optimal candidate prompt words, thereby forming the optimal candidate prompt word set.

[0069] After obtaining the optimal candidate prompt word set, the candidate prompt word with the highest final score is selected as the final output based on the final score of each candidate prompt word in the optimal candidate prompt word set. This process belongs to the existing technology in the APE algorithm and will not be repeated here.

[0070] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A large model prompt word automatic optimization method based on distributed intelligent agents, characterized by: The method comprises: In the candidate prompt word scoring process of APE, a candidate prompt word set generated based on the large model is obtained, and an initial score value sequence of each candidate prompt word in the candidate prompt word set is obtained based on a preset basic evaluation source set; For any candidate prompt word in the candidate prompt word set, obtaining a score consensus index for the candidate prompt word based on the difference in initial score values in the initial score value sequence of the candidate prompt word; obtaining an evaluation influence value index for the candidate prompt word based on the mean and sample skewness of the initial score value sequence of the candidate prompt word, combined with the score consensus index; dividing the candidate prompt word set into a high-consensus prompt word set and a low-consensus prompt word set based on the score consensus index and the evaluation influence value index of each candidate prompt word in the candidate prompt word set; performing adaptive scoring processing on the high-consensus prompt word set and the low-consensus prompt word set to obtain a final score for each candidate prompt word; According to the final score of each candidate prompt word, the optimal candidate prompt word is screened in the candidate prompt word set to obtain the optimal candidate prompt word set; The step of obtaining a score consensus index of any candidate prompt word according to a difference in initial score values in the initial score value sequence of any candidate prompt word includes: Calculating the standard deviation of the initial score value sequence of any candidate prompt word, normalizing the standard deviation using a preset theoretical maximum score value to obtain a relative dispersion degree of any candidate prompt word, and using the negation of the product of a preset positive scalar adjustment parameter and the relative dispersion degree as the independent variable of an exponential function with a natural constant as the base to obtain a score consensus index for any candidate prompt word; The step of obtaining the evaluation impact value index of any candidate prompt word based on the mean and sample skewness of the initial score value sequence of any candidate prompt word and combining the score consensus index includes: The uncertainty index of any candidate prompt word is obtained based on the difference between the constant 1 and the score consensus index. The uncertainty index is used as the base and the preset value is the exponential function value corresponding to the exponent, which is recorded as the uncertainty contribution degree of any candidate prompt word. Obtaining the mean of the initial score value sequence of any candidate prompt word, and using a preset key decision threshold to perform sensitivity quantification on the mean to obtain the decision sensitivity of any candidate prompt word; obtaining the sample skewness of the initial score value sequence of any candidate prompt word, and using the sample skewness to correct the decision sensitivity to obtain the sensitivity contribution degree of any candidate prompt word; The uncertainty contribution degree and the sensitivity contribution degree of any candidate prompt word are combined to obtain an evaluation impact value index of any candidate prompt word.

2. The method for automatically optimizing large model prompt words based on distributed intelligent agents according to claim 1, characterized in that: The method of using a preset key decision threshold to quantify the sensitivity of the mean value to obtain the decision sensitivity of any candidate prompt word includes: ； ； in, represents the decision sensitivity of the i-th candidate prompt word, represents an exponential function with a natural constant as base, represents the mean of the initial score value sequence of the i-th candidate prompt word, Indicates the preset critical decision threshold, Indicates the width of the sensitive area of the i-th candidate prompt word, Indicates the width of the basic sensitive area, 1 indicates a constant, A non-negative parameter representing the width of the sensitive region affected by control uncertainty, Represents the uncertainty indicator of the i-th candidate prompt word.

3. The method for automatically optimizing large model prompt words based on distributed intelligent agents according to claim 1, characterized in that: The method of using the sample skewness to correct the decision sensitivity to obtain the sensitivity contribution of any candidate prompt word includes: The absolute value of the sample skewness is used as the independent variable of an exponential function with a natural constant as the base to obtain a corresponding function value, the reciprocal of the sum of the function value and the constant 1 is used as an adjustment factor, and the product of the adjustment factor and the decision sensitivity is used as the sensitivity contribution degree of any candidate prompt word.

4. The method for automatically optimizing large model prompt words based on distributed intelligent agents according to claim 1, characterized in that: Combining the uncertainty contribution degree and the sensitivity contribution degree of any candidate prompt word to obtain the evaluation impact value index of any candidate prompt word includes: A weighted geometric mean process is performed on the uncertainty contribution degree and the sensitivity contribution degree of any candidate prompt word, and the obtained weighted geometric mean is used as the evaluation impact value index of any candidate prompt word.

5. The method for automatically optimizing large model prompt words based on distributed intelligent agents according to claim 1, characterized in that: The step of dividing the candidate prompt word set into a high-consensus prompt word set and a low-consensus prompt word set according to the score consensus index and the evaluation impact value index of each candidate prompt word in the candidate prompt word set includes: Obtaining a preset high consensus threshold, and forming a high consensus prompt word set from the candidate prompt word set with a score consensus index greater than or equal to the high consensus threshold; According to the evaluation influence value indexes of the candidate prompt words in the candidate prompt word set whose score consensus index is less than the high consensus threshold, the candidate prompt words with score consensus index less than the high consensus threshold are sorted in descending order to obtain a low consensus prompt word set.

6. The method for automatically optimizing large model prompt words based on distributed intelligent agents according to claim 1, characterized in that: The adaptive scoring process is performed on the high-consensus prompt word set and the low-consensus prompt word set to obtain a final score for each candidate prompt word, including: For any candidate prompt word in the high-consensus prompt word set, the average of the initial score value sequence of any candidate prompt word is used as the final score; for any candidate prompt word in the low-consensus prompt word set, the score of any candidate prompt word is obtained based on all evaluation sources of the large model and recorded as the final score.

7. The method for automatically optimizing large model prompt words based on distributed intelligent agents according to claim 1, characterized in that: The optimal candidate prompt word set is screened based on the final score of each candidate prompt word to obtain the optimal candidate prompt word set, including: For any candidate prompt word in the candidate prompt word set, a preset low score threshold is obtained. If the final score of any candidate prompt word is greater than the low score threshold, the candidate prompt word is marked as an optimal candidate prompt word. All optimal candidate prompt words in the candidate prompt word set are obtained to form an optimal candidate prompt word set.

Images