Autonomous evolution method and device of large language model, electronic equipment and storage medium

Abstract

The application relates to a large language model autonomous evolution method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring current environment information and a global strategy; inputting the current environment information and the global strategy into a multi-model set; generating an action instruction based on the current environment information and the global strategy by each large language model, and generating an action instruction based on the current environment information and a mathematical model by each reinforcement learning algorithm; selecting an optimal instruction with the highest accuracy from all the action instructions output by the multi-model set, updating the global strategy based on the optimal instruction, and completing one evolution cycle of the global strategy; based on the updated global strategy and the current environment information, the multi-model set is used to generate the action instruction again, and the steps of selecting the optimal instruction and updating the global strategy are repeatedly executed until a preset cycle termination condition is reached. The application improves the efficiency and stability of the large language model evolution process.

Description

Technical Field

[0001] This application relates to the field of large model technology, and in particular to a method, apparatus, electronic device and storage medium for the autonomous evolution of a large language model. Background Technology

[0002] In the field of autonomous decision-making and policy evolution for intelligent agents, existing methods based on Large Language Models (LLMs) generally suffer from technical drawbacks, such as reliance on human intervention and difficulty in achieving fully autonomous evolution. Especially in specialized scenarios, such as medical consultations and knowledge-based question answering, LLMs cannot autonomously acquire external authoritative knowledge or independently verify the correctness of decisions. Their policy optimization process often requires human intervention for feedback and correction to ensure the accuracy of decisions.

[0003] To reduce human intervention, existing technologies often employ a single agent architecture, relying on a single large language model for policy iteration and environmental interaction, improving decision-making effectiveness through iterative policy correction. However, this approach entirely depends on a single model for judgment and iteration. On one hand, the decision-making process is entirely dictated by the single large language model, which is susceptible to its own knowledge illusions and logical biases, resulting in the output of erroneous instructions. On the other hand, the system directly iterates the policy based on these erroneous instructions, leading to the continuous accumulation and amplification of biases. Ultimately, this causes the large language model to deviate from the correct direction during the autonomous policy evolution process, resulting in low efficiency and poor stability during autonomous evolution. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for the autonomous evolution of large language models, in order to solve the problems of low efficiency and poor stability in the autonomous evolution process of large language models.

[0005] Firstly, this application provides an autonomous evolution method for a large language model, the method comprising: Obtain current environment information and global strategy, wherein the current environment information is real-time interaction status data in the task environment, and the global strategy is a unified behavioral guideline for the large language model to perform decision-making operations; The current environment information and the global policy are input into a multi-model set, which includes multiple large language models and multiple reinforcement learning algorithms. Each of the large language models generates action instructions based on the current environment information and the global policy, and each of the reinforcement learning algorithms generates action instructions based on the current environment information and the mathematical model. The accuracy of the action instructions generated by the reinforcement learning algorithm is higher than a preset accuracy threshold. The action instructions are used to indicate the decision-making operations that need to be performed to complete the goal in the current task environment. The optimal instruction with the highest accuracy is selected from all action instructions output by the multi-model set, and the global policy is updated based on the optimal instruction to complete one evolution cycle of the global policy. Based on the updated global strategy and current environment information, action instructions are regenerated through the multi-model set. The steps of optimal instruction selection and global strategy update are repeated until the preset loop termination condition is reached. The autonomous evolution of the large language model is achieved through the autonomous evolution of the global strategy.

[0006] Optionally, selecting the optimal instruction with the highest accuracy from all action instructions output by the multi-model set includes: All action commands output by the multi-model set are aggregated to form an action command set; Each large language model and each reinforcement learning algorithm in the multi-model set is used to score each non-self-generated action instruction in the action instruction set according to a preset scoring rule; The final score of each action instruction is obtained based on all the scores corresponding to each action instruction. The action instruction with the highest final score is selected by the intelligent agent and used as the optimal instruction, wherein the intelligent agent is a large language model outside the multi-model set.

[0007] Optionally, the action instructions generated by the large language model include the instruction content and the reason for instruction generation, while the action instructions generated by the reinforcement learning algorithm only include the instruction content. Scoring each action instruction includes: Obtain the instruction type for each action instruction, wherein the instruction type is used to indicate whether the action instruction belongs to a large language model or a reinforcement learning algorithm; If the action instruction is determined to belong to the large language model based on the instruction type, a score is given based on the matching degree between the instruction content and the current environment information, as well as the logical rigor of the reason for the instruction generation. If the action instruction is determined to belong to a reinforcement learning algorithm based on the instruction type, a score is given based on the matching degree between the instruction content and the current environment information.

[0008] Optionally, scoring each non-self-generated action instruction in the action instruction set according to a preset scoring rule using each large language model and each reinforcement learning algorithm in the multi-model set includes: Randomly shuffle all action instructions in the action instruction set; Each large language model and each reinforcement learning algorithm in the multi-model set performs blind scoring on each action instruction that is not generated by itself and whose order is shuffled, according to preset scoring rules.

[0009] Optionally, based on the updated global policy and current environment information, regenerating action instructions through the multi-model set includes: Based on the updated global strategy, the self-awareness and world awareness of each large language model are optimized to generate a new round of self-awareness and world awareness for each large language model. The self-awareness is the cognitive judgment formed by the large language model based on its own state, and the world awareness is the comprehensive analysis and judgment of the large language model on the current task environment and its own executable decision-making operations in combination with the action instructions output by other large language models. The new round of self-awareness, world awareness, and current environmental information are input into the corresponding large language model to obtain the action instructions regenerated by the large language model; The current environment information is input into each reinforcement learning algorithm to obtain the action instructions regenerated by each reinforcement learning algorithm.

[0010] Optionally, the method further includes: After detecting that the current loop count has reached the preset round threshold, the final score corresponding to the action instructions generated by each language model in each round is calculated. The average score of the action instructions generated by the same large language model in multiple rounds of loops is calculated to obtain the target score corresponding to each language model. The agent eliminates the large language model with the lowest target score, and a new multi-model set is formed based on the remaining large language models and the multiple reinforcement learning algorithms. Based on the new multi-model set, the next loop will execute the steps of generating action instructions, selecting the optimal instruction, and updating the global policy.

[0011] Optionally, the preset loop termination condition includes: The agent detects that the final score of each action command generated by the multi-model set is higher than a set score threshold; or, The agent detects that the current cycle count has reached a set round threshold.

[0012] Secondly, this application provides an autonomous evolution device for a large language model, the device comprising: The acquisition module is used to acquire current environment information and global strategy, wherein the current environment information is real-time interaction status data in the task environment, and the global strategy is a unified behavior rule for the large language model to perform decision-making operations; The input module is used to input the current environment information and the global policy into a multi-model set, which includes multiple large language models and multiple reinforcement learning algorithms. The generation module is used to generate action instructions based on the current environment information and the global policy through each of the large language models, and to generate action instructions based on the current environment information and the mathematical model through each of the reinforcement learning algorithms. The accuracy of the action instructions generated by the reinforcement learning algorithms is higher than a preset accuracy threshold. The action instructions are used to indicate the decision operation that needs to be performed to complete the goal in the current task environment. The update module is used to select the optimal instruction with the highest accuracy from all action instructions output by the multi-model set, and update the global policy based on the optimal instruction to complete one evolution cycle of the global policy. The repetitive execution module is used to regenerate action instructions based on the updated global strategy and current environment information through the multi-model set, and repeatedly execute the steps of optimal instruction selection and global strategy update until the preset loop termination condition is reached. The autonomous evolution of the large language model is achieved through the autonomous evolution of the global strategy.

[0013] Thirdly, this application provides an electronic device, comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus.

[0014] Fourthly, this application also provides a computer storage medium storing computer-executable instructions for executing the autonomous evolution method of the large language model described in any of the preceding claims of this application.

[0015] The technical solutions provided in this application have the following advantages compared with the prior art: In this application, current environmental information and global policy are synchronously input into a multi-model ensemble, driving multiple large language models and multiple reinforcement learning algorithms to execute instruction generation operations in parallel. In the early stages of evolution, because the global policy has not yet undergone iterative optimization and its accuracy is insufficient, the large language models are susceptible to knowledge illusions and logical biases, resulting in low accuracy of generated action instructions. In contrast, the reinforcement learning algorithms do not rely on the global policy, but make purely rational decisions based solely on the current environmental information and mathematical models, without subjective judgment or knowledge fabrication. Therefore, the accuracy of their output action instructions is higher than a preset accuracy threshold. Subsequently, the optimal instruction with the highest accuracy is selected from the action instructions output by the multi-model ensemble, and the original global policy is modified based on this optimal instruction, completing one round of evolutionary loop. Finally, based on the updated global policy, the entire process of generating action instructions, selecting the optimal instruction, and updating the global policy is repeated to achieve multiple rounds of iterative iteration. In this process, on the one hand, a multi-source instruction game mechanism involving multiple large language models and multiple reinforcement learning algorithms is constructed. This eliminates the need to rely on the arbitrary decisions of a single large language model, instead comprehensively selecting the optimal instruction from action instructions of different types and logical perspectives. The rational results of reinforcement learning algorithms can correct the intuitive biases of the large language model, effectively preventing the global policy from deviating from the correct direction due to continuous iteration of erroneous instructions from a single model, thus improving the stability of the large language model's autonomous policy evolution. On the other hand, reinforcement learning algorithms can output highly accurate action instructions in the early stages of evolution, which can be directly used to update the global policy, providing reliable guidance for the large language model. This significantly reduces the number of ineffective trial and error rounds in the early stages, accelerating the convergence speed and iteration efficiency of the global policy. Ultimately, through multiple rounds of iterative iteration, the large language model simultaneously improves efficiency and stability during the autonomous policy evolution process. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0019] Figure 1A flowchart illustrating an autonomous evolution method for a large language model, as provided in this application embodiment; Figure 2 A schematic diagram of the control results of the comparative experiment provided in the embodiments of this application; Figure 3 A flowchart illustrating the autonomous evolution of a large language model, as provided in this application embodiment; Figure 4 A schematic diagram of the structure of an autonomous evolution device for a large language model provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0022] To address the issues of low efficiency and poor stability of large language models during autonomous evolution mentioned in the background art, this application's embodiments use high-accuracy instructions from reinforcement learning algorithms as a decision fallback and provide reliable guidance for the large language model. Ultimately, through multiple rounds of iterative cycles, the large language model simultaneously improves efficiency and stability during the autonomous evolution of its policies.

[0023] The application scenarios of this application include, but are not limited to: professional knowledge Q&A scenarios, intelligent customer service and human-computer interaction scenarios, game and competitive intelligent interaction scenarios, content creation and optimization scenarios, etc.

[0024] The embodiments of this application can be applied to terminals or servers. The main application includes three major modules: a large language model, a reinforcement learning algorithm, and an intelligent agent. Each module is functionally independent and works in concert, as detailed below.

[0025] Large Language Model: This is the decision-making entity in a multi-model set. It has semantic understanding and logical reasoning capabilities. It combines current environmental information with global strategies to generate action instructions that include the content of the instructions and the reasons for their generation. At the same time, it participates in the cross-scoring of action instructions generated by other decision-making entities.

[0026] Reinforcement learning algorithm: As a decision-making subject in a multi-model set, it has the ability to make decisions with high accuracy. Based on the current environmental information and the built-in mathematical model, it generates action instructions that contain only instructions and meet the accuracy standards, and simultaneously participates in the cross-scoring of action instructions generated by other decision-making subjects.

[0027] Intelligent Agent: An independent decision-making and selection entity that does not participate in the generation and cross-scoring of action instructions. It is responsible for summarizing the scoring results of all action instructions, selecting the optimal action instruction, and executing the dynamic survival-of-the-fittest decision of the large language model.

[0028] The following will describe in detail a method for the autonomous evolution of a large language model provided in this application, using specific implementation methods. The example below will be applied to a terminal. Figure 1 As shown, the specific steps are as follows: Step 101: Obtain the current environment information and global policy. The current environment information is the real-time interaction state data in the task environment, and the global policy is the unified behavior rule for the large language model to perform decision-making operations. Step 102: Input the current environment information and global policy into the multi-model set, which includes multiple large language models and multiple reinforcement learning algorithms; Step 103: Generate action instructions based on the current environment information and global policy through each large language model, and generate action instructions based on the current environment information and mathematical model through each reinforcement learning algorithm. The accuracy of the action instructions generated by the reinforcement learning algorithm is higher than the preset accuracy threshold. The action instructions are used to indicate the decision operation to be performed in order to complete the goal in the current task environment. Step 104: Select the optimal instruction with the highest accuracy from all action instructions output by the multi-model set, and update the global policy based on the optimal instruction to complete one evolution cycle of the global policy; Step 105: Based on the updated global strategy and current environment information, regenerate action instructions through a multi-model set, and repeat the steps of optimal instruction selection and global strategy update until the preset loop termination condition is reached. The autonomous evolution of the large language model is achieved through the autonomous evolution of the global strategy.

[0029] In step 101, the terminal first obtains the currently running task environment and extracts real-time and valid current environment information from the task environment; at the same time, the terminal obtains the global strategy uniformly followed by all large language models.

[0030] Among them, the task environment refers to an interactive environment with incomplete information where multiple decision-making entities coexist. The coexistence of multiple decision-making entities means that multiple entities capable of independent decision-making exist simultaneously in the same task environment. Each decision-making entity can generate decision instructions autonomously, and they are independent of each other and can interact. Incomplete information means that a single decision-making entity can only obtain its own information and publicly available information about the environment, and cannot know the private decision information of other decision-making entities. The interactive environment refers to a dynamic interactive scenario where users input information and operation instructions through a terminal, and the system responds and provides feedback on the processing results in real time.

[0031] Current environment information refers to real-time interaction status data within the task environment, reflecting the current interaction status, user needs, and publicly available information. For example, in a medical dialogue scenario, current environment information may include all publicly available interactive content such as the user's currently described symptoms, provided medical history, current dialogue round, and preliminary judgments already provided to the user by the terminal.

[0032] The global strategy is a unified decision-making behavior guideline followed by all large language models. It is used to standardize the task objectives, decision directions, and output boundaries of large language models, avoid the problem of divergent decisions and logical confusion caused by the lack of unified constraints among different large language models, and ensure that multiple large language models generate action instructions under the same rule framework.

[0033] In step 102, the terminal synchronously inputs the acquired current environment information and global policy into a preset multi-model set, which contains two types of decision-making entities: multiple large language models and multiple reinforcement learning algorithms.

[0034] Among them, the large language model can be a model with semantic understanding, logical reasoning and scene adaptation capabilities. It can include general dialogue large models and domain fine-tuning large models, such as Tongyi Qianwen, Zhipu Qingyan, and general pre-trained generative large model GPT-4, which can combine environmental information and global strategies to generate action instructions that fit the scene.

[0035] Reinforcement learning algorithms can be selected based on historical professional data and have stable decision-making capabilities, such as disease diagnosis algorithms trained on medical clinical data and symptom matching algorithms based on mathematical programming. Their decision-making process is based on pure mathematical models and historical data, without subjective bias or fabricated knowledge, and can stably output action instructions with high accuracy.

[0036] The two types of decision-making entities complement each other. Specifically, large language models have the ability to understand scenarios and can generate flexible instructions that meet user needs; reinforcement learning algorithms have high stability and high accuracy and can provide reliable basic decision support, thus jointly solving the drawbacks of the evolution of a single large language model.

[0037] In step 103, the terminal drives all large language models and reinforcement learning algorithms within the multi-model set to execute action instruction generation operations in parallel. The specific generation logic is as follows.

[0038] Each large language model independently generates action instructions based on the current environment information and global strategy. In the early stages of evolution, the global strategy has not undergone multiple rounds of optimization and has problems of imperfection and inaccuracy. The large language model itself has the inherent characteristics of knowledge illusion and logical bias. Under these circumstances, the large language model is prone to generating action instructions that do not match the actual task scenario or are even wrong, and the reliability of the action instructions cannot be guaranteed.

[0039] Each reinforcement learning algorithm does not rely on a global policy; it independently generates action commands based solely on current environmental information, its built-in mathematical model, and historical professional data. Because the decision-making process of reinforcement learning algorithms is entirely based on pure mathematical calculations and verified historical data, without any subjective judgment, knowledge illusion, or fabrication of false information, the accuracy of its generated action commands consistently exceeds a preset accuracy threshold. In this embodiment, this accuracy can reach as high as 70%. Such high-accuracy action commands can serve as a rational reference for selecting subsequent action commands, providing a stable decision safety net and reducing the proportion of erroneous commands from the source.

[0040] In step 104, after the multi-model set generates action instructions in parallel, the output instruction pool contains two types of instructions: instructions generated by various language models (which may include erroneous instructions) and high-accuracy instructions generated by various reinforcement learning algorithms. This constitutes a multi-source instruction. These multi-source instructions form a game-theoretic decision-making mechanism, so that the terminal no longer relies on the arbitrary decision of a single model, but selects the optimal instruction with the highest credibility and rationality through the integration of multiple models. Then, the original global strategy is updated based on the optimal instruction, completing one round of evolutionary cycle.

[0041] On the one hand, since the multi-model set always contains reinforcement learning algorithms that can output action commands with high accuracy, the command pool always contains stable and reliable fallback commands. The terminal identifies and selects the optimal command in the correct direction, allowing the global policy to iterate along a reasonable path. This is equivalent to using the rationality of reinforcement learning algorithms to correct the intuitive bias of large language models, avoiding continuous policy deviations due to errors in a single model. On the other hand, compared to a single large language model that lacks other perspectives and can only judge right and wrong on its own, making it prone to getting stuck on a single path, the addition of reinforcement learning algorithms forms a multi-source command game mechanism involving multiple large language models and reinforcement learning algorithms. The terminal does not need to rely on the arbitrary decision-making of a single large language model, but instead selects the optimal command from commands of different types and perspectives. This is equivalent to multiple people voting for the best, making the decision more objective and naturally preventing large deviations, thus significantly improving stability.

[0042] For example, in medical consultation, a single large language model may misdiagnose viral pharyngitis as bronchitis in the early stages due to insufficient information; while traditional algorithms, based on clinical data, can reliably output the correct diagnosis of viral pharyngitis. When the terminal is screening, it is highly likely to select this correct instruction, and the global strategy will evolve in the correct direction without deviation.

[0043] On the other hand, a drawback of the early evolution of large language models is the lack of experience and correct instruction references. Instructions can only be generated through random trial and error, with most instructions being invalid or incorrect. Repeated iterations are needed to filter out a small number of correct instructions, resulting in slow updates to the global policy. In contrast, reinforcement learning algorithms do not require prior experience accumulation. They can directly output high-quality action instructions based on the current environment information, essentially providing the answer directly to the early stage of the large language model. The terminal does not need to repeatedly filter out incorrect instructions and can quickly obtain the correct and optimal instructions, which can be directly used to update the global policy. This completes the foundational construction of the early global policy in one step, significantly reducing the number of trial and error rounds and improving the evolutionary efficiency of the large language model.

[0044] For example, in the medical field, large language models have no experience in diagnosing symptoms such as sore throat and low-grade fever in the early stages. If they rely solely on trial and error, it may take 5 to 10 rounds to find the correct diagnostic approach. However, reinforcement learning algorithms can directly output the correct diagnostic instructions with 70% accuracy. The terminal can select the correct instructions in the first round and update the global policy, which improves the efficiency of early evolution by several times.

[0045] In step 105, the terminal re-executes steps 101-104 based on the updated global policy. First, it obtains the latest environmental information and the updated global policy for the current task environment. As the interaction progresses, the environmental information may change dynamically. Then, it inputs the new environmental information and the updated global policy into the multi-model set to drive various language models and reinforcement learning algorithms to generate a new round of action instructions in parallel. Next, through multi-source instruction game, it selects the optimal instruction for this round and updates the global policy again. This process is repeated until the preset loop termination condition is reached.

[0046] As the iterative cycle continues, the stability and evolutionary efficiency of the global strategy and the large language model will gradually improve, forming a virtuous cycle: On the one hand, each iteration updates the global strategy based on high-quality optimal instructions, making the decision logic of the global strategy increasingly perfect and accurate, providing more reliable decision criteria for the large language model, significantly reducing knowledge illusions and logical biases caused by imperfect strategies, thereby reducing the probability of generating erroneous instructions, continuously increasing the proportion of correct instructions in multi-source instruction games, greatly improving the efficiency of terminal selection of optimal instructions, and accelerating the pace of global strategy updates; on the other hand, the high accuracy of reinforcement learning algorithms continues to play a supporting role, combined with the continuously optimized global strategy, further enhancing the objectivity of multi-source instruction games, effectively avoiding the accumulation of biases from a single model, ensuring that the iteration of the large language model always proceeds in the right direction, and gradually improving its stability.

[0047] When the loop terminates, the global strategy has been autonomously improved, forming a unified behavioral guideline that is adapted to the task environment, makes accurate decisions, and is logically rigorous. All large language models always generate action instructions based on the global strategy. In multiple iterations, as the global strategy is continuously optimized, the knowledge illusions and logical deviations of the large language models are continuously corrected, and their decision-making ability and scene adaptation ability are continuously improved. Finally, the large language models complete their own autonomous evolution in sync.

[0048] In this application, current environmental information and global policy are synchronously input into a multi-model ensemble, driving multiple large language models and multiple reinforcement learning algorithms to execute instruction generation operations in parallel. In the early stages of evolution, because the global policy has not yet undergone iterative optimization and its accuracy is insufficient, the large language models are susceptible to knowledge illusions and logical biases, resulting in low accuracy of generated action instructions. In contrast, the reinforcement learning algorithms do not rely on the global policy, but make purely rational decisions based solely on the current environmental information and mathematical models, without subjective judgment or knowledge fabrication. Therefore, the accuracy of their output action instructions is higher than a preset accuracy threshold. Subsequently, the optimal instruction with the highest accuracy is selected from the action instructions output by the multi-model ensemble, and the original global policy is modified based on this optimal instruction, completing one round of evolutionary loop. Finally, based on the updated global policy, the entire process of generating action instructions, selecting the optimal instruction, and updating the global policy is repeated to achieve multiple rounds of iterative iteration. In this process, on the one hand, a multi-source instruction game mechanism involving multiple large language models and multiple reinforcement learning algorithms is constructed. This eliminates the need to rely on the arbitrary decisions of a single large language model, instead comprehensively selecting the optimal instruction from action instructions of different types and logical perspectives. The rational results of reinforcement learning algorithms can correct the intuitive biases of the large language model, effectively preventing the global policy from deviating from the correct direction due to continuous iteration of erroneous instructions from a single model, thus improving the stability of the large language model's autonomous policy evolution. On the other hand, reinforcement learning algorithms can output highly accurate action instructions in the early stages of evolution, which can be directly used to update the global policy, providing reliable guidance for the large language model. This significantly reduces the number of ineffective trial and error rounds in the early stages, accelerating the convergence speed and iteration efficiency of the global policy. Ultimately, through multiple rounds of iterative iteration, the large language model simultaneously improves efficiency and stability during the autonomous policy evolution process.

[0049] As an optional implementation, step 104, selecting the optimal instruction with the highest accuracy from all action instructions output by the multi-model set, includes: Step S11: Summarize all action instructions output by the multi-model set to form an action instruction set; Step S12: Using each large language model and each reinforcement learning algorithm in the multi-model set, score each action instruction in the action instruction set that is not generated by itself according to the preset scoring rules; Step S13: Obtain the final score for each action instruction based on all the scores corresponding to it; Step S14: Select the action instruction with the highest final score through the agent and use it as the optimal instruction. Here, the agent is a large language model outside of the multi-model set.

[0050] In step S11, the terminal collects all action commands output by all large language models and reinforcement learning algorithms in the multi-model set through the intelligent agent, summarizes and organizes these commands to form a complete action command set, and ensures that the subsequent scoring process can cover all generated action commands.

[0051] In step S12, each large language model and each reinforcement learning algorithm in the terminal-driven multi-model set independently scores the collected action commands. The scoring process strictly follows preset scoring rules, which can be determined according to the application scenario. For example, in a medical dialogue scenario, the scoring rules include the suitability of medication to the user's symptoms, the degree to which medication conforms to clinical treatment guidelines, and the absence of allergy risk. Furthermore, to ensure fairness in the scoring, all participating entities do not participate in scoring their own generated commands; they only evaluate the action commands output by other entities, avoiding subjective bias.

[0052] The action instructions generated by the large language model include both the instruction content and the reason for its generation, while the action instructions generated by the reinforcement learning algorithm only include the instruction content. Scoring each action instruction involves: obtaining the instruction type for each action instruction, where the instruction type indicates whether the action instruction belongs to the large language model or the reinforcement learning algorithm; if the action instruction belongs to the large language model based on its instruction type, then a score is given based on the matching degree between the instruction content and the current environment information, as well as the logical rigor of the reason for its generation; if the action instruction belongs to the reinforcement learning algorithm based on its instruction type, then a score is given based on the matching degree between the instruction content and the current environment information.

[0053] Large language models possess powerful semantic understanding, logical reasoning, and language expression capabilities, enabling them to deeply interpret the decision-making logic behind instructions. Therefore, the generated action instructions must simultaneously include the instruction content and the reason for instruction generation. The instruction content refers to the specific decision-making operations and execution requirements that need to be performed to achieve the goal in the current task environment. The reason for instruction generation refers to the logical basis and reasoning process for deriving the decision-making operation by combining current environmental information, global strategies, and one's own cognitive judgment.

[0054] The function of reinforcement learning algorithms is to perform purely rational calculations based on built-in mathematical models and historical professional data. There is no subjective reasoning process, nor does it have the ability to understand semantics or interpret logic. It cannot interpret the reasons for the generation of instructions output by other large language models. Therefore, the action instructions it generates only need to contain the instruction content and do not need to generate additional reasons.

[0055] Each decision-making agent in the multi-model set, upon receiving an action instruction, parses the instruction type of that action instruction. This instruction type indicates whether the action instruction belongs to a large language model or a reinforcement learning algorithm.

[0056] If the decision-making body determines that the action instruction belongs to the large language model based on the instruction type, it not only needs to judge whether the instruction content itself meets the task requirements of the current environment information, but also needs to combine the reason for the instruction generation to verify whether the reasoning logic of the instruction content is rigorous, whether the basis is sufficient, and whether it fits the current environment information and global strategy, so as to more accurately score the accuracy and rationality of the instruction.

[0057] Taking a medical dialogue scenario as an example, in the action command output by a certain large language model, the instruction content is to suggest that the user take XX capsules, 3 times a day, 0.5g each time. The reason for generating the instruction is that it combines the user's described symptoms of sore throat, low-grade fever, and tonsillitis, as well as the global strategy's principle of prioritizing penicillin antibiotics for bacterial upper respiratory tract infections, and the user's lack of drug allergy history, therefore this medication regimen is recommended. Other large language models, when scoring, will combine the user's symptoms and the global strategy to judge the rationality of the medication recommendation, and at the same time verify whether the logic in the generated reason is coherent and whether the basis is accurate, thus giving a realistic score.

[0058] If the decision-making body determines that the action instruction belongs to the reinforcement learning algorithm based on the instruction type, it only judges whether the instruction content itself meets the task requirements of the current environment information, focuses on the rationality and feasibility of the instruction content itself, and obtains a score for the action instruction.

[0059] Taking a medical dialogue scenario as an example, the action command output by the reinforcement learning algorithm is simply to suggest that the user take XX capsules, 3 times a day, 0.5g each time, without adding any reason for its generation. When scoring, the reinforcement learning algorithm only judges the accuracy and suitability of the instruction content itself, and combines its own built-in clinical data to determine whether the medication plan is suitable for the user's current symptoms, without considering the reasoning logic behind the instruction. This not only conforms to the functional positioning of the reinforcement learning algorithm, but also avoids meaningless scoring processes and effectively improves scoring efficiency.

[0060] Large language models can interpret complex reasoning logic, so the action instructions they generate need to include the reason for their generation so that other decision-makers can conduct a comprehensive evaluation and judge the rationality of their reasoning process. Reinforcement learning algorithms do not have semantic understanding capabilities, so the action instructions they generate only need to include specific content without adding a reason for their generation. This not only meets their functional positioning but also avoids redundant evaluation steps and improves scoring efficiency.

[0061] In addition, the process of each decision-making body scoring each action instruction also includes the following: randomly shuffling all action instructions in the action instruction set; and blindly scoring each non-self-generated, shuffled action instruction according to preset scoring rules using each large language model and each reinforcement learning algorithm in the multi-model set.

[0062] In this application, a blind scoring mechanism is further introduced during the scoring process of each action instruction by the decision-making entities in the multi-model ensemble. The agent obtains the source of each action instruction, which refers to which large language model or reinforcement learning algorithm the action instruction specifically originates from. The agent first randomly shuffles the order of all action instructions in the action instruction set, while simultaneously hiding the source of each action instruction, so that neither the large language model nor the reinforcement learning algorithm participating in the scoring can identify which decision-making entity generated a particular action instruction. Based on this, each large language model and each reinforcement learning algorithm in the multi-model ensemble independently scores only the shuffled action instructions that were not generated by themselves, according to preset scoring rules.

[0063] This application can effectively avoid subjective bias or scoring discrimination by the scoring decision-maker due to knowledge of the source of the action instruction. For example, it will not deliberately give a high score to an action instruction because it comes from a well-known large language model, nor will it ignore the actual rationality of an action instruction because it comes from a reinforcement learning algorithm. This ensures that all action instructions are evaluated under completely equal conditions, greatly improves the fairness and objectivity of the scoring process, and further guarantees that the best instruction selected in the end is real and reliable.

[0064] In step S13, the terminal collects all scores corresponding to each action command through the intelligent agent, that is, the scores of each large language model and each reinforcement learning algorithm for the action command, and then calculates the final score of each command according to the preset calculation rules. There are two commonly used calculation rules: one is to take the average of all scores, and the other is to use weighted summation.

[0065] In step S14, the terminal selects the action instruction with the highest final score as the optimal instruction for this round of evolutionary loop through an intelligent agent. Here, the intelligent agent is a large language model independent of the multi-model set. Setting the intelligent agent as an independent entity outside the multi-model set completely eliminates the scoring correlations and biases within the multi-model set, ensuring the neutrality of the selection process. Simultaneously, as a dedicated selection entity, the intelligent agent possesses stronger comprehensive decision-making capabilities, enabling it to accurately select the instruction most suitable for the current scenario and with the highest accuracy based on the final score of each action instruction, combined with the current task objective and environmental information. This avoids selection errors caused by differences in scores from multiple execution entities or extreme scores, ensuring the reliability of the optimal instruction.

[0066] This application sets out differentiated action instruction formats for large language models and reinforcement learning algorithms. The action instructions generated by the large language model include both the instruction content and the reason for its generation, which fully leverages its logical reasoning advantages and allows other decision-makers to accurately verify the logical rigor and rationality of the action instructions during scoring. The action instructions generated by the reinforcement learning algorithm only include the instruction content, which aligns with its function of purely rational calculation. At the same time, the dual mechanism of cross-blind evaluation by multiple decision-makers and selection by independent agents effectively avoids subjective bias and scoring deviation, ensuring that the selected optimal instructions have high accuracy and strong rationality.

[0067] The preset loop termination conditions include: the agent detects that the final score of each action instruction generated by the multi-model set is higher than the set score threshold; or the agent detects that the current loop count has reached the set round threshold.

[0068] In this application, the preset loop termination conditions include the following two cases, and the iteration will terminate when either case is met.

[0069] In the first scenario, the agent scores and monitors all action commands generated by the multi-model set in each round. When it is detected that the final score corresponding to each action command is higher than the set score threshold, it indicates that the action commands generated by the current multi-model set have high accuracy and rationality, the global strategy is becoming more perfect, and the decision bias of the large language model has been effectively corrected. At this time, the agent determines that the loop termination condition has been met and ends the evolution iteration.

[0070] In the second scenario, the agent counts the number of loops in the current strategy evolution in real time. When the cumulative number of loops reaches a set threshold, the agent directly determines that the loop termination condition has been met and stops the subsequent process in order to avoid meaningless repeated iterations and reduce system resource consumption.

[0071] As an optional implementation, in step 105, regenerating action instructions based on the updated global policy and current environment information using a multi-model set includes: Step S21: Based on the updated global strategy, optimize the self-awareness and world awareness of each large language model respectively, and generate a new round of self-awareness and world awareness for each large language model. Among them, self-awareness is the cognitive judgment formed by the large language model based on its own state, and world awareness is the comprehensive analysis and judgment of the large language model in combination with the action instructions output by other large language models, on the current task environment and its own executable decision-making operations. Step S22: Input the new round of self-awareness, world awareness, and current environmental information into the corresponding large language model to obtain the action instructions regenerated by the large language model; Step S23: Input the current environment information into each reinforcement learning algorithm to obtain the action instructions regenerated by each reinforcement learning algorithm.

[0072] In step S21, the terminal optimizes the self-awareness and world awareness of each large language model based on the updated global strategy, and generates a new round of self-awareness and world awareness for each large language model.

[0073] Self-awareness, in this context, refers to the internal cognitive judgment formed by the large language model regarding its own decision-making ability, knowledge boundaries, output risks, and task suitability under the constraints of the updated global strategy. Taking a medical dialogue scenario as an example, the self-awareness of the large language model can be manifested as: I possess the ability to inquire about and make preliminary judgments about common upper respiratory tract infections, but I do not have prescription authority. My diagnostic conclusions must be conservative and in accordance with treatment guidelines, avoiding making absolute judgments beyond my own capabilities.

[0074] World awareness is a comprehensive analysis and judgment by the large language model, combining the updated global strategy, historical action instructions from other large language models, and current public environmental information, regarding the overall task environment, interaction dynamics, the behavior of other agents, and its own executable decision-making operations. In the same medical dialogue scenario, world awareness can manifest as follows: the user has expressed symptoms such as sore throat and low-grade fever; other large language models in the environment tend to diagnose upper respiratory tract infection; the global strategy requires prioritizing the investigation of common causes; and the model should provide a reasonable diagnostic direction within this scope to avoid deviating from the overall consensus. As the global strategy is continuously optimized, the large language model's self-awareness becomes clearer and more restrained, and its world awareness becomes more aligned with the environmental situation and task objectives.

[0075] In step S22, the terminal inputs the new self-awareness, new world awareness, and current environmental information of the large language model into the large language model, enabling the large language model to perform comprehensive reasoning based on more accurate and standardized cognition, and finally generate a new round of action instructions. These action instructions are constrained by the global strategy and conform to the user's own cognitive positioning and environmental state, which can reduce knowledge illusions, subjective assumptions, and unreasonable outputs.

[0076] In step S23, at the same time, the terminal inputs the current environment information into each reinforcement learning algorithm. The reinforcement learning algorithm does not participate in global policy optimization, does not form self-awareness or world awareness, and still only performs purely rational calculations based on the built-in mathematical model and historical clinical data to directly generate action instructions with high accuracy, continuously providing stable and reliable decision support for the entire iterative system.

[0077] This application simultaneously optimizes the self-awareness and world awareness of the large language model through an updated global strategy. This upgrades the large language model from simply generating action instructions based on the initial global strategy to making rational outputs based on a clear self-positioning and environmental cognition, significantly improving the rationality and consistency of action instructions. At the same time, it maintains the mechanism of independent and stable generation of action instructions by the reinforcement learning algorithm, forming a double guarantee of cognitive evolution and rational fallback. This ensures that the large language model becomes more and more stable and the deviation becomes smaller in continuous iteration, while also ensuring that the action instructions generated in each round have reliable references, further improving the stability of the overall evolutionary process.

[0078] As an optional implementation, the method further includes the following steps throughout the cycle: Step S31: After detecting that the current loop count has reached the preset round threshold, calculate the final score corresponding to the action instructions generated by each language model in each round of loop; Step S32: Calculate the average of the final scores of the action instructions generated by the same large language model in multiple rounds of loops to obtain the target scores corresponding to each language model; Step S33: Eliminate the large language model with the lowest target score through the agent, and form a new multi-model set based on the remaining large language models and multiple reinforcement learning algorithms; Step S34: Based on the new multi-model set, execute the steps of generating action instructions, selecting the optimal instruction, and updating the global policy in the next round of the loop.

[0079] In step S31, the agent performs real-time statistics on the number of completed iterations during each iteration. When the cumulative number of iterations reaches a preset threshold, it initiates an overall performance evaluation of all major language models within the multi-model set. The terminal retrieves the scoring data from each iteration within the current evaluation cycle, calculates the final score corresponding to the action commands generated by each major language model in each iteration, and fully records the output performance of each major language model across multiple iterations.

[0080] In step S32, the agent, for the same large language model, summarizes the final scores of all rounds within the current evaluation cycle and calculates the average score, using this average as the target score for the large language model. Using the average of multiple rounds rather than a single score for evaluation avoids misjudgments caused by accidental errors or abnormal fluctuations in a single instance, and more realistically reflects the long-term decision-making ability, output stability, and adaptability to global strategies of the large language model in complex scenarios such as medical consultation. For example, in a medical dialogue scenario, if a large language model consistently misjudges symptoms such as sore throat, low-grade fever, and cough in multiple rounds, its final score will remain consistently low, and its average score will also be lower than other models.

[0081] In step S33, the agent sorts all large language models from highest to lowest target score, identifying and eliminating one or more large language models with the lowest target score. The reason for eliminating these models is that they consistently generate action instructions with low accuracy, poor logic, and knowledge illusions across multiple iterations. This not only fails to contribute effectively to global policy optimization but also continuously introduces incorrect decision-making directions, increasing selection costs, slowing evolution, and even causing the global policy to shift in the wrong direction. After elimination, the agent recombines the remaining, better-performing, and more stable large language models with multiple reinforcement learning algorithms that consistently provide high-accuracy fallback instructions, forming a new multi-model set with a more rational structure and stronger overall performance.

[0082] In step S34, in subsequent iterations, the agent will continue to perform cyclical steps such as action instruction generation, optimal action instruction selection, and global policy update based on this new set of multiple models that has undergone the process of natural selection, so that the entire evolutionary system always remains in a state of high-quality model collaboration.

[0083] This application introduces a multi-round evaluation and dynamic elimination mechanism to continuously clean up large language models that have long performed poorly and are prone to generating erroneous instructions. This structurally optimizes the overall decision-making ability of the multi-model set, avoiding repeated interference from low-quality models in the iteration process, which would lead to ineffective computation and waste of resources. At the same time, the retained large language models work more stably with reinforcement learning algorithms, which can further reduce the proportion of erroneous action instructions, accelerate the selection speed of optimal action instructions, make global policy updates more accurate and evolutionary directions more reliable, and improve the efficiency and stability of the overall autonomous evolution process.

[0084] Figure 2 This is a schematic diagram showing the control results of the comparative experiment. Figure 2 The horizontal axis represents the three large language models tested: Qwen, Zhipu, and GPT-4. Each large language model corresponds to two sets of bars, representing two evolutionary strategies: In the two sets of pillars corresponding to each large language model, the left pillar represents the collaborative self-evolution strategy of multiple large language models, and the right pillar represents the self-evolution strategy of a single large language model. Figure 2 The vertical axis represents percentages (%), with each bar having a total height of 100%, representing the win-loss-draw ratio over 100 rounds of gameplay. Each bar is layered, from top to bottom representing the percentage of draws, losses, and wins. The values ​​in the bars indicate specific percentages. For example, the bottom values ​​of the two bars for Qwen are 38 and 35, respectively, indicating that Qwen's win percentage is 38% under the collaborative self-evolution strategy of multiple large language models and 35% under the self-evolution strategy of a single large language model.

[0085] This application compares the left and right pillars of each model to see which strategy makes the model perform better. The comparison dimensions include three aspects, including the following: 1. Comparing the bottom layer (win rate): The higher the percentage of wins in the left pillar, the more significant the effect of co-evolution on improving the model's win rate; 2. Comparing the middle layer (loss rate): The lower the percentage of losses in the left pillar, the more effectively co-evolution can reduce the model's failure rate; 3. Comparing the top layer (draw rate): The lower the percentage of draws in the left pillar, the more decisive the model's decisions are, and the fewer draws occur.

[0086] This application employs a large language model × each strategy, conducting two rounds of game experiments, with 100 games per round. The final data is the average of the two rounds, with a total sample size of 200 games, eliminating random errors.

[0087] (1) Win rate: Co-evolution strategy is superior to single-model evolution in all aspects.

[0088] Global average: The average win rate of the co-evolutionary strategy is 42.0%, while the average win rate of the single-model evolutionary strategy is only 36.8%. Co-evolution brought a 5.2 percentage point improvement in win rate.

[0089] Qwen: Co-evolution win rate 38.0%, single model evolution win rate 35.0%, an improvement of 3.0 percentage points; Zhipu: Co-evolution win rate 46.5%, single model evolution win rate 35.0%, an improvement of 11.5 percentage points (the largest improvement). GPT-4: Co-evolutionary win rate 41.5%, single-model evolutionary win rate 39.5%, an improvement of 2.0 percentage points; All models achieved a positive improvement in win rate under the co-evolution strategy, verifying the enhancing effect of multi-model collaboration on decision-making ability.

[0090] (2) Failure rate: Co-evolutionary strategy effectively reduces decision-making errors.

[0091] Global average: The average failure rate of the co-evolutionary strategy was 51.3%, while the average failure rate of the single-model evolutionary strategy was 55.2%. Co-evolution reduced the failure rate by 3.9 percentage points. Qwen: The failure rates of the two strategies are similar (55.5% vs 56.0%), with no significant difference; Zhipu: Co-evolution failure rate was 46.0%, and single-model evolution failure rate was 57.0%, a significant reduction of 11.0 percentage points; GPT-4: Co-evolutionary failure rate was 52.5%, and single-model evolutionary failure rate was 52.0%, basically the same; Co-evolution has the most significant effect on optimizing the failure rate of medium-to-high performance models (Zhipu), effectively reducing decision-making errors.

[0092] (3) Draw rate: Co-evolutionary strategy reduces conservative decision-making.

[0093] Global average: The average draw rate of the co-evolutionary strategy was 6.7%, the average draw rate of the single-model evolutionary strategy was 8.5%, and the co-evolutionary strategy reduced the draw rate by 1.8 percentage points; Qwen: Co-evolutionary draw rate 6.5%, single-model evolution draw rate 9.0%; Zhipu: Co-evolution draw rate 7.5%, Single model evolution draw rate 8.0%; GPT-4: Co-evolutionary draw rate 6.0%, Single-model evolutionary draw rate 8.5%; The results show that the model's decision-making under the co-evolutionary strategy is more aggressive, reducing conservative draw choices and tending to actively engage in the game to strive for victory.

[0094] Therefore, the collaborative self-evolution strategy of multiple large language models can comprehensively improve the decision-making performance of large language models in game tasks, namely, increase the win rate, effectively reduce the loss rate, and reduce conservative draws. The optimization effect is particularly prominent on medium-performance models, which verifies the effectiveness and superiority of the multi-model collaborative evolution mechanism.

[0095] This application provides a flowchart of the autonomous evolution of a global strategy for a large language model, such as... Figure 3 As shown, the steps include the following.

[0096] Step 301: Obtain current environment information and global policies.

[0097] The current environment information is the real-time interactive state data in the task environment, and the global strategy is the unified behavioral guideline for the large language model to execute decision-making operations.

[0098] Step 302: Input the multi-model set to generate action instructions.

[0099] The agent inputs current environmental information and global policy into a multi-model set, driving various language models to generate action instructions based on the current environmental information and global policy. The action instructions include the instruction content and the reason for the instruction generation. The agent also drives various reinforcement learning algorithms to generate action instructions based on the current environmental information and mathematical models. The action instructions only contain the instruction content, and the accuracy of the action instructions of the reinforcement learning algorithms is higher than a preset accuracy threshold.

[0100] Step 303: Summarize the action instructions to form an instruction set.

[0101] The intelligent agent comprehensively collects and unifies the action instructions output by all large language models and reinforcement learning algorithms in the multi-model set to form a complete action instruction set.

[0102] Step 304: Multiple decision-making entities perform cross-blind scoring on the action instructions in the instruction set.

[0103] Each large language model and each reinforcement learning algorithm in the multi-model set independently scores the non-self-generated action instructions in the action instruction set according to the preset scoring rules. The scoring process is blind scoring, and the specific process is as follows.

[0104] All action instructions in the action instruction set are randomly shuffled, and the source of each action instruction is hidden, but the instruction type is not hidden. This avoids subjective bias or scoring discrimination by the scoring decision-maker who knows the source of the action instruction. If the action instruction is determined to be generated by a large language model based on its instruction type, the score is based on the matching degree between the instruction content and the current environment information and the logical rigor of the instruction generation reason. If the action instruction is determined to be generated by a reinforcement learning algorithm based on its instruction type, the score is based on the matching degree between the instruction content and the current environment information.

[0105] Step 305: Calculate the final score for each instruction.

[0106] The agent collects all the scores corresponding to each action command, summarizes and processes all the scores according to the preset calculation rules, and obtains the final score for each action command.

[0107] Step 306: Select the optimal instruction based on the final score.

[0108] The agent compares the final scores of all action commands and selects the action command with the highest final score as the optimal command for this round.

[0109] Step 307: Update the global policy based on the optimal instruction.

[0110] The agent optimizes and updates the global policy based on the optimal instruction, completing a full evolutionary cycle.

[0111] Based on the updated global policy and combined with current environmental information, the agent drives a multi-model set to regenerate action instructions, which are specifically divided into two types of execution logic: large language model and reinforcement learning algorithm.

[0112] For large language models: Based on the updated global strategy, optimize the self-awareness and world awareness of each large language model to generate a new round of self-awareness and world awareness; input the new round of self-awareness, world awareness and current environmental information into the corresponding large language model to obtain its regenerated action instructions.

[0113] For reinforcement learning algorithms: directly input the current environment information into each reinforcement learning algorithm to obtain its regenerated action instructions.

[0114] Based on the action instructions regenerated in step 307, the agent repeatedly executes steps 302-307 to complete multiple rounds of evolutionary cycles. Each round is based on the latest global policy, and the global policy is optimized and updated after each round.

[0115] Step 308: If any preset loop termination condition is met, the loop terminates.

[0116] During multiple rounds of iterative cycles, the agent monitors the cycle status in real time. If any preset cycle termination condition is met, all cycle processes are immediately terminated, and the autonomous evolution of the large language model is achieved through the autonomous evolution of the global strategy.

[0117] In addition, in the above process, if the current loop count reaches a preset round threshold, the final scores corresponding to the action instructions generated by each language model in each round are calculated; the average of the final scores of the same language model in the above multiple rounds is calculated to obtain the target scores corresponding to each language model; the agent sorts them according to the target scores and eliminates the language model with the lowest target score; the remaining high-performing language models are recombined with the original multiple reinforcement learning algorithms to form a new multi-model set; based on the new multi-model set, the entire loop process continues.

[0118] Based on the same technical concept, this application provides an autonomous evolution device for a large language model, such as... Figure 4 As shown, the device includes: The acquisition module 401 is used to acquire current environment information and global strategy. The current environment information is real-time interaction status data in the task environment, and the global strategy is the unified behavior rule for the large language model to perform decision-making operations. Input module 402 is used to input the current environment information and global policy into a multi-model set, which includes multiple large language models and multiple reinforcement learning algorithms. The generation module 403 is used to generate action instructions based on the current environment information and global policy through each large language model, and to generate action instructions based on the current environment information and mathematical model through each reinforcement learning algorithm. The accuracy of the action instructions generated by the reinforcement learning algorithm is higher than a preset accuracy threshold. The action instructions are used to indicate the decision operation that needs to be performed to complete the goal in the current task environment. The update module 404 is used to select the optimal instruction with the highest accuracy from all action instructions output by the multi-model set, and update the global policy based on the optimal instruction to complete one evolution cycle of the global policy. The repetitive execution module 405 is used to regenerate action instructions through a multi-model set based on the updated global strategy and current environment information, and repeatedly execute the steps of optimal instruction selection and global strategy update until the preset loop termination condition is reached. The autonomous evolution of the large language model is achieved through the autonomous evolution of the global strategy.

[0119] Optionally, update module 404 is used for: All action commands output from multiple model sets are aggregated to form an action command set; Each large language model and each reinforcement learning algorithm in the multi-model set is scored according to the preset scoring rules for each action instruction not generated by itself in the action instruction set; The final score for each action instruction is obtained based on all the scores corresponding to each action instruction. The action instruction with the highest final score is selected by the agent as the optimal instruction. The agent is a large language model outside of the multi-model set.

[0120] Optionally, the action instructions generated by the large language model include both the instruction content and the reason for instruction generation, while the action instructions generated by the reinforcement learning algorithm only include the instruction content. Update module 404 is used for: Obtain the instruction type for each action instruction, where the instruction type indicates whether the action instruction belongs to a large language model or a reinforcement learning algorithm; If the action instruction is determined to belong to the large language model based on the instruction type, then a score is given based on the matching degree between the instruction content and the current environment information, as well as the logical rigor of the reason for the instruction generation. If the action instruction is determined to belong to a reinforcement learning algorithm based on the instruction type, then a score is given based on the degree of matching between the instruction content and the current environment information.

[0121] Optionally, update module 404 is used for: Randomly shuffle all action instructions in the action instruction set; Each large language model and each reinforcement learning algorithm in the multi-model set is used to perform blind scoring on each action instruction that is not generated by itself and whose order is shuffled, according to the preset scoring rules.

[0122] Optionally, the repetitive execution module 405 is used for: Based on the updated global strategy, the self-awareness and world awareness of each large language model are optimized to generate a new round of self-awareness and world awareness for each large language model. Among them, self-awareness is the cognitive judgment formed by the large language model based on its own state, and world awareness is the comprehensive analysis and judgment of the large language model in combination with the action instructions output by other large language models, on the current task environment and its own executable decision-making operations. The new round of self-awareness, world awareness, and current environmental information are input into the corresponding large language model to obtain the action instructions regenerated by the large language model. The current environment information is input into each reinforcement learning algorithm to obtain the action instructions regenerated by each reinforcement learning algorithm.

[0123] Optionally, the device is also used for: After detecting that the current loop count has reached the preset round threshold, the final score corresponding to the action instructions generated by each language model in each round is calculated. The average score of the action instructions generated by the same large language model in multiple rounds of loops is calculated to obtain the target score corresponding to each language model. The agent eliminates the large language model with the lowest target score, and a new multi-model set is formed based on the remaining large language models and multiple reinforcement learning algorithms. Based on the new multi-model set, the next loop will execute the steps of generating action instructions, selecting the optimal instruction, and updating the global policy.

[0124] Optionally, the preset loop termination conditions include: The agent detects that the final score of each action instruction generated by the multi-model set is higher than a set score threshold; or, The agent detects that the current loop count has reached the set round threshold.

[0125] like Figure 5 As shown, this application provides an electronic device including a processor 501, a communication interface 502, a memory 503, and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504.

[0126] Memory 503 is used to store computer programs.

[0127] In one embodiment of this application, when the processor 501 executes the program stored in the memory 503, it implements the autonomous evolution method of the large language model provided in any of the foregoing method embodiments.

[0128] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the autonomous evolution method for a large language model as provided in any of the foregoing method embodiments.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0131] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0132] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for the autonomous evolution of a large language model, characterized in that, The method includes: Obtain current environment information and global strategy, wherein the current environment information is real-time interaction status data in the task environment, and the global strategy is a unified behavioral guideline for the large language model to perform decision-making operations; The current environment information and the global policy are input into a multi-model set, which includes multiple large language models and multiple reinforcement learning algorithms. Each of the large language models generates action instructions based on the current environment information and the global policy, and each of the reinforcement learning algorithms generates action instructions based on the current environment information and the mathematical model. The accuracy of the action instructions generated by the reinforcement learning algorithm is higher than a preset accuracy threshold. The action instructions are used to indicate the decision-making operations that need to be performed to complete the goal in the current task environment. The optimal instruction with the highest accuracy is selected from all action instructions output by the multi-model set, and the global policy is updated based on the optimal instruction to complete one evolution cycle of the global policy. Based on the updated global strategy and current environment information, action instructions are regenerated through the multi-model set. The steps of optimal instruction selection and global strategy update are repeated until the preset loop termination condition is reached. The autonomous evolution of the large language model is achieved through the autonomous evolution of the global strategy.

2. The method according to claim 1, characterized in that, The optimal instruction with the highest accuracy is selected from all action instructions output by the multi-model set, including: All action commands output by the multi-model set are aggregated to form an action command set; Each large language model and each reinforcement learning algorithm in the multi-model set is used to score each non-self-generated action instruction in the action instruction set according to a preset scoring rule; The final score of each action instruction is obtained based on all the scores corresponding to each action instruction. The action instruction with the highest final score is selected by the intelligent agent and used as the optimal instruction, wherein the intelligent agent is a large language model outside the multi-model set.

3. The method according to claim 2, characterized in that, The action instructions generated by the large language model include the instruction content and the reason for instruction generation, while the action instructions generated by the reinforcement learning algorithm only include the instruction content. Scoring each action instruction includes: Obtain the instruction type for each action instruction, wherein the instruction type is used to indicate whether the action instruction belongs to a large language model or a reinforcement learning algorithm; If the action instruction is determined to belong to the large language model based on the instruction type, a score is given based on the matching degree between the instruction content and the current environment information, as well as the logical rigor of the reason for the instruction generation. If the action instruction is determined to belong to a reinforcement learning algorithm based on the instruction type, a score is given based on the matching degree between the instruction content and the current environment information.

4. The method according to claim 2, characterized in that, Each large language model and each reinforcement learning algorithm in the multi-model set is used to score each non-self-generated action instruction in the action instruction set according to a preset scoring rule, including: Randomly shuffle all action instructions in the action instruction set; Each large language model and each reinforcement learning algorithm in the multi-model set performs blind scoring on each action instruction that is not generated by itself and whose order is shuffled, according to preset scoring rules.

5. The method according to claim 1, characterized in that, Based on the updated global policy and current environment information, the action instructions are regenerated through the multi-model set, including: Based on the updated global strategy, the self-awareness and world awareness of each large language model are optimized to generate a new round of self-awareness and world awareness for each large language model. The self-awareness is the cognitive judgment formed by the large language model based on its own state, and the world awareness is the comprehensive analysis and judgment of the large language model on the current task environment and its own executable decision-making operations in combination with the action instructions output by other large language models. The new round of self-awareness, world awareness, and current environmental information are input into the corresponding large language model to obtain the action instructions regenerated by the large language model; The current environment information is input into each reinforcement learning algorithm to obtain the action instructions regenerated by each reinforcement learning algorithm.

6. The method according to claim 2, characterized in that, The method further includes: After detecting that the current loop count has reached the preset round threshold, the final score corresponding to the action instructions generated by each language model in each round is calculated. The average score of the action instructions generated by the same large language model in multiple rounds of loops is calculated to obtain the target score corresponding to each language model. The agent eliminates the large language model with the lowest target score, and a new multi-model set is formed based on the remaining large language models and the multiple reinforcement learning algorithms. Based on the new multi-model set, the next loop will execute the steps of generating action instructions, selecting the optimal instruction, and updating the global policy.

7. The method according to claim 2, characterized in that, The preset loop termination conditions include: The agent detects that the final score of each action command generated by the multi-model set is higher than a set score threshold; or, The agent detects that the current cycle count has reached a set round threshold.

8. An autonomous evolution device for a large language model, characterized in that, The device includes: The acquisition module is used to acquire current environment information and global strategy, wherein the current environment information is real-time interaction status data in the task environment, and the global strategy is a unified behavior rule for the large language model to perform decision-making operations; The input module is used to input the current environment information and the global policy into a multi-model set, which includes multiple large language models and multiple reinforcement learning algorithms. The generation module is used to generate action instructions based on the current environment information and the global policy through each of the large language models, and to generate action instructions based on the current environment information and the mathematical model through each of the reinforcement learning algorithms. The accuracy of the action instructions generated by the reinforcement learning algorithms is higher than a preset accuracy threshold. The action instructions are used to indicate the decision operation that needs to be performed to complete the goal in the current task environment. The update module is used to select the optimal instruction with the highest accuracy from all action instructions output by the multi-model set, and update the global policy based on the optimal instruction to complete one evolution cycle of the global policy. The repetitive execution module is used to regenerate action instructions based on the updated global strategy and current environment information through the multi-model set, and repeatedly execute the steps of optimal instruction selection and global strategy update until the preset loop termination condition is reached. The autonomous evolution of the large language model is achieved through the autonomous evolution of the global strategy.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images