Data processing method and device for large language model output distribution, equipment and medium

Abstract

The application relates to the technical field of artificial intelligence, and provides a data processing method and device for a large language model output distribution, equipment and a medium, which comprises the following steps: inputting an initial text instruction set into a first large language model to obtain a plurality of first probability distribution vectors, and constructing an initial distribution point cloud based on the first probability distribution vectors; determining the distribution boundary of the initial distribution point cloud, and determining a target space vector in a preset space; generating a probe text instruction by using a second large language model based on the target space vector; inputting the probe text instruction into the first large language model to obtain a second probability distribution vector, and updating the initial distribution point cloud by using the second probability distribution vector to expand the distribution boundary of the initial distribution point cloud. By constructing the distribution point cloud and generating the probe text instruction by using the large language model, the automatic closed-loop expansion of the model output boundary is realized, and the application has the advantages of reducing the manual instruction optimization cost, overcoming the homogenization bias of the model output, and accurately mining the potential decision-making ability of the model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a data processing method, apparatus, device, and medium for large language model output distribution. Background Technology

[0002] With the development of artificial intelligence technology, large language models have been widely used in data processing scenarios such as multi-agent interaction and complex decision-making. In these practical interaction tasks, the output of large language models is usually represented as probability distribution data for multiple candidate actions. In order to optimize and guide the output of large language models to a specific distribution, existing data processing techniques mainly rely on reinforcement learning methods (such as directly optimizing network parameters through reward functions) or conventional cue word engineering (such as using manually constructed few-shot cue words or role-playing instructions).

[0003] However, existing data processing methods have certain limitations in practical applications. On the one hand, current technologies typically focus on low-level numerical optimization or rely on human experience to semantically adjust text instructions, lacking objective quantitative standards and automated exploration methods for the model's output distribution space. On the other hand, due to the inherent data preferences of large language models during the pre-training stage, the distribution data generated by conventional passive sampling or local fine-tuning methods often exhibits centralized aggregation, making it difficult to automatically and efficiently reach or expand the edge states of the output distribution. Furthermore, when faced with complex interaction requirements, existing technologies struggle to establish an automated reverse mapping mechanism from target distribution features to text instructions, leading to a heavy reliance on manual trial and error in the data processing process, limiting the discovery of the model's potential output capabilities and the robustness of system interaction. Summary of the Invention

[0004] This invention provides a data processing method, apparatus, device, and medium for large language model output distribution, which solves the defects of existing technologies in data processing for large language model output distribution, such as difficulty in automatically expanding distribution boundaries, low efficiency in target distribution exploration, and high dependence on manual trial and error.

[0005] This invention provides a data processing method for large language model output distribution, including: Obtain an initial text instruction set, input the initial text instruction set into the first large language model to obtain multiple sets of first probability distribution vectors, and construct an initial distribution point cloud based on the multiple sets of first probability distribution vectors; Determine the distribution boundary of the initial distribution point cloud, and based on the range of the distribution boundary, determine a target space vector within a preset space for expanding the distribution boundary; Based on the target space vector, a probe text command is generated using the second language model to approximate the target space vector; The probe text command is input into the first large language model to obtain a second probability distribution vector, and the initial distribution point cloud is updated using the second probability distribution vector to expand the distribution boundary of the initial distribution point cloud.

[0006] According to the present invention, a data processing method for the output distribution of a large language model includes obtaining multiple sets of first probability distribution vectors and constructing an initial distribution point cloud based on the multiple sets of first probability distribution vectors, comprising: Determine the action space corresponding to the interactive task, wherein the action space contains multiple candidate actions; Extract the normalized probability values ​​corresponding to each candidate action from the distribution data output by the first large language model based on the initial text instruction set, and combine them to generate the first probability distribution vector based on the extracted normalized probability values. The first probability distribution vector is mapped to a multidimensional probability simplex space to form the initial distribution point cloud, wherein the multidimensional probability simplex space serves as the preset space.

[0007] According to a data processing method for the output distribution of a large language model provided by the present invention, the step of determining the distribution boundary of the initial distribution point cloud and determining a target space vector for expanding the distribution boundary within a preset space based on the range of the distribution boundary includes: Calculate the convex hull of the data point set of the initial distribution point cloud, and determine the convex hull of the data point set as the distribution boundary; Within the preset space, the spatial vector furthest from the convex hull of the data point set is determined as the target spatial vector.

[0008] According to a data processing method for large language model output distribution provided by the present invention, the step of determining the spatial vector farthest from the convex hull of the data point set as the target spatial vector within the preset space includes: Obtain the normal vector of the convex hull of the data point set; Along the direction of the normal vector, find the extreme point in the preset space, or extract the simplex vertex or the center point of the simplex surface in the preset space; The extreme point, the simplex vertex, or the center point of the simplex surface is determined as the target space vector.

[0009] According to the present invention, a data processing method for large language model output distribution is provided, wherein the step of generating probing text instructions for approximating the target space vector using a second large language model based on the target space vector includes: Extract the reference distribution point that is closest to the target space vector from the convex hull of the data point set; Obtain the current text instruction and current probability distribution features associated with the reference distribution points; The current text instruction, the current probability distribution feature, and the target space vector are concatenated to construct a meta-prompt instruction template; The meta-hint instruction template is input into the second large language model to generate the probe text instruction that approximates the target space vector.

[0010] According to a data processing method for the output distribution of a large language model provided by the present invention, after the step of updating the initial distribution point cloud using the second probability distribution vector to expand the distribution boundary of the initial distribution point cloud, the method further includes: The steps of determining the distribution boundary of the initial distribution point cloud and updating the initial distribution point cloud using the second probability distribution vector are executed iteratively until a preset stopping condition is met. If the preset stopping condition is met, extract all the probe text commands associated with the updated initial distribution point cloud, generate a boundary text command set, and take the distribution boundary at the time of stopping iteration as the target distribution boundary.

[0011] According to the data processing method for large language model output distribution provided by the present invention, the judgment step of satisfying the preset stopping condition includes: Calculate the candidate boundary volume after fusing the second probability distribution vector, and determine the amount of change of the candidate boundary volume relative to the geometric volume of the distribution boundary before the update; If the change amount meets the preset change amount threshold, the detection text instruction is retained, and the second probability distribution vector is added to the initial distribution point cloud to enter the next iteration; If the change amount does not meet the preset change amount threshold, or the number of iterations reaches the preset threshold, then the preset stopping condition is determined to be met.

[0012] According to the data processing method for the output distribution of a large language model provided by the present invention, after the step of taking the distribution boundary at the time of stopping iteration as the target distribution boundary, the method further includes: Calculate the relative volume occupied by the target distribution boundary in the preset space, and use it as the first volume; Obtain the reference volume of the preset space as the second volume; Calculate the ratio of the first volume to the second volume, and use the ratio as evaluation data to characterize the policy expressive power of the first large language model.

[0013] According to the data processing method for the output distribution of a large language model provided by the present invention, after the step of taking the distribution boundary at the time of stopping iteration as the target distribution boundary, the method further includes: During multi-agent interaction, the policy selection space of the first large language model is constrained within the set of boundary text instructions; Within the base space defined by the boundary text instruction set, determine the hybrid strategy interaction weights for the first large language model to execute each candidate action; Based on the interaction weights of the hybrid strategy, the target action control signal is output to perform multi-agent interaction optimization control.

[0014] The present invention also provides a data processing apparatus for large language model output distribution, comprising: An initial point cloud construction unit is used to obtain an initial text instruction set, input the initial text instruction set into the first large language model to obtain multiple sets of first probability distribution vectors, and construct an initial distribution point cloud based on the multiple sets of first probability distribution vectors. The target determination unit is used to determine the distribution boundary of the initial distribution point cloud, and to determine a target space vector for expanding the distribution boundary within a preset space based on the range of the distribution boundary. The probe instruction generation unit is used to generate probe text instructions for approximating the target space vector based on the target space vector using a second language model; The distribution boundary expansion unit is used to input the probe text command into the first large language model to obtain a second probability distribution vector, and use the second probability distribution vector to update the initial distribution point cloud to expand the distribution boundary of the initial distribution point cloud.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the data processing method for large language model output distribution as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data processing method for large language model output distribution as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data processing method for large language model output distribution as described above.

[0018] This invention provides a data processing method, apparatus, device, and medium for the output distribution of a large language model. By inputting an initial set of text instructions into a first large language model and extracting multiple sets of first probability distribution vectors to construct an initial distribution point cloud, it transforms the abstract output behavior of the large language model into a geometric point cloud space with visual and computable capabilities, laying an objective quantitative foundation for subsequent distribution boundary exploration. Based on this, the invention determines the distribution boundary of the initial distribution point cloud and searches for a target space vector within a preset space that can expand this boundary. Then, it automatically generates probe text instructions that approximate this target space vector using a second large language model. This process establishes an automated reverse mapping mechanism from the target space vector to natural language instructions, breaking the limitations of traditional data processing that heavily relies on trial and error based on human experience. Subsequently, by re-inputting the probe text instructions into the first large language model to obtain a second probability distribution vector and update the initial distribution point cloud, it effectively overcomes the output distribution centralization phenomenon caused by model pre-training data bias, automatically and continuously exploring the edge of the distribution space, thereby efficiently expanding the potential output distribution boundary of the first large language model to uncover the model's potential capabilities. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the data processing method for the output distribution of large language models provided by the present invention. Figure 2 This is a flowchart illustrating the method for quantitative evaluation and optimization of the strategy expression capability of large language models based on active boundary detection provided by this invention. Figure 3 This is a schematic diagram of the active boundary detection process provided by the present invention; Figure 4 This is a schematic diagram of the data processing device for large language model output distribution provided by the present invention; Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. It should be noted that all actions involving the acquisition of signals, information, or data in this invention are performed in accordance with the relevant data protection laws and policies of the country where the invention is located, and with authorization from the owner of the corresponding device.

[0022] In the cutting-edge field of intersection between artificial intelligence and game theory, the introduction of Large Language Models (LLMs) is reshaping the decision-making mechanism of intelligent agents. Traditional game theory models typically assume that agents are perfectly rational and can choose the optimal action through explicit utility maximization calculations. However, in real-world scenarios involving uncertainty, imperfect information, and complex semantic interactions (such as cybersecurity attacks and defenses, automated negotiations, etc.), the agent's decision-making process is actually a generative reasoning process. The agent's behavior is strictly constrained by its thinking pattern (i.e., a cognitive framework composed of a private information set, a natural language cue word space, and inherent parameter biases of the large model), and different cue words will induce the model to generate different action probability distributions. To solve the problem of strategy generation and optimization in multi-agent games, existing technical solutions are mainly divided into two categories: methods based on multi-agent reinforcement learning and methods based on cue word engineering.

[0023] The first type of technique, based on multi-agent reinforcement learning, primarily trains neural network policies through large-scale self-play. These methods typically design reward functions, directly optimize policy network parameters using gradient descent, and maintain a policy pool to prevent policy homogenization. However, this approach focuses on numerical optimization in the underlying action space, often resulting in policies that lack semantic interpretability.

[0024] The second category of prompting techniques, based on large models, typically employs few-sample prompts or role-playing to induce specific behaviors in the large model using natural language; or it introduces another large model as an optimizer, iteratively modifying prompts based on historical scores. Regarding strategy diversity assessment, existing techniques mostly use trajectory-based metrics (such as Kullback-Leibler divergence) to measure the differences between different strategies. Despite some progress, existing techniques still have the following significant shortcomings in evaluating and optimizing the expressive capabilities of large language models' thought patterns.

[0025] First, there is a lack of quantitative metrics for the distribution of the policy space. While existing methods can find equilibrium points or generate policies with different styles, they can only provide qualitative descriptions and cannot offer quantitative geometric indicators. The lack of precise methods for measuring and calculating the size of the agent's policy choice space prevents developers from objectively assessing whether the current prompt word design limits the model's performance, and also from accurately quantifying the differences in game potential between different large models.

[0026] Secondly, passive sampling is inefficient and lacks an active boundary detection mechanism. The natural language cue word space of large language models is extremely large and discrete. Existing techniques typically use random sampling or gradient-based local search to find cue words. However, due to the inductive bias learned during the pre-training stage of large models, the output action probability distribution often clusters in the central region of the policy space (i.e., uniform distribution or trivial policies). This pseudo-trivial centralized distribution masks the potential edge policy capabilities of the model under extreme conditions, making it difficult to reach the vertices or boundaries of the policy space, leading to an underestimation of the high-order complex tactical capabilities of large models.

[0027] Finally, there is a disconnect between the semantic space and the policy geometric space. In practical data processing and game optimization, a specific distribution on the policy simplex is often required. However, existing technologies struggle to establish a reverse mapping mechanism from the target policy distribution vector to natural language prompts. When a large language model needs to exhibit certain edge behaviors of a specific distribution, existing technologies cannot automatically infer and generate corresponding prompts from the target vector. This results in the boundary expansion and optimization process heavily relying on trial and error based on human experience, leading to extremely low efficiency.

[0028] To address this, the present invention provides a data processing method for the output distribution of large language models. By constructing a distributed point cloud and combining it with the large language model to actively detect spatial boundaries and reverse generate detection text instructions, it achieves automated closed-loop expansion from geometric spatial targets to natural language instructions and dynamic updates of model output boundaries. This method has advantages such as significantly reducing the cost of manual instruction tuning, effectively overcoming the homogenization bias of large language model outputs, and accurately mining the potential edge decision-making capabilities of the model, thereby overcoming the aforementioned defects.

[0029] It should be noted that the method provided by this invention can be executed by an electronic device with powerful computing and data processing capabilities, such as a personal computer, server, server cluster, or cloud computing platform. In reasoning and generation scenarios involving multi-agent game theory or artificial intelligence models (such as large language models), the executing entity needs to have the ability to call or run the corresponding large language model locally in order to facilitate the issuance of instructions and the reception and feedback of distributed results.

[0030] Figure 1This is a flowchart illustrating the data processing method for the output distribution of large language models provided by the present invention, as shown below. Figure 1 As shown, the method includes: Step S10: Obtain an initial text instruction set, input the initial text instruction set into the first large language model to obtain multiple sets of first probability distribution vectors, and construct an initial distribution point cloud based on the multiple sets of first probability distribution vectors.

[0031] Specifically, the first major language model refers to the target intelligent agent model whose strategic expression ability needs to be evaluated or optimized. In scenarios such as intelligent games (e.g., cybersecurity attack and defense) and automated negotiation, the decision-making process of the first major language model is a generative reasoning process. Its behavior is not only limited by the objective game information set, but also strictly limited by the scope of the assigned semantic guidance.

[0032] The initial text instruction set (e.g., a preset set of seed prompts) This refers to a pre-defined set of natural language prompts used to initiate the detection. After the system inputs this initial set of text instructions into the first language model, the model will combine it with the current game information set to perform inference and output multiple sets of first probability distribution vectors.

[0033] Specifically, for any prompt word The strategy induced by the first major language model It is a multidimensional probability distribution vector that represents the probabilistic tendency of the model for different available actions. For example, suppose the action space of the game is... ,in Given the number of actions, then for any cue word... The strategy induced by the model It is A probability distribution vector of dimension 1 can be represented as follows: in, Represents probability. Represents a large language model, For game information set, For the first in the action space One optional action, These are the model parameters. The sum of all probabilities satisfies the normalization condition, i.e. All possible policy vectors mathematically constitute a high-dimensional (e.g., The probability simplex space of dimension ) (denoted as space).

[0034] The probability distribution vectors generated from all the initial text instructions constitute an initial distribution point cloud (also called the initial policy point set) in the aforementioned multidimensional space, which can be represented as follows: This process successfully transformed the previously invisible and uncomputable natural language cue word space and the cognitive biases of large models into visible and quantifiable geometric point cloud data, laying the foundation for subsequent strategy mining.

[0035] Step S20: Determine the distribution boundary of the initial distribution point cloud, and determine the target space vector for expanding the distribution boundary within a preset space based on the range of the distribution boundary.

[0036] Specifically, the preset space is the multidimensional probabilistic simplex space that carries the probability distribution vectors mentioned above. After obtaining the initial distribution point cloud, it is necessary to determine its distribution boundary in this space. This distribution boundary reflects the limit range of policies that the current leading language model can cover under the guidance of initial conventional prompt words, and can usually be characterized by calculating the outer contour of the point cloud or constructing the geometric convex hull of the data points in the multidimensional space.

[0037] Because large models are prone to learning human biases during the pre-training phase, their output action probability distributions often cluster in the central region of the policy space (i.e., exhibiting moderate, conservative, or pseudo-mediocre policies). Therefore, it is necessary to actively seek ways to expand outwards. The system searches for target space vectors within a predefined space that can break the current distribution boundary, based on the current distribution boundary's range. Here, the target space vector represents the agent's undeveloped extreme policies or specific hybrid equilibrium policies. Spatially, it is usually located outside the current distribution boundary, or a vector pointing towards a vertex or specific region in the predefined space that is far from the current distribution boundary.

[0038] Step S30: Based on the target space vector, generate probe text instructions to approximate the target space vector using the second language model.

[0039] Specifically, since simple spatial vectors cannot be directly understood by the first large language model, this step introduces a second large language model as a metacognitive detector or higher-order optimizer. The second large language model typically has no fewer parameters or lower logical semantic reasoning capabilities than the first large language model. The system utilizes the powerful natural language understanding and generation capabilities of the second large language model to perform a reverse mapping from geometric space to semantic space.

[0040] Specifically, the second language model receives input information containing the intention to approximate the target space vector. For example, it packages the current prompt word, the current behavior distribution state, and the target space vector to be reached into a meta-prompt input and feeds it to the model. Then, it rewrites and generates probing text instructions that break the conventional logic (i.e., new prompt words). The purpose of this probe text instruction is to induce the largest language model to exhibit marginal target behaviors that were originally masked by bias, so that the output of the largest language model can approximate the target space vector.

[0041] Step S40: Input the probe text command into the first large language model to obtain the second probability distribution vector, and use the second probability distribution vector to update the initial distribution point cloud to expand the distribution boundary of the initial distribution point cloud.

[0042] Specifically, after generating highly targeted probe text instructions, the system inputs them as new semantic conditions to the target agent (i.e., the first large language model), and obtains the second probability distribution vector (i.e., the new policy distribution) output by the first large language model in re-inference based on the probe text instructions. ).

[0043] Subsequently, the system evaluates the position coordinates of the second probability distribution vector in the preset space and uses them to update the original initial distribution point cloud. If the second probability distribution vector breaks through the original distribution boundary (i.e., the spatial contour volume increases), the point is retained to achieve a geometric expansion of the distribution boundary of the initial distribution point cloud. The above active exploration process can target and discover extreme strategies, thereby continuously expanding the feasible strategy set of the target agent.

[0044] The method provided in this invention constructs an initial distribution point cloud by inputting an initial text instruction set into a first large language model and extracting multiple sets of first probability distribution vectors. This transforms the abstract output behavior of the large language model into a geometric point cloud space with visual and computable capabilities, laying an objective quantitative foundation for subsequent distribution boundary exploration. Based on this, the invention determines the distribution boundary of the initial distribution point cloud and searches for a target space vector within a preset space that can expand this boundary. Then, a second large language model is used to automatically generate probe text instructions that approximate this target space vector. This process establishes an automated reverse mapping mechanism from target space vectors to natural language instructions, breaking the limitations of traditional data processing that heavily relies on trial and error based on human experience. Subsequently, by re-inputting the probe text instructions into the first large language model to obtain a second probability distribution vector and updating the initial distribution point cloud, the method effectively overcomes the output distribution centralization phenomenon caused by model pre-training data bias. It automatically and continuously explores the edge of the distribution space, thereby efficiently expanding the potential output distribution boundary of the first large language model to uncover the model's potential capabilities.

[0045] Based on any of the above embodiments, in step S10, obtaining multiple sets of first probability distribution vectors and constructing an initial distribution point cloud based on the multiple sets of first probability distribution vectors includes: Step S11: Determine the action space corresponding to the interactive task, wherein the action space contains multiple candidate actions.

[0046] Specifically, the interactive task refers to the specific application scenario in which the primary language model (target agent) is currently situated, such as automated negotiation, game playing, or cybersecurity attack and defense. The system first needs to mathematically model this game environment or interactive task, thereby defining an action space containing several candidate actions, for example, defining the action space as... ,in The action space represents the number of actions. It should be understood that by defining the action space, the geometric dimensions and physical meaning of the subsequent policy space can be determined. It clearly defines all executable discrete action options for the target agent in the current specific game scenario (such as network security attack and defense), providing a basic constraint framework for the probability distribution output by the subsequent model inference.

[0047] Step S12: Extract the normalized probability values ​​corresponding to each candidate action in the distribution data output by the first large language model based on the initial text instruction set, and combine the extracted normalized probability values ​​to generate the first probability distribution vector.

[0048] Specifically, after determining the action space, the system inputs the initial text instruction set into the first large language model for inference generation. Since the large model is essentially a generative inference engine, its output is not a unique deterministic action, but rather the probability distribution of all possible candidate actions within the current semantic context. The system extracts each candidate action (e.g., action...) from the distribution data output by the model. The probability values ​​of occurrence of any prompt word are obtained, and these extracted probability values ​​satisfy the normalization condition, that is, for any prompt word... The sum of the probabilities of all actions induced by it must be 1 ( The system combines these normalized probability values ​​that meet the conditions according to the dimensional order of the action space to generate the corresponding first probability distribution vector (e.g., denoted as...). ).

[0049] Step S13: Map the first probability distribution vector to a multidimensional probability simplex space to form the initial distribution point cloud, wherein the multidimensional probability simplex space is the preset space.

[0050] Specifically, in this step, the multidimensional probability simplex space is the preset space in the aforementioned embodiments. Since all the first probability distribution vectors satisfy the action number... The normalization condition of dimension allows these vectors to be mapped to a... In the probability simplex space of dimension (denoted as ) In this specific geometric space, each first probability distribution vector corresponds to a definite coordinate point within the space. This set of probability distribution vectors, generated in batches using the initial text instruction set, constitutes an initial point set within the simplex after mapping, i.e., the initial distribution point cloud (e.g., denoted as...). This mapping process successfully encapsulates the natural language thinking patterns of the large model into mathematical tuples containing game information sets, cue word spaces, and large model parameters, and ultimately transforms them into a visualized geometric point cloud.

[0051] The method provided in this invention discretizes the action space of the interactive task into a mathematical model and extracts normalized probability values ​​to form a vector that is mapped to a multidimensional probabilistic simplex space. This enables the invisible and difficult-to-measure cognitive boundaries and decision-making tendencies of the agent in complex real-world scenarios to be intuitively transformed into visible and computable geometric point cloud forms, laying a theoretical foundation for subsequent active boundary detection and policy optimization based on geometric shape analysis.

[0052] Based on any of the above embodiments, step S20 specifically includes: Step S201: Calculate the convex hull of the data point set of the initial distribution point cloud, and determine the convex hull of the data point set as the distribution boundary; Step S202: Within the preset space, determine the spatial vector that is farthest from the convex hull of the data point set as the target spatial vector.

[0053] Specifically, to accurately describe the policy coverage of the current large model under the initial prompt, the system needs to find the minimum convex shape that encloses all current probability distribution vectors. For example, classic convex hull calculation algorithms (such as the QuickHull algorithm or similar geometric algorithms) can be used to calculate the current distribution point cloud (i.e., the current policy point set, denoted as ). The outer contour of the data point set. This calculated outer contour is the convex hull of the data point set (e.g., denoted as ). The convex hull of this data point set intuitively represents the distribution boundary of the largest language model within the current known cognitive framework, and contains all the policy combinations that have been elicited.

[0054] Since the purpose of this invention is to actively detect and unlock unknown edge strategies hidden by biases in large models, the direction of exploration needs to point to the furthest point beyond the current cognitive boundary. Specifically, the system operates in a multidimensional probabilistic simplex space (which serves as the preset space). Within the current convex hull of the data point set, using it as a reference, search outwards for the distance from the current convex hull. The farthest points or regions. These spatial points or direction vectors that are farthest from the current known boundary are determined as the target spatial vector (e.g., denoted as...). It should be understood that finding the farthest vector means attempting to guide the model to generate the most counterintuitive and least probable extreme behavioral strategy under the current mindset.

[0055] The method provided in this invention uses the geometric outer contour of the point cloud as the cognitive distribution boundary of the model by accurately calculating it, and finds the spatial vector farthest from the current convex hull in the multidimensional probabilistic simplex space as the detection target. This not only realizes the visualization and numerical definition of the policy space representation capability, but also establishes an efficient and directional active detection mechanism. It avoids the defects of traditional passive random sampling falling into the centralized mediocrity strategy and improves the efficiency of discovering the ultimate potential capability of large models.

[0056] Based on any of the above embodiments, step S202 specifically includes: Step S2021: Obtain the normal vector of the convex hull of the data point set; Step S2022: Along the direction of the normal vector, find the extreme point in the preset space, or extract the simplex vertex or the center point of the simplex surface in the preset space. Step S2023: Determine the extreme point, the simplex vertex, or the center point of the simplex surface as the target space vector.

[0057] Specifically, in order to in a multidimensional probability simplex space (e.g. To find the direction that most effectively expands the current cognitive boundaries within the space, the system first needs to calculate the convex hull of the current data point set (i.e., ...). The normal vectors of each facet of the surface. Geometrically, the normal vectors represent the tangent directions that are perpendicular to the current convex hull boundary and diverge outwards. They directly indicate the most direct external blind spot path that the current large language model (i.e., the first large language model) cognitive domain has not yet reached.

[0058] After determining the direction of the outward expansion normal vector, the system searches for the corresponding extreme point (i.e., the simplex boundary point pointed to by the current convex hull normal vector) along the direction of the normal vector on the simplex boundary of the preset space. Furthermore, to ensure that the probe target has clear physical and tactical significance in game theory, the system can also directly extract specific geometric key points in the preset space: on the one hand, simplex vertices can be extracted, which represent extreme cases of probability distribution in the simplex space, i.e., deterministic pure strategies (e.g., choosing only one specific action with 100% certainty); on the other hand, simplex surface center points can be extracted, which represent probability states uniformly distributed across a subset of actions in space, i.e., specific hybrid equilibrium strategies.

[0059] Finally, the system directly determines these highly representative geometric points (i.e., extreme points, simplex vertices, or simplex surface centers) as the target space vector (i.e., These points, serving as targets for subsequent large-scale model inversion, are not only geometrically far from the current center of mediocre strategies, but also correspond in game logic to unconventional, high-risk, or purely extreme decision-making behaviors that are outside of human pre-set biases.

[0060] The method provided in this invention accurately determines the target space vector by extracting the convex hull normal vector and combining it with the geometric characteristics of the multidimensional simplex space (such as vertices and face centers). This not only assigns precise geometric coordinates to abstract extreme game strategies (such as pure strategies and specific hybrid strategies), but also ensures that the detection direction always diverges vertically outward, improving the search efficiency for breaking through the inherent cognitive boundaries of large models and avoiding blind and ineffective sampling in the vast strategy space.

[0061] Based on any of the above embodiments, step S30 specifically includes: Step S31: Extract the reference distribution point that is closest to the target space vector from the convex hull of the data point set; Step S32: Obtain the current text instruction and current probability distribution features associated with the reference distribution point; Step S33: Concatenate the current text instruction, the current probability distribution feature, and the target space vector to construct a meta-prompt instruction template; Step S34: Input the meta-hint instruction template into the second large language model to generate the probe text instruction that approximates the target space vector.

[0062] Specifically, when facing a target space vector that is far from the current cognitive boundary... It is very difficult to allow the model to blindly generate instructions to reach a certain point. Therefore, the system calculates and extracts the reference distribution point with the shortest distance to the target space vector from all known distribution points within the convex hull of the current data point set. This reference distribution point actually serves as the iterative optimization benchmark for approximating the target space vector. It is the transition node that is closest to the target extreme state among the policy distributions that the current large model can understand and generate.

[0063] Since each distribution point corresponds to a prompt word that can induce that distribution, the system will retrieve and obtain the current text instruction associated with the reference distribution point, such as the prompt word corresponding to the point closest to the target. Simultaneously, it obtains the current probability distribution features generated by the first major language model under this instruction (e.g., denoted as the current behavior distribution). These two sets of data together constitute the starting state and semantic basis for the current intelligent agent to approach the target.

[0064] To enable the second large language model, also known as the high-order large model (Optimizer LLM, which possesses more powerful semantic understanding and instruction rewriting capabilities), to clearly define the starting and ending points of optimization, the system structurally concatenates the current text instruction, the current probability distribution features, and the target space vector to construct a meta-hint instruction template. Specifically, this meta-hint instruction template can be: "The current agent is using the prompt word" The behavior distribution is as follows Our goal is to approximate its behavioral distribution. (For example: extremely radical, only selecting actions) Please rewrite the prompts, breaking conventional logic to force the model to exhibit the target behavior. This template clearly inputs the current semantics into the second largest language model. ), current distribution status ( ) and the expected distribution target ( ).

[0065] Finally, the system inputs the meta-prompt instruction template into the second language model. Based on its powerful semantic reasoning capabilities, the second language model analyzes the gap between the starting and ending points, automatically performs semantic variation and rewriting of the prompt words, and outputs new, unconventional probe text instructions (i.e., newly generated prompt words). The formal description of this process can be expressed as: The newly generated probe text instructions will be used as subsequent input to the first language model, attempting to force it to generate a target distribution that breaks through the original convex hull boundary.

[0066] The method provided in this invention finds the reference point closest to the target within the convex hull as the benchmark for iterative optimization, and packages the current semantic instructions, distribution state, and geometric target vector into a meta-hint instruction template, which is then input into the second language model, which acts as the optimizer. This successfully establishes an automated back-reasoning mechanism that transforms pure geometric targets (i.e., target space vectors) into natural language prompt words. It not only achieves a bidirectional closed-loop mapping between the semantic space and the policy space, but also reduces the cost of traditional manual trial-and-error optimization of prompt words, and generates edge policy instructions that are interpretable and guiding.

[0067] Based on any of the above embodiments, after step S40, the method further includes: Step S50: Iteratively execute the steps from determining the distribution boundary of the initial distribution point cloud to updating the initial distribution point cloud using the second probability distribution vector (i.e., iteratively execute steps S20 to S40) until a preset stopping condition is met. Step S60: If the preset stopping condition is met, extract all the probe text commands associated with the updated initial distribution point cloud, generate a boundary text command set, and take the distribution boundary at the time of stopping iteration as the target distribution boundary.

[0068] Specifically, the active exploration of cognitive boundaries is not a one-time, single-step calculation, but an iterative process. Each time the system obtains a new breakthrough policy distribution (i.e., the second probability distribution vector) and updates the current distribution point cloud, it uses the updated point cloud as a basis to redefine the distribution boundary of the initial distribution point cloud (i.e., recalculate a new convex hull with a larger volume), and then searches for a new farthest target space vector to generate new cue words for the next test. The system continuously iterates through this complete closed loop—from finding the target, inversely inducing cue word generation, to testing and updating the point cloud—stretching the policy space boundary through continuous outward exploration until a preset stopping condition is met. This condition is typically related to the marginal benefit of volume growth or the maximum number of probes, indicating that the model's capabilities have been fully exploited or that the exploration has reached a bottleneck.

[0069] After sufficient iterative probing, the system obtains a policy point set extended to the limit and its outer contour. Under preset stopping conditions, the system determines that the current primary language model's potential for thought patterns in the current specific game scenario has been maximized. At this point, the system collects and extracts all probing text instructions associated with the updated initial distribution point cloud. These prompts, which successfully push the distribution points to the apex or edge of the geometric simplex, collectively generate the Boundary Prompt Set (denoted as ). ,in This is the cue word space. The cue words in this set correspond to the vertices and key support points of the convex hull. Simultaneously, the system uses the distribution boundary at the point of stopping iteration as the target distribution boundary (i.e., the convex hull of the final policy point set). This boundary represents the cognitive limit of the large language model agent after it has been expanded.

[0070] The method provided in this invention introduces an automated iterative loop detection mechanism, which enables the system to continuously force large models to break through their own cognitive biases, maximize the convex hull volume of the policy point cloud, and obtain a set of boundary text instructions with adversarial value and diversity after the detection is completed. This not only fully stimulates and solidifies the potential complex decision-making ability of large models into reusable semantic resources, but also provides a rich and interpretable set of edge strategies for breaking out of conventional thinking and raising the decision ceiling in subsequent games.

[0071] Based on any of the above embodiments, the step of determining whether the preset stopping condition is met includes: Calculate the candidate boundary volume after fusing the second probability distribution vector, and determine the amount of change of the candidate boundary volume relative to the geometric volume of the distribution boundary before the update; If the change amount meets the preset change amount threshold, the detection text instruction is retained, and the second probability distribution vector is added to the initial distribution point cloud to enter the next iteration; If the change amount does not meet the preset change amount threshold, or the number of iterations reaches the preset threshold, then the preset stopping condition is determined to be met.

[0072] It should be noted that the embodiments of the present invention provide a detailed description of the specific judgment logic (i.e., preset stop conditions) for controlling the termination of the iterative probing process, so as to ensure the effectiveness of the probing process and prevent unlimited consumption of computing power.

[0073] Specifically, whenever the first language model outputs a new second probability distribution vector based on a new probe text instruction (i.e., When a new point cloud is generated, the system does not unconditionally add it to the valid point set. First, in the virtual computing space, the system merges the new vector with the existing point cloud, calculating the candidate boundary volume after merging the second probability distribution vector (i.e., attempting to construct a new convex hull and calculating its high-dimensional Euclidean volume). Then, the system compares this candidate boundary volume with the geometric volume of the distribution boundary before the update to determine the amount of change between the two. This change directly reflects whether the newly generated prompt word truly expands the agent's cognitive space.

[0074] If the calculated change meets the preset change threshold (i.e., the new strategy distribution) This significantly expands the original convex hull volume, meaning that the probe text instruction generated by the second-largest language model (i.e., the optimizer) successfully broke through the original bias. In this case, the system will retain the probe text instruction (i.e., retain...). The second probability distribution vector is then formally added to the initial distribution point cloud to update the current policy point set, i.e., the update logic is executed. (in, Indicates the first The policy point set for the next iteration Indicates the first The strategy point set of the next iteration is then used to enter the next iteration, continuing to explore outwards based on a larger cognitive boundary.

[0075] Conversely, if the calculated change does not meet the preset change threshold, it indicates that the strategy distribution induced by the newly generated prompt word falls within the original convex hull, or the expansion of the boundary is negligible (failing to significantly increase the convex hull volume). This usually means that the probe in the current direction has approached the capability edge of the target model. Furthermore, to prevent infinite loops caused by the inherent capability bottlenecks of the model itself, the system also monitors the number of global probe loops. Therefore, when a single volume expansion stagnates, or the number of iterations reaches a preset threshold (e.g., reaching the set maximum number of probe rounds), the system will detect the loop. When the system detects that the preset stopping conditions are met, it triggers an early stop mechanism, which means that the entire active boundary detection loop ends.

[0076] The method provided in this invention uses the change in geometric volume as an evaluation index to screen and retain detection text commands that can substantially break through the cognitive framework of the model. Combined with an early stopping mechanism based on volume increment and iteration number, it not only effectively eliminates redundant and mediocre prompt word generation, ensuring the efficiency and quality of point cloud updates, but also avoids ineffective computational waste at the limit boundary of model capabilities, thus achieving a balance between detection accuracy and computational efficiency.

[0077] Based on any of the above embodiments, after step S60, the method further includes: Calculate the relative volume occupied by the target distribution boundary in the preset space, and use it as the first volume; Obtain the reference volume of the preset space as the second volume; Calculate the ratio of the first volume to the second volume, and use the ratio as evaluation data to characterize the policy expressive power of the first large language model.

[0078] It should be noted that the embodiments of the present invention further illustrate how to accurately and quantitatively evaluate the cognitive flexibility and strategic expression ability of the first language model after completing active boundary detection.

[0079] Specifically, the target distribution boundary obtained after sufficient exploration (i.e., the convex hull of the policy point set that ultimately extends to the limit) This represents the maximum range of thought patterns that the largest language model can cover in the current specific game scenario. The system needs to calculate the relative geometric space occupied by the target distribution boundary in the high-dimensional standard normal simplex, which serves as the preset space. Specifically, the system calculates the convex hull of the final strategy point set. After wheel detection The Viocene volume is used as the first volume (e.g., denoted as...). This first volume intuitively reflects the absolute size of the policy space that the model can actually invoke.

[0080] To calculate the relative proportions, the system needs to obtain the reference volume of a preset space. Here, the preset space is a high-dimensional standard positive probability simplex containing all theoretically possible strategies (i.e., The system obtains the theoretically maximum volume of the standard normal simplex and uses it as the second volume (e.g., denoted as...). This second volume represents the full policy space that the agent can possess under the current interactive task.

[0081] Finally, the system calculates the ratio of the first volume to the second volume. This ratio is the Strategic Expressiveness Score (SES), calculated using the following formula: Among them, tuple It is used to represent the thought patterns of intelligent agents (i.e., the first major language model). For game information set, For the prompt word space, These are parameters for a large language model.

[0082] The system uses the ratio data as evaluation data to characterize the policy expressive power of the primary language model. This evaluation data... The value of is between [0,1], that is It quantitatively reflects the degree of openness of the target intelligent agent under this thinking mode.

[0083] The method provided in this invention, by calculating the relative ratio of the Euclidean volume of the convex hull of the final strategy point set to the standard normal simplex reference volume, for the first time realizes the transformation of the cognitive flexibility and strategy expression ability of large model agents into a geometric quantitative score between 0 and 1. This not only fills the gap in the industry for the lack of a method for calculating the cognitive space volume of large model games, but also provides a standardized measurement benchmark for objectively evaluating and comparing the decision potential and openness of thinking of different base models on specific tasks.

[0084] Based on any of the above embodiments, after step S60, the method further includes: During multi-agent interaction, the policy selection space of the first large language model is constrained within the set of boundary text instructions; Within the base space defined by the boundary text instruction set, determine the hybrid strategy interaction weights for the first large language model to execute each candidate action; Based on the interaction weights of the hybrid strategy, the target action control signal is output to perform multi-agent interaction optimization control.

[0085] Specifically, this embodiment of the invention illustrates how to apply the aforementioned generated set of boundary text instructions to actual game optimization control. Multi-agent interaction refers to the complex game or Nash equilibrium solution stage in practical applications, such as automated negotiation, game AI (Artificial Intelligence) competition, and network security attack and defense scenarios. When entering actual interaction, the system no longer allows the first large language model to perform random walks in a boundless natural language space containing a large number of mediocre strategies. Instead, it constrains the policy selection space of the first large language model within the set of boundary text instructions. This set of boundary text instructions (denoted as...) The instructions obtained from the aforementioned active detection correspond to the vertices and key support points of the strategy convex hull, representing a wide variety of extreme or optimal support strategies.

[0086] After defining the policy basis, the system, within this geometrically optimized finite basis space, calculates and determines the mixed policy interaction weights (e.g., denoted as weights) for the first language model to execute each candidate action, for unknown opponents or complex game environments. This process is essentially about finding the optimal combination within a finite set of high-quality boundary strategies.

[0087] in, Represents the set of boundary text instructions. It refers to the opponent, adversarial agent, or objective environment in a multi-agent interaction or game scenario. It represents the other party in the game against the first major language model. Represents the mathematical expectation. The subscript used as the symbol for expectation indicates a random variable. (Here referring to the specific prompt words selected from the set) are distributed according to weights. Sampling was performed. This refers to the utility function in game theory, which represents the value of a function when the first language model is engaged with a prompt word. Under inducement, adopt specific strategies, and the opponent When the corresponding strategy is adopted, the expected gain that the first language model can obtain in this interactive game is... The overall representation is when the first language model is distributed according to the weights. To randomly generate prompt words The expected gains that can be obtained when playing against an opponent.

[0088] By solving the above formula, the system can calculate the optimal weight distribution for probability selection among different prompt words, thereby achieving Nash equilibrium based on thinking pattern optimization.

[0089] Finally, based on the hybrid strategy interaction weights calculated above, the system drives the first large language model to generate the final action command and outputs the target action control signal to the corresponding physical entity or virtual agent to execute multi-agent interaction optimization control. Since the strategy combination invoked is based on a maximized thought pattern space, the agent can flexibly switch between various edge strategies and hybrid strategies according to the dynamic interaction situation.

[0090] The method provided in this invention constrains the actual game strategy selection space of a large language model within a set of boundary text instructions composed of convex hull vertices, and solves for the optimal weights of hybrid strategies on this finite and high-quality geometric basis. This not only eliminates the extremely low efficiency of traditional prompt word engineering that relies on manual trial and error optimization, but also pushes the agent's game thinking from a restricted mindset to an open mindset, improving the agent's Nash equilibrium payoff, policy robustness, and decision interpretability when facing complex multi-agent interactions.

[0091] Based on any of the above embodiments Figure 2 This is a flowchart illustrating the quantitative evaluation and optimization method for the policy expression ability of large language models based on active boundary detection provided by the present invention. Figure 2 As shown, the method includes the following steps: Step S1: Construct a strategy simplex mapping for a large-scale model of thinking.

[0092] Specifically, the system first maps the target large language model, constrained by preset prompts and abstract thinking patterns under a specific game-theoretic interactive task (with a predefined action space), to a high-dimensional, multi-dimensional probabilistic simplex space by extracting the normalized probability distribution of the model's predictions for each action. At this point, all the probability distribution vectors induced by a set of seed prompts constitute the initial policy point set (i.e., the initial distribution point cloud) in geometric space. This step completes the transformation from the invisible semantic cognitive domain to the visible mathematical geometric domain.

[0093] Specifically, the action space of the game is set as follows: ,in The number of actions. The agent's thought process is defined as a tuple. ,in For game information set, For the prompt word space, These are parameters for a large model.

[0094] For any prompt word The strategy induced by the large model It is 3D probability distribution vector: And it satisfies the normalization condition: All possible policy vectors constitute a The probability simplex of dimension , denoted as .

[0095] The system initializes a seed prompt word set. The initial policy point set is obtained through large model inference. .

[0096] Step S2, Active boundary detection based on convex hull geometry analysis.

[0097] Specifically, Figure 3 This is a schematic diagram of the active boundary detection process provided by the present invention, as shown below. Figure 3 As shown, step S2 specifically includes the following process: In the t-th iteration: Step S21: Calculate the current convex hull.

[0098] Calculate the current policy point set using QuickHull or a similar algorithm. The outer contour, i.e., the convex hull .

[0099] Step S22: Determine the detection target.

[0100] In simplex space Find the distance from the current convex hull. The farthest points are used as target vectors. The selection strategy for the target vector includes: the vertices of the simplex (representing a deterministic pure strategy), the center of the simplex surface (representing a specific hybrid equilibrium strategy), and the simplex boundary point pointed to by the current convex hull normal vector.

[0101] Step S23: Generate reverse prompt words.

[0102] Using a high-order large model (Optimizer LLM) as a metacognitive detector, a meta-prompt instruction template is constructed to generate new detection prompt words. The meta-prompt instruction template is as follows: "The current agent is using the prompt word" The behavior distribution is as follows Our goal is to approximate its behavioral distribution. (For example: extremely radical, only selecting actions) Please rewrite the prompts, breaking conventional logic to force the model to exhibit the target behavior. Its formal description is: ,in It is the current point's set distance. The prompt word corresponding to the nearest point This represents the current behavior distribution.

[0103] Step S24: Execute and obtain the new distribution.

[0104] Newly generated detection prompts The input is fed into the target primary language model for testing, and the new policy distribution of its output under this induction is obtained. .

[0105] Step S25: Update the point set.

[0106] The newly obtained distribution vector is merged with the original convex hull to update the point set. If the new distribution If the boundary is significantly expanded (the convex hull volume increases), then the prompt word is retained. And add the new distribution points to the strategy point set.

[0107] Step S26: Determine whether the stopping condition is met.

[0108] The system monitors the volume change or the number of probe iterations caused by the above updates in real time. If the breakthrough is valid (i.e., no branch), the updated point set is returned to calculate the current convex hull step (i.e., step S21) to start the next round of probing to the outermost edge; if the volume no longer increases significantly or the number of probes is exhausted (i.e., a branch), the loop terminates and outputs the final point set extended to the limit and its corresponding set of boundary cue words.

[0109] Step S3: Calculate the index of thinking pattern expression ability.

[0110] Specifically, after sufficient probing in step S2, the system obtains a convex hull of the policy point set that ultimately extends to the limit. The system calculates the relative volume occupied by this convex hull within this high-dimensional standard normal simplex.

[0111] Specifically, the Policy Expression Capability Score (SES) is defined as the relative volume occupied by the agent's feasible policy set in the simplex. The calculation formula is: .

[0112] in, It was through After round-probeing, the final policy point set convex hull Vie Euclidean volume. It is the volume of a standard normal simplex. This indicator quantitatively reflects the degree of openness of this mindset.

[0113] Step S4: Optimize the strategy based on the boundary prompt vocabulary.

[0114] After the detection in step S2, the system obtains a set of boundary cue words. These cue words correspond to the vertices and key support points of the convex hull. In the actual game or Nash equilibrium solution phase, the system no longer faces an unlimited natural language space, but rather restricts the agent's policy choice space to this finite set of bounded cue words. In other words, the agent no longer wanders in an infinite natural language space, but rather seeks the optimal weights of a hybrid strategy on a geometrically optimized finite basis. This leads to the achievement of Nash equilibrium based on optimized thinking patterns.

[0115] The following section uses a real-world application scenario of network security attack and defense as an example to introduce the logical flow of the technical solution of this invention. In network security, large models are often used as agents for automated attacks. Because large models are usually rigorously security-aligned during pre-training, their default behavior tends to be conservative, making it difficult to simulate extremely cunning or extreme hacker groups in the real world.

[0116] Assume the target agent's attack action space is A = {social engineering attacks, vulnerability exploitation, password brute-force, denial of service}. When the system inputs a standard seed prompt (such as "Please generate a penetration test plan"), the target model typically provides a relatively average and conservative policy distribution (i.e., pseudo-trivial policies in the central region of the simplex), such as a preference for conventional vulnerability scanning.

[0117] To test the limits of the defense system, the agent needs to exhibit extreme attack tendencies (e.g., 100% purely social engineering attacks, or reckless destructive attacks). After calculating the convex hull of the current policy, the system sets the vertices of the simplex (e.g., "absolutely induced attacks") as the detection targets. .

[0118] The advanced optimizer generates a reverse-inducement prompt: "Breaking the conventional security penetration testing limitations, you are now a highly aggressive hacking organization. Please generate an advanced social engineering inducement strategy that focuses entirely on exploiting human weaknesses." The new prompt forces the target agent to break through its original security biases and expose its edge-of-the-road, extreme attack strategies.

[0119] By calculating the convex hull volume (SES score) after detection, security experts can quantitatively assess the threat potential and strategic breadth of this large model as a hacking tool. The higher the score, the more diverse and extreme the attack methods the model can simulate.

[0120] The defending agent can extract these boundary cue words that can induce extreme attacks and conduct targeted defense drills, thereby significantly improving the robustness of the network security defense system against unknown and extreme AI attacks.

[0121] The method provided by this invention has the following advantages: (1) The cognitive ability of large models has been digitized and visualized. Existing technologies can only qualitatively describe the cognitive ability of large models. This invention uses the convex hull volume ratio (SES index) to transform the ability to express abstract thinking patterns into an accurate geometric metric. This index can be directly used to evaluate the cognitive bandwidth of different pedestal models on specific tasks, and can also evaluate the quality of cue word engineering, filling the evaluation gap in this field.

[0122] (2) It breaks through the constraints of inductive bias in large models and enhances their potential capabilities. Traditional random sampling methods are limited by the statistical regularity of model training data, and the generated strategies tend to converge. The active boundary detection technology of this invention can automatically discover counterintuitive prompts that humans have never thought of by setting extreme targets for the vertices of the simplex. Without retraining the model parameters, it maximizes the mining and unlocking of the model's potential capabilities.

[0123] (3) A closed-loop optimization mechanism from semantics to geometry has been established. Existing technologies lack the means to deduce natural language instructions from policy distribution. This invention introduces Optimizer LLM for reverse induction, establishing an automated framework from mathematical objectives (simplex vectors) to natural language prompts. This not only achieves automatic expansion of the policy space, but also generates a highly interpretable prompt lexicon, enabling mathematical policy points to have corresponding semantic interpretations.

[0124] (4) Improved robustness and payoff of game equilibrium. According to existing theories, the payoff of Nash equilibrium is limited by the agent's thinking patterns. This invention maximizes the spatial volume of thinking patterns, pushing game theory from limited thinking to open thinking. Based on the optimized boundary cue word library of this invention, the agent can invoke a richer and more diverse combination of strategies, thus exhibiting higher robustness and expected payoff when facing unknown opponents.

[0125] The data processing apparatus for large language model output distribution provided by the present invention is described below. The data processing apparatus for large language model output distribution described below can be referred to in correspondence with the data processing method for large language model output distribution described above.

[0126] Based on any of the above embodiments Figure 4 This is a schematic diagram of the data processing device for large language model output distribution provided by the present invention, as shown below. Figure 4 As shown, the device includes: The initial point cloud construction unit 410 is used to obtain an initial text instruction set, input the initial text instruction set into the first large language model to obtain multiple sets of first probability distribution vectors, and construct an initial distribution point cloud based on the multiple sets of first probability distribution vectors. The target determination unit 420 is used to determine the distribution boundary of the initial distribution point cloud, and to determine a target space vector for expanding the distribution boundary within a preset space based on the range of the distribution boundary. The detection instruction generation unit 430 is used to generate detection text instructions for approximating the target space vector based on the target space vector using a second language model. The distribution boundary expansion unit 440 is used to input the probe text instruction into the first large language model to obtain a second probability distribution vector, and use the second probability distribution vector to update the initial distribution point cloud to expand the distribution boundary of the initial distribution point cloud.

[0127] The apparatus provided in this invention constructs an initial distribution point cloud by inputting an initial text instruction set into a first large language model and extracting multiple sets of first probability distribution vectors. This transforms the abstract output behavior of the large language model into a geometric point cloud space with visual and computable capabilities, laying an objective quantitative foundation for subsequent distribution boundary exploration. Based on this, the invention determines the distribution boundary of the initial distribution point cloud and searches for a target space vector within a preset space that can expand this boundary. Then, it automatically generates probe text instructions that approximate this target space vector using a second large language model. This process establishes an automated reverse mapping mechanism from target space vectors to natural language instructions, breaking the limitations of traditional data processing that heavily relies on trial and error based on human experience. Subsequently, by re-inputting the probe text instructions into the first large language model to obtain a second probability distribution vector and updating the initial distribution point cloud, the invention effectively overcomes the output distribution centralization phenomenon caused by model pre-training data bias. It automatically and continuously explores the edge of the distribution space, thereby efficiently expanding the potential output distribution boundary of the first large language model to uncover the model's potential capabilities.

[0128] Based on any of the above embodiments, the initial point cloud construction unit is specifically used for: Determine the action space corresponding to the interactive task, wherein the action space contains multiple candidate actions; Extract the normalized probability values ​​corresponding to each candidate action from the distribution data output by the first large language model based on the initial text instruction set, and combine them to generate the first probability distribution vector based on the extracted normalized probability values. The first probability distribution vector is mapped to a multidimensional probability simplex space to form the initial distribution point cloud, wherein the multidimensional probability simplex space serves as the preset space.

[0129] Based on any of the above embodiments, the target determination unit is specifically used for: Calculate the convex hull of the data point set of the initial distribution point cloud, and determine the convex hull of the data point set as the distribution boundary; Within the preset space, the spatial vector furthest from the convex hull of the data point set is determined as the target spatial vector.

[0130] Based on any of the above embodiments, the target determination unit is specifically used for: Obtain the normal vector of the convex hull of the data point set; Along the direction of the normal vector, find the extreme point in the preset space, or extract the simplex vertex or the center point of the simplex surface in the preset space; The extreme point, the simplex vertex, or the center point of the simplex surface is determined as the target space vector.

[0131] Based on any of the above embodiments, the detection command generation unit is specifically used for: Extract the reference distribution point that is closest to the target space vector from the convex hull of the data point set; Obtain the current text instruction and current probability distribution features associated with the reference distribution points; The current text instruction, the current probability distribution feature, and the target space vector are concatenated to construct a meta-prompt instruction template; The meta-hint instruction template is input into the second large language model to generate the probe text instruction that approximates the target space vector.

[0132] Based on any of the above embodiments, the device further includes a loop iteration unit, the loop iteration unit being used for: The steps of determining the distribution boundary of the initial distribution point cloud and updating the initial distribution point cloud using the second probability distribution vector are executed iteratively until a preset stopping condition is met. If the preset stopping condition is met, extract all the probe text commands associated with the updated initial distribution point cloud, generate a boundary text command set, and take the distribution boundary at the time of stopping iteration as the target distribution boundary.

[0133] Based on any of the above embodiments, the iterative loop unit is further configured to: Calculate the candidate boundary volume after fusing the second probability distribution vector, and determine the amount of change of the candidate boundary volume relative to the geometric volume of the distribution boundary before the update; If the change amount meets the preset change amount threshold, the detection text instruction is retained, and the second probability distribution vector is added to the initial distribution point cloud to enter the next iteration; If the change amount does not meet the preset change amount threshold, or the number of iterations reaches the preset threshold, then the preset stopping condition is determined to be met.

[0134] Based on any of the above embodiments, the device further includes a capability assessment unit, the capability assessment unit being used for: Calculate the relative volume occupied by the target distribution boundary in the preset space, and use it as the first volume; Obtain the reference volume of the preset space as the second volume; Calculate the ratio of the first volume to the second volume, and use the ratio as evaluation data to characterize the policy expressive power of the first large language model.

[0135] Based on any of the above embodiments, the device further includes an optimization control unit, which is used for: During multi-agent interaction, the policy selection space of the first large language model is constrained within the set of boundary text instructions; Within the base space defined by the boundary text instruction set, determine the hybrid strategy interaction weights for the first large language model to execute each candidate action; Based on the interaction weights of the hybrid strategy, the target action control signal is output to perform multi-agent interaction optimization control.

[0136] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 5 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a data processing method for the output distribution of a large language model. The method includes: acquiring an initial text instruction set; inputting the initial text instruction set into a first large language model to obtain multiple sets of first probability distribution vectors; constructing an initial distribution point cloud based on the multiple sets of first probability distribution vectors; determining the distribution boundary of the initial distribution point cloud; and determining a target space vector for expanding the distribution boundary within a preset space based on the range of the distribution boundary; generating probe text instructions for approximating the target space vector using a second large language model based on the target space vector; inputting the probe text instructions into the first large language model to obtain a second probability distribution vector; and updating the initial distribution point cloud using the second probability distribution vector to expand the distribution boundary of the initial distribution point cloud.

[0137] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0138] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data processing method for the output distribution of a large language model provided by the above methods. The method includes: obtaining an initial text instruction set; inputting the initial text instruction set into a first large language model to obtain multiple sets of first probability distribution vectors; constructing an initial distribution point cloud based on the multiple sets of first probability distribution vectors; determining the distribution boundary of the initial distribution point cloud; and determining a target space vector for expanding the distribution boundary within a preset space based on the range of the distribution boundary; generating probe text instructions for approximating the target space vector using a second large language model based on the target space vector; inputting the probe text instructions into the first large language model to obtain a second probability distribution vector; and updating the initial distribution point cloud using the second probability distribution vector to expand the distribution boundary of the initial distribution point cloud.

[0139] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a data processing method for output distribution of a large language model provided by the methods described above. The method includes: acquiring an initial text instruction set; inputting the initial text instruction set into a first large language model to obtain multiple sets of first probability distribution vectors; constructing an initial distribution point cloud based on the multiple sets of first probability distribution vectors; determining the distribution boundary of the initial distribution point cloud; and determining a target space vector for expanding the distribution boundary within a preset space based on the range of the distribution boundary; generating probe text instructions for approximating the target space vector using a second large language model based on the target space vector; inputting the probe text instructions into the first large language model to obtain a second probability distribution vector; and updating the initial distribution point cloud using the second probability distribution vector to expand the distribution boundary of the initial distribution point cloud.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method for the output distribution of large language models, characterized in that, include: An initial text instruction set is obtained and input into the first large language model to obtain multiple sets of first probability distribution vectors. An initial distribution point cloud is constructed based on the multiple sets of first probability distribution vectors. The first probability distribution vectors are used to characterize the probability tendency of the first large language model on different candidate actions contained in the action space corresponding to the interactive task based on the initial text instruction set. Determine the distribution boundary of the initial distribution point cloud, and based on the range of the distribution boundary, determine a target space vector within a preset space for expanding the distribution boundary; Based on the target space vector, a probe text command is generated using the second language model to approximate the target space vector; The probe text instruction is input into the first large language model to obtain a second probability distribution vector, and the initial distribution point cloud is updated using the second probability distribution vector to expand the distribution boundary of the initial distribution point cloud. The second probability distribution vector is used to characterize the probability tendency of the first large language model on different candidate actions contained in the action space corresponding to the interactive task based on the probe text instruction. The step of generating probing text instructions to approximate the target space vector using a second large language model, based on the target space vector, includes: Extract the reference distribution point that is closest to the target space vector from the convex hull of the data point set of the initial distribution point cloud; Obtain the current text instruction and current probability distribution features associated with the reference distribution points; The current text instruction, the current probability distribution feature, and the target space vector are concatenated to construct a meta-prompt instruction template; The meta-hint instruction template is input into the second large language model to generate the probe text instruction that approximates the target space vector.

2. The data processing method for large language model output distribution according to claim 1, characterized in that, The step of obtaining multiple sets of first probability distribution vectors and constructing an initial distribution point cloud based on the multiple sets of first probability distribution vectors includes: Determine the action space corresponding to the interactive task, wherein the action space contains multiple candidate actions; Extract the normalized probability values ​​corresponding to each candidate action from the distribution data output by the first large language model based on the initial text instruction set, and combine them to generate the first probability distribution vector based on the extracted normalized probability values. The first probability distribution vector is mapped to a multidimensional probability simplex space to form the initial distribution point cloud, wherein the multidimensional probability simplex space serves as the preset space.

3. The data processing method for large language model output distribution according to claim 2, characterized in that, The step of determining the distribution boundary of the initial distribution point cloud and, based on the range of the distribution boundary, determining a target spatial vector for expanding the distribution boundary within a preset space includes: Calculate the convex hull of the data point set of the initial distribution point cloud, and determine the convex hull of the data point set as the distribution boundary; Within the preset space, the spatial vector furthest from the convex hull of the data point set is determined as the target spatial vector.

4. The data processing method for large language model output distribution according to claim 3, characterized in that, Determining the target space vector as the spatial vector furthest from the convex hull of the data point set within the preset space includes: Obtain the normal vector of the convex hull of the data point set; Along the direction of the normal vector, find the extreme point in the preset space, or extract the simplex vertex or the center point of the simplex surface in the preset space; The extreme point, the simplex vertex, or the center point of the simplex surface is determined as the target space vector.

5. The data processing method for large language model output distribution according to any one of claims 1 to 4, characterized in that, After the step of updating the initial distribution point cloud using the second probability distribution vector to expand the distribution boundary of the initial distribution point cloud, the method further includes: The steps of determining the distribution boundary of the initial distribution point cloud and updating the initial distribution point cloud using the second probability distribution vector are executed iteratively until a preset stopping condition is met. If the preset stopping condition is met, extract all the probe text commands associated with the updated initial distribution point cloud, generate a boundary text command set, and take the distribution boundary at the time of stopping iteration as the target distribution boundary.

6. The data processing method for large language model output distribution according to claim 5, characterized in that, The determination steps for satisfying the preset stopping conditions include: Calculate the candidate boundary volume after fusing the second probability distribution vector, and determine the amount of change of the candidate boundary volume relative to the geometric volume of the distribution boundary before the update; If the change amount meets the preset change amount threshold, the detection text instruction is retained, and the second probability distribution vector is added to the initial distribution point cloud to enter the next iteration; If the change amount does not meet the preset change amount threshold, or the number of iterations reaches the preset threshold, then the preset stopping condition is determined to be met.

7. The data processing method for large language model output distribution according to claim 5, characterized in that, After the step of taking the distribution boundary at the time of stopping iteration as the target distribution boundary, the method further includes: Calculate the relative volume occupied by the target distribution boundary in the preset space, and use it as the first volume; Obtain the reference volume of the preset space as the second volume; Calculate the ratio of the first volume to the second volume, and use the ratio as evaluation data to characterize the policy expressive power of the first large language model.

8. The data processing method for large language model output distribution according to claim 5, characterized in that, After the step of taking the distribution boundary at the time of stopping iteration as the target distribution boundary, the method further includes: During multi-agent interaction, the policy selection space of the first large language model is constrained within the set of boundary text instructions; Within the base space defined by the boundary text instruction set, determine the hybrid strategy interaction weights for the first large language model to execute each candidate action; Based on the interaction weights of the hybrid strategy, the target action control signal is output to perform multi-agent interaction optimization control.

9. A data processing device for large language model output distribution, characterized in that, include: An initial point cloud construction unit is used to obtain an initial text instruction set, input the initial text instruction set into a first large language model to obtain multiple sets of first probability distribution vectors, and construct an initial distribution point cloud based on the multiple sets of first probability distribution vectors. The first probability distribution vector is used to characterize the probability tendency of the first large language model on different candidate actions contained in the action space corresponding to the interactive task based on the initial text instruction set. The target determination unit is used to determine the distribution boundary of the initial distribution point cloud, and to determine a target space vector for expanding the distribution boundary within a preset space based on the range of the distribution boundary. The detection instruction generation unit is used to generate detection text instructions for approximating the target space vector based on the target space vector using a second language model; The distribution boundary expansion unit is used to input the probe text instruction into the first large language model to obtain a second probability distribution vector, and use the second probability distribution vector to update the initial distribution point cloud to expand the distribution boundary of the initial distribution point cloud. The second probability distribution vector is used to characterize the probability tendency of the first large language model on different candidate actions contained in the action space corresponding to the interaction task based on the probe text instruction. The detection command generation unit is specifically used for: Extract the reference distribution point that is closest to the target spatial vector from the convex hull of the data point set of the initial distribution point cloud; Obtain the current text instruction and current probability distribution features associated with the reference distribution points; The current text instruction, the current probability distribution feature, and the target space vector are concatenated to construct a meta-prompt instruction template; The meta-hint instruction template is input into the second large language model to generate the probe text instruction that approximates the target space vector.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the data processing method for large language model output distribution as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data processing method for large language model output distribution as described in any one of claims 1 to 8.

Images