Resource optimization method and device based on multi-agent active reasoning

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a resource optimization method and device based on multi-agent active reasoning, and the method comprises the steps: obtaining the local observation information of a plurality of agents; generating a local belief corresponding to each agent according to each piece of local observation information; performing belief communication based on each local belief to generate a global belief; optimizing decision-making actions for each agent based on global belief by taking minimization of the overall free energy of the multi-agent system as an optimization target through a decision-making model; wherein the overall free energy is obtained by calculating the comprehensive external income and the internal belief error; controlling each agent to execute the optimized decision action, and receiving a feedback result generated after the environment executes the decision action; and updating the decision model based on the decision action, the overall free energy and the feedback result. According to the invention, stable, efficient and coordinated optimization decision can be realized in a multi-agent dynamic collaborative edge resource allocation scene.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a resource optimization method and apparatus based on multi-agent active reasoning. Background Technology

[0002] With the rapid development of mobile internet and the Internet of Things (IoT), the demand for computing and communication resources from various intelligent applications is increasing daily. Multi-access edge computing provides users with low-latency, high-response services by deploying distributed resources at the network edge. In multi-user or multi-server edge computing scenarios, how to coordinate the decisions of multiple users (or task agents) to rationally offload computing tasks and dynamically allocate resources has become a key technical issue for improving service efficiency and user experience.

[0003] Currently, existing technologies propose using multi-agent reinforcement learning methods to achieve distributed resource optimization. However, in such dynamic, multi-agent collaborative edge network environments, existing technologies still have some shortcomings. On the one hand, because each agent makes decisions based only on its own local information, and the environment is constantly changing due to the behavior of other agents, the learning process of the agents is difficult to converge stably, resulting in large policy fluctuations. On the other hand, the localized decision-making pattern easily leads to resource allocation conflicts among agents and overall inefficiency. Therefore, existing technologies struggle to achieve stable, efficient, and coordinated optimization decisions in dynamic, multi-agent collaborative edge resource allocation scenarios.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] This application provides a resource optimization method and apparatus based on multi-agent active reasoning, which can effectively solve the problems of limited local decision-making by multi-agents, single optimization objective, and insufficient dynamic adaptability, and improve the globality, stability and effectiveness of resource optimization decision-making.

[0006] In a first aspect, embodiments of this application provide a resource optimization method based on multi-agent active reasoning, including: Acquire local observation information from multiple intelligent agents; Based on the local observation information described above, generate local beliefs corresponding to each agent; Based on the aforementioned local beliefs, belief communication is performed to generate global beliefs; The decision model aims to minimize the overall free energy of the multi-agent system, and optimizes the decision actions of each agent based on the global beliefs; wherein the overall free energy is calculated by combining external benefits and internal belief errors. Control each intelligent agent to execute optimized decision actions, and receive feedback results generated by the environment after the decision actions are executed; The decision model is updated based on the decision action, the overall free energy, and the feedback result.

[0007] Secondly, embodiments of this application provide a resource optimization device based on multi-agent active reasoning, comprising: The acquisition module is used to acquire local observation information from multiple agents; The local belief module is used to generate local beliefs for each agent based on the local observation information. The global belief module is used to perform belief communication based on the local beliefs and generate global beliefs. An optimization module is used to optimize decision-making actions for each agent based on the global beliefs, with the goal of minimizing the overall free energy of the multi-agent system through a decision model; wherein the overall free energy is calculated by combining external benefits and internal belief errors; The execution module is used to control each intelligent agent to execute the optimized decision-making actions and to receive feedback results generated by the environment after the decision-making actions are executed. An update module is used to update the decision model based on the decision action, the overall free energy, and the feedback result.

[0008] This application provides a resource optimization method and apparatus based on multi-agent active reasoning. First, by acquiring local observation information from each agent and generating corresponding local beliefs, a basic representation is constructed for multi-agent collaborative decision-making. Then, through belief communication, all local beliefs are integrated to generate global beliefs, breaking the limitation of a single agent relying solely on its own local information, allowing decisions to be made based on global information, and fundamentally reducing resource conflicts or inefficient allocation problems caused by localized decision-making. Simultaneously, the overall free energy of minimizing the combined external benefits and internal belief errors is used as the optimization objective, taking into account both the actual effect of task execution and ensuring the consistency of system beliefs, effectively balancing the relationship between exploration and utilization, and avoiding performance imbalances caused by a single optimization objective. Finally, the decision model is updated through environmental feedback results, enabling the model to dynamically adapt to environmental changes and continuously optimize the decision strategy. In summary, this application transforms the local observations of each agent into global beliefs, optimizes decision-making with the goal of minimizing the overall free energy that combines external benefits and internal belief errors, and dynamically updates the model. This effectively solves the problems of limitations in local decision-making, single optimization objectives, and insufficient dynamic adaptability of multi-agent systems, improves the globality, stability, and effectiveness of resource optimization decision-making, and ensures the overall performance of multi-agent systems in resource allocation and task processing. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is an application environment diagram of the resource optimization method based on multi-agent active reasoning provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the resource optimization method based on multi-agent active reasoning provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the resource optimization device based on multi-agent active reasoning provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0011] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with those detailed in the appended claims or with some aspects of this application.

[0012] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover descriptions such as non-exclusive inclusion, so that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0013] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0014] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0015] To address the aforementioned technical problems and overcome the shortcomings of existing technologies, this application provides a resource optimization method and apparatus based on multi-agent active reasoning. This method effectively solves the problems of limited local decision-making by multi-agents, single optimization objective, and insufficient dynamic adaptability, thereby improving the globality, stability, and effectiveness of resource optimization decisions.

[0016] Figure 1 This is an application environment diagram of a resource optimization method based on multi-agent active reasoning in one embodiment. (Refer to...) Figure 1 This resource optimization method based on multi-agent active reasoning is applied to a resource optimization system based on multi-agent active reasoning. The system includes a terminal 110 and a server 120. The terminal 110 and server 120 are connected via a network. The terminal 110 can be a desktop terminal or a mobile terminal, specifically a mobile phone, tablet computer, laptop computer, or at least one of these. The server 120 can be a standalone server or a server cluster composed of multiple servers. The server 120 is configured to execute the aforementioned resource optimization method based on multi-agent active reasoning, including: acquiring local observation information from multiple agents; generating local beliefs for each agent based on the local observation information; performing belief communication based on the local beliefs to generate global beliefs; optimizing decision actions for each agent based on the global beliefs using a decision model with the goal of minimizing the overall free energy of the multi-agent system; wherein the overall free energy is calculated by combining external benefits and internal belief errors; controlling each agent to execute the optimized decision actions and receiving feedback results generated by the environment after executing the decision actions; and updating the decision model based on the decision actions, the overall free energy, and the feedback results.

[0017] Please see Figure 2 , Figure 2 This is a flowchart illustrating a resource optimization method based on multi-agent active reasoning according to an embodiment of this application. This embodiment primarily uses the application of this resource optimization method based on multi-agent active reasoning to a computer device as an example for illustration. Specifically, the resource optimization method based on multi-agent active reasoning provided in this embodiment may include the following steps: S1. Obtain local observation information from multiple agents; Specifically, in step S1, each agent independently perceives key information about its environment and related tasks (i.e., local observation information). This information forms the basis for the agent's subsequent decisions and needs to comprehensively cover core dimensions related to resource allocation and task execution to ensure the completeness and timeliness of the information. Each agent typically corresponds to an entity that needs to process a task, such as a mobile device or application agent. For example, in a scenario where multiple devices collaboratively process audio-related tasks, each agent (such as a user's mobile phone or a portable audio processing device) obtains resource status information of available processing nodes in the vicinity (such as remaining computing power and storage space), connection status information between itself and each processing node (such as signal strength and data transmission rate), and basic information about its own task to be processed (such as task type and data size).

[0018] S2. Generate local beliefs for each agent based on local observation information; Specifically, in step S2, the local observation information acquired by each agent is abstracted and characterized, transforming the original environmental and task information into a local belief vector with representational meaning. This local vector accurately reflects the agent's perception of its local scene, laying the foundation for subsequent multi-agent information interaction. For example, an agent might transform the observed raw information such as "80% remaining computing power of processing node A, connection rate of 50Mbps, and 20MB of audio question-and-answer task data to be processed" into a set of fixed-dimensional abstract vectors (such as a 16-dimensional vector) through a specific processing method. This vector represents the agent's local belief, intuitively reflecting its perception of the current local task requirements and available resources.

[0019] S3. Perform belief communication based on local beliefs to generate global beliefs; Specifically, in step S3, each agent shares its local beliefs through a pre-defined communication method, breaking the information limitations of a single agent. By integrating and fusing the local beliefs of all agents, a global belief that reflects the overall state of the system is formed, allowing each agent to obtain information support from a global perspective. Agents do not make decisions in isolation but share their local beliefs through a specific communication mechanism (belief communication). The local belief vectors of all agents are collected, and these scattered cognitions are fused and processed through a well-designed communication model, ultimately refining a global belief that reflects the macroscopic state of the entire system and the collective information of all agents. For example, 10 agents can interactively share their respective local beliefs (such as abstract vectors of different dimensions). Through the information fusion mechanism, the local cognitions of all agents are integrated to generate a global belief. This belief includes information such as the total task volume of all agents, the overall remaining resources of the processing nodes in the entire system, and the global distribution of connections between each agent and the processing nodes, realizing the transformation of local information into global information.

[0020] S4. Using a decision model, the overall free energy of the multi-agent system is minimized as the optimization objective, and the decision actions of each agent are optimized based on global beliefs; wherein, the overall free energy is calculated by combining external benefits and internal belief errors; Specifically, for step S4, the decision model focuses on minimizing the overall free energy of the system. This overall free energy is calculated by considering two factors: external benefits (i.e., indicators related to the actual effectiveness of task execution) and internal belief error (i.e., the degree of deviation between the local beliefs and global beliefs of each agent). Based on global beliefs, the decision model selects the optimal decision action for each agent, ensuring that the action aligns with global resource allocation requirements while also considering the individual agent's task execution performance. Specifically, using a pre-built or online-learned decision model, and taking the global beliefs generated in the previous step as input, an optimized action plan (i.e., decision action) is calculated for each agent. The optimization objective of the decision is to minimize the overall free energy of the multi-agent system. This objective function is a composite indicator; its external benefit component encourages the selection of actions that bring higher task rewards (such as shorter latency and higher accuracy); its internal belief error component encourages the agent's individual cognition (local beliefs) to align with the system's collective cognition (global beliefs), thereby reducing uncertainty and conflict. The decision model seeks the optimal solution by balancing these two factors. For example, the decision model combines global beliefs. If the entire system has 3 processing nodes, a total remaining computing power of 200 GFLOPS, and 10 agents need to process 8 audio tasks, the overall free energy corresponding to different resource allocation actions is calculated. If the overall free energy obtained by combining the external benefit (the expected success rate of the task is 90%) and the internal belief error (the deviation between the agent's local belief and global belief is small) is the lowest when the audio classification task of a certain agent is allocated to processing node B, then this allocation action is determined as the optimal decision action of the agent.

[0021] S5. Control each agent to execute the optimized decision-making actions, and receive feedback results generated by the environment after the decision-making actions are executed; Specifically, for step S5, according to the optimized decision action, each agent is controlled to perform corresponding operations (such as task unloading, resource request, etc.), while the environment's response to the action is monitored in real time, and feedback data related to task execution effect and resource utilization is collected to provide a basis for subsequent model updates. For example, an agent is controlled to perform the decision action of "unloading the audio question-and-answer task to processing node C and requesting 50Mbps bandwidth and 30GFLOPS computing resources"; after the task is completed, the results of environmental feedback are received, including information such as task completion time (e.g., 6 seconds), execution success rate (e.g., 100%), and actual resource utilization (e.g., bandwidth utilization of 48Mbps).

[0022] S6. Update the decision model based on the decision action, overall free energy, and feedback results; Specifically, for step S6, key data from this decision-making process is used as the basis for model updates. The internal parameters of the decision model are adjusted. Key data includes the executed decision actions, the calculated overall free energy, and environmental feedback results. This allows the model to absorb the experience from this decision and further improve decision accuracy and efficiency when facing similar scenarios in the future. For example, the action of allocating resources to processing node C, the corresponding overall free energy value, and the feedback result of 6 seconds of task completion time and 100% success rate are used as data inputs to adjust the parameters of the decision model. This allows the model to generate resource allocation decisions that better meet actual needs more quickly when encountering the same or similar global belief scenarios in the future.

[0023] This embodiment integrates local information from multiple agents to form global beliefs, and takes the overall free energy that comprehensively considers task effectiveness and belief consistency as the optimization objective. Combined with a dynamic update model based on environmental feedback, it effectively solves the local limitations of multi-agent decision-making, balances system stability and resource utilization efficiency, and significantly improves the processing effect and service quality of related tasks.

[0024] Furthermore, in some embodiments, step S1, "acquiring local observation information of multiple agents," may specifically include: S11. Obtain the current resource status information of each edge server; Specifically, for step S11, the remaining availability and occupancy status of the core hardware resources of all available edge servers deployed in the edge computing network are collected in real time. This information is the basis for determining whether the server can handle the task and for rationally allocating resources. It needs to cover key resource dimensions directly related to task execution to ensure the real-time nature and accuracy of the data. For example, in an edge computing network that supports audio task processing, the current resource status of three edge servers in the network is collected, including: Server 1: remaining CPU computing power 30 GFLOPS, remaining memory 16 GB, remaining bandwidth 100 Mbps; Server 2: remaining CPU computing power 25 GFLOPS, remaining memory 8 GB, remaining bandwidth 80 Mbps; Server 3: remaining CPU computing power 40 GFLOPS, remaining memory 24 GB, remaining bandwidth 120 Mbps. The real-time occupancy ratio of each resource is recorded simultaneously.

[0025] S12. Obtain network link status information between each agent and the edge server; Specifically, for step S12, for each agent, key performance indicators of the communication link between it and each edge server in the network are collected one by one. This information directly affects the data transmission efficiency and task execution latency, and is an important basis for judging the target server for task unloading. It needs to comprehensively reflect the connectivity and transmission quality of the link. For example, an agent (such as a portable audio processing device carried by a user) obtains the link status with the above 3 servers respectively: with server 1, the link signal strength is -55dBm, the data transmission rate is 90Mbps, the link latency is 2ms, and the link connectivity status is "normal"; with server 2, the link signal strength is -68dBm, the data transmission rate is 60Mbps, the link latency is 5ms, and the link connectivity status is "normal"; with server 3, the link signal strength is -72dBm, the data transmission rate is 45Mbps, the link latency is 8ms, and the link connectivity status is "normal".

[0026] S13. Obtain the feature information of the tasks to be processed initiated by each agent; Specifically, for step S13, the core attribute information of the task that each agent needs to execute is collected. This information determines the type and intensity of the task's resource requirements and is a prerequisite for achieving accurate matching between resource allocation and task characteristics. It needs to cover key feature dimensions such as task type and data scale. For example, the characteristics of the tasks to be processed initiated by each agent are as follows: Agent A initiates an audio question-and-answer task with a data volume of 20MB and requires real-time response; Agent B initiates a music note analysis task with a data volume of 50MB and requires high computational accuracy; Agent C initiates a human voice classification task with a data volume of 15MB and moderate sensitivity to latency.

[0027] This embodiment comprehensively acquires the resource status of edge servers, the link status between agents and servers, and the characteristic information of tasks to be processed, providing complete and accurate basic data support for subsequent resource optimization decisions. This ensures that the decision-making process can fully align with the actual situation of network resources and task requirements, laying a solid foundation for improving the rationality of resource allocation and task processing efficiency.

[0028] Furthermore, in some embodiments, step S2, "generating local beliefs corresponding to each agent based on local observation information," may specifically include: S21. Perform feature normalization processing on the local observation information of each agent; Specifically, for step S21, the local observation information contains raw data of different dimensions and magnitudes. The differences in the dimensions of these data will affect the accuracy and efficiency of subsequent encoding processing. Through feature normalization, various raw features are uniformly mapped to a preset reasonable numerical range, eliminating the interference of dimensions and making different types of features comparable and fusionable, providing a standardized data foundation for subsequent belief generation. For example, the local observation information of a certain agent includes "remaining bandwidth of the edge server is 150Mbps, remaining CPU computing power is 40GFLOPS, distance from the server is 500 meters, and the amount of audio question-and-answer task data to be processed is 30MB". Normalize this information: normalize the remaining bandwidth (range 0-200Mbps), CPU computing power (range 0-100GFLOPS), distance (range 0-1000m), and task data volume (range 0-50MB) to the [0,1] interval, and obtain "remaining bandwidth 0.75, CPU computing power 0.4, distance 0.5, task data volume 0.6"; for classification features such as task type (audio question answering), convert them into fixed-dimensional one-hot encoded vectors to ensure that all observed features meet the standardization requirements.

[0029] S22. Input the normalized feature vector into the belief encoder; Specifically, in step S22, the belief encoder is the core carrier for transforming local observation information into local beliefs. It receives a normalized multi-dimensional feature vector, which integrates all key information related to the local environment and task perceived by the agent. The input process must ensure that the dimension of the feature vector matches the input layer of the belief encoder to ensure the integrity of data transmission. For example, the normalized "remaining bandwidth 0.75, CPU computing power 0.4, distance 0.5, task data volume 0.6" are integrated with the task type one-hot encoded vector (such as [1,0,0]) to form a dimension-matched 6-dimensional normalized feature vector. This vector is then completely input into the belief encoder as the original input data for belief generation.

[0030] S23. The feature vectors are nonlinearly transformed and encoded by a multi-layer perceptual network of the belief encoder to generate local belief vectors for each agent. Specifically, for step S23, the multilayer perceptron possesses powerful nonlinear fitting capabilities, enabling it to deeply mine the intrinsic connections and key patterns of information in each dimension of the feature vector. Through layer-by-layer linear transformation and nonlinear activation processing, it abstracts, refines, and compresses the standardized features, transforming the high-dimensional original feature vector into a low-dimensional, compact, and highly representative local belief vector. This vector can accurately summarize the agent's cognition of the local scene. For example, the multilayer perceptron of the belief encoder includes an input layer, a hidden layer, and an output layer. After receiving the aforementioned 6-dimensional normalized feature vector, it first passes the feature vector to the hidden layer through the input layer. The hidden layer captures the complex relationships between features (such as the matching relationship between the task data volume and the remaining bandwidth of the server) through a nonlinear activation function. After further encoding and compression by subsequent layers, it finally outputs a 16-dimensional local belief vector (such as [0.21, 0.53, 0.18, ..., 0.36]). This vector condenses the core connotation of the agent's local observation information, becoming an important basis for subsequent information interaction and decision-making.

[0031] This embodiment eliminates dimensional interference by normalizing local observation information, and then transforms and encodes it through a multilayer sensing network to convert the original observation information into a local belief vector that accurately represents local scene cognition. This provides a high-quality and standardized information foundation for belief communication among multiple agents and global belief generation, effectively improving the accuracy and efficiency of subsequent decision-making processes.

[0032] Furthermore, in some embodiments, step S3, "performing belief communication based on local beliefs to generate global beliefs," may specifically include: S31. Organize the local belief vectors of each agent into an input sequence; Specifically, in step S31, each agent has generated a local belief vector representing its local scene cognition, and all vectors have a uniform dimensional specification. To achieve collaborative processing of multi-agent information, the local belief vectors of all agents need to be organized into an ordered input sequence according to preset rules, such as agent numbering order and task priority association order, to ensure that the attribution of each vector in the sequence is clear, providing a structured data foundation for the subsequent unified analysis of the relationship between local beliefs by the network model. For example, in a scenario where multiple agents collaboratively process audio-related tasks, there are 8 agents, and each agent's local belief is a 16-dimensional vector. According to the numbering order of agents 1 to 8, the 8 16-dimensional local belief vectors are arranged sequentially to form an input sequence matrix with a dimension of 8×16. Each row in the matrix corresponds to the local belief of one agent, completely preserving the local cognitive information of all agents.

[0033] S32. The input sequence is processed using a network model based on an attention mechanism, the correlation weights between local beliefs are calculated and information is fused to generate a global belief vector that integrates information from all agents. Specifically, for step S32, the attention-based network model has the ability to accurately capture the correlation between information. It first analyzes each local belief vector in the input sequence, mining the correlation between this vector and all other local belief vectors in the sequence. For example, it compares the correlation between an agent's local belief about server resources and other agents' local beliefs about the server's task requirements, quantifying this correlation as a corresponding correlation weight. The higher the correlation, the greater the weight. Subsequently, based on the correlation weight, it weights and aggregates the effective information in all local belief vectors, integrating the scattered local information from each agent into unified and complete global information, ultimately outputting a global belief vector that comprehensively reflects the state of the entire system. For example, after the above 8×16 input sequence is input into the network model, the model analysis finds that: agent 1's local belief (involving the remaining CPU computing power of server X) has a very high correlation with the local beliefs of agents 2, 5, and 7 (all involving the need to call the audio task of server X), with corresponding correlation weights of 0.3, 0.28, and 0.25 respectively; the correlation with the local beliefs of other agents is low, with weights all less than 0.1. The model uses these weights to weight and fuse the information of all local beliefs, focusing on integrating the resource status and related task requirements of server X, while also taking into account the global distribution of other servers and tasks. Finally, a 16-dimensional global belief vector is generated, which fully covers the total task requirements of the eight agents, the overall distribution of server resources in the entire system, and the adaptation and association between each task and server, among other global information.

[0034] This embodiment organizes scattered local belief vectors into a structured input sequence, accurately captures the correlation between local beliefs using an attention mechanism, and performs weighted fusion. This effectively breaks through the information limitations of a single agent, achieves comprehensive integration of local information from all agents, and generates a global belief vector that provides complete global perspective support for subsequent decision-making, thereby improving the globality and rationality of the decision.

[0035] Furthermore, in some embodiments, step S32, "processing the input sequence using an attention-based network model, calculating the association weights between local beliefs, and performing information fusion," may specifically include: S321. Perform a linear transformation on the input sequence to generate a query vector group, a key vector group, and a value vector group, respectively; Specifically, for step S321, the input sequence is structured data composed of local belief vectors of each agent, whose original dimensions do not match the feature space required for attention calculation. Three independent learnable linear transformation matrices are used to perform dimensional mapping and feature transformation on the input sequence, generating three sets of vectors—query (Q), key (K), and value (V)—that are structurally consistent and adapted to the attention mechanism. The query vector set represents the information needs of the current agent, the key vector set describes the feature identifiers of each agent's local beliefs, and the value vector set carries the core information of each agent's local beliefs. These three vectors together lay the foundation for subsequent association weight calculation. For example, suppose the input sequence is an 8×16 dimensional matrix composed of local belief vectors of 8 agents (8 being the number of agents and 16 being the local belief dimension). The input sequence is linearly transformed using three sets of 64×16 linear transformation matrices (learnable parameters), generating three sets of 8×8 dimensional vectors: query vector set Q (8 query vectors, each 8-dimensional), key vector set K (8 key vectors, each 8-dimensional), and value vector set V (8 value vectors, each 8-dimensional). This completes the mapping of the input sequence to the attention computation feature space.

[0036] S322. Employ a multi-head attention mechanism to calculate the attention weights between elements in the input sequence based on the query vector group, key vector group, and value vector group, and perform weighted aggregation to obtain the weighted aggregated feature sequence. Specifically, for step S322, the multi-head attention mechanism sets up multiple attention heads in parallel, capturing the correlations between local beliefs in the input sequence from different dimensions, thus avoiding the limitations of single-dimensional correlation analysis. For each attention head, the similarity between the query vector and the key vector (i.e., attention weight) is first calculated. Normalization is then applied to ensure the sum of the weights is 1. Then, the value vectors are weighted and summed based on this weight to obtain the aggregation result of a single attention head. Finally, the aggregation results of all attention heads are concatenated and integrated to form a weighted aggregated feature sequence that comprehensively reflects multi-dimensional correlation information. For example, eight attention heads are set up, each independently processing three sets of 8×8 dimensional vectors: Q, K, and V. Taking one attention head as an example, the dot product of the query vector of the first agent and the key vectors of all agents is calculated to obtain the similarity score [1.2, 0.8, 1.5, 0.3, 1.1, 0.6, 1.3, 0.5]. After Softmax normalization, the attention weights are obtained [0.18, 0.11, 0.23, 0.04, 0.16, 0.08, 0.17, 0.03]. These weights indicate that the local belief correlation between the first agent and the third agent is the highest. Then, the eight value vectors are weighted and summed using these weights to obtain the 1×8 dimension aggregation result of this attention head. This process is repeated to obtain the aggregation results of eight attention heads, which are then concatenated into an 8×64 dimension weighted aggregation feature sequence (64 dimensions of aggregation features for each agent).

[0037] S323. Input the weighted aggregated feature sequence into the feedforward neural network layer for processing; Specifically, in step S323, although the weighted aggregated feature sequence integrates multi-dimensional correlation information, the nonlinear correlations between features have not been fully explored. The feedforward neural network layer possesses powerful nonlinear fitting capabilities. Through two fully connected layers and activation functions, it deeply processes the aggregated features, strengthening effective features and suppressing redundant information, transforming the aggregated features into more representative high-level features, providing high-quality input for subsequent residual connections and normalization processing. For example, the aforementioned 8×64-dimensional weighted aggregated feature sequence is input into the feedforward neural network layer, which contains two fully connected layers: the first layer maps the 64-dimensional features to 128 dimensions, introducing nonlinearity through the ReLU activation function; the second layer maps the 128-dimensional features back to 64 dimensions, outputting an 8×64-dimensional processed feature sequence, effectively exploring the complex nonlinear correlations between the dimensions of the aggregated features.

[0038] S324. Perform a residual connection between the output of the feedforward neural network layer and the weighted aggregated feature sequence; Specifically, in step S324, the residual connection constructs a "shortcut path" for feature transfer by directly adding the output of the feedforward neural network layer to the original weighted aggregated feature sequence. This not only preserves the effective correlation information in the original aggregated features but also superimposes high-level features mined by the feedforward network, avoiding the gradient vanishing problem in deep network training and ensuring the complete transfer and efficient fusion of feature information. For example, the 8×64-dimensional feature sequence output by the feedforward neural network layer is element-wise added to the original weighted aggregated 8×64-dimensional feature sequence (residual connection) to obtain an 8×64-dimensional residual connection result. This result preserves the basic information of the initial multi-dimensional correlations while incorporating high-level features from deep processing, achieving complementary enhancement of feature information.

[0039] S325. Perform layer normalization on the results of residual connections to obtain the global belief vector; Specifically, in step S325, the feature sequence after residual connection may suffer from inconsistent feature distributions and large numerical fluctuations across dimensions, affecting the stability of subsequent decision-making processes. Layer normalization normalizes the feature vectors corresponding to each agent, adjusting the feature values ​​to a uniform distribution range, such as a mean of 0 and a variance of 1. This eliminates dimensional differences and numerical fluctuations between feature dimensions, outputting a stable, regular, and highly representative global belief vector that fully integrates the local information of all agents. For example, layer normalization is performed on the 8×64-dimensional feature sequence after residual connection. For each agent's 64-dimensional feature vector, its mean and variance are calculated and standardized, ensuring that the values ​​of each dimension of each feature vector are within the range of [-1, 1]. Then, a linear transformation maps the 64-dimensional features to 16 dimensions, ultimately obtaining an 8×16-dimensional global belief vector. The 16-dimensional vector corresponding to each agent contains global information such as the resource distribution and task requirement relationships of the entire system.

[0040] This embodiment accurately captures the multi-dimensional correlations between the local beliefs of each agent through a coherent process of linear transformation, multi-head attention aggregation, feedforward network processing, residual connection, and layer normalization. It fully integrates global effective information to generate a stable and highly representative global belief vector, providing comprehensive and reliable global information support for subsequent optimization decisions and effectively improving the accuracy and rationality of the decisions.

[0041] Furthermore, in some embodiments, step S4, "optimizing decision-making actions for each agent based on global beliefs by using a decision model to minimize the overall free energy of the multi-agent system as the optimization objective," may specifically include: S41. Based on the current global belief, generate a set of candidate actions for each agent; Specifically, for step S41, the current global belief integrates the local information of all agents and the overall system state, such as the distribution of system resources, total task requirements, and the compatibility between agents and servers. This is the core basis for generating candidate actions. The generated candidate actions need to cover the possible decision directions of the agents, and each action needs to clearly define the key choices for resource allocation or task processing (such as target server, resource allocation quota, etc.) to ensure that the candidate action set has diversity and effectiveness, providing sufficient alternative space for subsequent optimization and screening. For example, in an edge computing scenario where multiple agents collaboratively process audio tasks, the current global belief of a certain agent includes "3 edge servers (servers 1-3), server 1 has 30 GFLOPS of remaining CPU computing power / 100 Mbps bandwidth, server 2 has 25 GFLOPS of remaining CPU computing power / 80 Mbps bandwidth, server 3 has 40 GFLOPS of remaining CPU computing power / 120 Mbps bandwidth, and the audio question-and-answer task to be processed by this agent requires at least 20 GFLOPS of computing power and 50 Mbps bandwidth." Based on this global belief, a set of candidate actions is generated for the agent, including 30 candidate actions such as "select server 1 and allocate 25 GFLOPS computing power + 60 Mbps bandwidth", "select server 2 and allocate 25 GFLOPS computing power + 50 Mbps bandwidth", "select server 3 and allocate 20 GFLOPS computing power + 50 Mbps bandwidth", and "select server 3 and allocate 30 GFLOPS computing power + 70 Mbps bandwidth", covering different combinations of server selection and resource allocation.

[0042] S42. For each candidate action in a set of candidate actions, based on global beliefs and the predicted system state after the candidate action is executed, calculate the corresponding external benefits and internal belief errors in the system state, and predict the overall free energy of the system after the candidate action is executed. Specifically, for step S42, firstly, by combining global beliefs and individual candidate actions, the dynamic changes of the system after the action is executed are deduced, including the task execution state of the agent (such as whether it can be completed normally, execution efficiency), the overall resource consumption of the system, and the degree of fit between the local beliefs and global beliefs of each agent, thus forming a predicted system state. Then, based on this predicted state, two core indicators are calculated: one is the external benefit (i.e., the actual effect of task execution, such as task completion rate, execution quality, etc.), and the other is the internal belief error (i.e., the degree of deviation between the local beliefs and global beliefs of the agent after executing the action). Finally, according to preset rules, the external benefit and internal belief error are comprehensively quantified to obtain the overall free energy of the system after executing the candidate action. The value of the free energy directly reflects the comprehensive optimization effect of the action. For example, regarding the candidate action "Select server 2 and allocate 25 GFLOPS computing power + 50 Mbps bandwidth": ① Predict the system state after execution based on global beliefs: Server 2's remaining computing power is exhausted, and its bandwidth usage is 62.5%. The agent's audio question-and-answer task can be executed normally, but there are no redundant resources. Other agents' tasks and server resources can still be matched; ② Calculate external benefits: Combining task requirements and server capabilities, the predicted task completion rate is 100%, and the inference accuracy is 85%. The quantified external benefit is 0.88; ③ Calculate internal belief error: The local belief (focusing on server 2's resource usage and its own task execution status) after the agent executes the action deviates little from the global belief (the distribution of system resources and task adaptation). The quantified internal belief error is 0.12; ④ Calculate the overall free energy according to the comprehensive rules. Finally, the free energy of this candidate action is 0.65. Repeat the above process to calculate the overall free energy corresponding to each of the 30 candidate actions.

[0043] S43. Select the action that minimizes the predicted overall free energy from a set of candidate actions, and use it as the optimized decision action for the agent; Specifically, for step S43, the overall free energy is the core optimization indicator that balances external benefits (task execution performance) and internal belief error (system belief consistency). A smaller free energy means that the action can ensure efficient task completion while maintaining the coordination of system beliefs, avoiding resource conflicts or efficiency waste caused by the disconnect between local decisions and the global state. By ranking the free energies of all candidate actions, the action with the smallest free energy is selected. This action is the optimal decision best suited to the agent under the current global belief. For example, ranking the free energies of the 30 candidate actions reveals that the action with the smallest free energy (0.52) is "selecting server 3 and allocating 20 GFLOPS computing power + 50 Mbps bandwidth": this action has an external benefit of 0.9 (100% prediction task completion rate and 90% inference accuracy) and an internal belief error of 0.08 (high consistency between local and global beliefs), resulting in the best overall optimization effect. Therefore, this action is determined as the optimized decision action for the agent.

[0044] This embodiment generates diverse candidate actions based on global beliefs, combines external benefits with internal belief errors to quantify the overall free energy, and selects the optimal action with the minimum free energy. This achieves precise optimization of decision-making actions, ensuring both task execution effectiveness and maintaining system belief consistency, significantly improving the rationality and effectiveness of multi-agent resource optimization decisions.

[0045] Furthermore, in some embodiments, step S41, "generating a set of candidate actions for each agent based on the current global belief," may specifically include: S411. Based on the statistical characteristics of historical action distribution, initialize a parameterized action sampling distribution; Specifically, for step S411, the statistical characteristics of the historical action distribution refer to the numerical patterns of all effective actions in past decisions across various dimensions (such as server selection, resource allocation, etc.), including core indicators such as mean and variance. The parameterized action sampling distribution initialized based on these characteristics can inherit the effective experience of historical decisions, ensuring the initial sampled actions have a certain degree of rationality and providing a valuable basic distribution for the generation of subsequent candidate actions. This action sampling distribution is a probability distribution with adjustable parameters, which will dynamically adapt with subsequent optimization processes. For example, in an edge computing resource allocation scenario, actions include four dimensions: "server selection, bandwidth allocation, CPU allocation, and memory allocation." Statistical analysis of the historical actions from the past 30 rounds of effective decisions yields the statistical characteristics of each dimension: the server selection dimension has a mean of "server 3" (quantized as 3) and a variance of 0.8; the bandwidth allocation dimension has a mean of 60Mbps and a variance of 10; the CPU allocation dimension has a mean of 25GFLOPS and a variance of 5; and the memory allocation dimension has a mean of 8GB and a variance of 2. Based on these characteristics, a four-dimensional Gaussian distribution N(μ,σ) is initialized as the action sampling distribution, where μ=[3,60,25,8] and σ=[0.8,10,5,2].

[0046] S412. Randomly sample from the action sampling distribution to generate a first candidate action subset; Specifically, in step S412, a certain number of actions are randomly sampled from the parameterized action sampling distribution. This sampling process fully utilizes the probabilistic characteristics of the distribution to ensure that the distribution of actions across all dimensions is consistent with the historical distribution trends of effective actions. Simultaneously, randomness introduces a degree of exploration, preventing actions from being limited to a single range and allowing the first candidate action subset to cover more potential effective decision-making directions. For example, 20 actions are randomly sampled from the aforementioned four-dimensional Gaussian distribution to generate the first candidate action subset. Some of these actions include "Server 2, bandwidth 55Mbps, CPU 22GFLOPS, memory 7GB", "Server 3, bandwidth 68Mbps, CPU 28GFLOPS, memory 9GB", and "Server 4, bandwidth 52Mbps, CPU 20GFLOPS, memory 6GB", etc., which not only conform to the distribution patterns of historical actions but also cover different server selection and resource allocation combinations through randomness.

[0047] S413. Input the current global belief vector into the pre-trained policy network, and output the second candidate action subset through forward propagation reasoning; Specifically, in step S413, the pre-trained policy network has learned the mapping relationship between global beliefs and optimal actions through a large amount of historical data. It can accurately capture key decision-making features based on the current global beliefs (including global information such as the overall system resource status and task requirements). After inputting the global belief vector into the network, feature extraction and inference calculation are completed through the forward propagation process. The output subset of second candidate actions has clear targeting and effectiveness, focusing on the potential optimal decision direction in the current scenario. For example, the current global belief vector contains global information such as "remaining resources of the 4 servers in the entire system, total task requirements of 8 agents, and the adaptation relationship between each task and the server." This 16-dimensional global belief vector is input into the pre-trained policy network. Through forward propagation inference, the network outputs 10 highly targeted actions as the subset of second candidate actions, including "Server 3, bandwidth 62Mbps, CPU 26GFLOPS, memory 8GB" and "Server 1, bandwidth 70Mbps, CPU 30GFLOPS, memory 10GB," etc. These actions are highly adapted to the current global resource status and task requirements.

[0048] S414. Merge the first candidate action subset and the second candidate action subset, and perform deduplication to form a set of candidate actions; Specifically, in step S414, merging the two candidate action subsets fully combines the advantages of both: the exploratory nature of the first candidate action subset and the targeted nature of the second candidate action subset, resulting in a more comprehensive coverage of the candidate action set. Deduplication removes duplicate actions, avoids redundant calculations, and ensures that each candidate action is unique and valuable, ultimately forming a set of candidate actions that combines diversity and effectiveness, providing a high-quality foundation for subsequent optimization and selection. For example, merging the aforementioned 20 first candidate actions with 10 second candidate actions yields a set of 30 actions; after deduplication, it is found that "Server 3, bandwidth 62Mbps, CPU 26GFLOPS, memory 8GB" exists in both subsets. After removing duplicate actions, a final set of candidate actions containing 29 unique actions is formed, covering both exploratory actions and retaining highly targeted and effective actions.

[0049] This embodiment generates a candidate action set that is both exploratory and targeted by combining random sampling based on historical experience with policy network reasoning based on global beliefs. This ensures both the diversity of actions to uncover potential optimal decisions and the effectiveness of actions to meet the needs of the current scenario, providing a high-quality alternative basis for subsequent free energy optimization screening and improving the efficiency and reliability of decision optimization.

[0050] Furthermore, in some embodiments, step S5, "controlling each agent to execute the optimized decision-making action and obtaining feedback results from the environment," may specifically include: S51. Based on the decision action, select a target processing server for the task to be processed; Specifically, for step S51, the optimized decision action clearly includes server selection information that adapts to the current task requirements and global resource status. This step requires accurately extracting the server identifier specified in the decision action and directing the task to be processed to the target server. The selection process must ensure that the target server is completely consistent with the specified decision action and that the server is in an available state, providing a stable hardware platform for the subsequent execution of the task. For example, the optimized decision action for an agent's audio question-and-answer task is "select server 3, allocate 20 GFLOPS computing power + 50 Mbps bandwidth". Based on this decision action, the server identifier "server 3" is directly extracted, and the agent's audio question-and-answer task is determined to be assigned to server 3 for processing, ensuring that the task is directed to the target hardware node.

[0051] S52. Based on the decision action, allocate computing and communication resources to the task on the target processing server; Specifically, for step S52, the decision action clearly indicates the specific amount of computing and communication resources required to complete the task, and this amount is already adapted to the remaining resource capacity of the target server. This step requires reserving corresponding resources on the target processing server according to the resource values ​​specified in the decision action. The computing resources must meet the task's computational processing requirements, and the communication resources must ensure the smooth transmission of task data. Resource allocation must accurately match the requirements of the decision action, not exceeding the specified amount and without wasting server resources. For example, based on the above decision action of "selecting server 3 and allocating 20 GFLOPS computing power + 50 Mbps bandwidth," resources are allocated on target server 3 for this audio question-and-answer task: In terms of computing resources, 20 GFLOPS of CPU computing power and 8 GB of memory are reserved (the decision action implicitly adapts to the memory requirements of this task); in terms of communication resources, 50 Mbps of dedicated bandwidth is reserved to ensure a stable and sufficient resource supply during task data transmission and computational processing.

[0052] This embodiment achieves the implementation of decision-making strategies by accurately executing server selection and resource allocation requirements in the decision-making process. It provides suitable hardware carriers and sufficient resource guarantees for the tasks to be processed, ensuring that the tasks can be started smoothly and run efficiently, thereby improving the stability and reliability of task processing.

[0053] Furthermore, in some embodiments, step S6, "updating the decision model based on the decision action, the overall free energy, and the feedback result," may specifically include: S61. Store the local observation information, global beliefs, decision actions, overall free energy and feedback results within the current decision-making cycle as a piece of empirical data in the empirical replay buffer; Specifically, for step S61, the core data of the current decision-making cycle covers key information throughout the entire decision-making process, including the inputs to the decision (local observation information, global beliefs), the actions executed (decision actions), the optimization index of the decision (overall free energy), and the actual effects of the decision (feedback results). This information is integrated into a complete set of empirical data and stored in a dedicated empirical replay buffer. This buffer uses a first-in, first-out (FIFO) storage rule to ensure the timeliness of the data while retaining sufficient historical experience to provide rich training data support for subsequent model updates. For example, in one decision-making cycle of an audio question-answering task, a piece of empirical data is integrated, including: local observation information (server 3 has 35 GFLOPS of remaining CPU computing power and 110 Mbps of bandwidth, the link latency between the agent and server 3 is 3 ms, and the task data volume is 25 MB), global beliefs (a 16-dimensional vector covering the resource distribution of the three servers in the entire system and the task requirements of the five agents), decision action (selecting server 3 and allocating 22 GFLOPS of computing power + 55 Mbps of bandwidth), overall free energy (0.58), and feedback results (task completion rate 100%, inference accuracy 88%, latency 5.2 seconds, and overall feedback value 0.85). This piece of data is stored in an experience replay buffer with a capacity of 10,000 pieces. When the data volume exceeds the capacity, the earliest stored empirical data is automatically removed.

[0054] S62. Sample multiple empirical data points from the empirical replay buffer; Specifically, for step S62, to avoid overfitting caused by continuously using related data during model training, multiple unrelated empirical data points are randomly selected from the experience replay buffer to form a training batch. This sampling process ensures the randomness and diversity of the data, enabling the training data to cover decision-making situations in different scenarios and allowing the model to learn more general decision-making patterns. For example, when the decision model needs to be updated, 32 empirical data points are randomly sampled from the experience replay buffer to form a training batch. These data cover different task types (audio question answering, voice classification), different server resource states (high load, low load), and different decision-making effects (excellent, moderate, poor), ensuring the diversity and representativeness of the training data.

[0055] S63. Based on sampled empirical data, calculate the loss of the policy network and / or value function network in the decision model, and update the parameters of the corresponding network through gradient backpropagation; Specifically, in step S63, the policy network is responsible for generating decision actions, and the value function network is responsible for evaluating the merits of these actions; together, they constitute the core of the decision model. Based on sampled empirical data, the actions predicted by the model or the evaluation values ​​of the actions are compared with the actual results in the empirical data (such as feedback results, overall free energy), quantifying the difference between the two, which is the network loss. Subsequently, through the gradient backpropagation algorithm, the network's weights, biases, and other parameters are adjusted along the direction of loss reduction, enabling the model to learn from historical experience and continuously improve the accuracy and adaptability of decisions. For example, for 32 sampled empirical data points, the global belief of each data point is input into the policy network to obtain the decision actions predicted by the model. This is compared with the actual decision actions executed in the empirical data to calculate the loss of the policy network. Simultaneously, the expected benefits of the predicted actions are evaluated through the value function network and compared with the actual feedback results in the empirical data to calculate the loss of the value function network. Based on these two types of losses, the parameters of the two networks are adjusted through the gradient backpropagation algorithm, making the actions subsequently generated by the network closer to the actual optimal decision, and the evaluation results more accurately reflect the actual action effects.

[0056] This embodiment retains experience data from the entire decision-making process and randomly samples it for training. By combining loss calculation and parameter updates, the decision-making model can continuously learn and optimize from historical experience, thereby continuously improving the accuracy of decision-making and its adaptability to dynamic scenarios, and ensuring the stability and efficiency of the system's long-term operation.

[0057] To facilitate understanding of the resource optimization method based on multi-agent active reasoning provided in this embodiment, this embodiment also provides a specific implementation of the resource optimization method based on multi-agent active reasoning. Taking the method executed in a resource optimization system based on multi-agent active reasoning as an example, the system includes a multi-agent module, a multi-agent communication generative model (MACGM), a Transformer belief communication module, an offloading planner, and an environment interaction module. The functions of each component are as follows: Multi-agent module: Each user device (UD) acts as an independent agent, responsible for perceiving its own task status (type, size) and edge server resource status (remaining bandwidth, CPU, memory, link status), and executing task offloading decisions.

[0058] The Multi-Agent Communication Generative Model (MACGM) includes a belief encoder, a local state dynamics module, a preference calculation module, and a belief error calculation module, which realizes the transformation and dynamic updating of local observations to global beliefs.

[0059] Transformer Belief Communication Module: Composed of multiple Transformer encoders, it integrates the local beliefs and action information of each agent through a multi-head attention mechanism to generate global beliefs.

[0060] Unloading Planner: Based on the RL-DrivenMPPI method, combined with Gaussian distribution sampling and policy network, candidate actions are generated and optimized by free energy sorting, and the optimal resource allocation and task unloading strategy is output.

[0061] Environment Interaction Module: Responsible for the interaction between the agent and the MEC environment, obtaining server resource status feedback, task execution results (latency, accuracy), and updating the experience replay buffer.

[0062] The specific workflow is as follows: Step 1. State Awareness and Local Belief Generation: Each agent acquires local observation information through the environment interaction module, including: the remaining bandwidth B, CPU resources C, and memory resources M of the edge server set S; the distance d between the agent and the server; the server link status L (1 = idle, 0 = occupied); its own task type T (audio question answering AQA, music note analysis MNA, human voice classification VSC); and the task size. The local observations are then transformed into local beliefs through a belief encoder. The network topology parameters of the belief encoder are a 3-layer fully connected perceptron (MLP): the input layer has 12 neurons (corresponding to the dimensions of local observations: server remaining bandwidth B, CPU resources C, memory resources M, distance d, link state L, 3-dimensional one-hot encoding of task type T, and task size, totaling 1+1+1+1+1+3+1=9? In reality, the observation dimensions need to be matched, but 12 dimensions are used here to accommodate extended features), the first hidden layer has 64 neurons with ReLU activation function, the second hidden layer has 32 neurons with ReLU activation function, and the output layer has 16 neurons (i.e., local beliefs). (Dimensions), and a Dropout layer (dropout rate of 0.1) is added to each layer to prevent overfitting.

[0063] The detailed process of transforming local observations into local beliefs is as follows: ① Normalize the features of each dimension of the local observations (remaining bandwidth B, CPU resources C, and memory resources M are normalized to the [0,1] interval, distance d is normalized to the [0,1] interval, link state L remains a binary value, task type T is transformed into 3-dimensional one-hot encoding, and task size is normalized to the [0,1] interval); ② Input the normalized feature vector into the input layer of the belief encoder; ③ Pass it through the first hidden layer (linear transformation + ReLU activation + Dropout) and the second hidden layer (linear transformation + ReLU activation + Dropout); ④ Obtain the 16-dimensional local belief through the linear transformation of the output layer. .

[0064] Step 2. Multi-agent belief communication and global belief generation: Each agent will have local beliefs In line with expected actions Send to the Transformer Belief Communication module to form the input sequence. and By calculating query Q, key K, and value V through a multi-head attention mechanism, weighted features of information from each agent are extracted. After processing through a residual network and normalization, global beliefs are generated, thus achieving the fusion of local and global information.

[0065] The detailed process of this step is as follows: ① Input sequence and ① Concatenate the data into a fusion sequence of dimension N×(16+4) (N is the number of agents, 16 is the local belief dimension, and 4 is the expected action dimension: server selection k, bandwidth b, CPU resource c, and memory resource m); ② Perform a linear transformation on the fusion sequence to generate a query matrix. Key matrix Value matrix ( , , (The transformation matrix is ​​a learnable linear matrix, with dimensions of 64×20); ③ Multi-head attention calculation is performed using 8 attention heads: For each attention head, Q, K, and V are split according to the head dimension (each head has a dimension of 8), and the attention score is calculated. ( =8 is the head dimension, Softmax is used to normalize the score), the outputs of the 8 attention heads are concatenated into a feature matrix of dimension N×64; ④ A linear transformation is performed on the concatenated feature matrix (matrix ⑤ Perform residual connections and layer normalization: add the attention output to the original fusion sequence (after dimension matching linear transformation), and then perform layer normalization (LN) to obtain the residual output; ⑥ Input the residual output into the feedforward network (2 fully connected layers: 32→128→32, activation function is ReLU), and perform residual connections and layer normalization again; ⑦ Through the final linear transformation layer, generate a global belief zt=Cθ(et,at) with a dimension of N×16, realizing the fusion of local information and global information.

[0066] Step 3. Free energy calculation and preference / belief error assessment: Based on global beliefs The external benefits of the agent are obtained through the preference calculation module. (Corresponding to rewards such as task completion rate and accuracy), the internal belief error is obtained through the belief error calculation module. (Quantifying the deviation between local and global beliefs). The formula for calculating the average free energy of a multi-agent system is: The first part is the expected information gain (belief error), and the second part is the external benefit (preference), which is explored and utilized by minimizing the free energy balance.

[0067] The detailed process of this step is as follows: The operation process of the preference calculation module: The preference calculation module adopts a 2-layer fully connected perceptron (MLP) structure: The input is local beliefs. (16-dimensional) and action The concatenated vector (4-dimensional) has 20 dimensions. The first hidden layer has 32 neurons with ReLU activation function, and the output layer has 1 neuron with Sigmoid activation function. The operation process is as follows: ① ... and ① Concatenate into a 20-dimensional vector; ② Obtain a 32-dimensional feature through the first hidden layer (linear transformation + ReLU activation); ③ Obtain the final feature through the output layer (linear transformation + Sigmoid activation). (Value range [0,1], corresponding to the normalized weighted sum of task completion rate (60% weight) and inference accuracy (40% weight)). The calculation process of the belief error calculation module: The belief error calculation module adopts a 2-layer fully connected perceptron (MLP) structure: the input is local belief. (16-dimensional) and global beliefs The concatenated vector is 16-dimensional (32-dimensional), with 64 neurons in the first hidden layer and ReLU activation function, and 1 neuron in the output layer and ReLU activation function.

[0068] The operation process is as follows: ① and ① Concatenate into a 32-dimensional vector; ② Obtain a 64-dimensional feature through the first hidden layer (linear transformation + ReLU activation); ③ Obtain the output layer (linear transformation)... (Values ​​≥ 0 correspond to the KL divergence between local and global beliefs, i.e.) .

[0069] The calculation process of the average free energy of a multi-agent system is as follows: ① Traverse all agents and obtain the average free energy of each agent through the above modules. and ② For all intelligent agents Summing and taking the negative yields the expected information gain term: ③ For all intelligent agents Summing and taking the negative yields the external benefit term: ④ Add the two items together to obtain the average free energy of the multi-agent system. .

[0070] Step 4. Unloading Planning and Action Generation: The planner is unloaded using the RL-DrivenMPPI method, with the following steps: a. Candidate action set generation: from Gaussian distribution Sample K = 20 candidate actions (where The initial value is the average of the best moves in the previous round. Initially set to 0.2 (an empirically learnable parameter); from the policy network (3-layer MLP structure: input is global belief) (16-dimensional), with 64 neurons in the first hidden layer, 32 neurons in the second hidden layer, and 4 neurons in the output layer corresponding to the action dimension) Sample P=10 candidate actions; after deduplicating the two types of actions, form a set containing 30 candidate actions.

[0071] b. Free energy calculation of candidate actions: ① Each agent calculates based on global beliefs. With candidate actions Through local state dynamic module (2-layer MLP structure: input is) (16-dimensional) and The concatenated (4-dimensional) vector outputs a 16-dimensional local belief, predicting the local state at the next time step. ; ② will and Substitute into the preference calculation module to obtain the corresponding candidate action. );③ will and Substituting into the belief error calculation module, we obtain the corresponding candidate action. ④ Substitute the formula for the average free energy of a multi-agent agent to calculate the corresponding action of the candidate action. value.

[0072] c. Candidate action distribution update: ① Update all actions in the candidate set according to... ① Sort the data from smallest to largest; ② Select the top k=5 candidate actions with the smallest free energy; ③ Calculate the mean of these 5 actions across the four dimensions of "server selection, bandwidth, CPU, and memory", and use this as the new Gaussian distribution mean. ④ Update the standard deviation of the Gaussian distribution to This completes the iterative optimization of the candidate action distribution.

[0073] d. Optimal action output: Repeat step ac for a total of 10 iterations (or until the difference in free energy between two adjacent rounds is less than...). After the iteration terminates, select the action with the minimum free energy from the last round of candidate sets and output the optimal action. Where k is the selected edge server (selected from the server set S), b is the allocated bandwidth (not exceeding the server's remaining bandwidth B), c is the allocated CPU resources (not exceeding the remaining CPU C), and m is the allocated memory resources (not exceeding the remaining memory M).

[0074] Step 5. Task Execution and Model Update: The agent executes optimal task unloading and resource allocation actions. The environment interaction module obtains the task execution results (latency, task completion rate, inference accuracy) and stores the empirical data (local observations, global beliefs, actions, free energy, feedback results) in the replay buffer. The model parameters are updated using the policy network loss and Q-value network loss. The policy network loss is: ; The Q-value network loss is: ;in The target Q value is calculated by combining the multi-step free energy with the target network output.

[0075] The detailed process of this step is as follows: Task execution and result acquisition: ① The agent will perform the optimal action ① Encapsulate the request as a resource allocation request and send it to the selected edge server k; ② Server k reserves the corresponding hardware resources and starts a large audio language model (ALMs) task instance (such as loading the Qwen-AudioChat7.7B model for the AQA task) based on the bandwidth b, CPU resources c, and memory resources m in the request; ③ During task execution, the environment interaction module monitors the task latency in real time; After the task is completed, the result is determined according to preset rules: Task completion rate: 1 (sufficient resources and normal task execution) / 0 (insufficient resources or abnormal execution); Inference accuracy: semantic similarity of the AQA task matching the standard answer (normalized to [0,1]), accuracy of note recognition in the MNA task, and accuracy of voice classification in the VSC task; ④ Integrate to obtain a comprehensive feedback result: (Normalized to the [0,1] interval).

[0076] Experience data is stored in the replay buffer: ① The replay buffer is a first-in, first-out queue with a capacity of 10,000, and each piece of experience data is in tuple format. ,in For current local observation, For the current overall belief, For the action to be performed, For the current multi-agent mean free energy, To provide comprehensive feedback results; ② After each task is executed, the tuple is stored in a buffer; when the buffer reaches its capacity limit, the earliest stored empirical data is removed to ensure data timeliness.

[0077] Policy network parameter update: Policy network A 3-layer fully connected perceptron (input: global belief) (16-dimensional); First hidden layer: 64 neurons + ReLU activation; Second hidden layer: 32 neurons + ReLU activation; Output layer: 4 neurons (corresponding to the action dimension); The update process is as follows: ① Randomly sample 32 empirical data points from the replay buffer to form a mini-batch; ② For each data point, calculate the action distribution output by the policy network. ③ Call the Q-value network Calculate the Q value corresponding to this action. ④ Regarding the policy network parameters Find the gradient: ⑤ The negative value is used as the loss gradient; ⑤ The Adam optimizer (learning rate) is used. The policy network parameters are updated in the opposite direction of the loss gradient.

[0078] Q-value network parameter update: Q-value network (Q_mu) A 2-layer fully connected perceptron (input: global belief) (16 dimensions) + Action (4-dimensional) concatenated vector (20-dimensional); First hidden layer: 64 neurons + ReLU activation; Output layer: 1 neuron (linear activation); Update process is: ① Randomly sample 32 empirical data points from the replay buffer to form a mini-batch; ② Calculate the target Q value y: (in Here, n=3 is the discount factor, and n=3 is the step size of the multi-step free energy. For the target Q-value network parameters, soft updates are used: ); ③ Calculate the current Q-value network output. The mean square error between the target Q-value y and the target Q-value y is used as the Q-value network loss. ④ Use the Adam optimizer (learning rate) The Q-value network parameters are updated in the opposite direction of the loss gradient.

[0079] Step 6. Repeat steps 1-5 iteratively, calculating the change in the average free energy of the multi-agent agent after each iteration; when the change in free energy is less than 5 consecutive iterations... When the model converges, the iteration stops. In subsequent dynamic scenarios, steps 1-5 are repeated every 10 minutes to continuously optimize the task processing strategy.

[0080] In summary, compared with the prior art, the resource optimization method based on multi-agent active reasoning provided in this embodiment transforms the local observations of each agent into global beliefs, optimizes decisions and dynamically updates the model with the goal of minimizing the overall free energy that combines external benefits and internal belief errors. This effectively solves the problems of limitations in local decision-making, single optimization objectives, and insufficient dynamic adaptability of multi-agent systems, improves the globality, stability, and effectiveness of resource optimization decisions, and ensures the overall performance of multi-agent systems in resource allocation and task processing.

[0081] To facilitate better implementation of the resource optimization method based on multi-agent active reasoning in the embodiments of this application, this application also provides a resource optimization apparatus based on multi-agent active reasoning, which is based on the aforementioned resource optimization method based on multi-agent active reasoning. The meanings of the terms used are the same as in the aforementioned resource optimization method based on multi-agent active reasoning, and specific implementation details can be found in the descriptions in the method embodiments.

[0082] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a resource optimization device based on multi-agent active reasoning provided in an embodiment of this application. Specifically, the resource optimization device based on multi-agent active reasoning may include an acquisition module 201, a local belief module 202, a global belief module 203, an optimization module 204, an execution module 205, and an update module 206, as follows: Acquisition module 201 is used to acquire local observation information of multiple intelligent agents; The local belief module 202 is used to generate local beliefs for each agent based on local observation information. The global belief module 203 is used to perform belief communication based on local beliefs and generate global beliefs. The optimization module 204 is used to optimize the decision actions of each agent based on global beliefs by using a decision model to minimize the overall free energy of the multi-agent system as the optimization objective; wherein, the overall free energy is calculated by combining external benefits and internal belief errors; The execution module 205 is used to control each intelligent agent to execute the optimized decision action and to receive the feedback results generated by the environment after the decision action is executed. Update module 206 is used to update the decision model based on decision actions, overall free energy, and feedback results.

[0083] Furthermore, in some embodiments, the acquisition module 201 is specifically used for: Obtain the current resource status information of each edge server; Obtain network link status information between each intelligent agent and the edge server; Obtain feature information of the tasks to be processed initiated by each agent.

[0084] Furthermore, in some embodiments, the local belief module 202 is specifically used for: Feature normalization is performed on the local observation information of each agent; The normalized feature vectors are input into the belief encoder; The feature vectors are nonlinearly transformed and encoded by a multilayer perceptual network of a belief encoder to generate local belief vectors for each agent.

[0085] Furthermore, in some embodiments, the global belief module 203 is specifically used for: Organize the local belief vectors of each agent into an input sequence; The input sequence is processed using an attention-based network model, the correlation weights between local beliefs are calculated and information is fused to generate a global belief vector that integrates information from all agents.

[0086] Furthermore, in some embodiments, the global belief module 203 is specifically used for: Perform a linear transformation on the input sequence to generate a query vector group, a key vector group, and a value vector group, respectively; A multi-head attention mechanism is adopted to calculate the attention weights between elements in the input sequence based on the query vector group, key vector group, and value vector group, and then perform weighted aggregation to obtain the weighted aggregated feature sequence. The weighted and aggregated feature sequence is input into the feedforward neural network layer for processing; The output of the feedforward neural network layer is residually connected to the weighted aggregated feature sequence. The results of the residual connections are subjected to layer normalization to obtain the global belief vector.

[0087] Furthermore, in some embodiments, the optimization module 204 is specifically used for: Based on the current global belief, generate a set of candidate actions for each agent; For each candidate action in a set of candidate actions, based on global beliefs and the predicted system state after the candidate action is executed, the external benefits and internal belief errors corresponding to the system state are calculated, and the overall free energy of the system after the candidate action is executed is predicted. Choose the action that minimizes the predicted overall free energy from a set of candidate actions and use it as the optimized decision action for the agent.

[0088] Furthermore, in some embodiments, the optimization module 204 is specifically used for: Based on the statistical characteristics of historical action distribution, initialize a parameterized action sampling distribution; Randomly sample from the action sampling distribution to generate the first candidate action subset; The current global belief vector is input into the pre-trained policy network, and the second candidate action subset is output through forward propagation reasoning. The first and second candidate action subsets are merged and deduplicated to form a set of candidate actions.

[0089] Furthermore, in some embodiments, the execution module 205 is specifically used for: Based on the decision action, select a target processing server for the task to be processed; Based on the decision action, computing and communication resources are allocated to the task on the target processing server.

[0090] Furthermore, in some embodiments, the update module 206 is specifically used for: The local observation information, global beliefs, decision actions, overall free energy and feedback results within the current decision-making cycle are stored as a piece of empirical data in the empirical replay buffer. Sample multiple empirical data points from the empirical replay buffer; Based on sampled empirical data, the loss of the policy network and / or value function network in the decision model is calculated, and the parameters of the corresponding network are updated through gradient backpropagation.

[0091] For specific limitations regarding the resource optimization device based on multi-agent active reasoning, please refer to the limitations of the resource optimization method based on multi-agent active reasoning mentioned above, which will not be repeated here. Each module in the aforementioned resource optimization device based on multi-agent active reasoning can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0092] The resource optimization device based on multi-agent active reasoning provided in this embodiment transforms the local observations of each agent into global beliefs. It optimizes decision-making and dynamically updates the model with the goal of minimizing the overall free energy that combines external benefits and internal belief errors. This effectively solves the problems of limitations in local decision-making, single optimization objectives, and insufficient dynamic adaptability of multi-agent systems. It improves the globality, stability, and effectiveness of resource optimization decision-making and ensures the overall performance of multi-agent systems in resource allocation and task processing.

[0093] Furthermore, embodiments of this application also provide an electronic device, such as... Figure 4 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically: The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, a power supply 303, and an input unit 304. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 301 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 302, and by calling data stored in the memory 302, thereby providing overall monitoring of the electronic device. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 301.

[0094] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and resource optimization methods based on multi-agent active reasoning by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0095] The electronic device also includes a power supply 303 that supplies power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 303 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0096] The electronic device may also include an input unit 304, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0097] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 302 according to the following instructions, and the processor 301 runs the applications stored in the memory 302 to realize various functions, as follows: The system acquires local observation information from multiple agents; generates local beliefs for each agent based on the local observation information; performs belief communication based on the local beliefs to generate global beliefs; optimizes decision actions for each agent based on the global beliefs by minimizing the overall free energy of the multi-agent system through a decision model; the overall free energy is calculated by combining external benefits and internal belief errors; controls each agent to execute the optimized decision actions and receives feedback results from the environment after the decision actions are executed; and updates the decision model based on the decision actions, overall free energy, and feedback results.

[0098] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0099] This application's embodiments transform the local observations of each agent into global beliefs, optimize decision-making with the goal of minimizing the overall free energy that combines external benefits and internal belief errors, and dynamically update the model. This effectively solves the problems of limitations in local decision-making, single optimization objectives, and insufficient dynamic adaptability of multi-agent systems, improves the globality, stability, and effectiveness of resource optimization decisions, and ensures the overall performance of multi-agent systems in resource allocation and task processing.

[0100] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0101] To this end, embodiments of this application provide a storage medium storing multiple instructions that can be loaded by a processor to execute steps in any of the resource optimization methods based on multi-agent active reasoning provided in embodiments of this application. For example, the instructions can execute the following steps: The system acquires local observation information from multiple agents; generates local beliefs for each agent based on the local observation information; performs belief communication based on the local beliefs to generate global beliefs; optimizes decision actions for each agent based on the global beliefs by minimizing the overall free energy of the multi-agent system through a decision model; the overall free energy is calculated by combining external benefits and internal belief errors; controls each agent to execute the optimized decision actions and receives feedback results from the environment after the decision actions are executed; and updates the decision model based on the decision actions, overall free energy, and feedback results.

[0102] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0103] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0104] Since the instructions stored in the storage medium can execute the steps in any of the resource optimization methods based on multi-agent active reasoning provided in the embodiments of this application, the beneficial effects that any of the resource optimization methods based on multi-agent active reasoning provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0105] The above provides a detailed description of a resource optimization method and apparatus based on multi-agent active reasoning provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A resource optimization method based on multi-agent active reasoning, characterized in that, include: Acquire local observation information from multiple intelligent agents; Based on the local observation information described above, generate local beliefs corresponding to each agent; Based on the aforementioned local beliefs, belief communication is performed to generate global beliefs; The decision model aims to minimize the overall free energy of the multi-agent system, and optimizes the decision actions of each agent based on the global beliefs; wherein the overall free energy is calculated by combining external benefits and internal belief errors. Control each intelligent agent to execute optimized decision actions, and receive feedback results generated by the environment after the decision actions are executed; The decision model is updated based on the decision action, the overall free energy, and the feedback result.

2. The resource optimization method based on multi-agent active reasoning according to claim 1, characterized in that, The acquisition of local observation information from multiple agents includes: Obtain the current resource status information of each edge server; Obtain network link status information between each intelligent agent and the edge server; Obtain feature information of the tasks to be processed initiated by each agent.

3. The resource optimization method based on multi-agent active reasoning according to claim 1, characterized in that, The step of generating local beliefs for each agent based on the local observation information includes: Feature normalization is performed on the local observation information of each agent; The normalized feature vectors are input into the belief encoder; The feature vector is nonlinearly transformed and encoded by the multilayer perceptual network of the belief encoder to generate local belief vectors corresponding to each agent.

4. The resource optimization method based on multi-agent active reasoning according to claim 1, characterized in that, The step of generating global beliefs through belief communication based on the aforementioned local beliefs includes: Organize the local belief vectors of each agent into an input sequence; The input sequence is processed using an attention-based network model to calculate the correlation weights between local beliefs and perform information fusion to generate a global belief vector that integrates information from all agents.

5. The resource optimization method based on multi-agent active reasoning according to claim 4, characterized in that, The process of using an attention-based network model to process the input sequence, calculate the association weights between local beliefs, and perform information fusion includes: A linear transformation is performed on the input sequence to generate a query vector group, a key vector group, and a value vector group, respectively. A multi-head attention mechanism is adopted to calculate the attention weights between each element in the input sequence based on the query vector group, the key vector group, and the value vector group, and then perform weighted aggregation to obtain the weighted aggregated feature sequence. The weighted and aggregated feature sequence is input into a feedforward neural network layer for processing. The output of the feedforward neural network layer is residually connected to the weighted aggregated feature sequence. The results of the residual connections are subjected to layer normalization to obtain the global belief vector.

6. The resource optimization method based on multi-agent active reasoning according to claim 1, characterized in that, The optimization objective of minimizing the overall free energy of the multi-agent system through a decision model, and the optimization of decision actions for each agent based on the global beliefs, includes: Based on the current global belief, generate a set of candidate actions for each agent; For each candidate action in the set of candidate actions, based on the global belief and the predicted system state after the candidate action is executed, the external benefit and internal belief error corresponding to the system state are calculated, and the overall free energy of the system after the candidate action is executed is predicted. The action that minimizes the predicted overall free energy is selected from the set of candidate actions and used as the optimized decision action for the agent.

7. The resource optimization method based on multi-agent active reasoning according to claim 6, characterized in that, Based on the current global belief, a set of candidate actions is generated for each agent, including: Based on the statistical characteristics of historical action distribution, initialize a parameterized action sampling distribution; Randomly sample from the action sampling distribution to generate a first candidate action subset; The current global belief vector is input into the pre-trained policy network, and the second candidate action subset is output through forward propagation reasoning. The first subset of candidate actions and the second subset of candidate actions are merged and deduplicated to form the set of candidate actions.

8. The resource optimization method based on multi-agent active reasoning according to claim 1, characterized in that, The process of controlling each intelligent agent to execute optimized decision-making actions and obtaining feedback results from the environment includes: Based on the decision action, select a target processing server for the task to be processed; Based on the decision action, computing and communication resources are allocated to the task on the target processing server.

9. The resource optimization method based on multi-agent active reasoning according to claim 1, characterized in that, The step of updating the decision model based on the decision action, the overall free energy, and the feedback result includes: The local observation information, global beliefs, decision actions, overall free energy and feedback results within the current decision-making cycle are stored as a piece of empirical data in the empirical replay buffer. Sample multiple pieces of empirical data from the empirical replay buffer; Based on sampled empirical data, the loss of the policy network and / or value function network in the decision model is calculated, and the parameters of the corresponding network are updated through gradient backpropagation.

10. A resource optimization device based on multi-agent active reasoning, characterized in that, include: The acquisition module is used to acquire local observation information from multiple agents; The local belief module is used to generate local beliefs for each agent based on the local observation information. The global belief module is used to perform belief communication based on the local beliefs and generate global beliefs. An optimization module is used to optimize decision-making actions for each agent based on the global beliefs, with the goal of minimizing the overall free energy of the multi-agent system through a decision model; wherein the overall free energy is calculated by combining external benefits and internal belief errors; The execution module is used to control each intelligent agent to execute the optimized decision-making actions and to receive feedback results generated by the environment after the decision-making actions are executed. An update module is used to update the decision model based on the decision action, the overall free energy, and the feedback result.

Images