A game incentive-based heterogeneous agent cooperative control method, controller and system

Abstract

The application provides a game incentive-based full-heterogeneous-body cooperative control method, a controller and a system. In view of the problem that the full-heterogeneous-body has low cooperative efficiency due to local target conflict in distributed interaction, an incentive injection scheme driven by evaluation signal reconstruction is proposed: a cooperative controller collects local feedback of multiple bodies and calculates a global cooperative reward in combination with a global target. A cooperative weight factor is dynamically updated by monitoring the public resource competition state, guiding each body to weight and fuse individual income and global reward, and realizing evaluation signal reconstruction at the decision-making bottom. The application converts the system-level cooperative constraint into an incentive guide that can be perceived by individuals, guiding the full-heterogeneous-body with an asymmetric action space to spontaneously adjust the strategy to tend to a Pareto optimal equilibrium. While ensuring distributed autonomy, the application significantly improves the cooperative response speed and joint action efficiency of the global system in complex task scenarios.

Description

Technical Field

[0001] This invention relates to the fields of cyber-physical systems (CPS) and multi-agent cooperative technology, and particularly to a method, controller and system for cooperative control of all heterogeneous agents based on game-theoretic incentives. Background Technology

[0002] With the rapid development of Cyber-Physical Systems (CPS), the Internet of Things (IoT), and embodied intelligence, system scale is growing exponentially. These systems typically consist of a massive number of distributed, mobile, heterogeneous agents (such as intelligent vehicles, drones, and mobile robots) and edge nodes (such as edge clouds and roadside units). The constituent agents exhibit significant heterogeneity; for example, some agents focus on precise control of real-time physical states, while others focus on the dynamic orchestration of macroscopic resource logic. To achieve efficient system operation, these agents need to synchronize state data via wireless networks and compete for limited edge computing resources, thus forming a typical heterogeneous multi-agent interaction environment.

[0003] Existing multi-agent cooperative schemes (such as reinforcement learning or joint optimization paradigms) typically aim to bring agents closer to a jointly optimal strategy through algorithmic iteration under pre-defined fixed game rules. However, due to the heterogeneous nature of the agents within the system, their state observation space, action control space, and local reward function all exhibit significant asymmetry. This structural conflict of "incommensurable states, non-substitutable actions, and irreconcilable interests" leads to an exponential increase in computational overhead for traditional centralized scheduling algorithms when faced with massive concurrent requests, and the system is highly susceptible to falling into severe Pareto inefficient equilibrium.

[0004] Therefore, there is an urgent need for a low-level collaborative mechanism that can internalize heterogeneous conflicts and achieve game-inducing behavior. Summary of the Invention

[0005] This invention provides a method, controller, and system for collaborative control of heterogeneous subjects based on game-theoretic incentives, which solves the problem of inefficient collaboration among heterogeneous subjects in the prior art due to the asymmetric heterogeneous characteristics of state space, action space, and reward function.

[0006] This invention provides a game-theoretic incentive-based collaborative control method for all heterogeneous entities, applied to a collaborative controller, which is communicatively connected to multiple distributed, heterogeneous entities. The method includes: The system receives first evaluation signals reported by multiple heterogeneous entities and collects the macroscopic environmental state generated by the joint actions of the multiple heterogeneous entities. The multiple heterogeneous entities have asymmetric heterogeneous characteristics, which are manifested in that the state observation space, control action space and local reward function of each entity are different and there are structural conflicts. The first evaluation signal is used to characterize the basic individual reward of the heterogeneous entity after performing local actions based on its private local reward function. Based on the macro-environmental state and the overall system optimization objective, a second evaluation signal is generated; wherein, the second evaluation signal is used to characterize the global collaborative reward for measuring the overall system performance; The system obtains the competition intensity status of public resources and dynamically adjusts the collaborative weight factor based on the competition intensity status; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization; The second evaluation signal and the collaborative weighting factor are distributed to the multiple heterogeneous agents, so that the multiple heterogeneous agents can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal. The reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract a multi-step forward-looking advantage signal. The advantage signal is used as the optimization objective of its own decision-making strategy network to output control actions, so as to guide the agents with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy.

[0007] According to the present invention, a heterogeneous agent collaborative control method based on game-theoretic incentives is provided, wherein generating a second evaluation signal based on the macroscopic environmental state and the overall system optimization objective includes: When the system is under high load, the second evaluation signal is quantified as the total system throughput within a preset time window; In a latency-sensitive scenario, the second evaluation signal is quantified as the success rate of the target task being completed on time. When the system is in an energy-constrained scenario, the second evaluation signal is quantified as the ratio of total system revenue to total energy consumption.

[0008] According to the present invention, a fully heterogeneous agent collaborative control method based on game-theoretic incentives includes, in the step of acquiring the competition intensity state of common resources in the system and dynamically adjusting the collaborative weight factor according to the competition intensity state, the method comprises: If an increase in the intensity of competition for the system's public resources is detected, the value of the collaborative weight factor is lowered to amplify the influence of collective interests on strategy optimization. When the intensity of competition for the system's public resources is detected to decrease, the value of the collaborative weight factor is increased to amplify the influence of individual interests on strategy optimization. The system's public resources include at least one of wireless communication resources, computing power resources, and storage resources.

[0009] According to the present invention, a collaborative control method for all heterogeneous subjects based on game-theoretic incentives is provided. The heterogeneous entity has asymmetric heterogeneous characteristics, and the action space of the heterogeneous entity includes at least one of the execution-side action space and the resource orchestration-side action space. The output of the control action aims to internalize the marginal social cost of resource competition into individual decision-making cost through the game-theoretic guidance of the reconstructed collaborative signal, so that the system spontaneously escapes the suboptimal Nash equilibrium under non-cooperative game.

[0010] According to the present invention, a collaborative control method for all heterogeneous subjects based on game-theoretic incentives is provided. The process of generating the reconstructed collaborative signal includes: multiplying the first evaluation signal by the collaborative weighting factor to obtain a first weighted value; multiplying the second evaluation signal by the difference between 1 and the collaborative weighting factor to obtain a second weighted value; and adding the first weighted value and the second weighted value to obtain the result.

[0011] The present invention also provides a cooperative controller, comprising: a global state aggregation module, an evaluation calculation module, and an incentive injection communication module; The global state aggregation module is used to receive first evaluation signals reported by multiple heterogeneous entities and collect the macroscopic environmental state generated by the joint actions of the multiple heterogeneous entities; wherein, the multiple heterogeneous entities have asymmetric heterogeneous characteristics, which are manifested in that the state observation space, control action space and local reward function of each entity are different and there are structural conflicts; the first evaluation signal is used to characterize the basic individual reward of the heterogeneous entity after performing local actions based on its private local reward function. The evaluation calculation module is used to generate a second evaluation signal based on the macro-environmental state and the overall system optimization objective; wherein, the second evaluation signal is used to characterize the global collaborative reward for measuring the overall system performance; The evaluation and calculation module is also used to obtain the competition intensity state of the system's public resources and dynamically adjust the collaborative weight factor according to the competition intensity state; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization; The incentive injection communication module is used to send the second evaluation signal and the collaborative weighting factor to the multiple heterogeneous entities, so that the multiple heterogeneous entities can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal; wherein, the reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract a multi-step forward-looking advantage signal, and use the advantage signal as the optimization target of its own decision-making strategy network to output control actions, so as to guide each entity with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy; The collaborative controller is used to execute the game-theoretic incentive-based collaborative control method for all heterogeneous agents as described above.

[0012] The present invention also provides a fully heterogeneous entity, characterized in that the fully heterogeneous entity is communicatively connected to the cooperative controller as described above; the fully heterogeneous entity includes: a local evaluation module, a message receiving module, a utility reconstruction calculation module, and a decision execution module; The local evaluation module is used to generate a first evaluation signal after the execution of a local action; The message receiving module is used to receive an incentive message issued by the cooperative controller, which includes a second evaluation signal and a cooperative weight factor. The utility reconstruction calculation module is used to weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal. The decision execution module is used to take the reconstructed collaborative signal as the optimization target of its own decision strategy network and output the underlying control action.

[0013] The present invention also provides a fully heterogeneous agent collaborative control system based on game-theoretic incentives, comprising: at least one collaborative controller as described above, and multiple distributed fully heterogeneous agents as described above. The collaborative controller is deployed in the edge cloud or information space; the fully heterogeneous entity is deployed in the mobile physical space, and the fully heterogeneous entity includes an end-side execution agent deployed in the vehicle and a cloud orchestration agent deployed in the edge cloud.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the game-theoretic incentive-based fully heterogeneous subject collaborative control method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the game-theoretic incentive-based fully heterogeneous agent collaborative control method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the game-theoretic incentive-based collaborative control method for all heterogeneous entities as described above.

[0017] This invention provides a game-theoretic incentive-based collaborative control method, controller, and system for heterogeneous agents. Applied to a collaborative controller, the controller communicates with multiple distributed, heterogeneous agents. It receives first evaluation signals reported by these agents and collects the macroscopic environmental state generated by their joint actions. These heterogeneous agents exhibit asymmetric heterogeneity, meaning their state observation spaces, control action spaces, and local reward functions are all different and structurally conflicting. The first evaluation signal characterizes the basic individual reward of each heterogeneous agent after performing local actions based on its private local reward function. A second evaluation signal is generated based on the macroscopic environmental state and the overall system optimization objective. This second evaluation signal characterizes and measures the overall system performance. The system obtains global collaborative rewards; acquires the competition intensity state of public resources in the system, and dynamically adjusts the collaborative weight factor according to the competition intensity state; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization; the second evaluation signal and the collaborative weight factor are distributed to the multiple heterogeneous subjects, so that the multiple heterogeneous subjects can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weight factor to generate a reconstructed collaborative signal; wherein, the reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract multi-step forward-looking advantage signals, and use the advantage signals as the optimization target of their own decision-making strategy network to output control actions, so as to guide the subjects with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy. Therefore, this invention receives individual reward signals reported by multiple heterogeneous entities and collects macroscopic environmental states through a collaborative controller, thereby generating a global collaborative reward that reflects the overall performance of the system. Simultaneously, it dynamically adjusts the collaborative weighting factor based on the real-time competition intensity of public resources, distributing both to each heterogeneous entity. This allows for internal weighted fusion to generate a reconstructed collaborative signal, which is then used as the basic input for calculating time-series differential errors to extract multi-step forward-looking advantage signals. Finally, these advantage signals are used as the optimization target of the decision-making strategy network, internalizing the overall resource competition cost of the system into individual decision-making considerations. This enables each entity to spontaneously consider collective interests while pursuing its own long-term gains, fundamentally resolving the "tragedy of the commons" problem caused by the conflict between individual rationality and collective interests, and achieving efficient collaboration among multiple heterogeneous entities in resource-constrained environments. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the collaborative control method for all heterogeneous entities based on game-theoretic incentives provided by the present invention.

[0020] Figure 2 This is a network topology diagram of a fully heterogeneous subject collaborative control system based on game-theoretic incentives provided by the present invention.

[0021] Figure 3 This is a communication timing diagram of the game-theoretic incentive-based collaborative control method for all heterogeneous subjects provided by the present invention.

[0022] Figure 4 This is the logic diagram for the reconstruction and weighted calculation of the internal utility of a fully heterogeneous subject provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the structure of the collaborative controller provided by the present invention.

[0024] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] The following description, in conjunction with the accompanying drawings, describes a collaborative control method, controller, and system for heterogeneous entities based on game-theoretic incentives according to the present invention.

[0027] It should be noted that with the rapid development of Cyber-Physical Systems (CPS), the Internet of Things (IoT), and embodied intelligence, systems typically consist of a massive number of distributed mobile heterogeneous agents (such as intelligent vehicles, drones, and mobile robots) and fixed heterogeneous agents (such as roadside units, edge clouds, regional clouds, and central clouds). All types of agents deploy intelligent agents and participate in system interaction. To achieve efficient system operation, these agents need to synchronize state data via wireless networks and compete for limited wireless communication and edge computing resources, thus forming a typical fully heterogeneous multi-agent interactive environment.

[0028] Related technologies typically employ Multi-Agent Reinforcement Learning (MARL) or Joint Optimization paradigms. Specifically, existing mainstream collaborative solutions generally aim to train agents to play an existing game better under given, fixed game rules or constraints, using complex algorithmic models (such as joint convex optimization and reinforcement learning). For example, in vehicle-to-cloud collaborative scenarios, existing technologies design a global reward function or joint objective function, attempting to enable the data synchronization module and the computation orchestration module to jointly find the optimal strategy for bandwidth allocation and computational offloading through extensive iterative learning.

[0029] Despite the progress made by joint optimization and learning-based collaboration in homogeneous systems, the following fundamental scientific and engineering shortcomings remain when dealing with resource-constrained interactions between entirely heterogeneous agents: 1. Pareto Inefficiency Due to Structural Goal Conflicts: In a fully heterogeneous system, the local optima of a subsystem do not necessarily lead to the global optima of the system. For example, the edge execution agent (DSA) inherently tends to reduce transmission frequency to conserve communication resources, while the cloud orchestration agent (COA) craves high-frequency data to maintain decision-making accuracy. Existing joint optimization algorithms struggle to resolve these inherent structural conflicts in objectives at the underlying level, causing agents to easily engage in mutual exclusion and vicious competition when resources are limited, falling into the "1+1<2" internal friction trap.

[0030] 2. Nash Equilibrium, where the "Tragedy of the Commons" is difficult to escape under fixed rules: Existing multi-agent collaboration mainly focuses on "policy optimization" rather than "rule reshaping." When the system faces limited edge computing power and wireless channels (i.e., the "commons"), individual rationality-driven policy choices will generate severe negative externalities. In this cyber-physical system, "negative externalities" are specifically manifested as follows: an agent, in pursuit of maximizing its own local interests, causes unmeasured and unpunished physical damage to other agents sharing system resources. For example, an edge execution agent, in order to ensure the integrity of its own sensed data, chooses to blindly increase the data transmission frequency; although this operation based on individual rationality increases the revenue of a single vehicle, it excessively occupies the shared wireless communication bandwidth and cloud queuing, causing critical data of other terminals to be delayed or discarded. Since existing algorithms lack effective mechanisms to constrain this behavior, agents do not need to pay for the system-level congestion costs (marginal social costs) they generate. This leads to a significant deviation between the optimal solution of individual rationality and the optimal solution of total social benefit. The system inevitably and spontaneously falls into a Pareto-inefficient stable state (i.e., "coordination failure"), resulting in a collective suboptimal trap caused by individual rationality.

[0031] 3. The failure of "Credit Assignment" caused by global reward sharing: In order to promote collaboration, some technologies adopt a fully shared global reward. However, when the action spaces of heterogeneous subjects are unequal, this "iron rice bowl" reward mechanism will cause subjects to be unable to accurately perceive the true contribution of their own actions to the overall situation, thus causing a serious "free-rider" problem, making it difficult for collaborative strategies to converge stably.

[0032] Based on this, the present invention provides a collaborative control method for all heterogeneous subjects based on game-theoretic incentives, which can solve at least one of the above problems.

[0033] Figure 1 This is a flowchart illustrating the heterogeneous agent collaborative control method based on game-theoretic incentives provided by the present invention, as shown below. Figure 1 As shown, this method is applied to a collaborative controller that communicates with multiple distributed, heterogeneous entities; it includes the following steps: Step 100: Receive the first evaluation signal reported by multiple heterogeneous entities, and collect the macroscopic environmental state generated by the joint actions of the multiple heterogeneous entities; wherein, the multiple heterogeneous entities have asymmetric heterogeneous characteristics, which are manifested in that the state observation space, control action space and local reward function of each entity are different and there is structural conflict; the first evaluation signal is used to characterize the basic individual reward of the heterogeneous entity after performing local actions based on the private local reward function.

[0034] Figure 2 This is a network topology diagram of the heterogeneous agent collaborative control system based on game-theoretic incentives provided by the present invention, such as... Figure 2 As shown, in a cyber-physical system, independently decision-making, heterogeneous entities are deployed, including an end-side execution agent configured on the vehicle (for optimizing actions within the execution-side action space) and a cloud orchestration agent configured at the edge cloud (for scheduling resources within the resource orchestration-side action space). The end-side execution agent is deployed on the mobile heterogeneous entity to perform data synchronization actions; the cloud orchestration agent is deployed on the edge computing node to perform computation orchestration actions.

[0035] It should be noted that, Figure 2 The specific implementation shown is only a typical application example of the present invention, which is illustrated by taking the end-side execution agent and cloud orchestration agent in the vehicle-cloud collaboration scenario as an example. However, the scope of protection of the present invention is not limited to this, and it can be applied to any multi-physical entity collaborative control scenario with asymmetric heterogeneous characteristics.

[0036] The direct resource consumption or local task benefits of heterogeneous entities when performing local actions (such as DSA adjusting data synchronization frequency or COA allocating computing resources) generate the first evaluation signal (local individual reward). R i At this point, the heterogeneous entities are in an information silo state and cannot perceive the negative externalities of their own behavior on the system.

[0037] Step 200: Generate a second evaluation signal based on the macro-environmental state and the overall system optimization objective; wherein the second evaluation signal is used to characterize the global collaborative reward for measuring the overall performance of the system.

[0038] Specifically, a centralized collaborative controller is set up to receive in real time the objective macroscopic results (i.e., macroscopic environmental state) generated by the joint actions of all heterogeneous entities. Based on the overall optimization objective of the system, the collaborative controller dynamically extracts system-level performance indicators and quantifies them into a second evaluation signal (global collaborative reward R). global ).

[0039] Step 200 specifically includes: Step 210: When the system is under high load, quantify the second evaluation signal into the total system throughput within a preset time window.

[0040] Step 220: When the system is in a latency-sensitive scenario, the second evaluation signal is quantified as the success rate of the target task being completed on time.

[0041] Step 230: When the system is in an energy-constrained scenario, the second evaluation signal is quantified as the ratio of total system revenue to total energy consumption.

[0042] The specific quantification rules are dynamically switched according to the engineering scenario: 1. In high-load, service-driven scenarios, count the total number of all system tasks successfully completed within a preset time window, and then use R... global Defined as the total system throughput.

[0043] 2. For latency-sensitive or high-priority task scenarios, calculate the percentage of tasks marked as critical and completed on time, and assign R... global Defined as the success rate of critical missions.

[0044] 3. In energy-constrained or green computing scenarios, calculate the ratio of the total value created by completing the task to the total energy consumption of the system, and then use R... global Defined as total system energy efficiency.

[0045] Step 300: Obtain the competition intensity status of the system's public resources, and dynamically adjust the collaborative weight factor according to the competition intensity status; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization.

[0046] Step 300 specifically includes: Step 310: If the intensity of competition for the system's public resources is detected to increase, the value of the collaborative weight factor is lowered to amplify the influence of collective interests on strategy optimization.

[0047] Step 320: If the intensity of competition for the system's public resources decreases, the value of the collaborative weight factor is increased to amplify the influence of individual interests on strategy optimization. The system's public resources include at least one of wireless communication resources, computing power resources, and storage resources.

[0048] Specifically, the collaborative controller monitors system public resources in real time, such as the occupancy rate of wireless communication channel bandwidth and the queuing length of edge node CPUs. Based on the incentive alignment principle of game theory, a collaborative weighting factor ω between [0, 1] is generated by the computing unit. When resource competition intensifies and causes a drop in system throughput, the value of ω is lowered to amplify the proportion of collective interests; when resources are sufficient, the value of ω is raised to ensure the execution quality of individual tasks.

[0049] Step 400: The second evaluation signal and the collaborative weighting factor are sent to the multiple heterogeneous entities, so that the multiple heterogeneous entities can perform weighted fusion of the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal; wherein, the reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract the multi-step forward-looking advantage signal, and use the advantage signal as the optimization target of its own decision-making strategy network to output control actions, so as to guide each entity with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy.

[0050] In a specific embodiment: Suppose that in a vehicle-road cooperative scenario, due to morning rush hour congestion, the competition intensity of edge cloud computing power surges (the competition intensity state is high). Based on the game-theoretic incentive-based fully heterogeneous agent cooperative control method provided by this invention, at this time: 1. Controller Action: The collaborative controller automatically adjusts the collaborative weighting factor. ω The rate was lowered from 0.8 to 0.2 (to suppress individual self-interest).

[0051] 2. Signal Fusion: A vehicle originally planned to upload non-critical video at a frequency of 30Hz (its individual reward) Ri (High), but ω Under the constraint of 0.2, the synergistic reward after fusion Ri ′ Decrease.

[0052] 3. Spontaneous Adjustment: The vehicle's strategy network aims to achieve... Ri To maximize, it automatically reduces the upload frequency to 5Hz (although) Ri Lower, but Rglobal The weighted returns from the improvement are higher.

[0053] 4. Emergent Results: Global channel congestion mitigation, improved success rate of critical tasks ( Rglobal The system rebounded, achieving a Pareto improvement.

[0054] Figure 3 This is a communication timing diagram of the heterogeneous agent collaborative control method based on game-theoretic incentives provided by this invention, specifically the signal path and communication interaction timing diagram of Collaborative Value Injection (SVI). Figure 3 As shown, the cooperative controller will use the second evaluation signal (R) global As a "messenger" of collective value, it is encapsulated along with the collaborative weighting factor ω into an incentive message. This incentive message is then injected and distributed indiscriminately to all participants (DSA and COA) in the system via a broadcast downlink control channel.

[0055] It should be noted that, Figure 3The specific interaction process shown is only a typical application example of the present invention. The communication timing of collaborative value injection is shown by taking the end-side execution agent and the cloud orchestration agent as an example. However, the scope of protection of the present invention is not limited to this. It can be applied to any multi-physical entity collaborative control scenario with asymmetric heterogeneous characteristics.

[0056] It should be noted that the process of generating the reconstructed collaborative signal includes: multiplying the first evaluation signal by the collaborative weighting factor to obtain a first weighted value; multiplying the second evaluation signal by the difference obtained by subtracting the collaborative weighting factor from 1 to obtain a second weighted value; and adding the first weighted value and the second weighted value to obtain the result.

[0057] Figure 4 This invention provides a logic diagram for the reconstruction and weighted calculation of the internal utility of a fully heterogeneous subject, such as... Figure 4 As shown, after receiving the injected message, each heterogeneous entity's internal value assessment module performs signal fusion calculation. Specifically, the process involves using a collaborative weighting factor ω to perform signal fusion calculation on the first evaluation signal (R) representing individual rationality. i , Figure 4 The blue input pathway in the image represents the second evaluation signal (R) representing collective value. global , Figure 4 The red input path in the signal is weighted and summed to output a new, single scalar signal, namely the cooperative reward (R). ’ i Its internal calculation logic is as follows: R ’ i =ω·R i +(1-ω)·R global It should be noted that, due to R i With R global Since the physical units may differ (e.g., throughput in Mbps, success rate in percentage), the system performs normalization on both before weighted fusion, mapping them to the same dimensionless interval [0, 1]. For example, Min-Max scaling or Z-Score normalization can be used to ensure that both are numerically on the same order of magnitude, thus guaranteeing the collaborative weighting factor. ω The adjustment possesses genuine mathematical sensitivity.

[0058] The restructured collaborative reward (R) ’ i This will replace the original local rewards and be passed on as the final utility to the underlying decision-making core (Actor network) of the heterogeneous subjects.

[0059] Each heterogeneous entity generates a collaborative reward (R) based on the aforementioned fusion. ’ i The system performs online updates of strategies and outputs actions. The output of control actions aims to internalize the marginal social cost of resource competition into individual decision-making costs by reconstructing the game-theoretic guidance of collaborative signals, enabling the system to spontaneously escape the suboptimal Nash equilibrium under non-cooperative game dynamics. Because this fused signal is forcibly associated with global variables affecting the overall system performance at the underlying level, heterogeneous entities, in pursuing their own cumulative returns, will naturally incorporate macro-level considerations such as alleviating system congestion and cooperating with heterogeneous nodes into their decision-making. Thus, without high-frequency point-to-point communication negotiation, it spontaneously generates optimal collaborative control instructions to avoid resource conflicts, resolving the resource mismatch problem of "having data but lacking power" or "having power but lacking resources" in vehicle-cloud interaction.

[0060] In this embodiment of the invention, "incentive injection" and "signal reconstruction" serve as the core mechanisms of the collaborative control method, together forming a complete value transmission loop from macro-system objectives to micro-individual decisions. First, through the "incentive injection" mechanism, the collaborative controller encapsulates the second evaluation signal (i.e., global collaborative reward) generated based on global state information and the dynamically calculated collaborative weight factor into an incentive message, which is then broadcast to every heterogeneous entity through a broadcast channel. This process essentially establishes a "collective perception" signal path at the underlying level, enabling entities that were originally isolated in information silos to perceive the overall operational status and resource competition intensity of the system in real time, transforming the abstract "system effectiveness" into a concrete, value-oriented signal that can be received by individuals. Subsequently, under the "signal reconstruction" mechanism, each subject internally weights and merges the first evaluation signal representing individual rationality with the second evaluation signal representing collective value according to the collaborative weight factor to generate a reconstructed collaborative signal. This process is not a simple numerical superposition, but a fundamental reshaping of the subject's utility function. By internalizing the marginal social cost of global resource competition into the cost factor of individual decision-making, the subject's behavior logic naturally incorporates macro-level considerations such as alleviating system congestion, ensuring key tasks, and cooperating with heterogeneous nodes into the decision-making scope in the process of pursuing its own long-term accumulated benefits. The beneficial effects of this dual "injection-reconstruction" mechanism are significant: On the one hand, it completely breaks down the structural conflict between individual and collective goals that is difficult to reconcile in traditional methods, internalizing the negative externalities originally caused by competition for public resources, thus fundamentally avoiding the "tragedy of the commons"; on the other hand, it endows the system with strong dynamic adaptability, with the collaborative weight factor adjusting in real time according to the intensity of resource competition, enabling the system to smoothly switch between "individual efficiency priority" and "collective collaboration priority" modes without interrupting operation or retraining the model, greatly improving the system's robustness in changing environments; more importantly, this mechanism transforms the multi-agent collaborative process from the traditional "black box optimization" to a "gray box control" with high physical interpretability. System managers can clearly trace the motivation behind each collaborative behavior, and every instance of an agent actively reducing frequency, yielding, or cooperating can be attributed to the specific global performance feedback and weight configuration issued at the current moment. This interpretability provides key support for the implementation of technology in scenarios with high safety requirements (such as autonomous driving and the industrial internet). Ultimately, through the synergistic effect of "incentive injection" and "signal reconstruction," the system achieves a fundamental shift from "individual rationality leading to collective irrationality" to "individual rationality naturally converging to collective optimality," enabling a massive number of heterogeneous entities to spontaneously form an efficient, stable, and predictable collaborative order in a resource-constrained environment.

[0061] The above describes the steps of the game-theoretic incentive-based fully heterogeneous agent collaborative control method provided by this invention. As can be seen from the above description, the game-theoretic incentive-based fully heterogeneous agent collaborative control method provided by this invention is applied to a collaborative controller. The collaborative controller is communicatively connected to multiple distributed, fully heterogeneous agents. It receives first evaluation signals reported by the multiple fully heterogeneous agents and collects the macroscopic environmental state generated by the joint actions of the multiple fully heterogeneous agents. The multiple fully heterogeneous agents have asymmetric heterogeneous characteristics, meaning that their state observation spaces, control action spaces, and local reward functions are all different and there are structural conflicts. The first evaluation signal is used to characterize the basic individual reward of the fully heterogeneous agent after performing local actions based on its private local reward function. A second evaluation signal is generated based on the macroscopic environmental state and the overall system optimization objective. The second evaluation signal is used to characterize and measure the overall system optimization objective. The system employs a global collaborative reward mechanism for overall performance; it acquires the competition intensity state of common system resources and dynamically adjusts the collaborative weighting factor based on the competition intensity state; wherein the collaborative weighting factor is used to balance the importance of individual and collective interests in strategy optimization; the second evaluation signal and the collaborative weighting factor are distributed to the multiple heterogeneous entities, so that the multiple heterogeneous entities can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal; wherein the reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract multi-step forward-looking advantage signals, and use the advantage signals as the optimization target of their own decision-making strategy network to output control actions, so as to guide the entities with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy. Therefore, this invention receives individual reward signals reported by heterogeneous entities and collects macro-environmental status through a collaborative controller, thereby generating a global collaborative reward reflecting the overall performance of the system. Simultaneously, it dynamically adjusts the collaborative weighting factor based on the real-time competition intensity of public resources, distributing both to each heterogeneous entity. This allows for internal weighted fusion to generate a reconstructed collaborative signal, which is then used as the basic input for calculating time-series differential errors to extract multi-step forward-looking advantage signals. Finally, these advantage signals serve as the optimization target of the decision-making strategy network, internalizing the overall resource competition cost of the system into individual decision-making considerations. This enables each entity to spontaneously consider collective interests while pursuing its own long-term gains, fundamentally resolving the "tragedy of the commons" problem caused by the conflict between individual rationality and collective interests, and achieving efficient collaboration among multiple heterogeneous entities in resource-constrained environments.

[0062] The following describes the cooperative controller provided by the present invention. The cooperative controller described below can be referred to in correspondence with the game-theoretic incentive-based fully heterogeneous subject cooperative control method described above.

[0063] Figure 5This is a schematic diagram of the structure of the collaborative controller provided by the present invention, as shown below. Figure 5 As shown, the cooperative controller provided by the present invention includes: a global state aggregation module 501, an evaluation calculation module 502, and an incentive injection communication module 503; The global state aggregation module 501 is used to receive first evaluation signals reported by multiple heterogeneous entities and collect the macroscopic environmental state generated by the joint actions of the multiple heterogeneous entities; wherein, the multiple heterogeneous entities have asymmetric heterogeneous characteristics, which are manifested in that the state observation space, control action space and local reward function of each entity are different and there are structural conflicts; the first evaluation signal is used to characterize the basic individual reward of the heterogeneous entity after performing local actions based on its private local reward function. The evaluation calculation module 502 is used to generate a second evaluation signal based on the macro-environmental state and the overall system optimization objective; wherein, the second evaluation signal is used to characterize the global collaborative reward for measuring the overall system performance; The evaluation and calculation module 502 is also used to obtain the competition intensity state of the system's public resources and dynamically adjust the collaborative weight factor according to the competition intensity state; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization; The incentive injection communication module 503 is used to send the second evaluation signal and the collaborative weighting factor to the multiple heterogeneous subjects, so that the multiple heterogeneous subjects can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal; wherein, the reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract a multi-step forward-looking advantage signal, and use the advantage signal as the optimization target of its own decision-making strategy network to output control actions, so as to guide each subject with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy; The collaborative controller is used to execute the game-theoretic incentive-based collaborative control method for all heterogeneous agents as described above.

[0064] The cooperative controller provided by this invention includes: a global state aggregation module, an evaluation calculation module, and an incentive injection communication module. The global state aggregation module receives first evaluation signals reported by multiple heterogeneous entities and collects the macroscopic environmental state generated by the joint actions of these entities. These heterogeneous entities exhibit asymmetric heterogeneity, meaning their state observation spaces, control action spaces, and local reward functions are all different and structurally conflicting. The first evaluation signal characterizes the basic individual reward of each heterogeneous entity after performing local actions based on its private local reward function. The evaluation calculation module generates a second evaluation signal based on the macroscopic environmental state and the overall system optimization objective. The second evaluation signal characterizes the global cooperative reward used to measure the overall system performance. The module is also used to obtain the competition intensity state of the system's public resources and dynamically adjust the collaborative weighting factor according to the competition intensity state; wherein, the collaborative weighting factor is used to balance the importance of individual interests and collective interests in strategy optimization; the incentive injection communication module is used to send the second evaluation signal and the collaborative weighting factor to the multiple heterogeneous subjects, so that the multiple heterogeneous subjects can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal; wherein, the reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract multi-step forward-looking advantage signals, and use the advantage signals as the optimization target of their own decision-making strategy network to output control actions, so as to guide the subjects with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy. Therefore, this invention receives individual reward signals reported by heterogeneous entities and collects macro-environmental status through a collaborative controller, thereby generating a global collaborative reward reflecting the overall performance of the system. Simultaneously, it dynamically adjusts the collaborative weighting factor based on the real-time competition intensity of public resources, distributing both to each heterogeneous entity. This allows for internal weighted fusion to generate a reconstructed collaborative signal, which is then used as the basic input for calculating time-series differential errors to extract multi-step forward-looking advantage signals. Finally, these advantage signals serve as the optimization target of the decision-making strategy network, internalizing the overall resource competition cost of the system into individual decision-making considerations. This enables each entity to spontaneously consider collective interests while pursuing its own long-term gains, fundamentally resolving the "tragedy of the commons" problem caused by the conflict between individual rationality and collective interests, and achieving efficient collaboration among multiple heterogeneous entities in resource-constrained environments.

[0065] Based on the above embodiments, in this embodiment, the evaluation calculation module 502 is specifically used for: When the system is under high load, the second evaluation signal is quantified as the total system throughput within a preset time window; In a latency-sensitive scenario, the second evaluation signal is quantified as the success rate of the target task being completed on time. When the system is in an energy-constrained scenario, the second evaluation signal is quantified as the ratio of total system revenue to total energy consumption.

[0066] Based on the above embodiments, in this embodiment, the evaluation calculation module 502 is further used for: If an increase in the intensity of competition for the system's public resources is detected, the value of the collaborative weight factor is lowered to amplify the influence of collective interests on strategy optimization. When the intensity of competition for the system's public resources is detected to decrease, the value of the collaborative weight factor is increased to amplify the influence of individual interests on strategy optimization. The system's public resources include at least one of wireless communication resources, computing power resources, and storage resources.

[0067] Based on the above embodiments, in this embodiment... The heterogeneous entity possesses asymmetric heterogeneous characteristics, and the action space of the heterogeneous entity includes at least one of the execution-side action space and the resource orchestration-side action space. The output of the control action aims to internalize the marginal social cost of resource competition into individual decision-making cost through the game-theoretic guidance of the reconstructed collaborative signal, so that the system spontaneously escapes the suboptimal Nash equilibrium under non-cooperative game.

[0068] Based on the above embodiments, in this embodiment... The process of generating the reconstructed collaborative signal includes: multiplying the first evaluation signal by the collaborative weighting factor to obtain a first weighted value; multiplying the second evaluation signal by the difference between 1 and the collaborative weighting factor to obtain a second weighted value; and adding the first weighted value and the second weighted value to obtain the result.

[0069] The fully heterogeneous entity provided by this invention is described below.

[0070] The heterogeneous entity provided by this invention is communicatively connected to the aforementioned cooperative controller; the heterogeneous entity includes: a local evaluation module, a message receiving module, a utility reconstruction calculation module, and a decision execution module; The local evaluation module is used to generate a first evaluation signal after the execution of a local action; The message receiving module is used to receive an incentive message issued by the cooperative controller, which includes a second evaluation signal and a cooperative weight factor. The utility reconstruction calculation module is used to weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal. The decision execution module is used to take the reconstructed collaborative signal as the optimization target of its own decision strategy network and output the underlying control action.

[0071] The following describes the fully heterogeneous subject collaborative control system based on game-theoretic incentives provided by this invention.

[0072] See also Figure 2 The present invention provides a game-theoretic incentive-based fully heterogeneous agent collaborative control system, comprising: at least one collaborative controller as described above, and multiple distributed fully heterogeneous agents as described above. The collaborative controller is deployed in the edge cloud or information space; the fully heterogeneous entity is deployed in the mobile physical space, and the fully heterogeneous entity includes an end-side execution agent deployed in the vehicle and a cloud orchestration agent deployed in the edge cloud.

[0073] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device can be a roadside computing unit (RSU), an onboard terminal of an autonomous vehicle, an industrial control host computer, an edge server cluster, or other electronic devices with computing capabilities. This electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions from the memory 630 to execute a game-theoretic incentive-based heterogeneous agent cooperative control method, applied to a cooperative controller. The cooperative controller is communicatively connected to multiple distributed, heterogeneous agents. The method includes: The system receives first evaluation signals reported by multiple heterogeneous entities and collects the macroscopic environmental state generated by the joint actions of the multiple heterogeneous entities. The multiple heterogeneous entities have asymmetric heterogeneous characteristics, which are manifested in that the state observation space, control action space and local reward function of each entity are different and there are structural conflicts. The first evaluation signal is used to characterize the basic individual reward of the heterogeneous entity after performing local actions based on its private local reward function. Based on the macro-environmental state and the overall system optimization objective, a second evaluation signal is generated; wherein, the second evaluation signal is used to characterize the global collaborative reward for measuring the overall system performance; The system obtains the competition intensity status of public resources and dynamically adjusts the collaborative weight factor based on the competition intensity status; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization; The second evaluation signal and the collaborative weighting factor are distributed to the multiple heterogeneous agents, so that the multiple heterogeneous agents can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal. The reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract a multi-step forward-looking advantage signal. The advantage signal is used as the optimization objective of its own decision-making strategy network to output control actions, so as to guide the agents with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy.

[0074] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the game-theoretic incentive-based fully heterogeneous agent cooperative control method provided by the above methods, applied to a cooperative controller, the cooperative controller being communicatively connected to multiple distributed fully heterogeneous agents; The method includes: The system receives first evaluation signals reported by multiple heterogeneous entities and collects the macroscopic environmental state generated by the joint actions of the multiple heterogeneous entities. The multiple heterogeneous entities have asymmetric heterogeneous characteristics, which are manifested in that the state observation space, control action space and local reward function of each entity are different and there are structural conflicts. The first evaluation signal is used to characterize the basic individual reward of the heterogeneous entity after performing local actions based on its private local reward function. Based on the macro-environmental state and the overall system optimization objective, a second evaluation signal is generated; wherein, the second evaluation signal is used to characterize the global collaborative reward for measuring the overall system performance; The system obtains the competition intensity status of public resources and dynamically adjusts the collaborative weight factor based on the competition intensity status; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization; The second evaluation signal and the collaborative weighting factor are distributed to the multiple heterogeneous agents, so that the multiple heterogeneous agents can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal. The reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract a multi-step forward-looking advantage signal. The advantage signal is used as the optimization objective of its own decision-making strategy network to output control actions, so as to guide the agents with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy.

[0076] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the game-incentive-based fully heterogeneous agent cooperative control method provided by the above methods, and is applied to a cooperative controller, the cooperative controller being communicatively connected to multiple distributed fully heterogeneous agents. The method includes: The system receives first evaluation signals reported by multiple heterogeneous entities and collects the macroscopic environmental state generated by the joint actions of the multiple heterogeneous entities. The multiple heterogeneous entities have asymmetric heterogeneous characteristics, which are manifested in that the state observation space, control action space and local reward function of each entity are different and there are structural conflicts. The first evaluation signal is used to characterize the basic individual reward of the heterogeneous entity after performing local actions based on its private local reward function. Based on the macro-environmental state and the overall system optimization objective, a second evaluation signal is generated; wherein, the second evaluation signal is used to characterize the global collaborative reward for measuring the overall system performance; The system obtains the competition intensity status of public resources and dynamically adjusts the collaborative weight factor based on the competition intensity status; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization; The second evaluation signal and the collaborative weighting factor are distributed to the multiple heterogeneous agents, so that the multiple heterogeneous agents can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal. The reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract a multi-step forward-looking advantage signal. The advantage signal is used as the optimization objective of its own decision-making strategy network to output control actions, so as to guide the agents with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy.

[0077] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0078] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A collaborative control method for all heterogeneous agents based on game-theoretic incentives, characterized in that, It is applied to a collaborative controller, which communicates with multiple distributed, heterogeneous entities; The method includes: The system receives first evaluation signals reported by multiple heterogeneous entities and collects the macroscopic environmental state generated by the joint actions of the multiple heterogeneous entities. The multiple heterogeneous entities have asymmetric heterogeneous characteristics, which are manifested in that the state observation space, control action space and local reward function of each entity are different and there are structural conflicts. The first evaluation signal is used to characterize the basic individual reward of the heterogeneous entity after performing local actions based on its private local reward function. Based on the macro-environmental state and the overall system optimization objective, a second evaluation signal is generated; wherein, the second evaluation signal is used to characterize the global collaborative reward for measuring the overall system performance; The system obtains the competition intensity status of public resources and dynamically adjusts the collaborative weight factor based on the competition intensity status; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization; The second evaluation signal and the collaborative weighting factor are distributed to the multiple heterogeneous agents, so that the multiple heterogeneous agents can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal. The reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract a multi-step forward-looking advantage signal. The advantage signal is used as the optimization objective of its own decision-making strategy network to output control actions, so as to guide the agents with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy.

2. The method for collaborative control of all heterogeneous agents based on game-theoretic incentives according to claim 1, characterized in that, The step of generating a second evaluation signal based on the macro-environmental state and the overall system optimization objective includes: When the system is under high load, the second evaluation signal is quantified as the total system throughput within a preset time window; In a latency-sensitive scenario, the second evaluation signal is quantified as the success rate of the target task being completed on time. When the system is in an energy-constrained scenario, the second evaluation signal is quantified as the ratio of total system revenue to total energy consumption.

3. The method for collaborative control of all heterogeneous agents based on game-theoretic incentives according to claim 1, characterized in that, The process of acquiring the competition intensity state of public resources in the system and dynamically adjusting the collaborative weighting factor based on the competition intensity state includes: If an increase in the intensity of competition for the system's public resources is detected, the value of the collaborative weight factor is lowered to amplify the influence of collective interests on strategy optimization. When the intensity of competition for the system's public resources is detected to decrease, the value of the collaborative weight factor is increased to amplify the influence of individual interests on strategy optimization. The system's public resources include at least one of wireless communication resources, computing power resources, and storage resources.

4. The method for collaborative control of all heterogeneous agents based on game-theoretic incentives according to claim 1, characterized in that, The action space of the heterogeneous entity includes at least one of the execution-side action space and the resource orchestration-side action space; The output of the control action aims to internalize the marginal social cost of resource competition into individual decision-making cost through the game-theoretic guidance of the reconstructed collaborative signal, so that the system spontaneously escapes the suboptimal Nash equilibrium under non-cooperative game.

5. The method for collaborative control of all heterogeneous agents based on game-theoretic incentives according to claim 1, characterized in that, The process of generating the reconstructed collaborative signal includes: multiplying the first evaluation signal by the collaborative weighting factor to obtain a first weighted value; multiplying the second evaluation signal by the difference between 1 and the collaborative weighting factor to obtain a second weighted value; and adding the first weighted value and the second weighted value to obtain the result.

6. A cooperative controller, characterized in that, include: The module consists of a global state aggregation module, an evaluation and calculation module, and an incentive injection and communication module. The global state aggregation module is used to receive first evaluation signals reported by multiple heterogeneous entities and collect the macroscopic environmental state generated by the joint actions of the multiple heterogeneous entities; wherein, the multiple heterogeneous entities have asymmetric heterogeneous characteristics, which are manifested in that the state observation space, control action space and local reward function of each entity are different and there are structural conflicts; the first evaluation signal is used to characterize the basic individual reward of the heterogeneous entity after performing local actions based on its private local reward function. The evaluation calculation module is used to generate a second evaluation signal based on the macro-environmental state and the overall system optimization objective; wherein, the second evaluation signal is used to characterize the global collaborative reward for measuring the overall system performance; The evaluation and calculation module is also used to obtain the competition intensity state of the system's public resources and dynamically adjust the collaborative weight factor according to the competition intensity state; wherein, the collaborative weight factor is used to balance the importance of individual interests and collective interests in strategy optimization; The incentive injection communication module is used to send the second evaluation signal and the collaborative weighting factor to the multiple heterogeneous entities, so that the multiple heterogeneous entities can weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal; wherein, the reconstructed collaborative signal serves as the basic input for calculating the time-series difference error, and is then used to extract a multi-step forward-looking advantage signal, and use the advantage signal as the optimization target of its own decision-making strategy network to output control actions, so as to guide each entity with asymmetric heterogeneous characteristics to spontaneously converge to the global Pareto optimal strategy; The collaborative controller is used to execute the game-theoretic incentive-based collaborative control method for all heterogeneous entities as described in any one of claims 1 to 5.

7. A completely heterogeneous entity, characterized in that, The heterogeneous entity is communicatively connected to the cooperative controller as described in claim 6; the heterogeneous entity includes: a local evaluation module, a message receiving module, a utility reconstruction calculation module, and a decision execution module; The local evaluation module is used to generate a first evaluation signal after the execution of a local action; The message receiving module is used to receive an incentive message issued by the cooperative controller, which includes a second evaluation signal and a cooperative weight factor. The utility reconstruction calculation module is used to weight and fuse the first evaluation signal and the second evaluation signal according to the collaborative weighting factor to generate a reconstructed collaborative signal. The decision execution module is used to take the reconstructed collaborative signal as the optimization target of its own decision strategy network and output the underlying control action.

8. A fully heterogeneous agent collaborative control system based on game-theoretic incentives, characterized in that, include: At least one collaborative controller as described in claim 6, and multiple distributed, fully heterogeneous entities as described in claim 7; The collaborative controller is deployed in the edge cloud or information space; the fully heterogeneous entity is deployed in the mobile physical space, and the fully heterogeneous entity includes an end-side execution agent deployed in the vehicle and a cloud orchestration agent deployed in the edge cloud.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the game-theoretic incentive-based collaborative control method for all heterogeneous entities as described in any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the game-theoretic incentive-based collaborative control method for all heterogeneous entities as described in any one of claims 1 to 5.

Images