Brain-like structure driven multi-agent hierarchical decision-making method and system

Abstract

This invention provides a brain-like structure-driven multi-agent hierarchical decision-making method and system, relating to the field of artificial intelligence technology. The system comprises multiple single agents. Within each single agent, a multi-source perception module processes multimodal real-time observation data. A memory module dynamically updates the memory feature vector based on the environmental feature vector and the memory state of the previous time step. An analysis and reasoning module generates the target posterior probability and decision risk vector, thereby generating a contextual understanding vector. A communication mechanism based on decision risk or contextual changes is introduced to trigger communication with neighboring agents. By fusing the individual's contextual understanding vector with global collaborative information from neighboring agents, a fused contextual understanding vector is formed. Through modular perception, memory, analysis and reasoning, and context-aware adaptive communication and collaborative decision-making mechanisms, the system provides stronger autonomous decision-making capabilities, contextual understanding capabilities, and team collaboration efficiency in complex, dynamic, and communication-constrained scenarios.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a brain-like structure-driven multi-agent hierarchical decision-making method and system. Background Technology

[0002] The core structure of existing intelligent agents is a single-path serial structure without independent module division. The structure of the decision-making unit remains simple, lacking explicit memory management, contextual understanding, or hierarchical planning capabilities. It also lacks the simulation of the hierarchical and modular information processing mechanisms crucial in biological intelligence, limiting the agent's autonomous decision-making ability in complex scenarios that are fully distributed, communication-constrained, or require advanced cognition (such as experience-based planning and contextual understanding). Therefore, although this structure is effective in simulated environments, in real-world environments, environmental information often has high dimensionality, high dynamism, and multimodal characteristics. Decision-making objectives are complex and requirements are diverse, making it difficult to simultaneously meet the requirements of real-time response, long-term dependency, and cross-task generalization using only a single flat model. Summary of the Invention

[0003] The problem that this invention aims to solve is that the core overall structure of existing intelligent agents is a single-path serial structure, which makes it difficult to simultaneously meet the requirements of real-time response, long-term dependency, and cross-task generalization.

[0004] To address the aforementioned problems, in a first aspect, the present invention provides a brain-like structure-driven multi-agent hierarchical decision-making system, comprising multiple single agents, each single agent comprising a multi-source perception module, a memory module, an analysis and reasoning module, and a decision execution module; Within the current single intelligent entity, the multi-source perception module is used to extract environmental feature vectors based on multimodal real-time observation data acquired from the environment; the memory module is used to obtain updated memory feature vectors based on the environmental feature vectors and the memory state of the previous moment; and the analysis and reasoning module is used to generate target posterior probability and decision risk vectors based on the environmental feature vectors and memory feature vectors, and to generate context understanding vectors by combining the target posterior probability and decision risk vectors. When the decision risk vector is greater than the decision risk threshold or the change in the context understanding vector at adjacent time steps is greater than the change amount threshold, communication between the current single agent and its neighboring agents is triggered. The state representation vectors of multiple neighboring agents within the communication neighbor set of the current single agent are weighted and fused to obtain global collaborative information. The context understanding vector generated by the agent's own analysis and reasoning module is fused with the global collaborative information to obtain a fused context understanding vector. The state representation vector includes environmental feature vectors, environmental understanding vectors, and action instructions. The decision-making and execution module in a single intelligent body is used to generate action instructions based on the understanding of vectors in the fused context.

[0005] Optionally, the step of extracting the environmental feature vector based on multimodal real-time observation data obtained from the environment includes: Image features are obtained by performing convolution and global average pooling on the image data; After the point cloud data is encoded in terms of location, it is then processed by a multilayer perceptron and global max pooling to obtain the point cloud features. Image features and point cloud features are weighted and fused to obtain fused features, and then linear dimensionality reduction is performed to obtain environmental feature vectors. Cross-entropy loss is calculated based on the credibility of the fused features and the effectiveness of the training samples. The modal weights of the weighted fusion are the weights that minimize the cross-entropy loss.

[0006] Optionally, obtaining the updated memory feature vector based on the environmental feature vector and the memory state at the previous time step includes: The environmental feature vector and the memory state of the previous time step are processed through a gating mechanism to obtain the historical memory retention weight; The environmental feature vector and the memory state of the previous time step are processed through an update gate to obtain the update strength of the new memory; By utilizing historical memory retention weights, the environmental feature vector and the memory state of the previous time step are fused, and candidate memory features are obtained through linear transformation and function activation. By leveraging the updating intensity of new memories, candidate memory features are fused with the memory state of the previous moment to obtain more detailed short-term memories; If the immediate reward of the current decision is greater than the reward threshold, the environmental feature vector, situational understanding vector, action instruction and immediate reward are combined into a set of long-term memory data and saved to the long-term knowledge memory bank. Extract a predetermined number of long-term memory data that are most relevant to the current environment feature vector from the long-term knowledge memory base. Then, weightedly fuse the contextual understanding vector in the long-term memory data with the short-term finer memory to obtain the current contextual understanding vector.

[0007] Optionally, the step of generating a target posterior probability and a decision risk vector based on the environmental feature vector and the memory feature vector, and then combining the target posterior probability and the decision risk vector to generate a contextual understanding vector includes: The environmental feature vector and the memory feature vector are processed separately by a perceptron and then fused to obtain the analysis input vector; The analysis input vector is fed into a Bayesian neural network to obtain the target posterior probability. The analysis input vector is fed into a two-layer perceptron of the risk assessment network to obtain the decision risk vector; The target expectation and decision risk vector are fused together to obtain the context understanding vector.

[0008] Optionally, the filtering method for the communication neighbor set includes: Encode the current state representation vector of the single agent into a low-dimensional vector and broadcast it to the surroundings; Based on the current location information of a single agent and the coverage area of ​​its communication signals, an initial set of neighbors is obtained; The communication link quality is obtained based on the packet loss rate, communication distance, and average communication distance between the current single agent and the candidate neighbor single agents in the initial neighbor set during the communication process. Based on the task objective vector currently being executed by a single agent and the task objective vectors of candidate neighbor single agents in the initial neighbor set, the task relevance is obtained; When the communication link quality is greater than or equal to the link quality threshold, or the task relevance is greater than or equal to the relevance threshold, the candidate neighbor single agent is retained; otherwise, the candidate neighbor single agent is removed from the initial neighbor set; thus obtaining the communication neighbor set.

[0009] Optionally, the step of weightedly fusing the state representation vectors of multiple neighboring agents within the current single agent's communication neighbor set to obtain global collaborative information, and fusing the context understanding vector generated by the self-analysis and reasoning module with the global collaborative information to obtain the fused context understanding vector includes: Based on the communication link quality and task relevance between the current single agent and all neighboring agents, determine the information weight corresponding to each neighboring agent; The state representation vectors of all neighboring agents are weighted and fused according to the information weights corresponding to each neighboring agent to obtain global collaborative information. After processing the global collaborative information using the Sigmoid activation function, the vector is concatenated with the context understanding vector generated by the current single-individual-body analysis and reasoning module to obtain the fused context understanding vector.

[0010] Optionally, generating action instructions based on the fused contextual understanding vector includes: The fused contextual understanding vector is input into the Transformer encoder, and the encoder output is sampled using Gumbel-Softmax to obtain the probability of the high-level action intent and determine the high-level action intent. The context-awareness vector and the high-level action intent are concatenated and input into the multi-layer fully connected layer in the action network to obtain the low-level action, and then the low-level action is transformed into action instructions.

[0011] Optionally, the system further includes: The global objective function is obtained based on the immediate rewards from multiple individual agents and the collaborative rewards from the group. To maximize the global objective function and obtain ideal coordination and ideal action instructions; The analysis and reasoning module and the decision execution module are trained and optimized with the ideal action instructions as the target.

[0012] Optionally, the method for training the decision execution module separately includes: Based on the action instructions output by the current single agent and the expert-demonstrated action or ideal action instructions, a demonstration fitting term is obtained; Based on the pre-defined target priority weights of the task, the basic prior distribution is obtained; Determine the historical return coefficient based on the average return of the selected target according to historical interaction data; The target prior distribution is obtained by correcting the basic prior distribution using historical return coefficients. Calculate the KL divergence based on the target posterior distribution and target prior distribution output by the analysis and reasoning module; The cumulative loss of the decision execution module is calculated based on the exemplary fit term and the KL divergence. The decision execution module is trained to minimize its cumulative loss until the cumulative loss converges, thus obtaining the parameters of the trained and optimized decision execution module.

[0013] Secondly, the present invention also provides a brain-like structure-driven multi-agent hierarchical decision-making method, including: In the current single intelligent body: Environmental feature vectors are extracted based on real-time multimodal observation data acquired from the environment. Based on the environmental feature vector and the memory state of the previous time step, the updated memory feature vector is obtained; Based on the environmental feature vector and the memory feature vector, the target posterior probability and decision risk vector are generated. Combining the target posterior probability and decision risk vector, the context understanding vector is generated. When the decision risk vector is greater than the decision risk threshold or the change in the context understanding vector at adjacent time steps is greater than the change amount threshold, communication between the current single agent and its neighboring agents is triggered. The state representation vectors of multiple neighbor agents within the communication neighbor set are weighted and fused to obtain global collaborative information. The state representation vectors include environmental feature vectors, environmental understanding vectors, and action instructions. The context understanding vector and global collaborative information are fused to obtain the fused context understanding vector; Based on the understanding of the fusion context, vectors are used to generate action instructions.

[0014] This invention provides a brain-like structure-driven multi-agent hierarchical decision-making method and system. Compared with existing technologies, it has the following advantages: The multi-source perception module can process multimodal real-time observation data, significantly improving adaptability to complex and highly dynamic environments. The memory module dynamically updates its memory feature vector based on the environmental feature vector and the memory state of the previous time step. The analysis and reasoning module can generate the target posterior probability and decision risk vector, and further generate a contextual understanding vector, enabling the agent to perform more advanced cognition and judgment. A communication mechanism based on decision risk or contextual changes is introduced. When a significant contextual change is detected, communication with neighboring agents can be proactively triggered, greatly improving the efficiency and relevance of communication, while ensuring timely sharing of key information, especially suitable for distributed environments with limited communication. By fusing its own contextual understanding vector with global collaborative information from neighboring agents, a fused contextual understanding vector can be formed. This collaborative decision-making capability enables the entire robot team to operate efficiently as a whole, significantly improving the overall performance and robustness of the multi-agent system in complex tasks, and addressing the pain point that existing single flat models cannot meet the needs of cross-task generalization and collaborative decision-making. Through its modular perception, memory, analytical reasoning, and context-aware adaptive communication and collaborative decision-making mechanisms, it effectively simulates the hierarchical information processing of biological intelligence, thereby providing stronger autonomous decision-making capabilities, contextual understanding capabilities, and team collaboration efficiency in complex, dynamic, and communication-constrained multi-agent scenarios. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a brain-like structure-driven multi-agent hierarchical decision-making system provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a brain-like structure-driven multi-agent hierarchical decision-making method provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application are described clearly and completely. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0019] like Figure 1 As shown in the embodiment of this application, a brain-like structure-driven multi-agent hierarchical decision-making system is provided, which includes multiple single agents. Each single agent includes a multi-source perception module, a memory module, an analysis and reasoning module, and a decision execution module. Within the current single intelligent entity, the multi-source sensing module is used to extract environmental feature vectors based on real-time multimodal observation data acquired from the environment. The memory module is used to obtain the updated memory feature vector based on the environmental feature vector and the memory state of the previous time step. The analysis and reasoning module is used to generate the target posterior probability and decision risk vector based on the environmental feature vector and the memory feature vector, and to generate the context understanding vector by combining the target posterior probability and the decision risk vector. ; When the decision risk vector is greater than the decision risk threshold or the change in the context understanding vector at adjacent time steps is greater than the change amount threshold, communication between the current single agent and its neighboring agents is triggered. The state representation vectors of multiple neighboring agents within the communication neighbor set of the current single agent are weighted and fused to obtain global collaborative information. The context understanding vector generated by the agent's own analysis and reasoning module is fused with the global collaborative information to obtain a fused context understanding vector. The state representation vector includes environmental feature vectors, environmental understanding vectors, and action instructions. The decision-making and execution module in a current single-agent body is used to generate action instructions based on the context-aware vector. .

[0020] Specifically, such as Figure 1 As shown, the system consists of multiple structurally identical brain-like single agents, each containing four core functional modules: a multi-source perception module, a memory module, an analysis and reasoning module, and a decision execution module. These sub-modules form a hierarchical processing relationship: the multi-source perception module provides real-time environmental features, the memory module provides long-term experience support, the analysis and reasoning module is responsible for comprehensive judgment, and the decision execution module forms the final action command based on the comprehensive information. A collaborative mechanism is designed at the multi-agent level, enabling stable and flexible group decision-making among single agents in scenarios with limited communication, uncertain adversarial relationships, or dynamically changing task divisions. This can be used to improve the autonomy, interpretability, and adaptability to complex tasks of multi-agent systems. The invention will be described below from three aspects: network architecture design, brain-like single agent design, and multi-agent collaborative mechanism.

[0021] Multiple single agents in the system operate in parallel in a distributed environment and coordinate through communication protocols. The system design follows a hierarchical principle, forming a layered processing structure from the functional modules within a single agent to the interactions between multiple agents.

[0022] Suppose that the multi-agent system BHM includes A homogeneous brain-like intelligent agent Intelligent agents are distributed in dynamic environments In this model, the core mathematical definition is as follows: (1)) in, For the first A brain-like intelligent agent, satisfying ,in, These represent the perception, memory, analysis, and decision-making execution modules, respectively. It is a set of intelligent agent coordination mechanisms, including communication protocols, conflict resolution rules, etc. The total input set of the model. For intelligent agents The local observation input satisfies ( For the first (Observation dimension of each agent) This is the total output set of the model. For intelligent agents The action output satisfies ( For the first (The action space dimension of an individual agent).

[0023] The hierarchical information flow mechanism designed in this application is the core link of a brain-like multi-agent decision-making network. It simulates the hierarchical information processing logic of the biological brain from sensory input to cognitive decision-making, overcoming the technical shortcomings of the flat, sequential "perception-decision" model in traditional deep reinforcement learning models. This mechanism consists of two core components: cross-module information transmission within a single agent and global information fusion constraints among multiple agents. Through differentiable mapping and dynamic weight fusion, it achieves structured and interpretable information flow, ensuring both the hierarchical nature of the cognitive process within a single agent and the high efficiency of collaborative decision-making among multiple agents.

[0024] Cross-module information transfer functions serve as the information interaction criteria for the four functional modules of perception, memory, analysis, and decision-making within a single agent. By defining differentiable mathematical mapping relationships, hierarchical processing from real-time environmental observation to final action commands is achieved, while ensuring continuous gradient backpropagation during model training. This is illustrated in Equations 2-5.

[0025] (2)) Formula 2 is the information output formula for the multi-source sensing module, where the input is the multimodal real-time observation data acquired by the agent from the environment, specifically including camera images, radar point clouds, communication link status information, etc., the dimensions of which are determined by the sensor configuration of the actual application scenario; the output... The environmental feature vector extracted by the multi-source perception module. For feature dimensions, the preferred value in this invention is 256~1024, which can be adaptively adjusted according to task complexity; model parameters It is a set of learnable parameters for multi-source perception modules, including the weights and biases of sub-networks such as Convolutional Neural Network (CNN), PointNet, and Multilayer Perceptron (MLP). The design of the function is inspired by the partitioning processing mechanism of the visual cortex and somatosensory cortex of the biological brain. It adapts to observation data of different modalities through multi-branch networks to realize the structured feature extraction of environmental information, which is different from the coupled design of a single network processing multimodal data in traditional models.

[0026] (3)) Formula 3 is the information update formula for the memory module, and it is one of the core manifestations of brain-like design. The input includes the environmental feature vector output by the multi-source perception module. and Memory state at any moment , For the memory dimension, the best option is... Feature Dimensions Maintain consistency to reduce the cost of dimensional transformation in cross-module information interaction; output The updated memory feature vector integrates real-time environmental features and historical experience information; model parameters It is a set of learnable parameters for the memory module, including the weights and biases of the improved gated recurrent unit (GRU) and the memory network. The function design simulates the memory encoding and retrieval mechanism of the hippocampus in the biological brain. It realizes the storage of short-term episodic memory through time-dependent state updates, and realizes the retrieval of long-term knowledge memory through the screening and retrieval of the memory bank. It solves the technical defects of traditional models that rely only on current observations and lack historical experience support.

[0027] (4)) Formula 4 is the information output formula of the analysis and reasoning module, where the input is the environmental feature vector of the multi-source perception module. Memory feature vectors of memory modules This achieves the fusion of two dimensions: real-time context and historical experience; output... To analyze the contextual understanding vectors generated by the reasoning module. For the analysis results dimensions, a value of 512~2048 is preferred to meet the needs of intent inference and risk assessment in complex scenarios; model parameters The set of learnable parameters for the analysis and reasoning module includes the weights and biases of the Bayesian Neural Network (BNN) and the Transformer encoder. The function design simulates the cognitive analysis function of the prefrontal cortex of the biological brain, realizes the intention inference of environmental goals through probabilistic reasoning, and realizes the pre-judgment of decisions through risk assessment. It is different from the "black box" analysis process of direct mapping in traditional models, and improves the interpretability of decisions.

[0028] (5)) Formula 5 is the action output formula of the decision execution module, where the input is the context understanding vector of the analysis and reasoning module. It is the final feature input for information processing within a single agent; the output... The dimensions of the action instructions generated for the agent. The complexity of the action space determines the dimension (e.g., three-degree-of-freedom actions like steering, acceleration, and braking in autonomous driving; six-degree-of-freedom actions like position, velocity, and heading in a drone swarm); model parameters. The set of learnable parameters for the decision execution module includes the weights and biases of the policy network and the multilayer perceptron. The function design simulates the action planning function of the motor cortex of the biological brain. Through hierarchical reinforcement learning, it achieves the refinement from high-level intentions to low-level actions, solving the problem of the disconnect between action generation and scene understanding in traditional models.

[0029] In engineering implementation, the gradient backpropagation of the cross-module information transfer function follows the reverse link of "decision-analysis-memory-perception". The joint optimization of the parameters of each module is achieved through the chain rule, and its gradient calculation is shown in Equation 6: (6)) in, This is the global loss function of the model. Represents any functional module. This gradient calculation method, representing the module downstream in the information flow chain, ensures the consistency and continuity of parameter optimization for each module.

[0030] In multi-agent systems, the local observations of a single agent are incomplete. By designing global information fusion constraints, information interaction and collaboration among agents can be achieved. The core of this is to fuse the state information of neighboring agents through dynamic weights to generate global collaborative information, and then integrate it into the information processing of the analysis and reasoning module and the decision execution module to ultimately correct local decision biases.

[0031] In this optional embodiment, the multi-source perception module can process multimodal real-time observation data, such as images and point clouds. This enables the system to obtain richer and more comprehensive environmental information than a single sensor, significantly improving its adaptability to complex and highly dynamic environments. The memory module can dynamically update its memory feature vector based on the environmental feature vector and the memory state of the previous moment. This means that a device equipped with a single intelligent agent (such as robot A or a drone) can accumulate experience and remember the locations of explored areas, dangerous areas, or important targets, thereby avoiding repetitive work or falling into known dangers. This overcomes the limitations of existing intelligent agents that lack explicit memory management, making long-term reliance and experience-based planning difficult. The analysis and reasoning module can generate target posterior probability and decision risk vectors, and further generate contextual understanding vectors. For example, robot A not only knows that there are obstacles ahead, but can also assess the success rate and potential risks of passing through obstacles. This deep understanding of the context far exceeds the ability of existing simple decision-making units to only execute preset rules, enabling the intelligent agent to perform more advanced cognition and judgment. This system introduces a communication mechanism triggered by decision risk or contextual changes. When a device equipped with a single agent detects a high risk of collision or a significant situational change such as smoke emanating from the device, it can proactively trigger communication with neighboring agents. Compared to the fixed-period communication or lack of communication mechanisms in existing technologies, this significantly improves communication efficiency and relevance, avoids unnecessary communication overhead, and ensures timely sharing of critical information, making it particularly suitable for distributed environments with limited communication. By fusing its own situational understanding vector with global collaborative information from neighboring agents, a fused situational understanding vector can be formed. For example, when deciding whether to proceed to a smoking device, robot A considers not only its own path but also information on whether other robots are already processing the issue or whether there are better paths. This collaborative decision-making capability enables the entire robot team to operate efficiently as a whole, avoiding global suboptimal results caused by local optima, and significantly improving the overall performance and robustness of multi-agent systems in complex tasks. This directly addresses the pain point that existing single, flat models struggle to meet the needs of cross-task generalization and collaborative decision-making.

[0032] Therefore, the brain-like structure-driven multi-agent hierarchical decision-making system proposed in this embodiment effectively simulates the hierarchical information processing process of biological intelligence through its modular perception, memory, analytical reasoning, and context-aware adaptive communication and collaborative decision-making mechanisms. Thus, in complex, dynamic, and communication-constrained multi-agent scenarios, it provides stronger autonomous decision-making capabilities, contextual understanding capabilities, and team collaboration efficiency, which is significantly better than existing technologies.

[0033] Neuromorphic single intelligent agent (denoted as The multi-agent system of this invention is the core basic unit, inspired by the functional partitioning and hierarchical cognitive mechanism of the biological brain. It simulates the specialized division of labor among the brain's sensory cortex, hippocampus, prefrontal cortex, and motor cortex, breaking through the limitations of the traditional flat structure of intelligent agents that rely on a single network. It constructs a system consisting of multi-source sensory modules (…). ), memory module ( ), Analytical Reasoning Module ( ), Decision Execution Module ( The modular architecture consists of several modules. Each module functions independently yet works in synergy. Through a hierarchical process of "perceiving and extracting environmental features - storing experiential knowledge - analyzing and implementing situational reasoning - generating precise actions," it achieves advanced cognitive decision-making capabilities that combine real-time response, long-term reliance, and cross-task generalization. The overall structure and detailed design of each module are as follows.

[0034] 1. The core function of the multi-source perception module is to transform the multimodal, high-dimensional raw observation data acquired by the agent from the environment into structured, low-redundancy environmental feature vectors, providing a reliable data foundation for subsequent memory storage and contextual analysis. This module adopts a "multi-branch adaptation + adaptive fusion" design approach, adapting to different types of observation inputs such as vision, radar, communication, and audio, thus overcoming the technical shortcomings of traditional single-modal perception models, such as poor robustness and low efficiency in multimodal data coupling processing. Specific details are as follows.

[0035] The extraction of environmental feature vectors based on real-time multimodal observation data obtained from the environment includes: Image features are obtained by performing convolution and global average pooling on the image data.

[0036] After the point cloud data is encoded at its location, it is then processed by a multilayer perceptron and global max pooling to obtain the point cloud features.

[0037] Specifically, for the structural characteristics of different modal observation data, a dedicated feature extraction branch is designed to ensure the effective parsing of information for each modality, as shown in Equations 7 and 8.

[0038] (7) Formula 7 is a formula for visual modality feature extraction, adapted to grid-type data such as camera images and infrared images. Its technical details are as follows: enter : (Image height) The preferred value for (image width) is 256×256~512×512 (adjusted according to hardware computing power), and 3 represents the number of RGB channels; if it is an infrared image, the number of channels is adjusted to 1, and the network structure remains consistent.

[0039] Convolution operation : Use depthwise separable convolution to reduce computational cost. The number of convolutional layers is preferred, ideally 8 to 12 layers (including 4 to 6 residual blocks); the... Layer convolution kernel middle, (Kernel size) The first 4 layers are 3×3, and the last layer is 1×1 (dimensionality reduction). (Number of input channels) starts from 3 and increases by 2 for each layer (e.g., 3→64→128→256→512). The number of output channels matches the number of input channels.

[0040] Global Average Pooling (GAP): Transforms the features output by the last convolutional layer into a one-dimensional vector, avoiding the risk of overfitting caused by fully connected layers.

[0041] Learnable parameters Includes the weights and biases of each convolutional layer, as well as the parameters of the BatchNorm layer (batch normalization layer). The He initialization method is used to improve training stability.

[0042] (8) Formula 8 is a feature extraction formula for radar point clouds, adapted to unstructured point cloud data from lidar and millimeter-wave radar. Its technical details are as follows: enter : The number of point clouds is preferably between 1024 and 4096 points (balancing accuracy and efficiency), and the three dimensions are three-dimensional spatial coordinates; if intensity information is included, the dimensions are expanded to 4, and the Transformer coding layer adapts synchronously.

[0043] Location Encoding (Transformer): Captures spatial relationships between point clouds through a self-attention mechanism. After the input point cloud is mapped to 256-dimensional features through a linear layer, the global dependency is calculated through a 6-layer Transformer encoder (with 8 multi-head attention heads), which solves the problem of insufficient local feature capture in traditional PointNet.

[0044] Multilayer Perceptron (MLP): It consists of 3 fully connected layers (256→512→1024), and uses LeakyReLU (negative slope 0.01) as the activation function to enhance the non-linear representation of features.

[0045] Global max pooling (max): Extracts the global maximum value from the 1024-dimensional features, outputs the global features of the 1024-dimensional point cloud, and preserves the key information of the geometric structure of the point cloud.

[0046] Learnable parameters : Includes Transformer encoder weights, MLP weights and biases, and uses the Xavier initialization method.

[0047] Image features and point cloud features are weighted and fused to obtain fused features, and then linear dimensionality reduction is performed to obtain environmental feature vectors. Cross-entropy loss is calculated based on the credibility of the fused features and the effectiveness of the training samples. The modal weights of the weighted fusion are the weights that minimize the cross-entropy loss.

[0048] Multimodal features are weighted, fused, and reduced in dimensionality to generate a unified environmental feature vector. As shown in Formulas 9 and 10.

[0049] (9) Formula 9 is the dual-modal feature fusion formula, and its technical details are as follows: Modal weights The contribution of visual and radar features is dynamically balanced; for example, the reliability of visual features decreases in strong light conditions. Automatically reduces radar signature; more reliable radar signature in obstructed environments. It increases automatically.

[0050] Feature-weighted summation: visual features With radar characteristics First, unify the dimensions through linear layers, then... Weighted summation avoids information loss caused by dimension mismatch.

[0051] Linear dimensionality reduction: Maps the weighted summed high-dimensional features to a low-dimensional dimension, outputting the final perceptual features. Dropout is used during dimensionality reduction to prevent overfitting. For multimodal (≥3) feature fusion, linear expansion can be achieved by increasing the dimension of the weights.

[0052] (10) Formula 10 is the modal weight. The adaptive learning formula, and its technical details are as follows: Training samples and labels: The number of labeled samples should be ≥10,000. The modality validity label (1 indicates that the corresponding modality information in the sample is reliable, and 0 indicates that it is unreliable, generated by manual annotation or automatic detection algorithm).

[0053] Credibility : Indicates the current The reliability of the data after feature fusion is determined by the output of the fused features through a 2-layer MLP (384→128→1), with the activation function being Sigmoid, and mapped to... Interval.

[0054] Cross-entropy loss: By minimizing the loss function, the model learns an adaptive strategy of "increasing the corresponding weight when the modality is reliable and decreasing the weight when it is unreliable", without the need for manual intervention.

[0055] Training method: weights Other parameters of the multi-source sensing module ( , Joint training is used, and gradients are backpropagated through the chain rule to ensure optimization consistency.

[0056] 2. The core function of the memory module is to store the agent's short-term context and long-term knowledge, providing historical experience support for the decision-making process and overcoming the technical shortcomings of traditional models that rely solely on real-time observation and lack "empirical" decision-making capabilities. This module is divided into two sub-modules: short-term contextual memory and long-term knowledge memory. Through a mechanism of "real-time updates + on-demand retrieval," it achieves precise matching between memory and the current context. Specific details are as follows.

[0057] The process of obtaining the updated memory feature vector based on the environmental feature vector and the memory state at the previous time step includes: By processing the environmental feature vector and the memory state of the previous time step through a gating mechanism, the historical memory retention weight is obtained.

[0058] The environmental feature vector and the memory state of the previous time step are processed through an update gate to obtain the update strength of the new memory.

[0059] By utilizing historical memory retention weights, the environmental feature vector and the memory state of the previous time step are fused, and candidate memory features are obtained through linear transformation and function activation.

[0060] By leveraging the updating intensity of new memories, candidate memory features are fused with the memory state of the previous moment to obtain more detailed short-term memories.

[0061] Specifically, short-term episodic memory mimics the short-term memory function of the hippocampus, storing near-term memories. The perception features, analysis results, and action commands at each time step are used to capture temporal dependencies (such as the movement trend of the vehicle in front in autonomous driving and the collaborative trajectory of a drone swarm), as shown in Formulas 11-14.

[0062] (11) (12) (13) (14) The above formula is the update rule for the improved GRU (Gated Recurrent Unit), which enhances the interaction between memory and perceptual features compared to the traditional GRU. Its technical details are as follows: Input variables: : The environmental feature vector output by the multi-source sensing module at the current time step; : The short-term memory state is initialized to a vector of all zeros. Gating mechanism: Reset door Controlling the degree to which historical memory is preserved. Preserve all historical memory at all times. Ignoring historical memory and focusing on current perceptual features; updating the door : Control the intensity of new memory updates, Completely replace historical memory, Preserve historical memory; activation function The Sigmoid function is used to achieve continuous adjustment of the gating. Candidate Memory : Historical memories filtered through the reset gate are spliced ​​with current perceptual features, and then linearly transformed and Activation generation includes information that integrates the current context with filtered historical experiences; Short-term memory update By updating the gate to balance historical memory and candidate memory, the short-term memory state of the current time step is generated. This state will be passed to the next time step, forming a temporal memory chain.

[0063] If the immediate reward of the current decision is greater than the reward threshold, the environmental feature vector, situational understanding vector, action instruction, and immediate reward are combined into a set of long-term memory data and saved to the long-term knowledge memory bank.

[0064] Specifically, long-term knowledge memory simulates the function of transferring long-term memory from the hippocampus to the neocortex, storing the "perception-classification-decision-reward" quadruple that has been verified in practice, forming a reusable knowledge base, as shown in Formula 15.

[0065] (15) Formula 15 is the update rule for the long-term memory, and its technical details are as follows: Memory structure It adopts a key-value pair storage structure, where the key is the perceptual feature. Value is a triple ( To analyze features, For action commands, (For decision-making rewards); a maximum capacity is set for the memory bank. When the capacity is exceeded, the earliest stored low-value memory is deleted according to the "first-in, first-out" principle.

[0066] Memory storage conditions : The immediate reward for the current decision (based on environmental feedback, such as rewards for safe driving in autonomous driving and rewards for completing drone collaborative tasks). A reward threshold is set; only when the decision reward exceeds the threshold will the corresponding quadruple be stored in the long-term memory, ensuring the effectiveness of the knowledge in the memory.

[0067] Extract a predetermined number of long-term memory data that are most relevant to the current environment feature vector from the long-term knowledge memory base. Then, weightedly fuse the contextual understanding vector in the long-term memory data with the short-term finer memory to obtain the current contextual understanding vector.

[0068] The output of the memory module is a fusion of short-term episodic memory and long-term knowledge memory, as shown in Equations 16 and 17.

[0069] (16) Formula 16 is the memory fusion formula, and its technical details are as follows: Fusion weights It balances the contributions of short-term and long-term memory, dynamically adjusting based on the uncertainty of the current situation. When the environment is familiar (such as in repetitive task scenarios), Reduce (preferably 0.3~0.5) and increase the proportion of long-term knowledge memory; when the environment is unfamiliar (such as sudden obstacles or new tasks), Increase (preferably 0.7~0.9) to increase the proportion of short-term episodic memory; Through reinforcement learning and adaptive learning, it is jointly optimized with other parameters of the model.

[0070] Long-term memory retrieval Extracting features from long-term memory that are currently perceived. Most relevant A memory ( The preferred value is 5 to 10 items. For the first The relevance weight of each memory item This is the analytical feature (contextual understanding vector) corresponding to this memory.

[0071] (17) Formula 17 is for calculating the relevance weights, and its technical details are as follows: Cosine similarity Calculate the current perceived features With the first in the long-term memory bank Perceptual characteristics of memory The higher the similarity between the two, the better the current situation matches the historical memory.

[0072] Weight normalization: satisfy (Only the first part is retained) One high-similarity memory is selected, and the weights of the remaining memories are considered to be 0, to ensure the stability of the fused features.

[0073] Extraction efficiency optimization: To improve the retrieval speed of the long-term memory, a KD-tree (K-Dimensional Tree) is used to optimize the retrieval of data in the long-term memory. Indexing reduces retrieval time complexity and meets real-time decision-making needs.

[0074] 3. The analysis and reasoning module is the "cognitive core" of the intelligent agent. Its core function is based on environmental feature vectors. With memory feature vector This enables environmental context understanding, target intent inference, and decision risk assessment, overcoming the black-box defect of the traditional model of "perception directly mapping to decision-making," and providing interpretable reasoning basis for the decision execution module. The specific content is as follows.

[0075] The step of generating a target posterior probability and a decision risk vector based on environmental feature vectors and memory feature vectors, and then combining the target posterior probability and the decision risk vector to generate a contextual understanding vector includes: The environmental feature vector and the memory feature vector are processed separately by a perceptron and then fused to obtain the analysis input vector.

[0076] Specifically, contextual features are the basic inputs to the analysis and reasoning module, generated through cross-dimensional fusion of environmental feature vectors and memory feature vectors, as shown in Equation 18: (18) Environmental feature vector With memory feature vector Dimensional expansion and nonlinear mapping are performed using two independent MLP layers, with GELU (Gaussian Error Linear Units) as the activation function, which better alleviates the gradient vanishing problem compared to ReLU. The outputs of the two MLPs are directly added, and the original feature information is preserved through residual connections to enhance gradient backpropagation. Layer normalization is applied to the fused features to make the feature distribution more stable and improve the model training convergence speed. The contextual feature output contains full-dimensional information including "real-time environment state + historical experience + temporal dependencies," providing rich data support for subsequent inference.

[0077] The analytical input vector is fed into a Bayesian neural network to obtain the target posterior probability. The analytical input vector is then fed into a two-layer perceptron of a risk assessment network to obtain the decision risk vector.

[0078] Specifically, the core reasoning process of the analytical reasoning module includes intent inference (distribution of environmental targets) and risk assessment (degree of decision risk), as shown in Formulas 19 and 20.

[0079] (19) Formula 19 is the intent inference formula, and its technical details are as follows: target set The environmental target categories are defined based on the application scenario. For example, in autonomous driving, these include following other vehicles, overtaking, obstacle avoidance, and parking; in drone swarms, they include collaborative reconnaissance, target tracking, formation keeping, and emergency avoidance. The optimal number of target categories is 4-8. A Bayesian Neural Network (BNN) uses Variational Bayesian Inference for probabilistic modeling. The network structure is a 3-layer MLP, with weights in each layer following a Gaussian distribution. The weight distribution is optimized by minimizing the lower bound of evidence (ELBO). Compared to traditional deterministic networks, BNN can output the uncertainty of target distribution, providing a basis for risk assessment. Target posterior probability. This indicates that the environmental goal in the current context is The probability of.

[0080] (20) Formula 20 is a risk assessment formula, and its technical details are as follows: Risk Vector The dimension is consistent with the target set. Indicates the execution target When making corresponding decisions, the degree of risk faced (such as the collision risk of the obstacle avoidance target, the safe distance risk of the overtaking target), and the range of values. The closer the value is to 1, the higher the risk. The risk assessment network is a 2-layer MLP with LeakyReLU activation function, and the output is normalized to 1 using Softmax. During network training, risk labels are jointly labeled by environmental feedback and expert knowledge.

[0081] The target expectation and decision risk vector are fused together to obtain the context understanding vector.

[0082] The final output of the analytical reasoning module is a context understanding vector. The results of the combined intent inference and risk assessment are shown in Formula 21: (twenty one) Formula 21 is the formula for outputting the analysis results, and its technical details are as follows: Target Expectations Taking the expectation of the posterior probability distribution of the target, we get A 3D vector, where each element represents the target. The probability weighting value highlights the priority of high-probability targets.

[0083] Risk avoidance items The risk vector is inverted and transformed into a "safety coefficient". The lower the risk, the higher the safety coefficient. This, together with the target expectation, guides decision-making towards optimization towards "high probability target + high safety coefficient".

[0084] The multi-agent collaborative mechanism uses a single agent with a brain-like structure as the basic unit. Through two core components—a brain-like heuristic communication protocol and a distributed collaborative decision function—it achieves stable and efficient group decision-making in scenarios with limited communication, dynamic environments, and varied tasks, ensuring that the multi-agent system achieves a balance between autonomy, flexibility, and robustness.

[0085] The core objective of the neuromorphic heuristic communication protocol is to address the problems of "full-time broadcasting, information overload, and bandwidth waste" in traditional multi-agent communication. It simulates the sparse interaction characteristics of biological groups (such as ant colonies communicating on demand via pheromones) to achieve a communication strategy of "prioritizing the transmission of high-value information and silently suppressing low-value information," while ensuring information effectiveness in communication-constrained scenarios. This protocol comprises three key components: a communication triggering mechanism, low-dimensional information encoding, and dynamic neighbor management. Therefore, before the decision-making execution module within a single agent makes a decision, the priority of the communication triggering mechanism must be determined. The specific details are as follows.

[0086] When the decision risk vector is greater than the decision risk threshold or the change in the context understanding vector at adjacent time steps is greater than the change amount threshold, communication between the current single agent and its neighboring agents is triggered.

[0087] Specifically, the communication triggering mechanism determines when an agent initiates communication, and information broadcasting is only initiated when the current situation meets the conditions of "high risk" or "high change," as shown in Formula 22: (twenty two) Formula 22 is the communication trigger function, and its technical details are as follows: Triggering condition 1: Decision risk threshold judgment ( ) To analyze the maximum value of the decision risk vector output by the reasoning module (i.e., the risk level of the highest risk item in the current situation), such as "collision risk" in autonomous driving and "mission failure risk" in drone swarms; An adaptive dynamic adjustment strategy is adopted to determine the risk trigger threshold. ,in The base threshold is preferably 0.6 to 0.7. This is an adjustment factor (preferably 0.1~0.2). The threshold is the current number of neighbors; the more neighbors, the higher the threshold, to avoid communication storms; the fewer neighbors, the lower the threshold, to ensure that no critical information is missed. Physical meaning: When an intelligent agent faces high-risk decisions (such as sudden obstacles or sudden changes in task objectives), it proactively initiates communication to seek cooperation from neighbors, thereby reducing the probability of decision-making errors.

[0088] Triggering condition 2: Threshold judgment of situational change ( ) The Euclidean distance between context understanding vectors analyzed at adjacent time steps characterizes the degree of change in context understanding results. The threshold for triggering changes is adaptively set according to the dimension of the analysis vector: ,in To analyze the vector dimension, The threshold value is set at 0.1 to 0.15 per unit dimension; the higher the dimension, the larger the threshold value should be to avoid invalid communication triggered by high-frequency small fluctuations. Physical meaning: When the environmental situation changes significantly (such as the location of the collaborative goal shifting or the division of tasks being adjusted), the latest situation is proactively broadcast to neighbors to ensure consistency in group cognition.

[0089] Communication trigger output The value can be 1 (trigger communication) or 0 (silent). The sparse interaction strategy of "must communicate in critical situations and less communication in normal situations" is implemented through the "OR logic" of the two conditions. The communication frequency is reduced by 60% to 80% compared with traditional full-time broadcast, which greatly saves bandwidth.

[0090] In addition, methods for filtering communication neighbor sets include: Encode the current state representation vector of the single agent into a low-dimensional vector and broadcast it to the surroundings.

[0091] Specifically, when At this time, the agent needs to encode its own state information into a low-dimensional vector for broadcasting, balancing communication efficiency and information integrity, as shown in Equation 23: (twenty three) Formula 23 is the communication information encoding formula, and its technical details are as follows: State representation vector The original dimension is too high, and direct transmission consumes too much bandwidth, so it needs to be embedded and quantized for compression. Low-dimensional embedding: Dimensionality compression is achieved using an autoencoder. The network structure is: input layer → encoder (MLP) → decoder (MLP); the training objective is to minimize the reconstruction error; the final dimensionality compression rate reaches over 80%. Quantization: Scalar quantization is used to further reduce transmission overhead, quantizing a 128-dimensional floating-point vector (32 bits / element) into an 8-bit integer vector, with a quantization range of [value missing]. The quantification formula is ,in For embedding vector elements, , This defines the quantization range for learning during the training phase; after quantization, the data volume of a single agent's single communication is reduced from 512 bytes to 128 bytes, resulting in a 75% reduction in bandwidth consumption. Learnable parameters It includes encoder and decoder weights of the autoencoder, which are trained together with other model parameters to ensure that the embedded vector retains the core information of the original state (such as key environmental information of the perceptual features, target and risk information of the analysis vector, and decision direction of the action command).

[0092] Based on the current location information of a single agent and the coverage range of communication signals, an initial set of neighbors is obtained.

[0093] Specifically, neighbor set It is not fixed, but dynamically adjusted according to the quality of the communication link and the relevance of the task to ensure the "reliability" and "usefulness" of the received information. Neighbor candidate set: The initial neighbor set consists of all agents within the coverage area of ​​the communication signal (such as all nodes within a 5km communication radius of a drone swarm).

[0094] The communication link quality is obtained by comparing the packet loss rate, communication distance, and average communication distance between the current single agent and the candidate neighbor single agents in the initial neighbor set during the communication process.

[0095] The task relevance is obtained by combining the task objective vector of the current single agent and the task objective vectors of the candidate neighbor single agents in the initial neighbor set.

[0096] When the communication link quality is greater than or equal to the link quality threshold, or the task relevance is greater than or equal to the relevance threshold, the candidate neighbor single agent is retained; otherwise, the candidate neighbor single agent is removed from the initial neighbor set; thus obtaining the communication neighbor set.

[0097] Specifically, link quality screening: computational agents With candidate neighbors Communication link quality ,in The sliding window size (preferably 10 time steps) is used. For a moment Communication distance, For average communication distance, For a moment Packet loss rate; only retain Candidate neighbors; Task-related filtering: computational agents With candidate neighbors Task relevance ,in , They are respectively , The mission objective vector (e.g., when the mission objective in a UAV swarm is "area reconnaissance," it is the coordinates and range of the center of the reconnaissance area); only retain the following: Candidate neighbors; Neighbor set update: Updated every 5 time steps to ensure that the neighbor set is always composed of agents that are "reliable in communication and related to the target", avoiding interference from invalid information.

[0098] The process involves weighted fusion of the state representation vectors of multiple neighboring agents within the current single agent's communication neighbor set to obtain global collaborative information. Then, the fused context understanding vector is obtained by fusing the context understanding vector generated by the agent's own analysis and reasoning module with the global collaborative information. This fused context understanding vector includes: Based on the communication link quality and task relevance between the current single agent and all neighboring agents, determine the information weight corresponding to each neighboring agent.

[0099] (twenty four) The information weight value is determined by two factors: communication link quality and task relevance, as shown in Formula 24. For intelligent agents and The quality of the communication link between them (calculated by indicators such as signal strength and packet loss rate). For intelligent agents and The task relevance (obtained by calculating the task target vectors of the two agents using cosine similarity) and the weight design ensure the "usefulness" and "reliability" of information fusion, avoiding interference from invalid information on local decisions. This is the state representation vector of the agent. It integrates environmental feature vectors, contextual understanding vectors, and action instructions through a first-to-last concatenation method. It is a low-dimensional representation of the agent's core state, and its dimension is preferably optimized. In engineering implementation, dimensionality can be reduced to a fixed dimension (e.g., 1024 dimensions) through an embedding layer to unify the state representation format of different agents; global collaborative information. Dimensions and After dimensionality reduction, the dimensions are consistent, and its physical meaning is that of an intelligent agent. The global collaborative information obtained from neighboring nodes solves the technical defect of undifferentiated global information fusion in traditional multi-agent models.

[0100] The state representation vectors of all neighboring agents are weighted and fused according to the information weights corresponding to each neighboring agent to obtain global collaborative information. (25) Formula 25 is the formula for generating global collaborative information, which is the core mathematical expression of multi-agent collaboration. This is the set of communication neighbors for an intelligent agent. Its value is determined by the coverage of the communication link, signal quality, and task relevance. In communication-constrained scenarios (such as long-distance communication of drone swarms and vehicle-to-vehicle communication interruption in autonomous driving), it is particularly relevant. The size can be dynamically reduced, and the preferred value is... To reduce communication bandwidth consumption; The information weights of neighboring agents satisfy the normalization constraint. .

[0101] After processing the global collaborative information using the Sigmoid activation function, the vector is concatenated with the context understanding vector generated by the current single-individual-body analysis and reasoning module to obtain the fused context understanding vector.

[0102] (26) Formula 26 is the formula for fusing global information and local analysis features, where This is a vector concatenation operation that enables contextual understanding of vectors for local analysis. Global collaborative information The dimensional expansion, in engineering implementation, results in the following feature dimensions after concatenation: ( for (The dimension of the line can be reduced to a linear layer) To maintain consistency in the output dimensions of the analysis and reasoning module; The Sigmoid activation function is used to incorporate global cooperative information. The value is mapped to The interval enables adaptive control of the global information fusion intensity. When When the proportion of global information increases, it is suitable for scenarios with high task collaboration; when At this time, the proportion of global information decreases, making it suitable for scenarios with limited communication or high task independence; The fused contextual understanding vector, which integrates global information, is the final input to the decision execution module. This fusion method ensures a balance between local decision-making and global collaboration, and solves the problem of inconsistent information between the training and execution stages in the traditional CTDE framework.

[0103] 4. After information interaction and fusion are completed among the multiple agents, the decision-making and execution module within a single agent begins to function. The core function of the decision-making and execution module is based on the contextual understanding vector output by the analysis and reasoning module. (Analysis features that integrate global information) generate action instructions that meet the task objectives, safety constraints and collaborative requirements. It adopts a hierarchical design of "high-level strategy planning + low-level action refinement" to overcome the technical defects of traditional single-level decision-making models, such as rough actions and poor adaptability. The specific content is as follows.

[0104] (27) Formula 27 is the action output formula of the decision execution module, where the input is a fusion contextual understanding vector that integrates global information. It is the final feature input for information processing within a single agent; the output... The dimensions of the action instructions generated for the agent. The complexity of the action space determines the dimension (e.g., three-degree-of-freedom actions like steering, acceleration, and braking in autonomous driving; six-degree-of-freedom actions like position, velocity, and heading in a drone swarm); model parameters. This design provides a set of learnable parameters for the decision-making and execution module, including the weights and biases of the policy network and multilayer perceptron. The function design simulates the action planning function of the motor cortex in the biological brain, achieving refinement from high-level intentions to low-level actions through hierarchical reinforcement learning, thus solving the problem of disconnect between action generation and scene understanding in traditional models. This design ensures the integrity of hierarchical information flow within a single agent while simultaneously enabling collaborative decision-making among multiple agents.

[0105] The generation of action instructions based on the fused contextual understanding vector includes: The fused contextual understanding vector is input into the Transformer encoder, and the encoder output is sampled using Gumbel-Softmax to obtain the probability of the high-level action intent and determine the high-level action intent.

[0106] Specifically, the goal of high-level strategic planning is to generate abstract action intentions (such as "collaboration," "exploration," and "avoidance") based on contextual understanding vectors, clarifying the core direction of decision-making, as shown in Formula 28: (28) Formula 28 is the formula for generating high-level strategies, and its technical details are as follows: enter : Contextual understanding vector that integrates global collaborative information, including local contextual analysis results and multi-agent collaborative information, to ensure the global consistency of high-level policies.

[0107] Transformer Encoder: Employs a 2-layer Transformer encoder to capture global dependencies in contextual understanding vectors (such as the association between task objectives and collaborative requirements); compared to traditional MLPs, Transformer's attention mechanism can adaptively focus on key information (such as high-priority objectives and high-risk items), improving the accuracy of policy generation.

[0108] Gumbel-Softmax sampling solves the problem of differentiable sampling of discrete actions. The sampling result is a continuous approximation of the discrete action intention, which not only ensures the interpretability of the action, but also supports gradient backpropagation training.

[0109] Strategy Output : dimensional probability distribution Indicates selecting the first The probability of a high-level action intent is obtained by sampling with Gumbel-Softmax to obtain the specific action intent.

[0110] The context-awareness vector and the high-level action intent are concatenated and input into the multi-layer fully connected layer in the action network to obtain the low-level action, and then the low-level action is transformed into action instructions.

[0111] Specifically, the goal of low-level motion refinement is to generate continuous and precise specific motion commands (such as steering angle and acceleration for autonomous driving, and position and speed commands for drones) based on high-level motion intentions, as shown in Formula 29: (29) Formula 29 is the formula for generating low-level actions, and its technical details are as follows: Input concatenation The contextual understanding that integrates global information is concatenated with the high-level action intent to form an input vector, ensuring the consistency between low-level actions and high-level intent.

[0112] MLP Action Network: A 3-layer fully connected network, using LeakyReLU activation functions (first two layers) and... (Output layer) The function maps the action output to The interval is used to facilitate subsequent scaling based on physical constraints.

[0113] Action scaling and constraints: Low-level actions After linear scaling, it is converted into physically feasible motion commands. For example, the range of steering angle and acceleration must be specified to ensure the safety and feasibility of the maneuver.

[0114] In an optional embodiment of this application, the system further includes: The global objective function is obtained based on the immediate rewards from multiple individual agents and the collective cooperative rewards.

[0115] To maximize the global objective function and obtain ideal coordination and ideal action instructions.

[0116] The analysis and reasoning module and the decision execution module are trained and optimized with the ideal action instructions as the target.

[0117] Specifically, the core objective of the distributed collaborative decision function is to address the problems of "centralized reliance on global information and disconnect between local decisions and global goals" in traditional multi-agent collaboration. It simulates the distributed collaboration model of "individual autonomous decision-making + group goal alignment" in human society, maximizing collective intelligence through global goal decomposition and local optimization. This function includes two key components: global goal definition and collaborative quantification, as shown in Equations 30 and 31. The global objective function defines the overall optimization direction of the multi-agent system, ensuring both the effectiveness of individual decisions and strengthening the consistency of group collaboration. The core formula is shown in Figure 33. (30) Formula 30 is the global objective function, and its technical details are as follows: Expected calculation ( Based on environmental observation The expectation of the joint distribution calculation with the communication mechanism is approximated in engineering by Monte Carlo sampling: ,in The sampling batch size is (preferably 128~256). The objective function value for the b-th sample; Individual cumulative reward items : Total task time; As a discount factor, it balances immediate rewards with long-term rewards. The larger the size, the more the intelligent agent focuses on long-term collaborative benefits (such as the overall completion rate of cluster tasks). For intelligent agents The localized, immediate rewards employ a multi-dimensional weighted design: ,in Rewards for task completion (such as progress towards achieving the objective). Rewards are imposed to ensure safety (such as successful obstacle avoidance or sufficient resources). For efficiency rewards (such as motion energy consumption, time loss); Group collaboration reward items : To coordinate reward weights and balance individual gains with group collaboration, The larger the value, the more inclined the intelligent agent is to engage in cooperative behavior (such as resource sharing and task support). For intelligent agents and The degree of collaboration quantifies the closeness of cooperation between two agents, as shown in Formula 31.

[0118] (31) Formula 31 is the formula for co-quantification, and its technical details are as follows: Action consistency item : The Euclidean distance between the action commands of the two agents is smaller; the more similar the actions (such as drones flying in coordinated formation with the same speed), the smaller the distance. The cooperative distance parameter is adaptively set to... ,in This is the maximum Euclidean distance in the motion space; this setting ensures that the value range of the motion consistency term is [value missing]. The value is 1 when the actions are completely identical, and approaches 0 when they are completely opposite.

[0119] Target intersection : Indicator Function The value is 1 when the global collaborative information of the two agents intersects with the target; otherwise, it is 0. The target information in the analysis and reasoning module comes from the intent inference result; physical meaning: the cooperative reward for action consistency is calculated only when the targets of the two agents are related (such as both performing the "area reconnaissance" task), thus avoiding ineffective cooperation between unrelated agents.

[0120] The higher the value of the collaboration degree output, the stronger the collaboration between the two agents, and vice versa.

[0121] Within a single agent, each module functions independently yet works in close coordination, ensuring both the advanced cognitive capabilities of the single agent and providing a reliable basic unit for collaborative decision-making in multi-agent systems. This approach is suitable for decision-making tasks in complex and dynamic environments such as autonomous driving, drone swarms, and intelligent manufacturing. Among multiple agents, the collaborative mechanism simulates the collaborative cognitive and decision-making patterns of biological groups. This ensures the autonomous decision-making capabilities of individual agents while enabling the emergence of swarm intelligence through communication protocols and collaborative functions. This approach is suitable for distributed decision-making tasks in complex and dynamic environments such as autonomous driving fleet collaboration, drone swarm cooperation, and multi-device collaboration in intelligent manufacturing, demonstrating significant engineering application value.

[0122] In an optional embodiment of this application, the method for training the decision execution module separately includes: Based on the action instructions output by the current single agent and the expert demonstration action or ideal action instruction, a demonstration fitting term is obtained.

[0123] Based on the target priority weights preset in the task, the basic prior distribution is obtained.

[0124] The historical return coefficient is determined by selecting the average return of the target based on historical interaction data.

[0125] The target prior distribution is obtained by correcting the basic prior distribution using historical return coefficients.

[0126] Calculate the KL divergence based on the target posterior distribution and target prior distribution output by the analysis and reasoning module.

[0127] The cumulative loss of the decision execution module is calculated based on the exemplary fit term and the KL divergence.

[0128] The decision execution module is trained to minimize its cumulative loss until the cumulative loss converges, thus obtaining the parameters of the trained and optimized decision execution module.

[0129] Specifically, the training objective of the decision execution module is to minimize the cumulative loss function and achieve a balance between local decision optimization and global collaborative optimization, as shown in Equation 32: (32) Formula 32 is the decision loss function, and its technical details are as follows: Expert demonstration of fitting items Ensure that the action instructions conform to the experience of human experts. Provide expert demonstrations of actions (such as steering and acceleration in human driving, and optimal action sequences for collaborative tasks); through expert demonstration fitting, accelerate model training convergence and improve the safety and rationality of actions.

[0130] Target alignment item : Ensure that the target distribution inferred by the decision-making and analytical reasoning modules is consistent. To analyze the target posterior distribution output by the inference module, The prior distribution of the target corresponding to the decision action; Refers to the KL divergence meter. The weighting coefficients (preferably 0.1~0.3) are used to balance the priority of action fitting and target alignment. The target prior distribution is also considered. It employs a two-tiered progressive logic of "basic prior initialization → historical experience correction" to ensure that the distribution not only aligns with the task's essence but also dynamically adapts to the environment and its own state. The core of basic prior initialization is to transform the task's preset goal priorities into an initial probability distribution, ensuring that prior beliefs are consistent with the core requirements of the task and preventing deviation from the initial direction. Let the preset goal priority weights of the task be... (satisfy (If the prior distribution is set by the task designer based on business requirements), then the basic prior distribution is: .

[0131] The basic priors are fixed and need to be adjusted through the agent's historical interaction experience to make the prior distribution fit its own decision-making habits and task performance (for example, if an agent has a high success rate in "obstacle avoidance actions" in the past, the prior probability of the corresponding target should be increased). Define the target. Historical return coefficient : (33) in, This is an indicator function (1 if the condition is met, 0 otherwise). Characterizing the selection of targets in history The average return (the higher the return, the larger the coefficient).

[0132] Basic Priors Historical return coefficient By merging, we obtain the corrected prior distribution. : (34) Training optimization: The AdamW optimizer is used. During training, the "input-action-reward-next state" quadruple is stored using experience replay (ReplayBuffer), and batch sampling is performed for offline updates to improve training stability.

[0133] like Figure 2 As shown in the embodiments of this application, a brain-like structure-driven multi-agent hierarchical decision-making method is provided, including: In the current single intelligent body: S1: Extract environmental feature vectors based on real-time multimodal observation data obtained from the environment.

[0134] S2: Based on the environmental feature vector and the memory state of the previous time step, obtain the updated memory feature vector.

[0135] S3: Based on the environmental feature vector and the memory feature vector, generate the target posterior probability and decision risk vector, and combine the target posterior probability and decision risk vector to generate the context understanding vector.

[0136] S4: When the decision risk vector is greater than the decision risk threshold or the change in the context understanding vector of adjacent time steps is greater than the change amount threshold, communication between the current single agent and neighboring agents is triggered.

[0137] S5: Weighted fusion of the state representation vectors of multiple neighbor agents within the communication neighbor set to obtain global collaborative information. The state representation vectors include environmental feature vectors, environmental understanding vectors, and action instructions.

[0138] S6: Fuse the context understanding vector and global collaborative information to obtain the fused context understanding vector.

[0139] S7: Based on the fusion context, understand the vector and generate action instructions.

[0140] In this optional embodiment, a dynamic and efficient multi-agent collaborative decision-making mechanism is achieved in a communication-constrained environment by adaptively combining a decision risk threshold triggering mechanism with a context understanding vector change threshold triggering mechanism. Specifically, this method avoids the redundant overhead of fixed-period communication, triggering communication only when the decision risk is too high or the environmental context undergoes significant changes, ensuring timely sharing of key information. Simultaneously, global collaborative information is generated by weighted fusion of the state representation vectors of neighboring agents and fused with their own context understanding vectors, forming a more comprehensive fused context understanding vector, enabling the decision execution module to generate better action instructions. This design simulates the hierarchical information processing mechanism of biological intelligence, solving the problems of insufficient real-time response, lack of long-term dependency, and weak cross-task generalization ability caused by the flat structure in existing technologies, significantly improving the autonomous decision-making ability of agents in high-dimensional, highly dynamic, and multimodal environments. Through modular process design and an adaptive communication mechanism based on risk / context changes, agents can achieve efficient collaboration in distributed, communication-constrained scenarios, meeting the comprehensive needs of complex tasks for real-time response, long-term experience utilization, and cross-scenario generalization.

[0141] Implementation Scenario 1: Application Example of UAV Swarm Collaborative Reconnaissance Mission The study demonstrates a specific application in collaborative reconnaissance by a swarm of drones, verifying the effectiveness of the brain-like multi-agent hierarchical decision-making model in complex, dynamic, and communication-constrained environments.

[0142] A reconnaissance swarm consisting of 16 multi-rotor UAVs was deployed to conduct collaborative reconnaissance and obstacle avoidance missions in complex terrain areas with moving targets, partial obstructions, and unstable signals. During the mission, the UAVs needed to perform various advanced behaviors, including area coverage, real-time risk avoidance, and collaborative path adjustment. The individual UAVs could only maintain partial interconnectivity under limited communication conditions.

[0143] The brain-like four-module structure is deployed as follows: Each UAV is equipped with four functional modules: perception, memory, analysis, and decision-making. The multi-source perception module processes multimodal data from cameras and millimeter-wave radar, and obtains unified perception features through adaptive weight fusion; the memory module relies on short-term contextual memory and long-term knowledge memory to store and retrieve historical trajectory trends and key experiences; the analysis and reasoning module uses Bayesian inference to output target intent and risk level; and the decision-making and execution module generates UAV speed, heading, and altitude adjustment commands through "high-level intent planning + low-level action refinement".

[0144] Sparse communication mechanism and collaborative strategy execution: When a UAV detects a high-risk target or a sudden change in the situation (such as sudden occlusion or high-speed approach to the target), its communication trigger module is automatically activated and broadcasts the compressed and encoded status to neighboring and task-related UAVs; neighbor selection is dynamically adjusted according to communication quality and task relevance to ensure that it only collaborates with effective nodes.

[0145] Collaborative reconnaissance and dynamic obstacle avoidance: The UAV determines the collaborative reconnaissance area allocation based on the output of the analysis and reasoning module, maintains the consistency of the formation structure at high levels, and autonomously avoids moving obstacles at low levels; when encountering local congestion, it adjusts its own speed and direction through the collaborative metric function, thereby improving the reconnaissance coverage and path consistency.

[0146] Through brain-like structures and collaborative mechanisms, drone swarms exhibit significantly better stability and coordination than traditional methods in complex environments: (1) In situations with complex terrain and strong target dynamics, the reconnaissance area coverage is increased by about 22% compared with the traditional CTDE strategy network, and the coverage distribution is more uniform; (2) The real-time avoidance success rate for high-speed moving targets is improved by about 39%, and the action delay in sudden risk scenarios is significantly reduced; (3) The communication load is only about one-third of that of the traditional all-time communication solution, and the communication bandwidth usage is significantly reduced in actual tasks; (4) The structural offset of the group formation in the interference environment is reduced by about 70%, showing more stable and coordinated group behavior.

[0147] In summary, the system architecture of this invention enables UAV swarms to have stronger environmental adaptability, evasion capabilities, and collaborative intelligence in dynamic reconnaissance missions.

[0148] Implementation Scenario 2: Application Example of Multi-AGV Collaborative Handling Tasks in Smart Manufacturing Workshops The invention demonstrates its application in collaborative tasks involving multiple autonomous handling robots (AGVs), effectively verifying its ability to improve equipment scheduling efficiency and system stability in industrial environments such as production workshops.

[0149] In a smart manufacturing workshop, 12 AGVs are deployed to perform material handling tasks between production workstations and buffer areas. The workshop presents uncertainties such as worker movement, forklift movement, and path intersections causing congestion. The AGVs are required to avoid congestion while maintaining overall transportation efficiency.

[0150] The four-module intelligent agent application: Each AGV has a built-in multi-source perception module that processes LiDAR point clouds and visual semantic information to form a unified 512-dimensional environmental feature; the memory module records recent congestion conditions and historical task experience; the analysis and reasoning module infers the optimal intention in the current situation, such as deceleration, avoidance, path switching, etc.; the decision execution module generates smooth trajectories and action instructions based on the intention, and meets physical constraints such as workshop speed and safe distance.

[0151] On-demand communication and collaborative scheduling: AGV communication is triggered by "congestion changes." When a path becomes significantly congested, the corresponding AGV automatically broadcasts low-dimensional status information to neighboring, task-related nodes. This information is used by neighboring AGVs for path selection and speed adjustment, thus preventing all robots from blocking each other. The neighbor set is dynamically updated every few time steps based on communication quality and task relevance, ensuring collaborative efficiency.

[0152] Distributed collaborative execution: At the high-level decision-making level, AGVs determine strategies such as "path change" or "keep moving forward" based on contextual understanding, while at the low-level execution, they smoothly avoid obstacles and adjust speed. During peak task periods, multiple AGVs automatically achieve yielding, queuing, and alternating passage in congested areas through coordination calculations, ensuring a stable and continuous logistics rhythm.

[0153] Significantly improved the efficiency and safety of multi-AGV collaboration in smart manufacturing environments: (1) Under the condition of multiple obstacles in the workshop, the average task completion time is reduced by about 27%, and the overall transportation rhythm is significantly accelerated; (2) In high-congestion paths, the queuing length of AGVs is reduced from about 3–4 vehicles in the traditional scheduling method to about 1 vehicle, and the congestion dissipation speed is significantly improved. (3) Because the analysis and reasoning module has the ability to predict risks, collision or emergency braking events are greatly reduced, with the occurrence rate decreasing by more than 80% per unit time. (4) Thanks to more reasonable path selection and smooth motion planning, the energy consumption of a single AGV is reduced by about 14%, and the overall energy efficiency of the workshop is significantly improved.

[0154] Through this invention, a multi-AGV system can maintain high efficiency, safety and stability in unstructured, dynamically changing workshop environments, meeting the needs of high-load production scenarios.

[0155] This application, through a triple design of brain-like structure, hierarchical architecture, and distributed collaborative mechanism, significantly improves the cognitive ability, collaborative efficiency, and decision-making performance of multi-agent systems in complex dynamic environments, and has the following beneficial effects: 1. Significantly improved decision-making accuracy and enhanced stability in complex scenarios. Because the four modules of perception, memory, analysis, and decision-making have clearly defined functions, and information is processed step by step in a hierarchical structure, the accuracy of action decision-making in multimodal environments is improved by an average of 15% to 35% compared with traditional end-to-end DRL. In high-risk scenarios such as high-speed dynamics and dense obstacles, the success rate of tasks is improved by more than 20%, and the ability to avoid conflicts is significantly enhanced.

[0156] 2. Possesses enhanced experience reuse capabilities, with significantly improved long-term reliance and cross-task transfer capabilities. The dual-memory system enables agents to recall historical experience and perform contextualized matching in new tasks. Experiments show that in similar task transfer scenarios, training convergence speed is improved by 40% to 60%; in scenarios with randomly changing tasks, policy stability is improved by more than 30%; and for tasks with strong long-term temporal dependencies (such as cooperative trajectory maintenance and predictive obstacle avoidance), decision consistency is improved by 25%.

[0157] 3. Communication volume is reduced by 60% to 80%, significantly improving collaboration efficiency and scalability. Through a brain-inspired heuristic on-demand communication mechanism, low-dimensional compression of ontology state, and dynamic neighbor selection: the communication volume per unit time of multi-agent systems is reduced by 60% to 80%; the collaborative task retention rate in communication congestion scenarios is improved by 25% to 40%; when the number of agents is expanded from 10 to 200, the system performance degradation remains within 10%, and the scalability is significantly better than traditional centralized or global communication methods.

[0158] 4. Enhanced consensus in collaborative decision-making leads to the natural emergence of swarm intelligence behaviors. The collaborative metric model enables agents to achieve distributed consensus decision-making without relying on centralized computing power. Experimental results show that the success rate of multi-agent collaborative tasks is improved by more than 30%; in dynamic task division scenarios (such as the switching between reconnaissance-tracking-avoidance tasks in a drone swarm), the group response latency is reduced by more than 35%; and the system maintains stable group behavior even with a 20% local communication interruption.

[0159] 5. The interpretability of the decision-making process is greatly enhanced, and security and engineering controllability are improved. Because the four-module structure has clear functional boundaries, the decision-making link can be traced to "observation → memory → contextual understanding → action", and its interpretability is significantly better than that of the traditional black box model.

[0160] This makes the system more reliable in the following engineering scenarios: autonomous driving, drone swarms, intelligent manufacturing, emergency rescue and other tasks with extremely high safety requirements. It can pinpoint erroneous decisions to specific modules, improving verification and debugging efficiency by more than 50%.

[0161] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0162] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A brain-like structure-driven multi-agent hierarchical decision-making system, characterized in that, It includes multiple single agents, each of which includes a multi-source perception module, a memory module, an analysis and reasoning module, and a decision execution module; Within the current single intelligent entity, the multi-source perception module is used to extract environmental feature vectors based on multimodal real-time observation data acquired from the environment; the memory module is used to obtain updated memory feature vectors based on the environmental feature vectors and the memory state of the previous moment; and the analysis and reasoning module is used to generate target posterior probability and decision risk vectors based on the environmental feature vectors and memory feature vectors, and to generate context understanding vectors by combining the target posterior probability and decision risk vectors. When the decision risk vector is greater than the decision risk threshold or the change in the context understanding vector at adjacent time steps is greater than the change amount threshold, communication between the current single agent and its neighboring agents is triggered. The state representation vectors of multiple neighboring agents within the current single agent's communication neighbor set are weighted and fused to obtain global collaborative information. The context understanding vector generated by the agent's own analysis and reasoning module is fused with the global collaborative information to obtain a fused context understanding vector. The state representation vector includes environmental feature vectors, environmental understanding vectors, and action instructions. The decision-making and execution module in a single intelligent body is used to generate action instructions based on the understanding of vectors in the fused context.

2. The brain-like structure-driven multi-agent hierarchical decision-making system as described in claim 1, characterized in that, The extraction of environmental feature vectors based on real-time multimodal observation data obtained from the environment includes: Image features are obtained by performing convolution and global average pooling on the image data; After the point cloud data is encoded in terms of location, it is then processed by a multilayer perceptron and global max pooling to obtain the point cloud features. Image features and point cloud features are weighted and fused to obtain fused features, and then linear dimensionality reduction is performed to obtain environmental feature vectors. Cross-entropy loss is calculated based on the credibility of the fused features and the effectiveness of the training samples. The modal weights of the weighted fusion are the weights that minimize the cross-entropy loss.

3. The brain-like structure-driven multi-agent hierarchical decision-making system as described in claim 1, characterized in that, The process of obtaining the updated memory feature vector based on the environmental feature vector and the memory state at the previous time step includes: The environmental feature vector and the memory state of the previous time step are processed through a gating mechanism to obtain the historical memory retention weight; The environmental feature vector and the memory state of the previous time step are processed through an update gate to obtain the update strength of the new memory; By utilizing historical memory retention weights, the environmental feature vector and the memory state of the previous time step are fused, and candidate memory features are obtained through linear transformation and function activation. By leveraging the updating intensity of new memories, candidate memory features are fused with the memory state of the previous moment to obtain more detailed short-term memories; If the immediate reward of the current decision is greater than the reward threshold, the environmental feature vector, situational understanding vector, action instruction and immediate reward are combined into a set of long-term memory data and saved to the long-term knowledge memory bank. Extract a predetermined number of long-term memory data that are most relevant to the current environment feature vector from the long-term knowledge memory base. Then, weightedly fuse the contextual understanding vector in the long-term memory data with the short-term finer memory to obtain the current contextual understanding vector.

4. The brain-like structure-driven multi-agent hierarchical decision-making system as described in claim 1, characterized in that, The step of generating a target posterior probability and a decision risk vector based on environmental feature vectors and memory feature vectors, and then combining the target posterior probability and the decision risk vector to generate a contextual understanding vector includes: The environmental feature vector and the memory feature vector are processed separately by a perceptron and then fused to obtain the analysis input vector; The analysis input vector is fed into a Bayesian neural network to obtain the target posterior probability. The analysis input vector is fed into a two-layer perceptron of the risk assessment network to obtain the decision risk vector; The target expectation and decision risk vector are fused together to obtain the context understanding vector.

5. The brain-like structure-driven multi-agent hierarchical decision-making system as described in claim 1, characterized in that, The filtering method for the communication neighbor set includes: Encode the current state representation vector of the single agent into a low-dimensional vector and broadcast it to the surroundings; Based on the current location information of a single agent and the coverage area of ​​its communication signals, an initial set of neighbors is obtained; The communication link quality is obtained based on the packet loss rate, communication distance, and average communication distance between the current single agent and the candidate neighbor single agents in the initial neighbor set during the communication process. Based on the task objective vector currently being executed by a single agent and the task objective vectors of candidate neighbor single agents in the initial neighbor set, the task relevance is obtained; When the communication link quality is greater than or equal to the link quality threshold, or the task relevance is greater than or equal to the relevance threshold, the candidate neighbor single agent is retained; otherwise, the candidate neighbor single agent is removed from the initial neighbor set; thus obtaining the communication neighbor set.

6. The brain-like structure-driven multi-agent hierarchical decision-making system as described in claim 5, characterized in that, The process involves weighted fusion of the state representation vectors of multiple neighboring agents within the current single agent's communication neighbor set to obtain global collaborative information. Then, the fused context understanding vector is obtained by fusing the context understanding vector generated by the agent's own analysis and reasoning module with the global collaborative information. This fused context understanding vector includes: Based on the communication link quality and task relevance between the current single agent and all neighboring agents, determine the information weight corresponding to each neighboring agent; The state representation vectors of all neighboring agents are weighted and fused according to the information weights corresponding to each neighboring agent to obtain global collaborative information. After processing the global collaborative information using the Sigmoid activation function, the vector is concatenated with the context understanding vector generated by the current single-individual-body analysis and reasoning module to obtain the fused context understanding vector.

7. The brain-like structure-driven multi-agent hierarchical decision-making system as described in claim 1, characterized in that, The generation of action instructions based on the fused contextual understanding vector includes: The fused contextual understanding vector is input into the Transformer encoder, and the encoder output is sampled using Gumbel-Softmax to obtain the probability of the high-level action intent and determine the high-level action intent. The context-awareness vector and the high-level action intent are concatenated and input into the multi-layer fully connected layer in the action network to obtain the low-level action, and then the low-level action is transformed into action instructions.

8. The brain-like structure-driven multi-agent hierarchical decision-making system as described in claim 1, characterized in that, The system also includes: The global objective function is obtained based on the immediate rewards from multiple individual agents and the collaborative rewards from the group. To maximize the global objective function and obtain ideal coordination and ideal action instructions; The analysis and reasoning module and the decision execution module are trained and optimized with the ideal action instructions as the target.

9. The brain-like structure-driven multi-agent hierarchical decision-making system as described in claim 8, characterized in that, The method for training the decision execution module separately includes: Based on the action instructions output by the current single agent and the expert-demonstrated action or ideal action instructions, a demonstration fitting term is obtained; Based on the pre-defined target priority weights of the task, the basic prior distribution is obtained; Determine the historical return coefficient based on the average return of the selected target according to historical interaction data; The target prior distribution is obtained by correcting the basic prior distribution using historical return coefficients. Calculate the KL divergence based on the target posterior distribution and target prior distribution output by the analysis and reasoning module; The cumulative loss of the decision execution module is calculated based on the exemplary fit term and the KL divergence. The decision execution module is trained to minimize its cumulative loss until the cumulative loss converges, thus obtaining the parameters of the trained and optimized decision execution module.

10. A brain-like structure-driven multi-agent hierarchical decision-making method, characterized in that, include: In the current single intelligent body: Environmental feature vectors are extracted based on real-time multimodal observation data acquired from the environment. Based on the environmental feature vector and the memory state of the previous time step, the updated memory feature vector is obtained; Based on the environmental feature vector and the memory feature vector, the target posterior probability and decision risk vector are generated. Combining the target posterior probability and decision risk vector, the context understanding vector is generated. When the decision risk vector is greater than the decision risk threshold or the change in the context understanding vector at adjacent time steps is greater than the change amount threshold, communication between the current single agent and its neighboring agents is triggered. The state representation vectors of multiple neighbor agents within the communication neighbor set are weighted and fused to obtain global collaborative information. The state representation vectors include environmental feature vectors, environmental understanding vectors, and action instructions. The context understanding vector and global collaborative information are fused to obtain the fused context understanding vector; Based on the contextual understanding of vectors, action instructions are generated.

Images