Unmanned system brain-like intelligent decision-making method based on triple learning mechanism
Through the unmanned system brain-like intelligent decision-making method with triple learning mechanism, the decision adaptability and coordination efficiency of the unmanned system in complex environments is solved, efficient strategy migration and task coordination are achieved, and the perceived response and task execution capabilities of the unmanned system are improved.
Patent Information
- Application Number
- CN202510569002.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-30
- Publication Date
- 2025-08-08
AI Technical Summary
When faced with complex and changing environments, existing unmanned system decision-making methods have poor environmental adaptability, low coordination efficiency and weak knowledge transfer capabilities, making it difficult to achieve efficient coordination of multitasking and rapid adaptation of strategy models.
Adopting an intelligent decision-making method of unmanned systems based on triple learning mechanisms, integrating physically enhanced feedback learning, task-driven collaborative learning and directional distillation teaching learning, through the construction of multi-dimensional state space and dynamic artificial potential field models, combined with federal attention mechanisms and characteristic sensitivity-driven knowledge distillation methods, we realize adaptive modeling of environmental disturbances and obstacles and efficient transfer of strategic parameters.
It significantly improves the decision-making flexibility, coordination stability and resource scheduling efficiency of unmanned systems in multi-task scenarios, and enhances the perceived response capabilities and task execution efficiency in complex dynamic environments.
Smart Images

Figure CN120447575A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a brain-like intelligent decision-making method for an unmanned system based on a triple learning mechanism, and belongs to the technical field of unmanned system decision-making. Background Art
[0002] In recent years, the rapid development of artificial intelligence and robotics has made unmanned systems an integral part of modern technology. Significant improvements in information processing speed and mechanical control capabilities have enabled drones to play a vital role in numerous fields, particularly in scenarios such as swarm warfare, flexible manufacturing, and emergency rescue. As drone technology continues to mature, it has transcended traditional operational models, enabling it to accomplish complex tasks through the collaboration of multiple sensors and control systems. In emergency rescue and disaster response, drones, with their rapid response and high flexibility, have become a crucial tool. However, in practical applications, faced with changing mission requirements and complex environmental conditions, unmanned system decision-making still faces numerous challenges, including uncertainty in environmental information, diverse and conflicting mission objectives, inefficient multi-agent collaboration, limited computing resources on terminal devices, and insufficient transferability and generalization of policy models to new tasks or environments.
[0003] Current unmanned system decision-making methods primarily focus on end-to-end policy learning and rely on large amounts of interactive data for training. While these methods possess a certain degree of autonomy, they generally lack modeling of physical environmental characteristics, making them difficult to adapt to dynamic disturbances and complex scenarios. Furthermore, existing methods ignore the semantic structure and priority differences between tasks, making it difficult to achieve efficient multi-task coordination. In multi-agent collaboration, strategies are highly homogenized, lacking modeling of node heterogeneity and task relevance, resulting in low overall collaborative efficiency. Furthermore, the knowledge transfer process generally lacks the identification and guidance of key features, making it difficult to achieve rapid model adaptation and lightweight deployment. Summary of the Invention
[0004] Purpose of the invention: In view of the shortcomings of existing unmanned system decision-making methods in terms of poor environmental adaptability, low collaborative efficiency and weak knowledge transfer capabilities, a brain-like intelligent decision-making method for unmanned systems based on a triple learning mechanism is proposed to improve the task response capability and intelligent collaboration level in complex and changing environments. The present invention integrates three mechanisms: physically enhanced feedback learning, task-driven collaborative learning and directional distillation teaching learning. By constructing a multi-dimensional state space and a dynamic artificial potential field model, it realizes adaptive modeling and response to environmental disturbances and obstacles; further, it constructs semantic task representations based on task type, priority and risk index, and combines the federated attention mechanism to dynamically aggregate heterogeneous strategies to improve task collaboration and information sharing capabilities among multiple agents; at the same time, it proposes a feature sensitivity-driven knowledge distillation method to achieve efficient knowledge transfer and strategy alignment from large models to lightweight end models. Through the system fusion design of brain-like intelligence, the present invention significantly enhances the decision-making flexibility, collaborative stability and resource scheduling efficiency of unmanned systems in multi-task scenarios.
[0005] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:
[0006] A brain-like intelligent decision-making method for unmanned systems based on a triple learning mechanism includes the following steps:
[0007] Step 1: Obtain the drone's state, environmental perception characteristics, and mission constraint information, map them into a multi-dimensional state space, and simultaneously define a continuous action space to establish a system model for autonomous decision-making.
[0008] Step 2: Construct a dynamic artificial potential field model composed of gravity, repulsion, and wind field disturbances to generate a resultant force vector. Design a reward function that integrates path efficiency and potential field gradient. This system model guides the drone's autonomous obstacle avoidance and path optimization through autonomous decision-making.
[0009] Step 3: Based on the gravity, repulsion, and curl characteristics of the dynamic artificial potential field, a potential field guidance signal is constructed. Combined with the drone's proprioception data, a dual-branch attention structure of environmental attention and potential field attention is constructed in the local reinforcement learning policy network, and local policy learning is implemented based on the improved APF-PPO algorithm.
[0010] Step 4: Construct a cross-modal representation by fusing potential fields and states, generate similarity weights based on task semantics, and implement federated weighted aggregation and heterogeneous updates of policy parameters.
[0011] Step 5: Construct the knowledge element weight matrix of the teacher model by constructing the feature component activation strength and state attention weight of the teacher model; and construct the knowledge element weight matrix of the student model through backpropagation calculation and normalization processing;
[0012] Step 6: Construct the knowledge element weight matrix of the teacher and student models in the end-cloud collaboration, build the distillation loss function based on feature alignment, soft target distillation, and hard target constraints, and design a targeted knowledge transfer and temperature regulation structure that adapts to dynamic scenarios in the end-cloud collaborative structure.
[0013] Preferably: the artificial potential field model in step 2 includes a gravitational field, a repulsive field and a wind field disturbance, wherein the gravitational field function U att (q) is:
[0014]
[0015] Where ξ>0 is the gravitational gain coefficient; q goal is the coordinate of the target point; κ>0 is the rotation adjustment factor, which generates a tangential component around the target point to break the symmetric balance; the corresponding gravitational formula is derived from the gravitational field function:
[0016] The repulsive field function U att (q) is:
[0017]
[0018] Among them, d0 is the maximum distance detected by the UAV; ||qq obs || n is the obstacle distance coupling term; at the same time, the corresponding gravitational gradient
[0019] The environmental wind force is added to the construction of the artificial potential field:
[0020]
[0021] Among them, ||qq goal || / ||q start -q goal || represents the current position of the drone q to the target point q goal The distance and q start To the target point q goal The ratio of the distances; is the wind speed vector; Δθ w The angle between the wind direction and the direction of the UAV's movement; therefore, the potential field gradient The net force acting on the drone is:
[0022]
[0023] Finally, the resultant force vector is obtained. The magnitude of the vector is used as the motion amplitude of the drone, and the angle between the vector and the direction of the drone's previous motion is used as the rotation angle of the drone.
[0024] The artificial potential field reward function of the drone is defined as follows:
[0025] r=ξ·U att (q t ,q goal )-η·∑U rep (q t ,q obs )-α·Δt+β·Δd
[0026] Where Δt is the time consumption, path length, Δd=d(q t -q start )-d(q t -q goal ), that is, the reduction of path length, encourages drones to fly in straight lines and avoid obstacles as much as possible.
[0027] Preferred: In step 3, the local reinforcement learning strategy is constructed using the potential field enhanced APF-PPO algorithm. The enhanced state vector is constructed by fusing the potential field feature extraction with the drone body state information as the input of the strategy network:
[0028]
[0029] Among them, W s and W h is the weight matrix of the state vector and the potential field eigenvector; The high-order physical feature encoding is achieved through the potential field feature extraction module, specifically:
[0030]
[0031] Among them, the potential field force amplitude characteristic ||F att ||,||F rep ||;∠F att ,∠F rep is the direction characteristic of the potential field force; is the curl characteristic of the potential field;
[0032] The environmental attention branch takes the real-time posture state of the unmanned system and the lidar point cloud data as input, encodes the real-time posture state of the drone into a query vector Q, extracts multi-scale geometric features of the lidar point cloud data through the PointNet++ network, and generates a key vector K and a value vector V; the query vector Q and the key vector K are connected through the projection matrix Mapped to a shared 64-dimensional feature space, generating the environment attention weight matrix:
[0033]
[0034] Among them, d k =64 is the feature dimension of the key vector, which is used to balance the computational efficiency and the expressive power of the attention weight; Matrix parameters for projection of value vectors;
[0035] The potential field attention will be the real-time potential field artificial gradient As input features, and through the projection matrix Mapped to a 64-dimensional feature space, generating the potential field attention weight matrix:
[0036]
[0037] Adjust the fusion weight of the two branches according to the exponential decay in the training stage λ(t) = e -0.003t , t is the number of training steps, and the final action probability distribution of the policy network is synthesized by the dynamic weighting of the two-branch attention results:
[0038]
[0039] The policy gradient loss after trimming is calculated based on the proximal policy optimization PPO algorithm. The formula is:
[0040]
[0041] Among them, the strategy ratio is the advantage function, clip(·) is the clipping function, ε is the clipping threshold, is the strategy alignment term, π APF (a t ) Reference action probability distribution generated by the artificial potential field.
[0042] Preferred: In step 4, based on the improved artificial potential field model, the drone uses onboard sensors to analyze the potential field gradient and rotation characteristics in real time:
[0043]
[0044] Physical potential field characteristics Jointly encoded by gradient and rotation:
[0045]
[0046] The UAV's state information and potential field feature extraction fusion vector are used to achieve cross-modal feature fusion by constructing a lightweight multi-layer perceptron:
[0047]
[0048] The above method combines task type, priority and risk index to transform the node's local task and execution environment into a structured semantic vector:
[0049] T (i) =Encoder(T goal ,T priority ,T risk )
[0050] in, is the task type unique hot encoding, Tpriority∈[0,1] represents the task priority coefficient, T risk ∈[0,1] is the environmental risk index; therefore, the local task encoding T (i) With the global target T g The semantic relevance of is expressed as:
[0051]
[0052] Among them, task similarity φ (i) ∈[-1,1], this indicator quantifies the contribution potential of UAV i to the global task. The larger its value, the higher the potential for collaboration between the local task of node i and the global goal.
[0053] The policy parameters uploaded by each end-side node i after the local task training is completed The server uses a task-similarity weighted approach to perform heterogeneous parameter fusion, defining the federation aggregation weight as α (i) , satisfying Σ i α (i) =1, which is used to measure the degree of correlation between each local model and the current global task; the global strategy parameter update formula obtained by aggregation is as follows:
[0054]
[0055] in, represents the global model parameters after the tth round update; Represents the global model parameters of the previous round; μ is the historical forgetting factor, which ranges from [0,1] and is used to balance the residual influence of the parameters of the previous round in the current round update;
[0056] The local model update formula is:
[0057]
[0058] Where β is the fusion rate coefficient; λ (i) is the dynamic fusion weight.
[0059] Preferably, step 5 includes the following steps:
[0060] First, a large cloud-side model is constructed as the knowledge source, integrating task feedback information from multiple terminals and environmental characteristics. The knowledge element weight matrix of the cloud-side teacher model is defined as follows:
[0061]
[0062] It includes two parts: feature component activation intensity and state attention weights The strength of feature activation is calculated by normalizing the L2 norm to quantify the relative importance of each component:
[0063]
[0064] in, represents the state representation vector of the teacher network layer l, k is the index of the feature component, and the L2 norm of the feature value is calculated and normalized to the range of [0,1];
[0065] The state attention weight represents the decision criticality of a specific component in the state representation vector, which is quantified by calculating the ratio of the L2 norm of each component to the sum of the global L2 norm and based on the policy gradient Dynamically adjust the weights; specifically, the state representation vector of the teacher network The state attention weight of its k-th component is defined as:
[0066]
[0067] This weight combines the static strength and dynamic sensitivity of the state component, guiding the student network to focus on imitating the task benefits. Key status characteristics with significant impact;
[0068] Similar to the feature weight modeling of the teacher network, the knowledge weight of the student network is dynamically adjusted by its policy sensitivity to the task loss; specifically, the loss function L is calculated by backpropagation. task The impact of , measuring its importance in strategy optimization:
[0069]
[0070] in, Represents the kth component of the state representation vector of the mth layer of the student network.
[0071] Preferably, step 6 includes the following steps:
[0072] First, based on the knowledge factor weight matrix of the teacher model and the student model defined in step 5, calculate the loss of the knowledge factor:
[0073]
[0074] Where Φ is the mapping set between the teacher layer l and the student layer m; D lis the dimension of the feature vector of the teacher network layer l; ψ(k) is the feature dimension projection function, which maps the high-dimensional feature index k of the teacher network to the corresponding dimension of the student network through linear transformation;
[0075] Feature consistency loss term:
[0076]
[0077] The total knowledge element alignment loss function is designed to combine the two loss terms of feature importance weight alignment and feature space alignment, dynamically adjust the importance of the two parts through the weight parameter ζ, and optimize the performance of the student model during the distillation process:
[0078] L KI =ζ·L align +(1-ζ)·L feat
[0079] A hierarchical adaptive temperature adjustment mechanism is proposed for the calculation of directed knowledge distillation loss under feature activation constraints. First, a state component difference sensitivity factor is constructed based on the policy gradient sensitivity and feature alignment difference to quantify the response deviation between the teacher model and the student model on key decision features. It is specifically defined as:
[0080]
[0081] Among them, σ(·) is the standard deviation operation, which is used to dynamically adjust the weight alignment strength; is the gradient amplitude of the teacher model policy network with respect to the state component k; ε is a numerical stability term; secondly, to consider the differences in the need for temperature regulation at different stages of training, a process-aware temperature basis function is designed:
[0082]
[0083] where τ max and τ min are the upper and lower bounds of temperature, k′ is the control annealing rate, t is the current training step number, T max is the maximum number of training steps; we further introduce the state component difference driving term and policy gradient sensitivity, dynamically adjust the temperature parameter through the task-aware modulation mechanism, and finally obtain the dynamic temperature expression for decision optimization:
[0084]
[0085] Where λ is the modulation intensity factor;
[0086] The output of the teacher and student models are defined as:
[0087] p T (a)=softmax(zT / τ (l,m) (t))
[0088] p S (a)=softmax(z S / τ (l,m) (t))
[0089] Among them, z T and z S are the output vectors of the teacher and student models respectively;
[0090] The dynamic temperature parameter is explicitly introduced into the distillation loss, and the soft target loss function is designed in the form of KL divergence under dynamic temperature:
[0091]
[0092] The hard target decision constraint ensures that the student model's decision in the task is consistent with the teacher model, thus ensuring its stability and reliability; the specific hard target loss is as follows:
[0093]
[0094] Among them, y a is the one-hot encoding of the reference action based on potential field planning; Var(·) is the variance of the action probability, and A is the set of all actions of the drone;
[0095] The comprehensive loss function is expressed by the following formula:
[0096] L total =L KI +L soft +L hard
[0097] Total loss function L total It is to combine various loss terms together to ensure the coordinated optimization of the student model in feature learning, semantic transfer and task decision-making.
[0098] Preferably, the state space in step 1 includes the geometric position and motion state of the drone and the artificial potential field characteristics, and an eight-dimensional state vector is constructed:
[0099]
[0100] Where x and y represent the Cartesian coordinates of the drone in the two-dimensional horizontal plane, and the real-time position of the drone is q = (x, y); v x , v y is the velocity component of the UAV in the x and y axis directions; θ is the heading angle, which represents the angle between the longitudinal axis of the body and the positive x direction; F att and F repare the gravitational and repulsive moduli calculated by the artificial potential field model; d goal =||q t -q goal ||2 represents the Euclidean distance from the current point to the target point.
[0101] Preferably, the action space in step 1 is a two-dimensional continuous action space, defined as follows:
[0102]
[0103] Among them, a T ∈[0,1] is the normalized main thrust, mapped to the actual thrust range [0,F max ], F max is the maximum thrust of the UAV; Δθ∈[-15°,15°] is the heading angle increment.
[0104] A brain-like intelligent decision-making system for unmanned systems based on a triple learning mechanism is used to implement a brain-like intelligent decision-making method for unmanned systems based on a triple learning mechanism, including an autonomous decision-making unit, a path optimization unit, a local strategy unit, a federated aggregation and heterogeneous update unit, a knowledge element weight unit, and an adjustment unit, wherein:
[0105] The autonomous decision-making unit is used to obtain the drone's body state, environmental perception characteristics and mission constraint information, map them into a multi-dimensional state space and simultaneously define a continuous action space, and establish a system model for the drone's autonomous decision-making;
[0106] The path optimization unit is used to construct a dynamic artificial potential field model composed of gravity, repulsion, and wind field disturbances, generate a resultant force vector, and design a reward function that integrates path efficiency and potential field gradient. This is then used to guide the drone's autonomous obstacle avoidance and path optimization through a system model of autonomous decision-making.
[0107] The local policy unit is used to construct a potential field guidance signal based on the gravity, repulsion and curl characteristics in the dynamic artificial potential field, and to build a dual-branch attention structure of environmental attention and potential field attention in the local reinforcement learning policy network in combination with the drone's proprioception data, and to implement local policy learning based on the improved APF-PPO algorithm;
[0108] The federated aggregation and heterogeneous update unit is used to construct a cross-modal expression by fusing potential fields and states, generate similarity weights based on task semantics, and implement federated weighted aggregation and heterogeneous update of policy parameters;
[0109] The knowledge element weight unit is used to construct the knowledge element weight matrix of the teacher model by constructing the feature component activation strength and state attention weight of the teacher model; and to construct the knowledge element weight matrix of the student model through back propagation calculation and normalization processing;
[0110] The adjustment unit is used to construct the knowledge element weight matrix of the teacher and student models of the end-cloud collaboration, construct the distillation loss function based on feature alignment, soft target distillation and hard target constraints, and design a directed knowledge transfer and temperature adjustment structure that adapts to dynamic scenarios in the end-cloud collaborative structure.
[0111] Compared with the prior art, the present invention has the following beneficial effects:
[0112] 1. This invention constructs a multidimensional state space that couples physical potential field characteristics and state information, and combines it with a dynamically adjustable artificial potential field model to guide the unmanned system's adaptive obstacle avoidance and path planning, effectively improving the perception and response capabilities and task execution efficiency in complex dynamic environments. It also introduces a reward mechanism based on path efficiency and potential field gradient, enhancing the physical interpretability and stability of strategy optimization.
[0113] 2. This paper proposes a dual-branch strategy network structure that integrates environmental attention and potential field attention, combined with an improved APF-PPO algorithm, to achieve joint modeling and strategy enhancement of key environmental and task features in dynamic scenarios, significantly improving the robustness and generalization ability of local reinforcement learning.
[0114] 3. This invention introduces a federated aggregation mechanism based on task semantic similarity to dynamically adjust the policy parameter contributions of edge nodes, enhancing the targetedness and coordination of policy synchronization. Furthermore, it constructs a knowledge distillation weight modeling method based on feature sensitivity to achieve efficient knowledge transfer and policy alignment between the teacher and student models. This improves the decision-making performance of edge devices under resource-constrained conditions and provides effective support for collaborative decision-making in unmanned system clusters in multi-task heterogeneous environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0115] Figure 1 This is a block diagram of the brain-like intelligent decision-making method for unmanned systems based on the triple learning mechanism of the present invention.
[0116] Figure 2 This is a comparison chart of the improved reinforcement learning algorithm of the present invention and other reinforcement learning algorithms.
[0117] Figure 3 This is a comparison chart of the task completion time of the federal attention fusion mechanism and the federal averaging mechanism of the present invention.
[0118] Figure 4 This is a comparison chart of the navigation success rate based on improved knowledge distillation in the present invention. DETAILED DESCRIPTION
[0119] The present invention is further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.
[0120] This embodiment provides a brain-like intelligent decision-making method for unmanned systems based on a triple learning mechanism, which obtains the drone's body state, environmental perception characteristics and task constraint information, constructs a multi-dimensional state space and defines a continuous action space, and establishes an autonomous decision-making system model; constructs a dynamic artificial potential field model that integrates gravity, repulsion and wind field disturbances to generate a physical resultant force vector, and designs a reward function that couples path efficiency with potential field gradient to guide the drone's adaptive obstacle avoidance and path planning; designs a dual-branch strategy network that integrates environmental attention and potential field attention, and implements local strategy learning based on the improved APF-PPO algorithm; combines task type, priority and risk index to construct a task semantic expression and similarity measurement mechanism to achieve federated weighted aggregation and heterogeneous update of strategy parameters; based on the feature sensitivity and strategy gradient of the teacher model and the student model, constructs a knowledge element weight matrix to quantify the key features of the task; designs a directional knowledge distillation strategy for the end-cloud collaborative structure, aligns feature responses through a dynamic temperature regulation mechanism, and realizes efficient migration and strategy inheritance of lightweight models in complex dynamic scenarios, such as Figure 1 As shown, the specific steps include:
[0121] Step 1: Obtain the drone's state, environmental perception characteristics, and mission constraint information, map them into a multi-dimensional state space, and simultaneously define a continuous action space to establish a system model for autonomous decision-making.
[0122] The state space includes the geometric position and motion state of the UAV and the fusion of the artificial potential field characteristics to construct an eight-dimensional state vector:
[0123]
[0124] Where x and y represent the Cartesian coordinates of the drone in the two-dimensional horizontal plane, and the real-time position of the drone is q = (x, y); v x , v y is the velocity component of the UAV in the x and y axis directions; θ is the heading angle, which represents the angle between the longitudinal axis of the body and the positive x direction; F att and F rep are the gravitational and repulsive moduli calculated by the artificial potential field model; d goal =||q t -q goal ||2 represents the Euclidean distance from the current point to the target point;
[0125] The action space is a two-dimensional continuous action space, defined as follows:
[0126]
[0127] Among them, a T ∈[0,1] is the normalized main thrust, mapped to the actual thrust range [0,F max ], F max is the maximum thrust of the UAV; Δθ∈[-15°,15°] is the heading angle increment.
[0128] Step 2: Construct a dynamic artificial potential field model composed of gravity, repulsion, and wind field disturbances to generate a resultant force vector. Design a reward function that integrates path efficiency and potential field gradient. This system model guides the drone's autonomous obstacle avoidance and path optimization through autonomous decision-making.
[0129] The artificial potential field model is a dynamic adjustment potential field, which is composed of gravitational field, repulsive field and wind field disturbance, wherein the gravitational field function U att (q) is defined as:
[0130]
[0131] Where ξ>0 is the gravitational gain coefficient; q goal is the coordinate of the target point; κ>0 is the rotation adjustment factor, which generates a tangential component around the target point to break the symmetric balance; the corresponding gravitational formula is derived from the gravitational field function:
[0132] The repulsive field function U att (q) is defined as:
[0133]
[0134] Among them, d0 is the maximum distance detected by the UAV; ||qq obs || n is the obstacle distance coupling term; at the same time, the corresponding gravitational gradient
[0135] In order to cope with the complex time-varying environment, the environmental wind force is also added to the construction of the artificial potential field:
[0136]
[0137] Among them, ||qq goal || / ||q start -q goal || represents the current position of the drone q to the target point q goal The distance and q start To the target point q goal The ratio of the distances; is the wind speed vector; Δθ w The angle between the wind direction and the direction of the UAV's movement; therefore, the potential field gradient The net force acting on the drone is:
[0138]
[0139] Finally, the resultant force vector is obtained. The magnitude of the vector is used as the motion amplitude of the drone, and the angle between the vector and the direction of the drone's previous motion is used as the rotation angle of the drone.
[0140] The artificial potential field reward function of the drone is defined as follows:
[0141] r=ξ·U att (q t ,q goal )-η·∑U rep (q t ,q obs )-α·Δt+β·Δd
[0142] Where Δt is the time consumption, path length, Δd=d(q t -q start )-d(q t -q goal ), that is, the reduction of path length, encourages drones to fly in straight lines and avoid obstacles as much as possible.
[0143] Step 3: Based on the gravity, repulsion, and curl characteristics of the dynamic artificial potential field, a potential field guidance signal is constructed. Combined with the drone's proprioception data, a dual-branch attention structure of environmental attention and potential field attention is constructed in the local reinforcement learning policy network, and local policy learning is implemented based on the improved APF-PPO algorithm.
[0144] The local reinforcement learning strategy is constructed using the potential field enhanced APF-PPO algorithm. By fusing the potential field feature extraction with the drone body state information, an enhanced state vector is constructed as the input of the policy network:
[0145]
[0146] Among them, W s and W h is the weight matrix of the state vector and the potential field eigenvector; The high-order physical feature encoding is achieved through the potential field feature extraction module, specifically:
[0147]
[0148] Among them, the potential field force amplitude characteristic ||Fatt ||,||F rep ||;∠F att ,∠F rep is the direction characteristic of the potential field force; is the curl characteristic of the potential field;
[0149] The environmental attention branch takes the real-time posture state of the unmanned system and the lidar point cloud data as input, encodes the real-time posture state of the drone into a query vector Q, extracts multi-scale geometric features of the lidar point cloud data through the PointNet++ network, and generates a key vector K and a value vector V; the query vector Q and the key vector K are connected through the projection matrix Mapped to a shared 64-dimensional feature space, generating the environment attention weight matrix:
[0150]
[0151] Among them, d k =64 is the feature dimension of the key vector, which is used to balance the computational efficiency and the expressive power of the attention weight; Matrix parameters for projection of value vectors;
[0152] The potential field attention will be the real-time potential field artificial gradient As input features, and through the projection matrix Mapped to a 64-dimensional feature space, generating the potential field attention weight matrix:
[0153]
[0154] Adjust the fusion weight of the two branches according to the exponential decay in the training stage λ(t) = e -0.003t , t is the number of training steps, and the final action probability distribution of the policy network is synthesized by the dynamic weighting of the two-branch attention results:
[0155]
[0156] The policy gradient loss after trimming is calculated based on the proximal policy optimization PPO algorithm. The formula is:
[0157]
[0158] Among them, the strategy ratio is the advantage function, clip(·) is the clipping function, ε is the clipping threshold, is the strategy alignment term, π APF (a t ) Reference action probability distribution generated by the artificial potential field.
[0159] Step 4: Construct a cross-modal representation by fusing potential fields and states, generate similarity weights based on task semantics, and implement federated weighted aggregation and heterogeneous updates of policy parameters.
[0160] Based on the improved artificial potential field model, the drone uses onboard sensors to analyze the potential field gradient and rotation characteristics in real time:
[0161]
[0162] Physical potential field characteristics Jointly encoded by gradient and rotation:
[0163]
[0164] The UAV's state information and potential field feature extraction fusion vector are used to achieve cross-modal feature fusion by constructing a lightweight multi-layer perceptron:
[0165]
[0166] The above method combines task type, priority and risk index to transform the node's local task and execution environment into a structured semantic vector:
[0167] T (i) =Encoder(T goal ,T priority ,T risk )
[0168] in, is the task type unique hot encoding, Tpriority∈[0,1] represents the task priority coefficient, T risk ∈[0,1] is the environmental risk index; therefore, the local task encoding T (i) With the global target T g The semantic relevance of is expressed as:
[0169]
[0170] Among them, task similarity φ (i) ∈[-1,1], this indicator quantifies the contribution potential of UAV i to the global task. The larger its value, the higher the potential for collaboration between the local task of node i and the global goal.
[0171] The policy parameters uploaded by each end-side node i after the local task training is completed , the server adopts a task-similarity weighted approach to perform heterogeneous parameter fusion, and defines the federation aggregation weight as α (i) , satisfying Σ i α (i)=1, which is used to measure the degree of correlation between each local model and the current global task; the global strategy parameter update formula obtained by aggregation is as follows:
[0172]
[0173] in, represents the global model parameters after the tth round update; Represents the global model parameters of the previous round; μ is the historical forgetting factor, which ranges from [0,1] and is used to balance the residual influence of the parameters of the previous round in the current round update;
[0174] The local model update formula is:
[0175]
[0176] Where β is the fusion rate coefficient; λ ( i ) is the dynamic fusion weight.
[0177] Step 5: Construct the knowledge element weight matrix of the teacher model by constructing the feature component activation strength and state attention weight of the teacher model; and construct the knowledge element weight matrix of the student model through backpropagation calculation and normalization processing;
[0178] First, a large cloud-side model is constructed as the knowledge source, integrating task feedback information from multiple terminals and environmental characteristics. The knowledge element weight matrix of the cloud-side teacher model is defined as follows:
[0179]
[0180] It includes two parts: feature component activation intensity and state attention weights The strength of feature activation is calculated by normalizing the L2 norm to quantify the relative importance of each component:
[0181]
[0182] in, represents the state representation vector of the teacher network layer l, k is the index of the feature component, and the L2 norm of the feature value is calculated and normalized to the range of [0,1];
[0183] The state attention weight represents the decision criticality of a specific component in the state representation vector, which is quantified by calculating the ratio of the L2 norm of each component to the sum of the global L2 norm and based on the policy gradient Dynamically adjust the weights; specifically, the state representation vector of the teacher network The state attention weight of its k-th component is defined as:
[0184]
[0185] This weight combines the static strength and dynamic sensitivity of the state component, guiding the student network to focus on imitating the task benefits. Key status characteristics with significant impact;
[0186] Similar to the feature weight modeling of the teacher network, the knowledge weight of the student network is dynamically adjusted by its policy sensitivity to the task loss; specifically, the loss function L is calculated by backpropagation. task The impact of , measuring its importance in strategy optimization:
[0187]
[0188] in, Represents the kth component of the state representation vector of the mth layer of the student network.
[0189] Step 6: Construct the knowledge element weight matrix of the teacher and student models in the end-cloud collaboration, build the distillation loss function based on feature alignment, soft target distillation, and hard target constraints, and design a targeted knowledge transfer and temperature regulation structure that adapts to dynamic scenarios in the end-cloud collaborative structure.
[0190] First, based on the knowledge factor weight matrix of the teacher model and the student model defined in step 5, calculate the loss of the knowledge factor:
[0191]
[0192] Where Φ is the mapping set between the teacher layer l and the student layer m, indicating which teacher layer features need to be aligned with which student layers; D l is the dimension of the feature vector of the teacher network layer l; ψ(k) is the feature dimension projection function, which maps the high-dimensional feature index k of the teacher network to the corresponding dimension of the student network through linear transformation;
[0193] To further ensure the alignment of the feature space, a feature consistency loss term based on cosine similarity is designed. Its core is to quantify the consistency of the response direction of the teacher and student models in key state components. This loss term is calculated by the following formula:
[0194]
[0195] This loss term forces the student model to be consistent with the teacher model in the response direction of the state component by calculating the cosine similarity between the teacher and student feature vectors in the corresponding dimensions;
[0196] The total knowledge element alignment loss function is designed to combine the two loss terms of feature importance weight alignment and feature space alignment, dynamically adjust the importance of the two parts through the weight parameter ζ, and optimize the performance of the student model during the distillation process:
[0197] L KI =ζ·L align +(1-ζ)·L feat
[0198] A hierarchical adaptive temperature adjustment mechanism is proposed for the calculation of directed knowledge distillation loss under feature activation constraints. First, a state component difference sensitivity factor is constructed based on the policy gradient sensitivity and feature alignment difference to quantify the response deviation between the teacher model and the student model on key decision features. It is specifically defined as:
[0199]
[0200] Among them, σ(·) is the standard deviation operation, which is used to dynamically adjust the weight alignment strength; is the gradient amplitude of the teacher model policy network to the state component k; ∈ is a numerical stability term; secondly, to consider the different requirements for temperature regulation at different stages of training, a process-aware temperature basis function is designed:
[0201]
[0202] where τ max and τ min are the upper and lower bounds of temperature, k′ is the control annealing rate, t is the current training step number, T max is the maximum number of training steps; we further introduce the state component difference driving term and policy gradient sensitivity, dynamically adjust the temperature parameter through the task-aware modulation mechanism, and finally obtain the dynamic temperature expression for decision optimization:
[0203]
[0204] Where λ is the modulation intensity factor. When the response difference of the feature dimension is large, the corresponding temperature is increased to alleviate the information interference caused by the feature deviation. When the feature match is good, the temperature is reduced to enhance the target focus. Based on this dynamic temperature scheme, the original fixed temperature softmax function is updated to a temperature-dynamically adjusted version. Therefore, in the soft target probability calculation, the output of the teacher and student models are defined as:
[0205] p T (a)=softmax(z T / τ (l,m) (t))
[0206] p S (a)=softmax(zS / τ (l,m) (t))
[0207] Among them, z T and z S are the output vectors of the teacher and student models, representing the model’s predictions for different actions;
[0208] The dynamic temperature parameter is explicitly introduced into the distillation loss, and the soft target loss function is designed in the form of KL divergence under dynamic temperature:
[0209]
[0210] The hard target decision constraint ensures that the student model's decision in the task is consistent with the teacher model, thus ensuring its stability and reliability; the specific hard target loss is as follows:
[0211]
[0212] Among them, y a is the one-hot encoding of the reference action based on potential field planning; Var(·) is the variance of the action probability, and A is the set of all actions of the drone;
[0213] In order to achieve directed knowledge distillation in the student model, it is necessary to combine soft target semantic distillation, hard target decision constraints, and feature activation constraints to guide the learning of the student model through a comprehensive loss function; the total loss function can be expressed as follows:
[0214] L total =L KI +L soft +L hard
[0215] Total loss function L total It is to combine various loss terms together to ensure the coordinated optimization of the student model in feature learning, semantic transfer and task decision-making.
[0216] In another embodiment of the present invention, a brain-like intelligent decision-making system for an unmanned system based on a triple learning mechanism is provided, which is used to implement a brain-like intelligent decision-making method for an unmanned system based on a triple learning mechanism, including an autonomous decision-making unit, a path optimization unit, a local strategy unit, a federated aggregation and heterogeneous update unit, a knowledge element weight unit, and an adjustment unit, wherein:
[0217] The autonomous decision-making unit is used to obtain the drone's body state, environmental perception characteristics and mission constraint information, map them into a multi-dimensional state space and simultaneously define a continuous action space, and establish a system model for the drone's autonomous decision-making;
[0218] The path optimization unit is used to construct a dynamic artificial potential field model composed of gravity, repulsion, and wind field disturbances, generate a resultant force vector, and design a reward function that integrates path efficiency and potential field gradient. This is then used to guide the drone's autonomous obstacle avoidance and path optimization through a system model of autonomous decision-making.
[0219] The local policy unit is used to construct a potential field guidance signal based on the gravity, repulsion and curl characteristics in the dynamic artificial potential field, and to build a dual-branch attention structure of environmental attention and potential field attention in the local reinforcement learning policy network in combination with the drone's proprioception data, and to implement local policy learning based on the improved APF-PPO algorithm;
[0220] The federated aggregation and heterogeneous update unit is used to construct a cross-modal expression by fusing potential fields and states, generate similarity weights based on task semantics, and implement federated weighted aggregation and heterogeneous update of policy parameters;
[0221] The knowledge element weight unit is used to construct the knowledge element weight matrix of the teacher model by constructing the feature component activation strength and state attention weight of the teacher model; and to construct the knowledge element weight matrix of the student model through back propagation calculation and normalization processing;
[0222] The adjustment unit is used to construct the knowledge element weight matrix of the teacher and student models of the end-cloud collaboration, construct the distillation loss function based on feature alignment, soft target distillation and hard target constraints, and design a directed knowledge transfer and temperature adjustment structure that adapts to dynamic scenarios in the end-cloud collaborative structure.
[0223] The present invention improves the reinforcement learning and compares it with other reinforcement learning algorithms. Figure 2 Comparison of task completion time between the federal attention fusion mechanism and the federal average mechanism of the present invention is shown in Figure 2. Figure 3 As shown in the figure, the navigation success rate comparison based on improved knowledge distillation is shown in the figure. Figure 4 The present invention significantly improves the perception and response capability, collaborative decision-making efficiency, and knowledge transfer adaptability of unmanned systems in multi-task heterogeneous environments, ensuring the stability and reliability of unmanned system task execution in complex dynamic environments.
[0224] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A brain-like intelligent decision-making method for unmanned systems based on a triple learning mechanism, characterized by: The steps include: Step 1: Obtain the drone's state, environmental perception characteristics, and mission constraint information, map them into a multi-dimensional state space, and simultaneously define a continuous action space to establish a system model for autonomous decision-making. Step 2: Construct a dynamic artificial potential field model composed of gravity, repulsion, and wind field disturbances to generate a resultant force vector. Design a reward function that integrates path efficiency and potential field gradient. This system model guides the drone's autonomous obstacle avoidance and path optimization through autonomous decision-making. Step 3: Based on the gravity, repulsion, and curl characteristics of the dynamic artificial potential field, a potential field guidance signal is constructed. Combined with the drone's proprioception data, a dual-branch attention structure of environmental attention and potential field attention is constructed in the local reinforcement learning policy network, and local policy learning is implemented based on the improved APF-PPO algorithm. Step 4: Construct a cross-modal representation by fusing potential fields and states, generate similarity weights based on task semantics, and implement federated weighted aggregation and heterogeneous updates of policy parameters. Step 5: Construct the knowledge element weight matrix of the teacher model by constructing the feature component activation strength and state attention weight of the teacher model; and construct the knowledge element weight matrix of the student model through backpropagation calculation and normalization processing; Step 6: Construct the knowledge element weight matrix of the teacher and student models in the end-cloud collaboration, build the distillation loss function based on feature alignment, soft target distillation, and hard target constraints, and design a targeted knowledge transfer and temperature regulation structure that adapts to dynamic scenarios in the end-cloud collaborative structure.
2. The brain-inspired intelligent decision-making method for unmanned systems based on the triple learning mechanism according to claim 1 is characterized by: The artificial potential field model in step 2 includes the gravitational field, repulsive field and wind field disturbance, wherein the gravitational field function U att (q) is: Where ξ>0 is the gravitational gain coefficient; q goal is the coordinate of the target point; κ>0 is the rotation adjustment factor, which generates a tangential component around the target point to break the symmetric balance; the corresponding gravitational formula is derived from the gravitational field function: The repulsive field function U att (q) is: Among them, d0 is the maximum distance detected by the UAV; ||qq obs || n is the obstacle distance coupling term; at the same time, the corresponding gravitational gradient The environmental wind force is added to the construction of the artificial potential field: Among them, ||qq goal || / ||q start -q goal || represents the current position of the drone q to the target point q goal The distance and q start To the target point q goal The ratio of the distances; is the wind speed vector; Δθ w The angle between the wind direction and the direction of the UAV's movement; therefore, the potential field gradient The net force acting on the drone is: Finally, the resultant force vector is obtained. The magnitude of the vector is used as the motion amplitude of the drone, and the angle between the vector and the direction of the drone's previous motion is used as the rotation angle of the drone. The artificial potential field reward function of the drone is defined as follows: r=ξ·U att (q t ,q goal )-η·∑U rep (q t ,q obs )-α·Δt+β·Δd Where Δt is the time consumption, path length, Δd=d(q t -q start )-d(q t -q goal ), that is, the reduction of path length, encourages drones to fly in straight lines and avoid obstacles as much as possible.
3. The brain-inspired intelligent decision-making method for unmanned systems based on the triple learning mechanism according to claim 2 is characterized by: In step 3, the local reinforcement learning strategy is constructed using the potential field enhanced APF-PPO algorithm. The enhanced state vector is constructed by fusing the potential field feature extraction with the drone body state information and used as the input of the policy network: Among them, W s and W h is the weight matrix of the state vector and the potential field eigenvector; The high-order physical feature encoding is achieved through the potential field feature extraction module, specifically: Among them, the potential field force amplitude characteristic ||F att ||,||F rep ||;∠F att ,∠F rep is the direction characteristic of the potential field force; is the curl characteristic of the potential field; The environmental attention branch takes the real-time posture state of the unmanned system and the lidar point cloud data as input, encodes the real-time posture state of the drone into a query vector Q, extracts multi-scale geometric features of the lidar point cloud data through the PointNet++ network, and generates a key vector K and a value vector V; the query vector Q and the key vector K are connected through the projection matrix Mapped to a shared 64-dimensional feature space, generating the environment attention weight matrix: Among them, d k =64 is the feature dimension of the key vector, which is used to balance the computational efficiency and the expressive power of the attention weight; Matrix parameters for projection of value vectors; The potential field attention will be the real-time potential field artificial gradient As input features, and through the projection matrix Mapped to a 64-dimensional feature space, generating the potential field attention weight matrix: Adjust the fusion weight of the two branches according to the exponential decay in the training stage λ(t) = e -0.003t , t is the number of training steps, and the final action probability distribution of the policy network is synthesized by the dynamic weighting of the two-branch attention results: The policy gradient loss after trimming is calculated based on the proximal policy optimization PPO algorithm. The formula is: Among them, the strategy ratio is the advantage function, clip(·) is the clipping function, ε is the clipping threshold, is the strategy alignment term, π APF (a t ) Reference action probability distribution generated by the artificial potential field.
4. The brain-inspired intelligent decision-making method for unmanned systems based on the triple learning mechanism according to claim 3 is characterized by: In step 4, based on the improved artificial potential field model, the drone uses onboard sensors to analyze the potential field gradient and rotation characteristics in real time: Physical potential field characteristics Jointly encoded by gradient and rotation: The UAV's state information and potential field feature extraction fusion vector are used to achieve cross-modal feature fusion by constructing a lightweight multi-layer perceptron: The above method combines task type, priority and risk index to transform the node's local task and execution environment into a structured semantic vector: T (i) =Encoder(T goal ,T priority ,T risk ) in, is the task type unique hot encoding, Tpriority∈[0,1] represents the task priority coefficient, T risk ∈[0,1] is the environmental risk index; therefore, the local task encoding T (i) With the global target T g The semantic relevance of is expressed as: Among them, task similarity φ (i) ∈[-1,1], this indicator quantifies the contribution potential of UAV i to the global task. The larger its value, the higher the potential for collaboration between the local task of node i and the global goal. The policy parameters uploaded by each end-side node i after the local task training is completed The server uses a task-similarity weighted approach to perform heterogeneous parameter fusion, defining the federation aggregation weight as α (i) , satisfying Σ i α (i) =1, which is used to measure the degree of correlation between each local model and the current global task; the global strategy parameter update formula obtained by aggregation is as follows: in, represents the global model parameters after the t-th round update; Represents the global model parameters of the previous round; μ is the historical forgetting factor, which ranges from [0,1] and is used to balance the residual influence of the parameters of the previous round in the current round update; The local model update formula is: Where β is the fusion rate coefficient; λ (i) is the dynamic fusion weight.
5. The brain-inspired intelligent decision-making method for unmanned systems based on the triple learning mechanism according to claim 4 is characterized by: The step 5 comprises the following steps: First, a large cloud-side model is constructed as the knowledge source, integrating task feedback information from multiple terminals and environmental characteristics. The knowledge element weight matrix of the cloud-side teacher model is defined as follows: It includes two parts: feature component activation intensity and state attention weights The strength of feature activation is calculated by normalizing the L2 norm to quantify the relative importance of each component: in, represents the state representation vector of the teacher network layer l, k is the index of the feature component, and the L2 norm of the feature value is calculated and normalized to the range of [0,1]; The state attention weight represents the decision criticality of a specific component in the state representation vector, which is quantified by calculating the ratio of the L2 norm of each component to the sum of the global L2 norm and based on the policy gradient Dynamically adjust the weights; specifically, the state representation vector of the teacher network The state attention weight of its k-th component is defined as: This weight combines the static strength and dynamic sensitivity of the state component, guiding the student network to focus on imitating the task benefits. Key status characteristics with significant impact; Similar to the feature weight modeling of the teacher network, the knowledge weight of the student network is dynamically adjusted by its policy sensitivity to the task loss; specifically, the loss function L is calculated by backpropagation. task The impact of , measuring its importance in strategy optimization: in, Represents the kth component of the state representation vector of the mth layer of the student network.
6. The brain-inspired intelligent decision-making method for unmanned systems based on the triple learning mechanism according to claim 5 is characterized by: The step 6 comprises the following steps: First, based on the knowledge factor weight matrix of the teacher model and the student model defined in step 5, calculate the loss of the knowledge factor: Where Φ is the mapping set between the teacher layer l and the student layer m; D l is the dimension of the feature vector of the teacher network layer l; ψ(k) is the feature dimension projection function, which maps the high-dimensional feature index k of the teacher network to the corresponding dimension of the student network through linear transformation; Feature consistency loss term: The total knowledge element alignment loss function is designed to combine the two loss terms of feature importance weight alignment and feature space alignment, dynamically adjust the importance of the two parts through the weight parameter ζ, and optimize the performance of the student model during the distillation process: L KI =ζ·L align +(1-ζ)·L feat A hierarchical adaptive temperature adjustment mechanism is proposed for the calculation of directed knowledge distillation loss under feature activation constraints. First, a state component difference sensitivity factor is constructed based on the policy gradient sensitivity and feature alignment difference to quantify the response deviation between the teacher model and the student model on key decision features. It is specifically defined as: Among them, σ(·) is the standard deviation operation, which is used to dynamically adjust the weight alignment strength; is the gradient amplitude of the teacher model policy network to the state component k; ∈ is a numerical stability term; secondly, to consider the different requirements for temperature regulation at different stages of training, a process-aware temperature basis function is designed: where τ max and τ min are the upper and lower bounds of temperature, k′ is the control annealing rate, t is the current training step number, T max is the maximum number of training steps; we further introduce the state component difference driving term and policy gradient sensitivity, dynamically adjust the temperature parameter through the task-aware modulation mechanism, and finally obtain the dynamic temperature expression for decision optimization: Where λ is the modulation intensity factor; The output of the teacher and student models are defined as: p T (a)=softmax(z T / τ (l,m) (t)) p S (a)=softmax(z S / τ (l,m) (t)) Among them, z T and z S are the output vectors of the teacher and student models respectively; The dynamic temperature parameter is explicitly introduced into the distillation loss, and the soft target loss function is designed in the form of KL divergence under dynamic temperature: The hard target decision constraint ensures that the student model's decision in the task is consistent with the teacher model, thus ensuring its stability and reliability; the specific hard target loss is as follows: Among them, y a is the one-hot encoding of the reference action based on potential field planning; Var(·) is the variance of the action probability, and A is the set of all actions of the drone; The comprehensive loss function is expressed by the following formula: L total =L KI +L soft +L hard Total loss function L total It is to combine various loss terms together to ensure the coordinated optimization of the student model in feature learning, semantic transfer and task decision-making.
7. The brain-inspired intelligent decision-making method for unmanned systems based on a triple learning mechanism according to claim 6, characterized in that: In step 1, the state space includes the geometric position and motion state of the UAV and the artificial potential field characteristics, and constructs an eight-dimensional state vector: Where x and y represent the Cartesian coordinates of the drone in the two-dimensional horizontal plane, and the real-time position of the drone is q = (x, y); v x , v y is the velocity component of the UAV in the x and y axis directions; θ is the heading angle, which represents the angle between the longitudinal axis of the body and the positive x direction; F att and F rep are the gravitational and repulsive moduli calculated by the artificial potential field model; d goal =||q t -q goal ||2 represents the Euclidean distance from the current point to the target point.
8. The brain-inspired intelligent decision-making method for unmanned systems based on a triple learning mechanism according to claim 1 is characterized by: The action space in step 1 is a two-dimensional continuous action space, defined as follows: Among them, a T ∈[0,1] is the normalized main thrust, mapped to the actual thrust range [0,F max ], F max is the maximum thrust of the UAV; Δθ∈[-15°,15°] is the heading angle increment.
9. A system for implementing the brain-inspired intelligent decision-making method for unmanned systems based on the triple learning mechanism of claim 1, characterized in that: It includes autonomous decision-making unit, path optimization unit, local strategy unit, federation aggregation and heterogeneous update unit, knowledge element weight unit, and regulation unit, among which: The autonomous decision-making unit is used to obtain the drone's body state, environmental perception characteristics and mission constraint information, map them into a multi-dimensional state space and simultaneously define a continuous action space, and establish a system model for the drone's autonomous decision-making; The path optimization unit is used to construct a dynamic artificial potential field model composed of gravity, repulsion, and wind field disturbances, generate a resultant force vector, and design a reward function that integrates path efficiency and potential field gradient. This is then used to guide the drone's autonomous obstacle avoidance and path optimization through a system model of autonomous decision-making. The local policy unit is used to construct a potential field guidance signal based on the gravity, repulsion and curl characteristics in the dynamic artificial potential field, and to build a dual-branch attention structure of environmental attention and potential field attention in the local reinforcement learning policy network in combination with the drone's proprioception data, and to implement local policy learning based on the improved APF-PPO algorithm; The federated aggregation and heterogeneous update unit is used to construct a cross-modal expression by fusing potential fields and states, generate similarity weights based on task semantics, and implement federated weighted aggregation and heterogeneous update of policy parameters; The knowledge element weight unit is used to construct the knowledge element weight matrix of the teacher model by constructing the feature component activation strength and state attention weight of the teacher model; and to construct the knowledge element weight matrix of the student model through back propagation calculation and normalization processing; The adjustment unit is used to construct the knowledge element weight matrix of the teacher and student models of the end-cloud collaboration, construct the distillation loss function based on feature alignment, soft target distillation and hard target constraints, and design a directed knowledge transfer and temperature adjustment structure that adapts to dynamic scenarios in the end-cloud collaborative structure.
Citation Information
Cited By
Unmanned aerial vehicle target sensing method based on beam reconstruction and environment sensing self-adaption
CN120993369A
Top opening and closing greenhouse artificial intelligence regulation and control method and system for peach tree planting
CN121455268A