Unmanned system cooperative SLAM method for lifelong federated learning
Through the unmanned system collaborative SLAM method of lifelong federated learning, combined with federated learning and lifelong learning, the positioning and mapping accuracy of unmanned system clusters in weak communication environments is solved, and precise positioning and mapping under weak communication conditions are achieved. Real-time reconstruction overcomes the impact of communication failures and improves the stability of three-dimensional reconstruction in the underground environment.
Patent Information
- Application Number
- CN202410980345.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-22
- Publication Date
- 2024-11-12
AI Technical Summary
In the weak underground communication environment, it is difficult for the prior art to realize real-time three-dimensional reconstruction and positioning of unmanned system clusters, especially in the absence of satellite images and complex terrain conditions, which makes it difficult to guarantee the accuracy and robustness of positioning and map construction.
The unmanned system collaborative SLAM method of lifelong federated learning is adopted. By collaborating the cloud and multiple edge information, combined with the concept of federated learning and lifelong learning, the unmanned system cluster is accurately positioned and real-time reconstruction in weak-connected large scenarios. Specific steps include data acquisition by edge drones, generating local maps, training of lifelong learning submodules, dynamic adjustment of model weights and adaptive fusion, scene completion and splicing to ensure the stability of positioning and map construction.
Under weak communication conditions, dynamic adjustment and precise positioning of unmanned system clusters are achieved, ensuring the accuracy and robustness of long-term positioning and map construction, overcoming the impact of communication failures, and improving the stability of three-dimensional reconstruction of underground environments sex.
Smart Images

Figure CN118941714A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of three-dimensional reconstruction, and in particular to an unmanned system collaborative SLAM method of lifelong federated learning. Background Art
[0002] With the spatial pressure brought by population growth and urbanization, human activities have begun to shift underground, and the detection and development of underground environments have become an important prerequisite for making full use of underground resources. However, underground spaces are often hypoxic, with poor lighting and complex terrain. There is a lack of satellite images when detecting underground spaces, so using drones to reconstruct three-dimensional underground environments is a very widespread and effective method. When exploring and reproducing large underground scenes through unmanned system clusters, on the one hand, due to the limitations of onboard capacity, the area that can be reached and recorded within the flight time of a single machine is very limited. On the other hand, the particularity of the underground environment often causes a decrease in communication quality and a reduction in communication coverage, which will affect the data synchronization and collaborative processing between various edge devices. The calculation and communication of massive sensor data are difficult to guarantee, and the real-time and stability of positioning and mapping will be greatly reduced. In such an unknown underground environment, since the communication quality cannot be guaranteed, it is still challenging to achieve efficient unmanned system cloud-edge collaborative SLAM.
[0003] Faced with the above difficulties, some studies have been devoted to solving the reconstruction problems in practical engineering applications. For example, a drone-based remote sensing technology has been proposed, which can be used for the detection and visualization of partially exposed underground structures in complex archaeological sites; using multiple drones to achieve dense back-end 3D reconstruction for visual tunnel inspection; using drone images to quickly build and monitor foundation pit construction. However, the methods currently used cannot achieve real-time splicing and establishment of point cloud maps in underground weak communication environments. Catastrophic forgetting problems are more likely to occur in long time series and large scenes, and the accuracy and robustness of positioning and mapping cannot be guaranteed. Summary of the invention
[0004] The purpose of the present invention is to provide a collaborative SLAM method for unmanned systems based on lifelong federated learning, which aims to combine the concepts of federated learning and lifelong learning with mapping algorithms through collaborative cloud and multiple edge information, so as to achieve precise positioning and real-time reconstruction of unmanned system clusters in weakly connected scenarios.
[0005] The purpose of the present invention can be achieved by the following technical solutions:
[0006] A lifelong federated learning unmanned system collaborative SLAM method, comprising the following steps:
[0007] Step S1: The edge drone deploys sensors and obtains data collected by the sensors;
[0008] Step S2: Based on the data collected by the sensor in step S1, the edge UAV implements laser inertial odometer estimation to generate a local map;
[0009] Step S3: The laser radar data collected in step S1 are accumulated as scene streams at preset intervals, and input into a lifelong learning submodule to complete model training and obtain historical model information, wherein the lifelong learning submodule uses a laser radar point cloud set as input and uses extracted features and scene types as output;
[0010] Step S4: According to the historical model information of the lifelong learning submodule of each edge node, the model weight of each lifelong learning submodule is dynamically adjusted based on reinforcement learning, and the adaptive model fusion is performed in the cloud lifelong learning center module of the central node, and the central node sends the updated model parameters to the edge nodes;
[0011] Step S5: Using the model obtained in step S4, output point cloud features according to the current scene type, complete the scene, and splice it with the local map output in step S2 to form a predicted global map.
[0012] The step S3 specifically includes:
[0013] Step S31: Obtain the scene flow at a preset time interval T, and define the intercepted n-th point cloud set as in for The i-th scene data in, is the corresponding scene type label, N n is the number of all scenes in this point cloud set, All point clouds in are converted into point cloud relays, which consist of position encoding and seed features extracted by the point cloud relay extractor; each point cloud set is removed from memory after training is completed;
[0014] Step S32: Input Use during initialization Train the feature extractor E0 and classifier C0 of the lifelong learning submodule, then freeze both E0 and C0, and then use the output of E0 to train the feature repeater R0; the flock algorithm is used to select the training samples of the feature repeater. represents the training samples of class α, N α Represents the number of samples, and its class mean can be obtained according to the following formula:
[0015]
[0016] Step S33: In the incremental learning stage n, the point cloud data is continuously obtained, through E n-1 and C n-1 Initialize E respectively nand C n , the knowledge distillation technology is used in the process to distill E n-1 Knowledge in preventing E n Severe forgetting occurs, and the difference in feature activations at corresponding positions in the teacher and student feature maps is used as the loss:
[0017]
[0018] where f t and f s Represent the feature graphs of teachers and students respectively, Φ t and Φ s Represent the transformation of teacher and student characteristics respectively;
[0019] use and R n-1 Generate replay features To train the feature extractor E n and classifier C n After the training is completed, both are frozen and further used. n The output h n and R n-1 Output To train the feature repeater R n , to prepare for feature fusion for the next task. This process uses the GAN network for adversarial training:
[0020]
[0021] in represents the loss of the discriminator, indicates the loss of the replayer, represents the loss of replay alignment;
[0022] In the classification prediction of the nth stage, the size of the last fully connected layer is expanded to accommodate the new type, and the Softmax layer is also expanded to the same size;
[0023] Step S34: After learning task n, the current optimal R n and C n The output feature extraction and scene classification results are stored as And store the historical results as Q n = {P0,P1,...,P n}.
[0024] In step S32, the classification loss in the initialization stage is:
[0025]
[0026] Where γ is the true label, is the predicted classification probability, and V0 is the number of scene types in the initialization stage.
[0027] In step S33, the classification loss of the classification prediction in the nth stage is calculated as follows:
[0028]
[0029] It includes the cross entropy loss of the current task and the previous tasks, ζ is the previous task factor, U n V is the number of all scene types obtained after learning the point cloud set n. n The number of scene types learned for the current point cloud set.
[0030] The step S4 specifically includes:
[0031] Step S41: Initialize weight W for each edge node model i , and initialize the reinforcement learning network parameters θ and sample library D. Each child node receives tasks from the central node and starts training. When the model training is completed, each child node uploads the trained model to the reinforcement learning network;
[0032] Step S42: In the reinforcement learning network, the state s is used to determine the action a and execute it to obtain the weight w i After execution, the network obtains the reward value r and the new state s', and stores {a, s, r, s'} in the sample library D, where the state s represents the current model fusion accuracy and the previously obtained weights, the action a represents the weight assigned to the node, and r represents the difference between the accuracy of the fused model and the previous accuracy;
[0033] Step S43: Train the network parameters and update θ by randomly selecting a set of samples from the sample library. Specifically, the value V of state s π (s) and the long-term reward function Q for performing action a in state s π (s,a) are as follows:
[0034] V π (s) = E π [G t ∣S t =s]
[0035] Q π (s,a)=E π [G t ∣S t =s,A t =a]
[0036] Among them, π represents the strategy of taking action a in state s, G trepresents the profit after executing actions according to the strategy, E π It represents the expected return under this strategy.
[0037] In order to achieve the best effect by executing action a in state s, the value function should be maximized, which is obtained according to the Bellman equation:
[0038] V(s)=E[R t+1 +γmaxπV(s t+1 )∣S t =s]
[0039] Q(s,a)=E[Rt +1 +γmaxa ′ Q(st +1 ,a′)|St=s,At=a]
[0040] Among them, γ∈[0,1] represents the discount weight.
[0041] The maximum Q value after executing the next action a′ is taken as the target solution, and the gradient descent is performed through the mean square error. The corresponding loss function is:
[0042] L(θ)=E[(yQ(s,a;θ)) 2 ]
[0043] Step S44: weight w updated by reinforcement learning i The central node carries out model fusion of each lifelong learning sub-module, and then sends the model parameters to the edge nodes by the central node, and repeats steps S41-S44 until the training is completed.
[0044] The step S5 specifically includes:
[0045] Step S51: Identify the current scene type label, output corresponding point cloud features according to the trained model, use incomplete seed features to generate smooth interpolation to fill in the local incomplete scan, and generate a rough prediction of the missing part;
[0046] Step S52: Input the incomplete seed feature into the missing feature generator, learn the missing part of the feature from the existing features, and form the missing point cloud relay with the random position code as the Transformer query source; the incomplete seed feature and the absolute position code form the existing point cloud relay as the Transformer key and value, and both are input into the Transformer to output the predicted missing point cloud relay;
[0047] Step S53: applying an alignment constraint with the map truth to the predicted missing point cloud relay obtained in step S52, and then inputting it as a feature into FoldingNet for coarse-to-fine conversion, combining it with the predicted coarse missing part obtained in step S51, and outputting the dense missing part;
[0048] Step S54: concatenate the dense missing part obtained in step S53 with the local map generated in step S2 to form a predicted complete scene map.
[0049] The method for generating a rough prediction of the missing part is expressed as:
[0050]
[0051] Where X mis represents missing features, X obs1 , X obs2 Represents the existing features, p mis represents the topological position corresponding to the missing part, p obs1 、p obs2 Indicates the topological location corresponding to the existing features.
[0052] The types of sensors include at least: laser radar and inertial measurement unit.
[0053] The information collected by the sensor includes at least: the UAV flight speed and acceleration in three-dimensional directions, the angular momentum and angular acceleration of the UAV pitch-roll-rotation, the UAV coordinate information, and the point cloud posture information.
[0054] The step S2 is specifically as follows: after the laser radar data is preprocessed, it participates in forward and backward propagation together with the IMU data, and then residual calculation and iterative state update are carried out, and finally the odometer output is obtained by convergence, and the local map is updated.
[0055] Compared with the prior art, the present invention has the following beneficial effects:
[0056] The present invention adopts the enhanced federated learning method, coordinates the cloud and multiple edge data associations, and realizes the dynamic adjustment and precise positioning of the unmanned system cluster. At the same time, it combines the lifelong learning mechanism with the SLAM algorithm to ensure the accuracy, stability and excellent robustness of long-term positioning and mapping in underground environments with weak communication conditions. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 is a flow chart of the method of the present invention;
[0058] Figure 2 Schematic diagram of the framework of the point cloud replay lifelong learning algorithm;
[0059] Figure 3To strengthen the schematic diagram of the federated learning model fusion process;
[0060] Figure 4 A schematic diagram of the overall architecture to complete the scene. DETAILED DESCRIPTION
[0061] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0062] This embodiment provides a lifelong federated learning unmanned system collaborative SLAM method, such as Figure 1 As shown, the following steps are included:
[0063] Step S1: The edge drone deploys sensors and obtains data collected by the sensors.
[0064] In this embodiment, the types of sensors provided include at least: laser radar, inertial measurement unit. The information collected by the sensor includes at least: the flight speed and acceleration of the drone in three-dimensional directions, the angular momentum and angular acceleration of the drone's pitch-roll-rotation, the drone's coordinate information, and the point cloud pose information.
[0065] Step S2: Based on the data collected by the sensor in step S1, the edge UAV implements laser inertial odometer estimation and generates a local map.
[0066] Specifically: After preprocessing, the lidar data participates in forward and backward propagation together with the IMU data, and then residual calculation and iterative state update are carried out, and finally converge to obtain the odometer output and update the local map.
[0067] Step S3: The lidar data collected in step S1 is accumulated as a scene stream at preset intervals and input into a lifelong learning submodule to complete model training and obtain historical model information, wherein the lifelong learning submodule uses a lidar point cloud set as input and extracted features and scene types as output.
[0068] Step S3 specifically includes:
[0069] Step S31: Obtain the scene flow at intervals of T, and define the intercepted n-th point cloud set as in for The i-th scene data in is the corresponding scene type label, N n is the number of all scenes in the point cloud set. In particular, All point clouds in are converted into point cloud relays, which consist of position encodings and seed features extracted by the point cloud relay extractor. Each point cloud set is removed from memory after training is completed.
[0070] Step S32: Input Use during initialization Train the feature extractor E0 and classifier C0, and the classification loss is:
[0071]
[0072] Where γ is the true label, is the predicted classification probability, V0 is the number of scene types in the initialization stage. Then, E0 and C0 are frozen, and the output of E0 is used to train the feature repeater R0. The flock algorithm is used to select the training samples of the feature repeater. represents the training samples of class α, N α Represents the number of samples, and its class mean can be obtained according to the following formula:
[0073]
[0074] Step S33: In the incremental learning stage n, the point cloud data is continuously obtained, through E n-1 and C n-1 Initialize E respectively n and C n In this process, knowledge distillation technology is used to distill E n-1 Knowledge in preventing E n Serious forgetting occurs. The difference in feature activations at corresponding positions in the teacher and student feature maps is taken as the loss:
[0075]
[0076] where f t and f s Represent the feature graphs of teachers and students respectively, Φ t and Φ s Represent the transformation of teacher and student characteristics respectively.
[0077] Use at the same time and R n-1 Generate replay features To train E n and C n After the training is completed, both are frozen and further used. n The output h n and R n-1 Output To train R n, preparing for feature fusion for the next task. This process uses the GAN network for adversarial training:
[0078]
[0079] in represents the loss of the discriminator, Indicates the loss of the replayer, Represents the loss of replay alignment.
[0080] In the classification prediction of the nth stage, the size of the last fully connected layer is expanded to accommodate the new type. At the same time, the Softmax layer is also expanded to the same size. The classification loss is calculated as follows:
[0081]
[0082] It includes the cross entropy loss of the current task and the previous tasks, ζ is the previous task factor, U n V is the number of all scene types obtained after learning the point cloud set n. n The number of scene types learned for the current point cloud set.
[0083] Step S34: After learning task n, the current optimal R n and C n The output feature extraction and scene classification results are stored as Historical results are stored as Q n = {P0,P1,...,P n}.
[0084] Step S4: According to the historical model information of the lifelong learning sub-module of each edge node, the model weight of each lifelong learning sub-module is dynamically adjusted based on reinforcement learning, and adaptive model fusion is performed in the cloud-based lifelong learning center module of the central node. The central node then sends updated model parameters to the edge nodes.
[0085] Step S4 specifically includes:
[0086] Step S41: First, initialize the weight W for each edge node model i , and initialize the reinforcement learning network parameters θ and sample library D. Next, each child node receives tasks from the central node and starts training. When the model training is completed, each child node uploads the trained model to the reinforcement learning module.
[0087] Step S42: In the reinforcement learning network, the state s is used to determine the action a and execute it to obtain the weight w iAfter execution, the network obtains the reward value r and the new state s', and stores {a, s, r, s'} in the sample library D. Here, the state s represents the current model fusion accuracy and the previously obtained weights, the action a represents the weight assigned to the node, and r represents the difference between the accuracy of the fused model and the previous accuracy.
[0088] Step S43: Train the network parameters and update θ by randomly selecting a set of samples from the sample library. Specifically, the value V of state s π (s) and the long-term reward function Q for performing action a in state s π (s,a) are as follows:
[0089] V π (s) = E π [G t ∣S t =s]
[0090] Q π (s,a)=E π [G t ∣S t =s,A t =a]
[0091] Among them, π represents the strategy of taking action a in state s, G t represents the profit after executing actions according to the strategy, E π It represents the expected return under this strategy.
[0092] In order to achieve the best effect by executing action a in state s, the value function should be maximized. According to the Bellman equation, it can be concluded that:
[0093] V(s)=E[R t+1 +γmaxπV(s t+1 )∣S t =s]
[0094] Q(s,a)=E[Rt +1 +γmaxa ′ Q(st +1 ,a′)|St=s,At=a]
[0095] Among them, γ∈[0,1] represents the discount weight.
[0096] The maximum q value after executing the next action a′ is taken as the target solution, which can be solved by gradient descent through the mean square error. The corresponding loss function is:
[0097] L(θ)=E[(yQ(s,a;θ)) 2 ]
[0098] Step S44: By w i Carry out model fusion, and then the central node sends model parameters. The above process is repeated until the training is completed.
[0099] Step S5: Using the model obtained in step S4, output point cloud features according to the current scene type, complete the scene, and splice it with the local map output in step S2 to form a predicted global map.
[0100] Step S5 specifically includes:
[0101] Step S51: Identify the current scene type label and call Q n Output the corresponding point cloud features, use the incomplete seed features to generate smooth interpolation to fill the local incomplete scan, and generate a rough prediction of the missing part.
[0102]
[0103] Where X mis represents missing features, X obs1 , X obs2 Represents the existing features, p mis represents the topological position corresponding to the missing part, p obs1 、p obs2 Indicates the topological location corresponding to the existing features.
[0104] Step S52: The incomplete seed feature is input into the missing feature generator, and the missing part of the feature is learned from the existing features, and the missing point cloud relay is composed of the random position code as the Transformer query source; and the incomplete seed feature and the absolute position code constitute the existing point cloud relay as the Transformer key and value. Both are input into the Transformer, and the predicted missing point cloud relay is output.
[0105] Step S53: Apply alignment constraints with the map truth to the predicted missing point cloud relay obtained in S52, and then input it as a feature into FoldingNet for coarse-to-fine conversion, and finally combine it with the predicted coarse missing part obtained in S51 to output the dense missing part. The result is spliced with the local map in step S2 to form a predicted complete scene map.
[0106] The goal of 3D reconstruction is to restore the 3D structure from known data information and describe the shape, surface features and spatial layout of the object as accurately as possible. When drones perform underground operations, 3D reconstruction is usually required at the back end. The particularity of the underground environment will limit the application of traditional communication technologies, resulting in a decrease in communication quality. This will affect the data synchronization and processing between edge devices, making it difficult to ensure the real-time and stability of reconstruction. At the same time, when performing long-term detection in a multi-scenario environment, the model is prone to catastrophic forgetting problems due to task changes, and the accuracy of the reconstruction work will also decrease. In this regard, the method of cloud-edge collaborative lifelong federated mapping is particularly important.
[0107] The cloud-edge collaborative lifelong federated mapping method aims to overcome the uncertainty that may be caused by communication shielding and realize real-time mapping and updating of underground environments. The present invention fully combines the advantages of edge mobile nodes and cloud computing through a cloud-edge collaborative mechanism based on federated learning, eliminates the information interaction of each edge node, and ensures real-time processing and analysis of sensor data. A point cloud replay module and a scene completion module that integrate the concept of lifelong learning are proposed to ensure the robustness and accuracy of three-dimensional reconstruction, and ultimately realize real-time detection, precise modeling and continuous updating of the underground environment, providing more reliable and effective support for the management and utilization of the underground environment.
[0108] Taking drones as an example, the sensor data collection link mainly uses lidar, inertial measurement units and other sensor units to collect the drone's flight speed and acceleration in three dimensions, the drone's pitch-roll-rotation angular momentum and angular acceleration, drone coordinate information, point cloud pose information, etc.
[0109] Taking drones as an example, the edge implements laser inertial odometer estimation based on FAST-LIO. The main process includes: After the lidar data is preprocessed, it participates in forward and backward propagation together with the IMU data, and then performs residual calculation and iterative state update. The filtering framework uses IEKF, and finally converges to obtain the odometer output and updates the global map.
[0110] Taking drones as an example, the algorithm framework of its lifelong learning module is as follows: Figure 2 As shown in the figure, the scene stream is obtained at intervals of T, and the lifelong learning method based on generative feature replay is used for scene classification training; the algorithm mainly includes generative feature replay and knowledge distillation, which are used to avoid catastrophic forgetting in classifiers and extractors; a feature replay device is introduced to generate pseudo data for joint training. During the training process, the generated features of the previous task are combined with the real features of the current task to ensure classification accuracy; finally, the feature extraction and scene classification results are stored for call when the scene is completed.
[0111] Taking drones as an example, the update process of the enhanced federated learning model is as follows: Figure 3As shown. Each edge node first receives tasks from the central node and starts training, and uploads the trained model to the reinforcement learning module. Then, in the reinforcement learning network, the state s is used to determine the action a, and it is executed to obtain the weight w i After execution, the network gets the reward value r and the new state s', and stores {a, s, r, s'} in the sample library D. A set of samples from the sample library is randomly selected to train the network parameters and update θ. Finally, w i Carry out model fusion, and then send it down from the central node. Repeat the above process until the training is completed.
[0112] Taking drones as an example, the overall architecture of its scene completion module is as follows: Figure 4 As shown. Identify the current scene type label, output the corresponding point cloud features, use the incomplete seed features to generate linear interpolation to get the predicted rough missing part; the incomplete seed features are input into the missing feature generator, and the predicted missing features are learned from the existing features, and the random position encoding constitutes the missing point cloud relay, while the incomplete seed features and the absolute position encoding constitute the existing point cloud relay, and the two are input into the Transformer together to output the predicted missing point cloud relay; impose alignment constraints on the predicted missing point cloud relay with the true value of the map, and then input it as a feature into FoldingNet for coarse-to-fine conversion, and finally combine it with the predicted rough missing part to output the dense missing part. Thus, it is spliced with the local map generated above to obtain the global map.
[0113] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.
Claims
1. A lifelong federated learning unmanned system collaborative SLAM method, characterized in that: The following steps are involved: Step S1: The edge drone deploys sensors and obtains data collected by the sensors; Step S2: Based on the data collected by the sensor in step S1, the edge UAV implements laser inertial odometer estimation to generate a local map; Step S3: The laser radar data collected in step S1 are accumulated as scene streams at preset intervals, and input into a lifelong learning submodule to complete model training and obtain historical model information, wherein the lifelong learning submodule uses a laser radar point cloud set as input and uses extracted features and scene types as output; Step S4: According to the historical model information of the lifelong learning submodule of each edge node, the model weight of each lifelong learning submodule is dynamically adjusted based on reinforcement learning, and the adaptive model fusion is performed in the cloud lifelong learning center module of the central node, and the central node sends the updated model parameters to the edge nodes; Step S5: Using the model obtained in step S4, output point cloud features according to the current scene type, complete the scene, and splice it with the local map output in step S2 to form a predicted global map.
2. According to claim 1, a lifelong federated learning unmanned system collaborative SLAM method is characterized in that: The step S3 specifically includes: Step S31: Obtain the scene flow at a preset time interval T, and define the intercepted n-th point cloud set as in for The i-th scene data in is the corresponding scene type label, N n is the number of all scenes in this point cloud set, All point clouds in are converted into point cloud relays, which consist of position encoding and seed features extracted by the point cloud relay extractor; each point cloud set is removed from memory after training is completed; Step S32: Input Use during initialization Train the feature extractor E0 and classifier C0 of the lifelong learning submodule, then freeze both E0 and C0, and then use the output of E0 to train the feature repeater R0; wherein, the sheep flock algorithm is used to select the training samples of the feature repeater; Step S33: In the incremental learning stage n, the point cloud data is continuously obtained, through E n-1 and C n-1 Initialize E respectively n and C n , the knowledge distillation technology is used in the process to distill E n-1 Knowledge in preventing E n Severe forgetting occurs, and the difference in feature activations at corresponding positions in the teacher and student feature maps is used as the loss: where f t and f s Represent the feature graphs of teachers and students respectively, Φ t and Φ s Represent the transformation of teacher and student characteristics respectively; use and R n-1 Generate replay features To train the feature extractor E n and classifier C n After the training is completed, both are frozen and further used. n The output h n and R n-1 Output To train the feature repeater R n , to prepare for feature fusion for the next task. This process uses the GAN network for adversarial training: in represents the loss of the discriminator, Indicates the loss of the replayer, represents the loss of replay alignment; In the classification prediction of the nth stage, the size of the last fully connected layer is expanded to accommodate the new type, and the Softmax layer is also expanded to the same size; Step S34: After learning task n, the current optimal R n and C n The output feature extraction and scene classification results are stored as And store the historical results as Q n = {P0,P1,...,P n }.
3. The unmanned system collaborative SLAM method of lifelong federated learning according to claim 2 is characterized in that: In step S32, the classification loss in the initialization stage is: Where γ is the true label, is the predicted classification probability, and V0 is the number of scene types in the initialization stage.
4. The unmanned system collaborative SLAM method of lifelong federated learning according to claim 2 is characterized in that: In step S33, the classification loss of the classification prediction in the nth stage is calculated as follows: It includes the cross entropy loss of the current task and the previous tasks, ζ is the previous task factor, U n V is the number of all scene types obtained after learning the point cloud set n. n The number of scene types learned for the current point cloud set.
5. The unmanned system collaborative SLAM method of lifelong federated learning according to claim 1, characterized in that: The step S4 specifically includes: Step S41: Initialize weight W for each edge node model i , and initialize the reinforcement learning network parameters θ and sample library D. Each child node receives tasks from the central node and starts training. When the model training is completed, each child node uploads the trained model to the reinforcement learning network; Step S42: In the reinforcement learning network, the state s is used to determine the action a and execute it to obtain the weight w i After execution, the network obtains the reward value r and the new state s', and stores {a, s, r, s'} in the sample library D, where the state s represents the current model fusion accuracy and the previously obtained weights, the action a represents the weight assigned to the node, and r represents the difference between the accuracy of the fused model and the previous accuracy; Step S43: Train the network parameters and update θ by randomly selecting a set of samples from the sample library. Specifically, the value V of state s π (s) and the long-term reward function Q for performing action a in state s π (s,a) are as follows: V π (s)=E π [G t ∣S t [s] Q π (s,a)=E π [G t ∣S t =s,A t =a] Among them, π represents the strategy of taking action a in state s, G t represents the profit after executing actions according to the strategy, E π It represents the expectation of the expected return under the strategy; In order to achieve the best effect by executing action a in state s, the value function should be maximized, which is obtained according to the Bellman equation: V(s)=E[R t+1 +γmaxπV(s t+1 )∣S t =s] Q(s,a)=E[Rt +1 +γmaxa ′ Q(st +1 ,a′)∣St=s,At=a] Among them, γ∈[0,1] represents the discount weight; The maximum Q value after executing the next action a′ is taken as the target solution, and the gradient descent is performed through the mean square error. The corresponding loss function is: L(θ)=E[(yQ(s,a;θ)) 2 ] Step S44: weight w updated by reinforcement learning i The central node carries out model fusion of each lifelong learning sub-module, and then sends the model parameters to the edge nodes by the central node, and repeats steps S41-S44 until the training is completed.
6. The unmanned system collaborative SLAM method of lifelong federated learning according to claim 1, characterized in that: The step S5 specifically includes: Step S51: Identify the current scene type label, output corresponding point cloud features according to the trained model, use incomplete seed features to generate smooth interpolation to fill in the local incomplete scan, and generate a rough prediction of the missing part; Step S52: Input the incomplete seed feature into the missing feature generator, learn the missing part of the feature from the existing features, and form the missing point cloud relay with the random position code as the Transformer query source; the incomplete seed feature and the absolute position code form the existing point cloud relay as the Transformer key and value, and both are input into the Transformer to output the predicted missing point cloud relay; Step S53: applying an alignment constraint with the map truth to the predicted missing point cloud relay obtained in step S52, and then inputting it as a feature into FoldingNet for coarse-to-fine conversion, combining it with the predicted coarse missing part obtained in step S51, and outputting the dense missing part; Step S54: concatenate the dense missing part obtained in step S53 with the local map generated in step S2 to form a predicted complete scene map.
7. The unmanned system collaborative SLAM method of lifelong federated learning according to claim 6, characterized in that: The method for generating a rough prediction of the missing part is expressed as: Where X mis represents missing features, X obs1 , X obs2 Represents the existing features, p mis represents the topological position corresponding to the missing part, p obs1 、p obs2 Indicates the topological location corresponding to the existing features.
8. The unmanned system collaborative SLAM method of lifelong federated learning according to claim 1, characterized in that: The types of sensors include at least: laser radar and inertial measurement unit.
9. The unmanned system collaborative SLAM method of lifelong federated learning according to claim 8, characterized in that: The information collected by the sensor includes at least: the UAV flight speed and acceleration in three-dimensional directions, the angular momentum and angular acceleration of the UAV pitch-roll-rotation, the UAV coordinate information, and the point cloud posture information.
10. The unmanned system collaborative SLAM method of lifelong federated learning according to claim 8, characterized in that: The step S2 is specifically as follows: after the laser radar data is preprocessed, it participates in forward and backward propagation together with the IMU data, and then residual calculation and iterative state update are carried out, and finally the odometer output is obtained by convergence, and the local map is updated.
Citation Information
Cited By
Warehousing feature learning method and system based on laser SLAM
CN119672548A
Resource-constrained edge network-oriented large-scene federated 3D Gaussian sputtering scene reconstruction method and system
CN122336171A
Large-scene federated 3d gaussian sputtering scene reconstruction method and system for resource-constrained edge network
CN122336171B
