An unmanned aerial vehicle end-to-end autonomous obstacle avoidance method based on a laser radar world model

Abstract

This invention discloses an end-to-end autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) based on a lidar world model. The method includes: acquiring raw point cloud data from a lidar system and UAV state information; rotating the coordinate system of each frame of raw point cloud data, and generating a one-dimensional distance array for each horizontal obstacle within a preset sector; accumulating multiple frames of raw point cloud data and concatenating them into a two-dimensional distance array to obtain a spatiotemporal perception input; generating a latent state representation based on the spatiotemporal perception input and UAV state information using a world model; generating three-dimensional acceleration commands for the UAV based on the latent state representation using a policy network, and performing value evaluation using a value network. This invention achieves environmental modeling and prediction through a world model, completing policy learning without requiring a large amount of real-world interaction data, effectively improving sample utilization. Simultaneously, combined with the accurate perception capabilities of lidar, it enhances stability in high-speed obstacle avoidance scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent control and reinforcement learning of unmanned aerial vehicles, and in particular relates to an unmanned aerial vehicle end-to-end autonomous obstacle avoidance method based on a laser radar world model. BACKGROUND

[0002] In recent years, unmanned aerial vehicle technology has achieved large-scale application in many fields and has become a hot research direction in the field of scientific research. With the continuous expansion of the application scenarios of unmanned aerial vehicles and the continuous improvement of the task complexity, the safety guarantee and fully autonomous operation capability in the flight process have become the key requirements for the development of unmanned aerial vehicles, and the performance optimization of autonomous obstacle avoidance, as a core technology for guaranteeing the safe flight and fully autonomous operation of unmanned aerial vehicles, has become a key research topic in the related technical field.

[0003] In the field of autonomous obstacle avoidance technology of unmanned aerial vehicles, the end-to-end obstacle avoidance method based on reinforcement learning is a research hotspot in recent years. In the field of single-agent reinforcement learning in continuous action space, the near-optimal policy optimization algorithm is widely recognized for its high efficiency. However, although the existing technology provides certain technical support for the autonomous obstacle avoidance of unmanned aerial vehicles, there are still many defects in actual application: Firstly, the model-free reinforcement learning obstacle avoidance method based on the near-optimal policy optimization algorithm requires a large amount of data, and is limited by the observation range of the sensor and the design of the reward function, and the stability of high-speed obstacle avoidance is insufficient, which is difficult to meet the safe flight requirements in complex scenarios; Secondly, the existing research on reinforcement learning obstacle avoidance relies on depth cameras, and the application effect is not good in complex lighting and bad weather environments; Thirdly, the existing strategy has limited generalization ability in complex scenarios, and it is difficult to balance global fast obstacle avoidance and local direction exploration, which limits the adaptability of unmanned aerial vehicles in diversified complex environments.

[0004] Therefore, the existing technology still needs to be improved and developed. SUMMARY

[0005] The technical problem to be solved by the present application is to provide an unmanned aerial vehicle end-to-end autonomous obstacle avoidance method based on a laser radar world model, which aims to solve the problem that the existing technology cannot balance data efficiency, perception stability, scene generalization, and task adaptability, and cannot fully meet the requirements of fully autonomous and high safety flight of unmanned aerial vehicles in complex scenarios.

[0006] The technical solution adopted by the present application to solve the problem is as follows: In a first aspect, the present application provides an unmanned aerial vehicle end-to-end autonomous obstacle avoidance method based on a laser radar world model, which comprises: obtaining raw point cloud data of a laser radar and unmanned aerial vehicle state information; The coordinate system is rotated for each frame of the original point cloud data. For a preset sector interval, a one-dimensional distance array is generated by the distance to the nearest obstacle in each horizontal direction. Multiple frames of the original point cloud data are accumulated and stitched together to form a two-dimensional distance array, which is used as the spatiotemporal sensing input of the lidar. Based on the spatiotemporal perception input of the lidar and the state information of the UAV, a spatiotemporally consistent hidden state representation is generated using a world model. The UAV's three-dimensional acceleration commands are generated based on the hidden state representation through the policy network, and its value is evaluated through the value network.

[0007] In one implementation, the world model includes: A perception encoder is used to extract features from the spatiotemporal perception input of the lidar to obtain a lidar perception feature tensor. A state encoder is used to extract features from the state information of the UAV to obtain a sensing feature tensor. The cyclic state space module is used to generate a spatiotemporally consistent hidden state representation based on the joint input features obtained by concatenating the radar sensing feature tensor and the sensing feature tensor. The state decoder and perception decoder are used to reconstruct information based on the hidden state representation in order to achieve joint modeling of the environmental space and the UAV state.

[0008] In one implementation, the cyclic state space module constitutes a sequence model through gated cyclic units; the step of generating a spatiotemporally consistent hidden state representation based on the joint input features obtained by concatenating the radar sensing feature tensor and the sensing feature tensor includes: Based on the joint input features, the cyclic state is obtained, and discrete latent variables are generated by combining prior and posterior. The hidden state representation is obtained based on the cyclic state and the discrete hidden variables.

[0009] In one implementation, the training objective of the world model is to optimize a total loss function, which includes: observation and reward reconstruction loss, dynamic consistency loss, and representation regularization loss.

[0010] In one implementation method, the training method for the policy network and the value network includes: Fragments are sampled from the historical flight trajectory cache, and an imagined trajectory is generated based on the sampled fragments using the world model; The parameters of the policy network and the value network are updated using the imagined trajectory.

[0011] In one implementation, the training objective of the policy network and the value network is to maximize the total reward function, which includes: survival reward, collision reward, speed reward, position reward, obstacle avoidance reward, and smoothness reward.

[0012] In one implementation, the survival reward is a fixed preset value.

[0013] Secondly, embodiments of the present invention also provide an end-to-end autonomous obstacle avoidance system for unmanned aerial vehicles based on a lidar world model, the system comprising: The acquisition module is used to acquire raw point cloud data from the LiDAR and UAV status information. The processing module is used to rotate the coordinate system of each frame of the original point cloud data, generate a one-dimensional distance array for each preset sector interval by the distance to the nearest obstacle in each horizontal direction, accumulate multiple frames of the original point cloud data and stitch them together to form a two-dimensional distance array, and use the two-dimensional distance array as the spatiotemporal perception input of the lidar. A world model is used to generate a spatiotemporally consistent latent state representation based on the spatiotemporal perception input of the lidar and the state information of the UAV. The reinforcement learning module is used to generate three-dimensional acceleration commands for the UAV based on the hidden state representation through the policy network, and to perform value evaluation through the value network.

[0014] Thirdly, embodiments of the present invention also provide a terminal, the terminal including a memory and one or more processors; the memory stores one or more programs; the programs include instructions for executing the UAV end-to-end autonomous obstacle avoidance method based on a lidar world model as described above; the processor is used to execute the programs.

[0015] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a plurality of instructions adapted to be loaded and executed by a processor to implement the steps of the end-to-end autonomous obstacle avoidance method for unmanned aerial vehicles based on a lidar world model as described above.

[0016] The beneficial effects of this invention are as follows: This invention acquires raw point cloud data from a LiDAR and UAV state information; it rotates the coordinate system of each frame of raw point cloud data, and for a preset sector interval, generates a one-dimensional distance array based on the distance to the nearest obstacle in each horizontal direction; it accumulates multiple frames of raw point cloud data and concatenates them into a two-dimensional distance array to obtain the spatiotemporal perception input; it generates a hidden state representation based on the spatiotemporal perception input and UAV state information using a world model; it generates three-dimensional acceleration commands for the UAV based on the hidden state representation using a policy network, and performs value evaluation using a value network. This invention achieves environmental modeling and prediction through a world model, completing policy learning without requiring a large amount of real interaction data, effectively improving sample utilization. Simultaneously, combined with the accurate perception capabilities of LiDAR, it enhances stability in high-speed obstacle avoidance scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the end-to-end autonomous obstacle avoidance method for UAVs based on a lidar world model provided in this embodiment of the invention.

[0019] Figure 2 This is a schematic diagram of the method for processing polar coordinate depth information of point clouds based on mid360 single-line lidar and the spatiotemporal representation method provided in the embodiments of the present invention.

[0020] Figure 3 This is a simulation training scene effect diagram of LiDAR under the DiffAero open source platform provided in the embodiment of the present invention.

[0021] Figure 4 This is a comparison curve of the world model reconstruction loss between the LiDAR spatiotemporal representation method and the original perception representation method provided in this embodiment of the invention.

[0022] Figure 5 This is a test scenario under the Gazebo portable simulation platform provided in this embodiment of the invention, as well as the point cloud effect diagram of the mid360 LiDAR in the simulation.

[0023] Figure 6 This is a schematic diagram of the flight trajectory and speed of a drone in a simulation migration experiment provided in this embodiment of the invention.

[0024] Figure 7This is a schematic diagram of the timing processing and motion generation of multimodal data for obstacle avoidance of unmanned aerial vehicles provided in an embodiment of the present invention.

[0025] Figure 8 This is a schematic diagram of the module of the end-to-end autonomous obstacle avoidance system for UAVs based on a lidar world model provided in an embodiment of the present invention.

[0026] Figure 9 This is a schematic diagram of the terminal provided in the embodiment of the present invention. Detailed Implementation

[0027] This invention discloses an end-to-end autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) based on a lidar world model. To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention.

[0028] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0029] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0030] To address the aforementioned deficiencies in existing technologies, this invention provides an end-to-end autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) based on a lidar world model. The method includes: acquiring raw point cloud data from a lidar system and UAV state information; rotating the coordinate system of each frame of the raw point cloud data, and generating a one-dimensional distance array for each horizontally nearest obstacle within a preset sector interval; accumulating multiple frames of the raw point cloud data and concatenating them into a two-dimensional distance array, using this two-dimensional distance array as the spatiotemporal perception input of the lidar; generating a spatiotemporally consistent latent state representation based on the lidar's spatiotemporal perception input and the UAV state information using a world model; generating a three-dimensional acceleration command for the UAV based on the latent state representation using a policy network, and performing value evaluation using a value network.

[0031] This invention addresses the issues of large data volume and insufficient stability in high-speed obstacle avoidance using model-free reinforcement learning methods. It employs a world-model-driven reinforcement learning architecture, constructing an abstract world model based on environmental observations to model and predict the environment. This eliminates the need for extensive real-world interaction data to learn policies, effectively improving data efficiency and sample utilization. Furthermore, by combining this with the accurate perception capabilities of LiDAR, the impact of sensor observation range limitations is reduced, enhancing stability in high-speed obstacle avoidance scenarios.

[0032] This invention addresses the problem of insufficient verification of reinforcement learning obstacle avoidance schemes that rely heavily on depth camera and LiDAR data in existing research. It uses LiDAR as the core sensing device to directly acquire raw point cloud data. Through targeted data processing methods such as coordinate system rotation, sector interval filtering, and multi-frame cumulative stitching, it generates spatiotemporal sensing input adapted to the world model. It leverages the sensing advantages of LiDAR under complex lighting and adverse weather conditions to achieve simulation training and strategy deployment.

[0033] This invention addresses the limitations of existing strategies in generalizing to complex scenarios and the difficulty in balancing global and local obstacle avoidance requirements. It leverages a world model to generate spatiotemporally consistent hidden state representations, combining a policy network and a value network to generate accurate 3D acceleration commands based on these hidden states, and then performs real-time value assessment on these commands. This effectively enhances the generalization ability of the strategy in complex scenarios, achieving a balance between rapid global obstacle avoidance and local directional exploration.

[0034] like Figure 1 As shown, the method specifically includes the following steps: Step S100: Obtain the raw point cloud data of the lidar and the status information of the UAV.

[0035] Specifically, for the untrained world model and / or policy network, a comprehensive initialization configuration is required before data acquisition: initializing obstacle feature information, model training process, and basic parameters of the simulation scene to build a basic framework for data acquisition and subsequent training. To improve the generalization ability and robustness of the trained policy, key features such as obstacle distribution density, geometric height, and angle can be randomized to ensure that subsequent data acquisition covers diverse obstacle environments, providing a comprehensive data foundation for adaptive training. Simultaneously, to optimize hardware resources during training, a specific configuration scheme for the graphics card buffer can be defined. By rationally allocating video memory resources and optimizing data read / write processes, operational efficiency can be improved, providing hardware support for the efficient acquisition and processing of large amounts of LiDAR point cloud data. Furthermore, the core perception parameters, such as the UAV model parameters, LiDAR sensing range, and detection accuracy, need to be configured. The perception algorithm design for different reference frames of the LiDAR sensor is implemented, clarifying the core parameter standards and perception logic for LiDAR data acquisition. At this stage, the UAV model can be represented as a point mass model. ; Where d represents the drag coefficient. This represents the control delay, and parameter calibration is performed before training. This represents the acceleration command, where p, v, and g represent position velocity and gravitational acceleration, respectively. This represents the current acceleration of the drone.

[0036] The aforementioned point quality model can provide a model foundation for subsequent acquisition of UAV status information. Finally, by setting the initial deployment position of the UAV and the coordinates of the target endpoint, a complete training scenario of the starting point, obstacles, and endpoint is constructed, defining the spatial range for data acquisition.

[0037] Data acquisition is conducted within the aforementioned framework and parameter configuration: raw point cloud data from the LiDAR is acquired to gather spatial location information of obstacles within the training scene, providing environmental perception data for subsequent obstacle recognition and avoidance decisions. Simultaneously, UAV status information is obtained based on the point quality model and configured UAV body parameters. The raw point cloud data and UAV status information together constitute the foundational data source for subsequent data processing and world model building, providing raw data support for later stages.

[0038] Step S200: Rotate the coordinate system of each frame of the original point cloud data, and generate a one-dimensional distance array for each preset sector interval by the distance to the nearest obstacle in each horizontal direction; accumulate multiple frames of the original point cloud data and stitch them together to form a two-dimensional distance array, and use the two-dimensional distance array as the spatiotemporal perception input of the lidar.

[0039] like Figure 2 As shown, after acquiring the raw point cloud data, this embodiment transforms the raw point cloud data into a structured spatiotemporal perception input adapted to the subsequent world model through polar coordinate depth information representation and spatiotemporal expansion processing of the LiDAR point cloud. Specifically, it performs secondary processing on the LiDAR perception data. Each frame of raw point cloud data undergoes coordinate system rotation. This rotation decouples the point cloud information in the radar coordinate system from the radar attitude information, effectively reducing perception information fluctuations caused by UAV attitude changes. Simultaneously, this rotation, combined with coordinate system changes, ultimately yields spatial perception information in the front-left-upper coordinate system, ensuring the coordinate system consistency and usability of the perception data. Furthermore, to achieve targeted processing of key perception areas in the UAV's flight direction, this embodiment divides the data into sector intervals of a specified dimension according to the right-hand coordinate system. By maintaining an array of a specified length and cyclically updating distance information, the nearest obstacle distance in each horizontal direction is extracted, generating a one-dimensional distance array. This one-dimensional distance array is essentially a one-dimensional tensor, accurately representing obstacle information in the XY directions of 3D space, while simultaneously optimizing the perception dimension: the original radar perception dimension is defined as... Where H represents the number of horizontal intervals and V represents the number of vertical intervals; after extracting the nearest distance in each horizontal direction, the perceptual dimension is corrected to... This process eliminates redundant information in the vertical direction, achieving dimensionality reduction and key information extraction from the perceptual data. Finally, multiple frames of raw point cloud data are accumulated and stitched together to form a two-dimensional distance array, completing the temporal dimension expansion of the perceptual information and constructing a complete spatiotemporal perception representation. In each training step, the one-dimensional distance array obtained from the single frame is continuously accumulated for N frames and then stitched together, ultimately yielding a dimension of... A two-dimensional tensor is used, where the horizontal index represents spatial obstacle information and the vertical index represents temporal information. This two-dimensional distance array is the spatiotemporal representation of the LiDAR's sensing information. As the spatiotemporal sensing input of the LiDAR, it can simultaneously reflect the spatial distribution information of obstacles and the dynamic changes of obstacles in the temporal dimension, providing accurate and comprehensive sensing data for the subsequent fusion of world model and UAV state information to generate a spatiotemporally consistent hidden state representation.

[0040] The secondary processing of the aforementioned raw point cloud data requires support from a suitable simulation environment. This embodiment uses a lidar simulation based on the open-source UAV reinforcement learning platform DiffAero to implement this process. Specifically, in the DiffAero simulation platform, the deployment of a 360-degree lidar sensor is first completed to ensure comprehensive coverage of the UAV's surrounding environment, providing a hardware simulation foundation for subsequent acquisition of full-dimensional raw point cloud data. Simultaneously, a distance information perception calculation method under a fixed attitude coordinate system is designed. This method is compatible with the aforementioned lidar point cloud polar coordinate depth information representation method, further enhancing the decoupling effect between point cloud information and radar attitude information, reducing the impact of UAV attitude changes on perception information fluctuations, and ensuring the stability and accuracy of distance perception data. Furthermore, spatiotemporal accumulation processing of perception data can be directly implemented at the sensor layer without additional cross-module data transmission, improving data processing efficiency and ensuring the temporal consistency of accumulated data, thus unifying the format of perception data generated in the simulation environment with the spatiotemporal representation requirements of the framework. Furthermore, the platform includes a radar visualization module, which can display key information such as the LiDAR's sensing range and obstacle distance distribution in real time. This facilitates technicians in debugging and optimizing the radar sensor's sensing accuracy and data acquisition range, and promptly identifying and resolving anomalies in the sensing data. Through this simulation design, not only can the LiDAR sensing process in real-world scenarios be accurately reproduced, but the simulation environment can also be deeply adapted to subsequent data processing, world model building, and policy learning frameworks, providing reliable support for the synchronous migration of policies from the simulation environment to the real-world scenario.

[0041] Step S300: Using a world model, generate a spatiotemporally consistent hidden state representation based on the spatiotemporal perception input of the lidar and the state information of the UAV.

[0042] Step S400: Generate three-dimensional acceleration commands for the UAV based on the hidden state representation through the policy network, and perform value assessment through the value network.

[0043] In this embodiment, the world model can be the DreamerV3 world model. The UAV reinforcement learning obstacle avoidance method based on this world model essentially combines model building with reinforcement learning. It utilizes fully connected networks and gated recurrent units to construct an observation encoder, decoder, recurrent state space model, and actor-critic policy training framework, thereby transforming the perception data that integrates spatiotemporal information and the UAV's own state into precise and evaluable obstacle avoidance control commands.

[0044] The observation encoder includes a perception encoder and a state encoder, which are specifically adapted to the two-dimensional spatiotemporal perception input of LiDAR and UAV state information, so as to complete the feature extraction and dimensionality compression of the original input data; the cyclic state space model can capture the spatiotemporal correlation in the data, and transform discrete perception data and state information into a continuous and unified hidden state representation by dynamically modeling the temporal changes of the environment and UAV state.

[0045] The hidden state representation serves as input data to the policy network. Combined with the constructed UAV point quality model and acceleration control loop, it accurately outputs 3D acceleration commands adapted to the obstacle avoidance task's action space, ensuring that the commands directly drive the UAV to complete actions such as attitude adjustment and path avoidance. The value network evaluates the value of the generated 3D acceleration commands. This evaluation process utilizes a reward function specifically designed for obstacle avoidance tasks, quantifying the value of the commands to provide feedback signals for the iterative optimization of the policy network. Both are implemented using a multilayer perceptron.

[0046] It should be noted that the model building process can be synchronized with the agent's policy training: the world model provides direction for policy optimization through prior predictions, making the hidden state representation more aligned with the actual needs of the obstacle avoidance task. Throughout the process, the hidden state representation learned from the world model continuously updates the policy network and value network, making the obstacle avoidance commands generated by the policy network increasingly accurate, and the evaluation of the value network increasingly aligned with actual obstacle avoidance requirements, ultimately ensuring the efficiency and reliability of the UAV's obstacle avoidance decision-making.

[0047] For example, to make the 3D acceleration commands generated by the policy network more closely match the obstacle avoidance requirements of the UAV flying towards the destination, the target velocity in the world coordinate system is... Defined as a drone location Finish line and specified maximum speed Relevant quantities: ; To ensure the drone maintains its flight trend toward the target point while avoiding obstacles, and to prevent meaningless deviations, the policy network uses hidden states to represent (…). and Using the target velocity as input, the three-dimensional acceleration command for the UAV is generated. At the same time, the value network estimates the expected long-term returns based on the implicit state representation. This allows for the evaluation of the value of current acceleration commands in long-term obstacle avoidance and flight missions, providing feedback for the iterative optimization of the policy network.

[0048] In one implementation, the world model includes: A perception encoder is used to extract features from the spatiotemporal perception input of the lidar to obtain a lidar perception feature tensor. A state encoder is used to extract features from the state information of the UAV to obtain a sensing feature tensor. The cyclic state space module is used to generate a spatiotemporally consistent hidden state representation based on the joint input features obtained by concatenating the radar sensing feature tensor and the sensing feature tensor. The state decoder and perception decoder are used to reconstruct information based on the hidden state representation in order to achieve joint modeling of the environmental space and the UAV state.

[0049] Furthermore, the cyclic state space module constitutes a sequence model through gated cyclic units; the step of generating a spatiotemporally consistent hidden state representation based on the joint input features obtained by concatenating the radar sensing feature tensor and the sensing feature tensor includes: Based on the joint input features, the cyclic state is obtained, and discrete latent variables are generated by combining prior and posterior. The hidden state representation is obtained based on the cyclic state and the discrete hidden variables.

[0050] Specifically, the world model includes: a perception encoder, a state encoder, a cyclic state-space model, and a multi-output decoder (including a state decoder and a perception decoder). Both the encoder and decoder are implemented using a multilayer perceptron, while the cyclic state-space model consists of a multilayer perceptron and gated cyclic units, specifically adapted to the multi-source data fusion and spatiotemporal modeling requirements in UAV obstacle avoidance scenarios. Specifically, the perception encoder processes radar spatiotemporal observation inputs with a dimension of 72*10. It uses a multilayer perceptron to extract features and reduce the dimensionality of the LiDAR spatiotemporal perception input, outputting a compact radar perception feature tensor. This transforms high-dimensional raw sensor data into structured feature representations. The state encoder, on the other hand, targets 9-dimensional UAV state information. (Including target velocity in world coordinate system) The speed of the drone in the current world coordinate system The unit vector of the body's Z-axis The sensor is encoded and its state features are extracted using a multilayer perceptron to generate a sensing feature tensor. The two types of features are concatenated into a joint input feature. The input is fed into the cyclic state space module to learn spatiotemporally consistent hidden state representations. The cyclic state space model is used to construct a sequence model through gated cyclic units. First, the cyclic state is obtained based on the joint input features. By leveraging the memory mechanism of gated recurrent units, temporal correlations in multi-source data can be captured, while also incorporating prior distributions. With posterior distribution Generate discrete latent variables This approach models the uncertainties and dynamic changes in the environment. Cyclic states and discrete latent variables together constitute a spatiotemporally consistent latent state representation. This method generates a latent state representation that incorporates both the spatiotemporal distribution information of environmental obstacles sensed by the lidar and the dynamic evolution of the UAV's own state, ensuring consistency of the latent state in both time and space. Finally, the state decoder and perception decoder reconstruct information based on this latent state representation. The state decoder reconstructs the UAV's state information, while the perception decoder reconstructs the spatiotemporal perception input from the lidar. This reconstruction enables joint modeling of the environmental space and the UAV's state, not only validating the effectiveness of the latent state representation but also enabling the world model to predict the environment and state, providing a reliable modeling foundation for the subsequent imaginary trajectory generation of the policy network.

[0051] In one implementation, the training objective of the world model is to optimize a total loss function, which includes: observation and reward reconstruction loss, dynamic consistency loss, and representation regularization loss.

[0052] Specifically, the training process of the world model is essentially a continuous optimization of the total loss function, with the aim of enabling the world model to accurately model the environmental dynamics and the drone's own dynamic characteristics in obstacle avoidance scenarios. Total Loss Function This can be represented as the observation and reward reconstruction loss. kinetic consistency loss and characterization of regularization loss The weighted sum of the three:

[0053] in, =1, =0.5, =0.1 represents the weight of each loss.

[0054] The observation and reward reconstruction loss oversees the reconstruction of observations, the prediction of reward information, and the prediction of termination signals, enhancing the world model's ability to characterize the environment. In this embodiment, observation reconstruction and reward prediction reconstruction are represented using mean squared error, while the termination signal prediction loss is represented using cross-entropy loss. The dynamic consistency loss utilizes KL divergence to measure the difference between the model's prior and posterior distributions of future latent states, constraining the dynamic consistency of the latent space. The characterization regularization loss improves the stability of latent representations, employing a latent state smoothing regularization form to constrain excessive changes in latent states.

[0055] In actual training, the optimization of the total loss function is accomplished through the GPU-accelerated DiffAero simulator. The system adopts a self-supervised learning approach, optimizing the world model parameters by minimizing the weighted sum of the dynamics loss (corresponding to dynamics consistency loss) and the representation loss (corresponding to observation and reward reconstruction loss, and representation regularization loss), enabling the world model to accurately reconstruct environmental obstacle information and UAV dynamic characteristics.

[0056] In one implementation, the training method for the policy network and the value network includes: Fragments are sampled from the historical flight trajectory cache, and an imagined trajectory is generated based on the sampled fragments using the world model; The parameters of the policy network and the value network are updated using the imagined trajectory.

[0057] In this embodiment, the training process of the policy network and value network is implemented by generating imagined trajectories using a world model. This eliminates the need for additional simulator interaction and enables efficient iterative optimization of network parameters. First, short context fragments are sampled from a historical flight trajectory cache storing the UAV's previous flight data to initialize the imagined trajectory. These context fragments may contain key data such as LiDAR spatiotemporal perception input, UAV state information, action commands, and reward feedback. Then, using a world model that has been trained and possesses accurate modeling capabilities, subsequent imagined trajectories are autonomously generated based on the sampled short context fragments. These imagined trajectories encompass hidden state transitions, rewards, and termination flag sequences. The reward calculation is based on a preset total reward function, and the termination flag is determined based on whether the UAV collides with an obstacle or reaches the target location. The entire imagined trajectory generation process relies entirely on the world model's environment and state modeling capabilities, eliminating the need for real-time interaction with a real simulator or simulation platform, significantly reducing the hardware resource consumption for training. Next, the policy network and value network are updated using imagined trajectories. The policy network optimizes the generation logic of 3D acceleration commands based on the hidden state representation in the imagined trajectory, making it easier to obtain higher cumulative rewards. The value network adjusts the estimation accuracy of long-term expected returns, making its predicted values ​​closer to the actual cumulative rewards in the imagined trajectory. The two networks achieve collaborative optimization with the support of the imagined trajectory. Throughout the training process, the initial and target positions of the UAV are randomly generated in the environment to simulate diverse obstacle avoidance scenarios. Simultaneously, the processes of world model feature extraction, policy interaction sampling, and network parameter updates are repeatedly executed until the policy converges. Ultimately, this enables the UAV to possess agile obstacle avoidance and navigation capabilities within a limited perception range.

[0058] In one implementation, the training objective of the policy network and the value network is to maximize the total reward function, which includes: survival reward, collision reward, speed reward, position reward, obstacle avoidance reward, and smoothness reward.

[0059] Furthermore, the survival reward is a fixed preset value.

[0060] Specifically, in the training of the actor-critic model, the total reward function is defined as follows: The training objective (i.e., the task objective) is to maximize the reward function. It includes survival bonuses, collision bonuses, speed bonuses, position bonuses, obstacle avoidance bonuses, and smoothness bonuses. The calculation methods for each type of bonus are as follows: Collision Rewards Represented as: To encourage drones to reduce collisions; Speed ​​Reward Represented as: To encourage drones to fly at speeds close to the target; Location Rewards Represented as: To encourage drones to approach the target endpoint; Obstacle Avoidance Rewards Represented as: To encourage drones to stay away from obstacles, among which, This represents the distance to the nearest perceived obstacle. This represents the radar's sensing range, set to 10m. Smooth rewards Represented as: To encourage drones to generate smooth motion commands; Finally, in the reward function, +0.3 represents the survival reward. In a reinforcement learning environment, a fixed reward can be used to encourage drones to survive longer during training, thereby reducing the likelihood of drones crashing or colliding with obstacles and triggering training termination.

[0061] like Figure 3 The diagram illustrates the effect of the simulation training process using the method described in this embodiment. Figure 4As shown, the world model reconstruction results during the training process demonstrate that the spatiotemporal distance information representation method proposed in this embodiment has better prediction performance, lower reconstruction loss compared to the original 3D spatial perception input, and the generated imagined trajectory is more consistent with the real flight scenario. To improve the inference speed of model deployment, after the model training converges and offline testing is completed, the trained policy network and value network models are exported as a high-performance inference model in ONNX format through a simulation platform. This format has the technical advantages of strong cross-platform compatibility and fast inference speed, and can effectively adapt to the embedded deployment scenario of UAVs. Subsequently, model migration deployment tests are carried out in the Gazebo high-fidelity simulation environment. The attitude control scheme natively supported by PX4 flight controller is adopted to convert the acceleration and yaw information output by the network into attitude thrust input, realizing precise closed-loop control of UAV attitude and position, and further verifying the actual obstacle avoidance effect of the trained policy. The running effect of the Gazebo instance scenario and the mid360 single-line LiDAR point cloud representation method is shown in the figure. Figure 5 As shown. The final model deployment effect, migration experiment flight trajectory, and flight speed are as follows. Figure 6 As shown, in complex obstacle scenarios, the drone can achieve high-speed operation of over 5m / s.

[0062] After the policy network and value network training converges and a high-performance inference model in ONNX format is exported, considering the uncertainties in the actual deployment of neural network models, which can easily lead to large errors and UAV crashes during transfer experiments, this embodiment also designs a Gazebo transferable simulation platform and control signal conversion interface. This constructs a Gazebo co-simulation experimental platform supporting cross-scenario transfer verification, providing technical support for a smooth transition from simulation training to real-world deployment. The platform is equipped with the PX4-Autopilot open-source flight controller, enabling precise simulation and control of the UAV's flight status. It also integrates a mid360 single-line LiDAR simulation module, restoring the detection range, data output format, and noise characteristics of real radar, ensuring consistency between the simulation environment and the actual UAV perception data. Furthermore, the platform supports ONNX format policy network inference, enabling software-in-the-loop verification experiments after world model obstacle avoidance strategy training.

[0063] To facilitate understanding of the method of the present invention, as follows: Figure 7As shown, the complete closed-loop process from multi-source input to latent state generation, information reconstruction, and action output is presented. At time Stept, the system first receives two types of core inputs: one is radar perception input, corresponding to a two-dimensional spatiotemporal distance array accumulated over multiple frames, presented in the form of a grayscale heatmap, where different bright and dark areas represent the distances to obstacles and their temporal changes in different horizontal sectors; the other is state input, containing multi-dimensional state information such as the UAV's x-axis component, velocity in the world coordinate system, and target velocity, presented in the form of a labeled array. Subsequently, the radar perception input is fed into the perception encoder, where features are extracted by a multilayer perceptron to generate a radar perception feature tensor; the state input is fed into the state encoder, where UAV state features are also extracted by a multilayer perceptron to generate a sensing feature tensor.

[0064] These two types of feature tensors are concatenated into a joint input feature, which is then fed into a recurrent state space module composed of gated recurrent units. The module evolves the recurrent state based on the joint input feature and generates discrete latent variables by combining prior and posterior distributions. Finally, these are fused to generate a spatiotemporally consistent latent state representation. To verify the effectiveness of the latent state representation, the system reconstructs the radar perception input and state input using a perception decoder and a state decoder, respectively. By comparing the similarity between the original input and the reconstructed output, self-supervised verification of the world model's modeling accuracy is achieved. Finally, based on the latent state representation at the current time step, the policy network generates the corresponding 3D acceleration command. At step 1, the system repeats the above process: receiving new radar perception input and state input, generating an updated latent state representation through encoding and the recurrent state space module, and outputting a new acceleration command after information reconstruction. This iterative temporal operation allows the world model to continuously capture the dynamic changes in the environment and the UAV's state, providing the policy network with a coherent spatiotemporally consistent latent state representation, thereby enabling continuous autonomous obstacle avoidance decisions by the UAV in complex scenarios.

[0065] This invention addresses the technical goal of high-speed fully autonomous obstacle avoidance in complex scenarios for unmanned aerial vehicles (UAVs). By optimizing perception representation and model architecture and designing cross-platform transfer, it improves perception stability, learning efficiency, obstacle avoidance performance, and deployment reliability. Specifically, this is reflected in the following aspects: Firstly, at the environmental perception level, this invention utilizes a polar coordinate depth representation of LiDAR point clouds and a spatiotemporal perception modeling method. Based on the UAV's current position, a 360-degree sector-shaped perception zone is divided. The distance to the nearest obstacle in each horizontal zone is extracted and accumulated over multiple frames as perception input, successfully decoupling the UAV's attitude changes from perception information and effectively reducing the interference of attitude fluctuations on environmental perception. Simultaneously, through an integrated 3D LiDAR simulation module, the actual radar detection range, data format, and noise characteristics are reproduced, ensuring the authenticity and reliability of the perception data and providing accurate data support for subsequent obstacle avoidance decisions.

[0066] Secondly, regarding learning efficiency and planning capabilities, this invention constructs a world model using the DreamerV3 hierarchical reinforcement learning architecture, integrating spatiotemporal observations from LiDAR and UAV status data (including multimodal information such as inertial navigation devices). It optimizes environmental dynamic modeling and long-term planning capabilities through self-supervised learning. Simultaneously, it builds a cyclic state space model and an actor-critic framework, designing a robust total loss function, enabling the UAV to accurately understand the environment and predict future obstacle distribution. This simultaneously improves policy learning efficiency and sample utilization, effectively solving the problem of high data requirements in traditional obstacle avoidance methods.

[0067] Thirdly, regarding obstacle avoidance performance, a multi-dimensional composite reward function was designed, incorporating survival, collision, speed, position, obstacle avoidance, and smoothness rewards. This balances obstacle avoidance safety, flight speed, and trajectory smoothness, enabling the system to support high-speed obstacle avoidance requirements exceeding 5 m / s, significantly improving the agility and stability of the UAV in complex and dense obstacle avoidance scenarios. Simultaneously, a simplified point mass model of the UAV was constructed, and an accelerometer controller and a parallelizable dense obstacle avoidance scenario were built based on the PyTorch deep learning framework, further ensuring the precise execution of obstacle avoidance actions.

[0068] Fourth, at the migration and deployment level, by building the DiffAero LiDAR simulation platform and the Gazebo high-fidelity migration verification platform, and using the PX4-Autopilot open-source flight control system to reproduce the real flight control logic, a smooth transition from simulation training to actual aircraft deployment was achieved. Leveraging ONNX format model export and software-in-the-loop verification, it combines cross-platform compatibility with high-speed inference advantages, effectively adapting to embedded deployment scenarios and significantly reducing performance degradation and operational risks during model migration.

[0069] Overall, this invention, through an end-to-end reinforcement learning method integrating 3D LiDAR and the DreamerV3 world model, effectively fuses multimodal sensing information and leverages the environmental understanding and prediction capabilities of the world model to successfully address key issues such as weak generalization ability and insufficient high-speed stability in traditional obstacle avoidance methods. Furthermore, through a complete link design encompassing sensor input design, model training optimization, and flight control command conversion, and based on the execution strategy output of the PX4 flight control system, it accurately converts acceleration and yaw angle into motor commands, ultimately achieving fully autonomous and safe flight of the UAV in complex environments. This provides an efficient and feasible technical solution for autonomous navigation applications of UAVs in the fields of robotics, artificial intelligence, and control technology.

[0070] Based on the above embodiments, the present invention also provides an end-to-end autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on a lidar world model, such as... Figure 8 As shown, the system includes: Acquisition module 01 is used to acquire raw point cloud data from the lidar and UAV status information; Processing module 02 is used to rotate the coordinate system of each frame of the original point cloud data, generate a one-dimensional distance array for each preset sector interval by the distance to the nearest obstacle in each horizontal direction; accumulate multiple frames of the original point cloud data and stitch them together to form a two-dimensional distance array, and use the two-dimensional distance array as the spatiotemporal perception input of the lidar; World Model 03 is used to generate a spatiotemporally consistent hidden state representation based on the spatiotemporal perception input of the lidar and the state information of the UAV. The reinforcement learning module 04 is used to generate three-dimensional acceleration commands for the UAV based on the hidden state representation through the policy network, and to perform value evaluation through the value network.

[0071] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 9 As shown, the terminal includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an end-to-end autonomous obstacle avoidance method for UAVs based on a LiDAR world model. The display screen can be an LCD screen or an e-ink screen.

[0072] Those skilled in the art will understand that Figure 9 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0073] In one implementation, the terminal's memory stores one or more programs, and these programs are configured to be executed by one or more processors, and the programs contain instructions for performing an end-to-end autonomous obstacle avoidance method for a drone based on a lidar world model.

[0074] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0075] In summary, this invention discloses an end-to-end autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) based on a lidar world model. The method includes: acquiring raw point cloud data from a lidar system and UAV state information; rotating the coordinate system of each frame of the raw point cloud data, and generating a one-dimensional distance array for each horizontal obstacle within a preset sector interval; accumulating multiple frames of the raw point cloud data and concatenating them into a two-dimensional distance array, using this two-dimensional distance array as the spatiotemporal perception input of the lidar; generating a spatiotemporally consistent latent state representation based on the lidar's spatiotemporal perception input and the UAV state information using a world model; generating a three-dimensional acceleration command for the UAV based on the latent state representation using a policy network, and performing value evaluation using a value network. This invention achieves environmental modeling and prediction by constructing an abstract world model based on environmental observation, completing policy learning without relying on a large amount of real interaction data, effectively improving data efficiency and sample utilization. Simultaneously, combined with the accurate perception capability of lidar, it reduces the impact of sensor observation range limitations and enhances stability in high-speed obstacle avoidance scenarios.

[0076] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. An end-to-end autonomous obstacle avoidance method for unmanned aerial vehicles (UAVs) based on a lidar world model, characterized in that, The method includes: Acquire raw point cloud data from the lidar and UAV status information; The coordinate system is rotated for each frame of the original point cloud data. For a preset sector interval, a one-dimensional distance array is generated by the distance to the nearest obstacle in each horizontal direction. Multiple frames of the original point cloud data are accumulated and stitched together to form a two-dimensional distance array, which is used as the spatiotemporal sensing input of the lidar. Based on the spatiotemporal perception input of the lidar and the state information of the UAV, a spatiotemporally consistent hidden state representation is generated using a world model. The UAV's three-dimensional acceleration commands are generated based on the hidden state representation through the policy network, and its value is evaluated through the value network.

2. The end-to-end autonomous obstacle avoidance method for UAVs based on a lidar world model according to claim 1, characterized in that, The world model includes: A perception encoder is used to extract features from the spatiotemporal perception input of the lidar to obtain a lidar perception feature tensor. A state encoder is used to extract features from the state information of the UAV to obtain a sensing feature tensor. The cyclic state space module is used to generate a spatiotemporally consistent hidden state representation based on the joint input features obtained by concatenating the radar sensing feature tensor and the sensing feature tensor. The state decoder and perception decoder are used to reconstruct information based on the hidden state representation in order to achieve joint modeling of the environmental space and the UAV state.

3. The end-to-end autonomous obstacle avoidance method for UAVs based on a lidar world model according to claim 2, characterized in that, The cyclic state space module constitutes a sequence model through gated cyclic units; the step of generating a spatiotemporally consistent hidden state representation based on the joint input features obtained by concatenating the radar sensing feature tensor and the sensing feature tensor includes: Based on the joint input features, the cyclic state is obtained, and discrete latent variables are generated by combining prior and posterior. The hidden state representation is obtained based on the cyclic state and the discrete hidden variables.

4. The end-to-end autonomous obstacle avoidance method for UAVs based on a lidar world model according to claim 1, characterized in that, The training objective of the world model is to optimize the total loss function, which includes: observation and reward reconstruction loss, dynamic consistency loss, and representation regularization loss.

5. The end-to-end autonomous obstacle avoidance method for UAVs based on a lidar world model according to claim 1, characterized in that, The training methods for the policy network and the value network include: Fragments are sampled from the historical flight trajectory cache, and an imagined trajectory is generated based on the sampled fragments using the world model; The parameters of the policy network and the value network are updated using the imagined trajectory.

6. The end-to-end autonomous obstacle avoidance method for UAVs based on a lidar world model according to claim 5, characterized in that, The training objective of the policy network and the value network is to maximize the total reward function, which includes: survival reward, collision reward, speed reward, position reward, obstacle avoidance reward, and smoothness reward.

7. The end-to-end autonomous obstacle avoidance method for UAVs based on a lidar world model according to claim 6, characterized in that, The survival reward is a fixed preset value.

8. An end-to-end autonomous obstacle avoidance system for unmanned aerial vehicles (UAVs) based on a lidar world model, characterized in that, The system includes: The acquisition module is used to acquire raw point cloud data from the LiDAR and UAV status information. The processing module is used to rotate the coordinate system of each frame of the original point cloud data, generate a one-dimensional distance array for each preset sector interval by the distance to the nearest obstacle in each horizontal direction, accumulate multiple frames of the original point cloud data and stitch them together to form a two-dimensional distance array, and use the two-dimensional distance array as the spatiotemporal perception input of the lidar. A world model is used to generate a spatiotemporally consistent latent state representation based on the spatiotemporal perception input of the lidar and the state information of the UAV. The reinforcement learning module is used to generate three-dimensional acceleration commands for the UAV based on the hidden state representation through the policy network, and to perform value evaluation through the value network.

9. A terminal, characterized in that, The terminal includes a memory and one or more processors; the memory stores one or more programs; the programs contain instructions for executing the UAV end-to-end autonomous obstacle avoidance method based on a lidar world model as described in any one of claims 1 to 7; the processors are used to execute the programs.

10. A computer-readable storage medium storing a plurality of instructions thereon, characterized in that, The instructions are applicable to be loaded and executed by a processor to implement the steps of the end-to-end autonomous obstacle avoidance method for unmanned aerial vehicles based on a lidar world model as described in any one of claims 1 to 7.

Images