Obstacle avoidance method and device based on laser radar and vector field histogram direction search, equipment and medium

Abstract

The invention discloses an obstacle avoidance method and device based on a laser radar and vector field histogram direction search, equipment and a medium. The method comprises the following steps: simulating an unmanned aerial vehicle based on strategy network and environment interaction to collect a track sequence; performing vector field histogram direction search on the laser radar sensing data of each time step to obtain a local obstacle avoidance direction of each time step; based on the local obstacle avoidance direction and state data of each time step, determining a state value of each time step through a value network; based on the data tuple and the state value of each time step, updating the policy network and the value network by using a near-end policy optimization algorithm to obtain a target policy network; and controlling the unmanned aerial vehicle to deploy the target strategy network so as to perform obstacle avoidance flight based on the target strategy network. The laser radar sensing data is added to the state data, and the obstacle avoidance flight effect is improved by using the memory ability of the long-short term memory neural network. Meanwhile, a local obstacle avoidance direction obtained by performing vector field histogram direction search on laser radar sensing data is added into state data input into the value network, the intelligent agent is guided to quickly learn a flight strategy fitting a vector field histogram safety direction, and the obstacle avoidance efficiency and stability are greatly improved.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) control technology, and in particular to an obstacle avoidance method, device, equipment, and medium based on lidar and vector field histogram direction search. Background Technology

[0002] Currently, most autonomous obstacle avoidance methods for drones divide the obstacle avoidance task into three stages: perception, planning, and control. The perception stage uses sensors (such as depth cameras and LiDAR modules) to acquire information about external obstacles; the planning stage uses this information to plan an obstacle avoidance trajectory suitable for stable drone flight; and the control stage uses PID control, model predictive control, and other methods to track the obstacle avoidance trajectory. However, the planning stage requires extensive real-time calculations to obtain the obstacle avoidance trajectory suitable for stable drone flight, and the control stage may fail to accurately track the obstacle avoidance trajectory due to air resistance and errors in the planning model, thus affecting the effectiveness of autonomous obstacle avoidance for drones.

[0003] Therefore, the existing technology still needs to be improved and enhanced. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide an obstacle avoidance method, device, equipment and medium based on laser radar and vector field histogram direction search, which addresses the shortcomings of the existing technology.

[0005] To address the aforementioned technical problems, the first aspect of this application provides an obstacle avoidance method based on laser radar and vector field histogram direction search, wherein the obstacle avoidance method based on laser radar and vector field histogram direction search specifically includes: The UAV interacts with the environment based on a policy network to collect trajectory sequences. Each time step in the trajectory sequence includes data tuples for state data, action, reward, and next state data. The state data includes feature data from LiDAR perception data, UAV state information, and target position information. The policy network includes a long short-term memory neural network and a multilayer perceptron. Perform vector field histogram direction search on the LiDAR sensing data at each time step to obtain the local obstacle avoidance direction at each time step; Based on the local obstacle avoidance direction and state data at each time step, the state value at each time step is determined through a value network. Based on the data tuples and state values ​​at each time step, the policy network and the value network are updated using the near-end policy optimization algorithm to obtain the target policy network. Control the drone to deploy the target policy network to perform obstacle avoidance flight based on the target policy network.

[0006] The obstacle avoidance method based on lidar and vector field histogram direction search, wherein the process of acquiring the state data specifically includes: Acquire lidar perception data of the obstacle scene, and extract features from the lidar perception data to obtain feature data. The lidar perception data is used to reflect obstacle information in the obstacle scene. The feature data, the UAV status information, and the target location information are concatenated to obtain the status data.

[0007] The obstacle avoidance method based on lidar and vector field histogram direction search includes UAV state data such as UAV position information, UAV attitude information, UAV speed information, UAV orientation information, and UAV thrust information.

[0008] The obstacle avoidance method based on lidar and vector field histogram direction search, wherein acquiring lidar perception data of the obstacle scene specifically includes: Obstacle distance distribution data in polar coordinates is obtained using lidar; A fixed-dimensional sector spatial division method is used to determine the nearest point cloud distance within each sector region in order to obtain LiDAR perception data for obstacle scenes.

[0009] The obstacle avoidance method based on lidar and vector field histogram direction search, wherein the step of performing vector field histogram direction search on the lidar sensing data at each time step to obtain the local obstacle avoidance direction at each time step specifically includes: For each time step, the optimal direction that is closest to the target direction and less than a preset distance threshold is searched based on the vector field histogram in the lidar sensing data. If an optimization direction is found, then the optimization direction is used as the local obstacle avoidance direction for the time step. If no optimization direction is found, the local obstacle avoidance direction of the previous time step is used as the local obstacle avoidance direction of the current time step.

[0010] The obstacle avoidance method based on lidar and vector field histogram direction search, wherein the reward is determined based on a pre-constructed reward function, which includes a high-speed distance reward function, a velocity direction reward function, a safety reward function, a normal vector constraint reward function, and a survival reward function. Specifically, the high-speed distance reward function is used to shorten the time it takes for the UAV to reach the target location; the velocity direction reward function is used to incentivize the UAV to maintain high speed and fly towards the target direction; the safety reward function is used to improve obstacle avoidance safety; the normal vector constraint reward function is used to limit the UAV's normal vector tilt angle; and the survival reward function is used to extend the UAV's survival time in the environment.

[0011] The obstacle avoidance method based on lidar and vector field histogram direction search, wherein, after updating the policy network and the value network using a near-end policy optimization algorithm based on the data tuples and state values ​​at each time step to obtain the target policy network, the method further includes: Export the target policy network; Build a drone obstacle flight simulation scenario in a simulation environment; Based on the target strategy, the flight control commands of the simulated UAV in the UAV obstacle flight simulation scenario are determined to realize the migration verification of the target strategy network from simulation to simulation, and the verified target strategy network is deployed on the UAV.

[0012] The obstacle avoidance method based on lidar and vector field histogram direction search includes the following steps: during the process of updating the policy network and the value network using the near-end policy optimization algorithm, a generalized advantage estimation is used to determine the advantage function to constrain the update of the policy network, and a mean squared error loss function is used to optimize the value network.

[0013] A second aspect of this application provides an obstacle avoidance device based on laser radar and vector field histogram direction search, wherein the obstacle avoidance device based on laser radar and vector field histogram direction search specifically includes: The collection module is used to simulate the interaction between the UAV and the environment based on the policy network to collect trajectory sequences. The data tuples of each time step in the trajectory sequence include state data, action, reward and next state data. The state data includes feature data of lidar perception data and UAV state information. The policy network includes a long short-term memory neural network and a multilayer perceptron. The training module is used to perform vector field histogram direction search on the LiDAR perception data at each time step to obtain the local obstacle avoidance direction at each time step. Based on the local obstacle avoidance direction and state data at each time step, the state value at each time step is determined through the value network. Based on the data tuples and state value at each time step, the policy network and the value network are updated using the near-end policy optimization algorithm to obtain the target policy network. The control module is used to control the UAV to deploy the target policy network so as to perform obstacle avoidance flight based on the target policy network.

[0014] A third aspect of this application provides a terminal device, which includes: a processor and a memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps in the obstacle avoidance method based on lidar and vector field histogram direction search as described above.

[0015] Beneficial effects: 1. This application determines state data based on lidar perception data and UAV state information, and uses the state data as input data for a strategy network composed of a long short-term memory neural network and a multi-layer perception mechanism. This allows the strategy network to effectively process radar perception information and improve obstacle avoidance flight performance by utilizing the memory capability of the long short-term memory neural network.

[0016] 2. This application constructs the obstacle distance distribution in polar coordinates based on radar point clouds, performs optimization direction search based on vector field histogram on lidar perception data to obtain local obstacle avoidance direction, and adds the local obstacle avoidance direction to the state data of the value network to guide the UAV to stable obstacle avoidance flight.

[0017] 3. The policy network in this application makes decisions based solely on the state data observed in the global direction, while the value network adds locally searchable and observable obstacle avoidance directions to the state data observed in the global direction. By using the deviation between the local obstacle avoidance direction and the actual flight path of the UAV as a penalty term, the agent is guided to quickly learn a flight strategy that fits the safe direction of the vector field histogram, thereby significantly improving obstacle avoidance efficiency and stability.

[0018] 4. In this application, a fixed-dimensional sector space is divided under the radar coordinate system. By maintaining an array of fixed length, the nearest point cloud distance under each region index is updated cyclically to obtain the lidar sensing data. This achieves the unification of the simulation and actual radar representation methods and improves the accuracy of the lidar sensing data obtained by simulation.

[0019] 5. This application supports network-to-network simulation migration and deployment verification to reduce migration errors in physical deployment through joint simulation debugging. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart of an obstacle avoidance method based on lidar and vector field histogram direction search provided in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of a real-time simulation training screen.

[0023] Figure 3 This is a schematic diagram of the network structure of a policy network.

[0024] Figure 4 This is a schematic diagram of the obstacle avoidance direction search process based on vector field histograms.

[0025] Figure 5 This is a screenshot showing the running effect of the point cloud representation method based on mid360 single-line LiDAR.

[0026] Figure 6 This is a diagram showing the execution trajectory of a strategy deployed in a Gazebo simulation scenario.

[0027] Figure 7 This is a block diagram illustrating the principle of an obstacle avoidance device based on lidar and vector field histogram direction search, provided in an embodiment of this application.

[0028] Figure 8 A schematic block diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0029] This application provides an obstacle avoidance method, apparatus, device, and medium based on lidar and vector field histogram direction search. To make the objectives, technical solutions, and effects of this application clearer and more explicit, the following detailed description is provided with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0030] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0031] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0032] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0033] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0034] This embodiment provides an obstacle avoidance method based on lidar and vector field histogram direction search, such as... Figure 1 As shown, the method includes: S10. Simulate the interaction between the UAV and the environment based on the policy network to collect trajectory sequences.

[0035] Specifically, the trajectory sequence is a simulated flight trajectory under obstacle scenarios, and during the collection of this trajectory sequence, the UAV interacts with the environment based on flight control commands generated by the policy network. Therefore, before collecting the trajectory sequence, an obstacle simulation scenario, a policy network, a UAV model, and LiDAR perception parameters are first constructed, such as... Figure 2 As shown, obstacle simulation scenarios can be obstacle avoidance scenarios with randomized obstacle density and height information to improve the generalization and robustness of policy networks.

[0036] like Figure 3 As shown, the policy network includes a Long Short-Term Memory (LSTM) neural network and a Multilayer Perceptron (MLP). The LSTM and MLP are cascaded. The input to the LSTM is state data, and the input to the MLP is the output of the LSTM. The output of the MLP is the action. This application uses a network composed of a cascaded LSTM and MLP as the policy network, enabling the policy network to effectively process radar sensing information and utilize the memory capability of the LSTM to improve obstacle avoidance flight performance.

[0037] In simulating the interaction between a UAV and its environment based on a policy network, multiple initial and target position pairs can be initialized for the UAV. Then, a trajectory sequence is formed based on each initial and target position. The trajectory sequence collection process involves first acquiring the state data of the initial position, then using the policy network to generate actions based on this state data, and determining the next state data based on these actions. Simultaneously, a reward is determined according to a preset reward function, and then the policy network generates the next action based on the next state data, and so on, until the UAV reaches the target position. After acquiring the trajectory sequence, for each time step in the trajectory sequence, the corresponding state data, actions, and rewards for that time step, as well as the state data for the next time step, are read to determine the data tuple corresponding to that time step. This data tuple includes state data, actions, rewards, and the next state data. The state data includes feature data from the LiDAR sensing data and UAV state information.

[0038] In one embodiment, the process of acquiring the state data specifically includes: Acquire lidar perception data of the obstacle scene, and extract features from the lidar perception data to obtain feature data. The lidar perception data is used to reflect obstacle information in the obstacle scene. The feature data, the UAV status information, and the target location information are concatenated to obtain the status data.

[0039] Specifically, lidar perception data is obtained through lidar sensing. LiDAR emits short-pulse laser beams, which reach surrounding objects, are reflected, and are detected by the lidar receiver. The generated point cloud signal is then processed into distance information. In other words, lidar perception data represents the distance information between the UAV and obstacles, which can be represented as a high-dimensional tensor. The number of columns in this high-dimensional tensor represents the number of longitudinal cross-sectional perception data points of the lidar, and the number of rows represents the number of lateral cross-sectional perception data points.

[0040] Furthermore, when extracting features from the LiDAR sensing data, the LiDAR sensing data can first be converted into a one-dimensional tensor, and then feature extraction can be performed on this one-dimensional tensor to obtain feature data. The number of elements in the one-dimensional tensor is the sum of the total number of all LiDAR data. Feature extraction can be performed using a preset feature extraction model (such as a multilayer perceptron), meaning the LiDAR sensing data can be input into the preset feature extraction model, and the preset feature extraction model can output feature data.

[0041] The UAV's state information is obtained through global observation of the UAV and can be determined by the mission it is performing. For example, when a UAV is performing a flight mission, its state information may include its position, attitude, speed, orientation, and thrust. Target position information reflects the positional relationship between the target's current position and its initial position; this information may include the target's direction and yaw angle. Therefore, the state data can be represented as follows: ,in, Indicates the location information of the drone. Indicates the attitude information of the drone. Indicates drone speed information, Indicates the drone's orientation information. Indicates the thrust information of the drone. Representing feature data, Indicates the direction of the target. Indicates the target yaw angle.

[0042] In one embodiment, due to size and weight limitations of the lidar, it is difficult to use a multi-line lidar in a live experiment. Therefore, a mid360 single-line cyclic scanning lidar is selected for live verification, and the lidar perception data is represented by the lidar polar coordinate point cloud distance. Based on this, the acquisition of lidar perception data for the obstacle scene specifically includes: Obstacle distance distribution data in polar coordinates is obtained using lidar; A fixed-dimensional sector spatial division method is used to determine the nearest point cloud distance within each sector region in order to obtain LiDAR perception data for obstacle scenes.

[0043] Specifically, the fixed-dimensional sector space partitioning method refers to dividing a fixed-dimensional sector region in the lidar coordinate system and configuring a perception array of fixed-dimensional length. This fixed-dimensional length perception array is used to store the nearest point cloud distance in each sector region. Then, by updating this fixed-dimensional length perception array, lidar perception data can be obtained, thus realizing the unification of simulation and actual lidar representation.

[0044] In one embodiment, the reward is determined based on a pre-built reward function, which includes a high-speed distance reward function, a velocity direction reward function, a safety reward function, a normal vector constraint reward function, and a survival reward function. The high-speed distance reward function aims to enable the UAV to reach the target location in the shortest possible time. This function includes tight position constraints and relaxed position constraints. By adjusting the ratio of these constraints, the UAV is encouraged to quickly approach the target location or perform a certain degree of location exploration to avoid lock-up. Therefore, the high-speed distance reward function can be expressed as: ,

[0045]

[0046] < , in, This represents the high-speed distance reward function. This indicates a positionally tight constraint. This indicates the position relaxation constraint term. Indicates the time factor. Indicates the initial round length. Indicates the distance between the drone and the target location. and All represent weighting coefficients. t< This is a conditional statement; if true, it returns 1; otherwise, it returns 0.

[0047] The high-speed distance reward function provided in this application is... t< By allocating constraints and weighting coefficients, the drone can reach the target location in a short time, while also mitigating collision and deadlock issues.

[0048] The velocity direction reward function incentivizes the drone to maintain high speed and fly towards the target direction. By controlling the drone to maintain a relatively high speed during obstacle avoidance and ensuring that the velocity direction is as close to the target point as possible, the obstacle avoidance task execution time is reduced. The velocity direction reward function can be expressed as follows: , in, This represents the reward function in the direction of velocity. Represents the drone's velocity state vector. This represents the direction vector of the UAV's optimal position, i.e., the unit direction vector obtained from the vector field histogram search. This indicates that the function sets a threshold. This indicates the preset function threshold. This represents the weighting coefficient.

[0049] The safety reward function is used to improve obstacle avoidance safety, controlling the drone to avoid collisions with obstacles and completing obstacle avoidance flight tasks along the safest possible path. The safety reward function can be expressed as: , in, This represents the security reward function. This indicates the distance information scanned by the drone's onboard lidar. This represents the average value of the distance information obtained. This is used to take the logarithm of the average to reduce the range of the distance term, thereby unifying the value of the reward function term.

[0050] The normal vector constraint reward function is used to limit the tilt angle of the UAV's normal vector. By limiting the direction of the UAV's normal vector, it prevents the UAV from tilting too much during obstacle avoidance, which could cause it to roll over and crash. The normal vector constraint reward function is expressed as follows:

[0051] in, The normal vector constraint reward function, Indicates the drone's orientation information. This represents the z-axis component of the UAV's normal vector, thereby constraining the orientation of the UAV's normal vector.

[0052] The survival reward function is used to extend the survival time of the drone in the environment. A fixed reward can be used to encourage the drone to survive longer during training, thereby reducing the possibility of the drone crashing or colliding with obstacles and triggering training termination. The survival reward function can be set to a default value, such as 0.1.

[0053] Based on this, the reward function can be expressed as: , in, Represents the reward function, This is the survival reward function.

[0054] S20. Perform vector field histogram direction search on the lidar sensing data at each time step to obtain the local obstacle avoidance direction at each time step.

[0055] Specifically, the local obstacle avoidance direction is an ideal search method provided by the vector field histogram, which is the direction closest to the target direction in the corresponding time step. The local obstacle avoidance direction is the local observation information for each time step, which forms the value network. The deviation between the optimized direction and the actual UAV heading is used as the core penalty term in the reward function, guiding the policy network to quickly learn a flight strategy that fits the safe direction of the vector field histogram, significantly improving obstacle avoidance efficiency and stability.

[0056] In one embodiment, the step of performing a vector field histogram direction search on the lidar sensing data at each time step to obtain the local obstacle avoidance direction at each time step specifically includes: For each time step, the optimal direction that is closest to the target direction and less than a preset distance threshold is searched based on the vector field histogram in the lidar sensing data. If an optimization direction is found, then the optimization direction is used as the local obstacle avoidance direction for the time step. If no optimization direction is found, the local obstacle avoidance direction of the previous time step is used as the local obstacle avoidance direction of the current time step.

[0057] Specifically, such as Figure 4 As shown, when performing a search based on the vector field histogram direction search, the vector field histogram is first determined based on the lidar sensing data. Then, the optimal direction that is closest to the target direction and less than a preset distance threshold is searched within this vector field histogram. Here, the target direction is the direction between the initial position and the target position, and the preset distance threshold is a pre-set value used to constrain the flight direction.

[0058] S30. Based on the local obstacle avoidance direction and state data at each time step, determine the state value at each time step through a value network.

[0059] Specifically, the value network is used to estimate the state value of a state. The value network and the policy network form an asymmetric structure. The policy network's state data includes radar perception data, UAV state information, and target position information, while the value network's state data includes radar perception data, UAV state information, target position information, and local obstacle avoidance direction. In other words, the policy network's state data includes global observation data, while the value network's state data includes both global and local observation data. This effectively reduces the collision probability in complex scenarios and improves the accuracy of obstacle avoidance flight.

[0060] S40. Based on the data tuples and state values ​​at each time step, update the policy network and the value network using the near-end policy optimization algorithm to obtain the target policy network.

[0061] Specifically, when updating the policy network and the value network using the proximal policy optimization algorithm, a generalized advantage estimation (GOP) function is used to constrain the policy network update. This involves calculating the advantage function at each time step based on the value state, and then using GOP to reduce variance and ensure training stability. The value network is optimized using a mean squared error loss function, which measures the difference between the estimated value and the actual reward to update the value network.

[0062] It should be noted that during training, the initial and target positions of the UAV are randomly generated in the environment. The UAV interacts with the environment based on these randomly generated initial and target positions to continuously optimize the policy network and value network. Furthermore, each time the trajectory sequence is re-collected, the interaction between the policy network and the environment, updated based on the previously collected trajectory sequence, can be captured.

[0063] In one embodiment, to improve the inference speed of model deployment, after training the target policy network, the target policy network is exported and validated. Based on this, after updating the policy network and the value network using a near-end policy optimization algorithm based on the data tuples and state values ​​at each time step to obtain the target policy network, the method further includes: Export the target policy network; Build a drone obstacle flight simulation scenario in a simulation environment; Based on the target strategy, the flight control commands of the simulated UAV in the UAV obstacle flight simulation scenario are determined to realize the migration verification of the target strategy network from simulation to simulation, and the verified target strategy network is deployed on the UAV.

[0064] Specifically, the target policy network is exported and deployed in a Gazebo scenario for migration testing. The PX4 thrust-angular velocity control method is used, and the target policy network output is sent to the UAV as a RoS topic. The target policy network is verified in a loop simulation to achieve migration verification from simulation to simulation. For example, based on the Gazebo instance scenario and the mid360 single-line LiDAR point cloud representation method, the operational effect of the UAV carrying the target policy network is as follows: Figure 5 As shown.

[0065] S50. Control the UAV to deploy the target policy network to perform obstacle avoidance flight based on the target policy network.

[0066] Specifically, a target policy network is deployed on UAVs. The UAVs interact with the environment based on this network to control their obstacle avoidance flight, thereby improving obstacle avoidance performance. For example, ... Figure 6 As shown, in complex obstacle scenarios, a drone deployed with the target policy network trained by this application can achieve a high speed of 4 m / s.

[0067] In summary, this embodiment provides an obstacle avoidance method based on LiDAR and vector field histogram direction search, applicable to high-speed autonomous flight of UAVs in complex scenarios, and relating to the fields of robotics, artificial intelligence, and control technology. The method includes simulating the interaction between the UAV and the environment based on a policy network to collect trajectory sequences; performing vector field histogram direction search on the LiDAR perception data at each time step to obtain the local obstacle avoidance direction at each time step; determining the state value at each time step through a value network based on the local obstacle avoidance direction and state data at each time step; updating the policy network and the value network using a near-end policy optimization algorithm based on the data tuples and state value at each time step to obtain a target policy network; and controlling the UAV to deploy the target policy network for obstacle avoidance flight. This application combines LiDAR perception data, UAV state information, and target position information into state data of the policy network, extracts obstacle distance features using a multilayer perceptron, and improves obstacle avoidance performance using a long short-term memory neural network; simultaneously, it adds local obstacle avoidance directions to the state data of the policy network to form the state data of the value network, and then effectively reduces the collision probability in complex scenarios through an asymmetric policy network and value network. In addition, this application constructs a reward function by weighting the high-speed distance reward function, velocity direction reward function, safety reward function, normal vector constraint reward function, and survival reward function. This guides the policy network to quickly learn a flight strategy that fits the safe direction of the vector field histogram, which greatly improves obstacle avoidance efficiency and stability, and effectively increases the flight speed of the UAV during obstacle avoidance.

[0068] Based on the above obstacle avoidance method using lidar and vector field histogram direction search, this embodiment provides an obstacle avoidance device based on lidar and vector field histogram direction search, such as... Figure 7 As shown, the obstacle avoidance device based on lidar and vector field histogram direction search specifically includes: The collection module 100 is used to simulate the interaction between the UAV and the environment based on the policy network to collect trajectory sequences. The data tuples of each time step in the trajectory sequence include state data, action, reward and next state data. The state data includes feature data of lidar perception data and UAV state information. The policy network includes a long short-term memory neural network and a multilayer perceptron. The training module 200 is used to perform vector field histogram direction search on the LiDAR perception data at each time step to obtain the local obstacle avoidance direction at each time step. Based on the local obstacle avoidance direction and state data at each time step, the state value at each time step is determined through the value network. Based on the data tuples and state value at each time step, the policy network and the value network are updated using the near-end policy optimization algorithm to obtain the target policy network. The control module 300 is used to control the UAV to deploy the target policy network so as to perform obstacle avoidance flight based on the target policy network.

[0069] Based on the above obstacle avoidance method based on lidar and vector field histogram direction search, this embodiment provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in the obstacle avoidance method based on lidar and vector field histogram direction search as described in the above embodiment.

[0070] Based on the aforementioned obstacle avoidance method using lidar and vector field histogram direction search, this application also provides a terminal device, such as... Figure 8 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0071] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0072] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0073] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0074] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0075] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An obstacle avoidance method based on lidar and vector field histogram direction search, characterized in that, The obstacle avoidance method based on lidar and vector field histogram direction search specifically includes: The UAV interacts with the environment based on a policy network to collect trajectory sequences. Each time step in the trajectory sequence includes data tuples for state data, action, reward, and next state data. The state data includes feature data from LiDAR perception data, UAV state information, and target position information. The policy network includes a long short-term memory neural network and a multilayer perceptron. Perform vector field histogram direction search on the LiDAR sensing data at each time step to obtain the local obstacle avoidance direction at each time step; Based on the local obstacle avoidance direction and state data at each time step, the state value at each time step is determined through a value network. Based on the data tuples and state values ​​at each time step, the policy network and the value network are updated using the near-end policy optimization algorithm to obtain the target policy network. Control the drone to deploy the target policy network to perform obstacle avoidance flight based on the target policy network.

2. The obstacle avoidance method based on lidar and vector field histogram direction search according to claim 1, characterized in that, The process of acquiring the status data specifically includes: Acquire lidar perception data of the obstacle scene, and extract features from the lidar perception data to obtain feature data. The lidar perception data is used to reflect obstacle information in the obstacle scene. The feature data, the UAV status information, and the target location information are concatenated to obtain the status data.

3. The obstacle avoidance method based on lidar and vector field histogram direction search according to claim 1 or 2, characterized in that, The UAV status data includes UAV position information, UAV attitude information, UAV speed information, UAV orientation information, and UAV thrust information.

4. The obstacle avoidance method based on lidar and vector field histogram direction search according to claim 2, characterized in that, The acquisition of lidar perception data for obstacle scenes specifically includes: Obstacle distance distribution data in polar coordinates is obtained using lidar; A fixed-dimensional sector spatial division method is used to determine the nearest point cloud distance within each sector region in order to obtain LiDAR perception data for obstacle scenes.

5. The obstacle avoidance method based on lidar and vector field histogram direction search according to claim 1, characterized in that, The process of performing a vector field histogram direction search on the lidar sensing data at each time step to obtain the local obstacle avoidance direction at each time step specifically includes: For each time step, the optimal direction that is closest to the target direction and less than a preset distance threshold is searched based on the vector field histogram in the lidar sensing data. If an optimization direction is found, then the optimization direction is used as the local obstacle avoidance direction for the time step. If no optimization direction is found, the local obstacle avoidance direction of the previous time step is used as the local obstacle avoidance direction of the current time step.

6. The obstacle avoidance method based on lidar and vector field histogram direction search according to claim 1, characterized in that, The reward is determined based on a pre-built reward function, which includes a high-speed distance reward function, a velocity direction reward function, a safety reward function, a normal vector constraint reward function, and a survival reward function. The high-speed distance reward function is used to shorten the time it takes for the UAV to reach the target location. The velocity direction reward function is used to incentivize the UAV to maintain high speed and fly towards the target direction. The safety reward function is used to improve obstacle avoidance safety. The normal vector constraint reward function is used to limit the UAV's normal vector tilt angle. The survival reward function is used to extend the UAV's survival time in the environment.

7. The obstacle avoidance method based on lidar and vector field histogram direction search according to claim 1, characterized in that, After updating the policy network and the value network using a near-end policy optimization algorithm based on the data tuples and state values ​​at each time step to obtain the target policy network, the method further includes: Export the target policy network; Build a drone obstacle flight simulation scenario in a simulation environment; Based on the target strategy, the flight control commands of the simulated UAV in the UAV obstacle flight simulation scenario are determined to realize the migration verification of the target strategy network from simulation to simulation, and the verified target strategy network is deployed on the UAV.

8. The obstacle avoidance method based on lidar and vector field histogram direction search according to claim 1, characterized in that, In the process of updating the policy network and the value network using the proximal policy optimization algorithm, the advantage function is determined by generalized advantage estimation to constrain the update of the policy network, and the mean squared error loss function is used to optimize the value network.

9. An obstacle avoidance device based on lidar and vector field histogram direction search, characterized in that, The obstacle avoidance device based on lidar and vector field histogram direction search specifically includes: The collection module is used to simulate the interaction between the UAV and the environment based on the policy network to collect trajectory sequences. The data tuples of each time step in the trajectory sequence include state data, action, reward and next state data. The state data includes feature data of lidar perception data and UAV state information. The policy network includes a long short-term memory neural network and a multilayer perceptron. The training module is used to perform vector field histogram direction search on the LiDAR perception data at each time step to obtain the local obstacle avoidance direction at each time step. Based on the local obstacle avoidance direction and state data at each time step, the state value at each time step is determined through the value network. Based on the data tuples and state value at each time step, the policy network and the value network are updated using the near-end policy optimization algorithm to obtain the target policy network. The control module is used to control the UAV to deploy the target policy network so as to perform obstacle avoidance flight based on the target policy network.

10. A terminal device, characterized in that, include: Processor and memory; The memory stores a computer-readable program that can be executed by the processor; When the processor executes the computer-readable program, it implements the steps of the obstacle avoidance method based on lidar and vector field histogram direction search as described in any one of claims 1-8.

Images