Obstacle avoidance method, device, equipment and medium based on imitation learning

Abstract

The present invention provides an obstacle avoidance method, device, equipment and medium based on imitation learning, including: collecting first navigation data of a drone; outputting a multi-mode first speed direction index according to scene image data, drone posture data and target direction data in the first navigation data through an obstacle avoidance model based on imitation learning, the multi-mode first speed direction index is used to locate a target expert trajectory segment to be executed from an expert trajectory library, the target expert trajectory segment includes multiple navigation points and their corresponding expert speed directions; controlling the navigation of the drone according to the navigation points in the target expert trajectory segment to be executed and their corresponding expert speed directions, and continuing to collect new first navigation data during the drone navigation process until the drone reaches the terminal position. The present invention can significantly improve the robustness, flexibility and trajectory obstacle avoidance effect of trajectory planning.

Description

Technical Field

[0001] The present invention relates to the field of trajectory planning technology, and in particular to an obstacle avoidance method, device, equipment and medium based on imitation learning. Background Art

[0002] With the rapid development of drone technology, the application of drones in logistics and transportation, environmental monitoring, disaster relief and other fields has gradually become popular. However, achieving precise control of drones in complex dynamic environments remains a difficult problem that needs to be solved. At present, traditional control algorithms usually rely on accurate dynamic modeling and environmental perception, but often show low robustness and flexibility when faced with variable obstacles and complex kinematic constraints. In addition, the development of deep learning has provided new solutions for drone control, but the end-to-end method based entirely on data-driven has problems such as unstable training and insufficient generalization ability, resulting in poor obstacle avoidance effect of the planned trajectory. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide an obstacle avoidance method, device, equipment and medium based on imitation learning, which can significantly improve the robustness, flexibility and trajectory obstacle avoidance effect of trajectory planning.

[0004] In a first aspect, the present invention provides an obstacle avoidance method based on imitation learning, comprising: Collecting first navigation data of the UAV, the first navigation data including scene image data, UAV posture data and target direction data, the target direction data being a vector pointing from the current position of the UAV to the end position; Through an obstacle avoidance model based on imitation learning, a multi-mode first speed direction index and its corresponding probability prediction value are output according to the scene image data, drone posture data and target direction data in the first navigation data; wherein the multi-mode first speed direction index is used to locate the target expert trajectory segment to be executed from the expert trajectory library, and the probability prediction value is used to characterize the possibility of calling the target expert trajectory segment corresponding to the first speed direction index, and the target expert trajectory segment includes multiple navigation points and their corresponding expert speed directions; the obstacle avoidance model is trained by using a multi-modal loss function, and the multi-modal loss function is obtained by fusing the classification loss function and the speed constraint loss function, and the classification loss function is used to describe the deviation between the second target expert trajectory segment corresponding to the multi-mode second speed direction index predicted by the obstacle avoidance model and the first target expert trajectory segment actually executed by the drone, and the speed constraint loss function is used to describe the deviation between the speed direction prediction value corresponding to the second speed direction index and the expert speed direction included in the first target expert trajectory segment; The UAV is controlled to navigate according to the navigation points in the target expert trajectory segment to be executed and their corresponding expert speed directions, and new first navigation data is continuously collected during the UAV navigation process until the UAV reaches the destination position.

[0005] In one embodiment, before outputting a multi-mode first speed direction index according to scene image data, drone posture data, and target direction data through an obstacle avoidance model based on imitation learning, the method further includes: Get pre-planned multiple expert trajectories; Controlling the UAV to navigate according to the expert trajectory in scenes of different scales, and collecting second navigation data during the navigation of the UAV, the second navigation data including timestamp, scene image data, UAV position data and target direction data; The expert trajectory is segmented based on the timestamp to construct an expert trajectory library; The expert trajectory library, as well as the scene image data, drone pose data and target direction data in the second navigation data, are used in combination with a multimodal loss function to train the obstacle avoidance model based on imitation learning.

[0006] In one implementation, the expert trajectory library includes a plurality of target expert trajectory segments; and the expert trajectory is segmented based on the timestamp to construct the expert trajectory library, including: Determine multiple groups of start timestamps and end timestamps according to the preset planning time to obtain multiple timestamp intervals to be intercepted; Extracting an initial expert trajectory segment within a timestamp interval from the expert trajectory; Reconstruct the trajectory based on the navigation points contained in the initial expert trajectory segment to obtain a smooth expert trajectory segment; A plurality of navigation points are sampled from the smooth expert trajectory segment, and the sampled navigation points and their corresponding expert speed directions constitute the target expert trajectory segment.

[0007] In one embodiment, the expert trajectory library, the scene image data, the drone pose data and the target direction data in the second navigation data are used in combination with a multimodal loss function to train an obstacle avoidance model based on imitation learning, including: Outputting a multi-mode second speed direction index according to the scene image data, the UAV posture data and the target direction data in the second navigation data through an obstacle avoidance model based on imitation learning; extracting, from an expert trajectory library, a first target expert trajectory segment matching the second navigation data based on a timestamp in the second navigation data; According to the multimodal loss function, based on the second speed direction index of the multimodal and the first target expert trajectory fragment, a classification loss value and a speed constraint loss value are determined, and the classification loss value and the speed constraint loss value are fused to obtain a multimodal loss value; The obstacle avoidance model based on imitation learning is trained using multimodal loss values.

[0008] In one embodiment, determining a classification loss value based on a multimodal second speed direction index and a first target expert trajectory segment according to a multimodal loss function includes: From the expert trajectory library, locate the second target expert trajectory segment corresponding to the second speed direction index of each mode; Determine a label corresponding to each second target expert trajectory segment based on the first target expert trajectory segment, where the label is used to indicate whether the second target expert trajectory segment is a first target expert trajectory segment; The classification loss value is determined according to the probability prediction value corresponding to the second speed direction index of each mode and the label corresponding to each second target expert trajectory segment.

[0009] In one embodiment, determining the speed constraint loss value based on the multi-modal second speed direction index and the first target expert trajectory segment according to the multi-modal loss function includes: The second speed direction index of each mode is interpreted to obtain a speed direction prediction value corresponding to the second speed direction index of each mode; Determine the matching degree between the speed direction prediction value corresponding to the second speed direction index of each mode and the expert speed direction included in the first target expert trajectory segment; A speed constraint penalty value is determined based on the degree of matching.

[0010] In one implementation, according to the multimodal loss function, based on the multimodal second speed direction index and the first target expert trajectory segment, a classification loss value and a speed constraint loss value are determined, and the classification loss value and the speed constraint loss value are fused to obtain a multimodal loss value, including: The multimodal loss value is determined according to the following formula: ; in, is the multimodal loss value, is the number of the second navigation data, is the number of modes, For the The second navigation data output The second speed direction index of the mode, Index for the second speed direction The label of the corresponding second target expert trajectory segment, Index for the second speed direction The corresponding probability prediction value of the second target expert trajectory segment, Index for the second speed direction The corresponding speed direction prediction value, For the The expert velocity direction of the first target expert trajectory segment matched by the second navigation data includes: Predicted value for velocity direction With expert speed direction The degree of match between is the weight coefficient, Is a constant.

[0011] In one embodiment, an obstacle avoidance model based on imitation learning is used to output a multi-mode first speed direction index according to scene image data, drone posture data, and target direction data, including: Extracting first feature information of scene image data, and extracting second feature information of drone posture data and target direction data; Performing downsampling and fusion processing on the first feature information and the second feature information to obtain target feature information; Based on the target feature information, a multi-mode first speed direction index and its corresponding probability prediction value are output.

[0012] In a second aspect, the present invention further provides an obstacle avoidance device based on imitation learning, comprising: A data acquisition module is used to collect first navigation data of the UAV, the first navigation data including scene image data, UAV posture data and target direction data, the target direction data is a vector pointing from the current position of the UAV to the end position; An index output module is used to output a multi-mode first speed direction index and its corresponding probability prediction value according to the scene image data, UAV posture data and target direction data in the first navigation data through an obstacle avoidance model based on imitation learning; wherein the multi-mode first speed direction index is used to locate the target expert trajectory segment to be executed from the expert trajectory library, and the probability prediction value is used to characterize the possibility of calling the target expert trajectory segment corresponding to the first speed direction index, and the target expert trajectory segment includes multiple navigation points and their corresponding expert speed directions; the obstacle avoidance model is trained using a multi-modal loss function, and the multi-modal loss function is obtained by fusing the classification loss function and the speed constraint loss function, and the classification loss function is used to describe the deviation between the second target expert trajectory segment corresponding to the multi-mode second speed direction index predicted by the obstacle avoidance model and the first target expert trajectory segment actually executed by the UAV, and the speed constraint loss function is used to describe the deviation between the speed direction prediction value corresponding to the second speed direction index and the expert speed direction included in the first target expert trajectory segment; The UAV control module is used to control the UAV navigation according to the navigation points in the target expert trajectory segment to be executed and their corresponding expert speed directions, and continue to collect new first navigation data during the UAV navigation process until the UAV reaches the terminal position.

[0013] In a third aspect, the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement any one of the methods provided in the first aspect.

[0014] In a fourth aspect, the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.

[0015] The obstacle avoidance method, device, equipment and medium based on imitation learning provided by the present invention first collect first navigation data of a UAV, the first navigation data including scene image data, UAV posture data and target direction data, the target direction data being a vector pointing from the current position of the UAV to the terminal position; then, through an obstacle avoidance model based on imitation learning, according to the scene image data, UAV posture data and target direction data in the first navigation data, a multi-mode first speed direction index is output, the multi-mode first speed direction index is used to locate a target expert trajectory segment to be executed from an expert trajectory library, the target expert trajectory segment including a plurality of navigation points and their corresponding expert speed directions; finally, the navigation of the UAV is controlled according to the navigation points in the target expert trajectory segment to be executed and their corresponding expert speed directions, and new first navigation data continues to be collected during the navigation of the UAV until the UAV arrives at the terminal position. The above method combines expert experience and deep learning technology, and outputs a multi-mode first speed direction index according to the scene image data, UAV posture data and target direction data in the first navigation data through an obstacle avoidance model based on imitation learning, and then controls the UAV navigation according to the target expert trajectory fragment obtained by locating the first speed direction index. The present invention realizes the decision-making process of imitating humans or algorithm experts, and not only has high robustness and flexibility, but also has better obstacle avoidance effect.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of an obstacle avoidance method based on imitation learning provided by an embodiment of the present invention; Figure 2 A schematic diagram of the structure of an obstacle avoidance model based on imitation learning provided by an embodiment of the present invention; Figure 3A schematic diagram of the architecture of an obstacle avoidance method based on imitation learning provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of an obstacle avoidance device based on imitation learning provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] At present, traditional control algorithms often show low robustness and flexibility when facing changeable obstacles and complex kinematic constraints. Control algorithms based on deep learning have problems such as unstable training and insufficient generalization ability. Based on this, the present invention implements an obstacle avoidance method, device, equipment and medium based on imitation learning, which can significantly improve the robustness, flexibility and trajectory obstacle avoidance effect of trajectory planning.

[0022] To facilitate understanding of this embodiment, firstly, a method for circumventing obstacles based on imitation learning disclosed in an embodiment of the present invention is described in detail. Figure 1 The flowchart of an obstacle avoidance method based on imitation learning is shown in FIG. 1 , and the method mainly includes the following steps S102 to S106: Step S102, collecting first navigation data of the UAV.

[0023] The first navigation data includes scene image data, drone posture data and target direction data. The drone is equipped with an image acquisition device, such as a monocular fisheye camera. The image acquired by the monocular fisheye camera for the scene in front of the drone is the scene image data. The content displayed by the scene image data may include obstacles in the direction of the drone's advance. The drone is equipped with a sensor system, such as a positioning device, a velocity sensor, an accelerometer, a gyroscope, etc. The sensor system is used to collect corresponding posture data during the drone's navigation process. The posture data includes position, quaternion, velocity, acceleration, angular velocity, etc. The target direction data is a vector pointing from the current position of the drone to the end position.

[0024] Step S104, outputting a multi-mode first speed direction index and its corresponding probability prediction value according to the scene image data, drone posture data and target direction data in the first navigation data through an obstacle avoidance model based on imitation learning.

[0025] Among them, the input of the obstacle avoidance model of imitation learning includes scene image data, drone posture data and target direction data, and the output includes multi-mode first speed direction index and its corresponding probability prediction value. Multi-mode can be understood as the number of output first speed direction indexes is multiple, the first speed direction index is also the speed direction index output by the model in the model application stage, the first speed direction index is used to locate the target expert trajectory segment to be executed from the expert trajectory library, the expert trajectory library includes multiple target expert trajectory segments, the target expert trajectory segment includes multiple navigation points and their corresponding expert speed directions; the probability prediction value is used to characterize the possibility of the target expert trajectory segment corresponding to the first speed direction index being called, that is, the higher the probability prediction value corresponding to the first speed direction index, the higher the possibility of controlling the drone navigation according to the target expert trajectory segment corresponding to the first speed direction index.

[0026] Among them, the obstacle avoidance model is trained by using a multimodal loss function, which is obtained by fusing the classification loss function and the speed constraint loss function. The classification loss function is used to describe the deviation between the second target expert trajectory segment corresponding to the multi-mode second speed direction index predicted by the obstacle avoidance model and the first target expert trajectory segment actually executed by the UAV. The speed constraint loss function is used to describe the deviation between the speed direction prediction value corresponding to the second speed direction index and the expert speed direction included in the first target expert trajectory segment.

[0027] In one example, the scene image data, UAV posture data and target direction data in the first navigation data are input into an obstacle avoidance model based on imitation learning. The model outputs a multi-mode first speed direction index and its corresponding probability prediction value through operations such as feature extraction, feature downsampling, feature fusion and feature decoding, and uses the first speed direction index with the highest probability prediction value to locate the target expert trajectory segment to be executed from the expert trajectory library.

[0028] Step S106, controlling the drone to navigate according to the navigation points in the target expert trajectory segment to be executed and their corresponding expert speed directions, and continuing to collect new first navigation data during the drone's navigation until the drone reaches the destination position.

[0029] In one example, while controlling the navigation of the UAV according to the navigation points and their corresponding expert speed directions within the target expert trajectory segment to be executed, new first navigation data continues to be collected, and through an obstacle avoidance algorithm based on imitation learning, a multi-mode new first speed direction index is output according to the new first navigation data, so as to locate the new target expert trajectory segment, and then control the navigation of the UAV according to the new target expert trajectory segment, and repeat the above process until the UAV reaches the terminal position.

[0030] The obstacle avoidance method based on imitation learning provided in the embodiment of the present invention combines expert experience and deep learning technology. Through the obstacle avoidance model based on imitation learning, a multi-mode first speed direction index is output according to the scene image data, UAV posture data and target direction data in the first navigation data, and then the UAV navigation is controlled according to the target expert trajectory fragment obtained by locating the first speed direction index. The present invention realizes the decision-making process of imitating humans or algorithm experts, and not only has high robustness and flexibility, but also has better obstacle avoidance effect.

[0031] In order to make the obstacle avoidance model based on imitation learning have a better obstacle avoidance effect, it is necessary to train the obstacle avoidance model based on imitation learning in advance. The embodiment of the present invention provides a specific implementation method for training the obstacle avoidance model based on imitation learning. See the following steps 1 to 4: Step 1: Obtain multiple pre-planned expert trajectories. The expert trajectories can be divided into expert trajectories based on pilot hand flying, expert trajectories based on polynomial fitting, and expert trajectories based on traditional obstacle avoidance algorithms.

[0032] Step 2: Control the UAV to navigate according to the expert trajectory in scenes of different scales, and collect second navigation data during the navigation of the UAV. The second navigation data includes a timestamp, scene image data, UAV posture data, and target direction data.

[0033] In one example, the embodiment of the present invention can control the drone to fly according to expert trajectories in large-scale scenes and small-scale scenes, such as forests, buildings, geometric arrays, road signs, electric towers and other scenes of different scales. Data collection is performed during the drone's flight. The data collection is in rounds. Each round flies for 1 minute, and the timestamp, scene image data, drone posture data (position, quaternion, speed, acceleration, angular velocity), target direction data, etc. during the flight are recorded. A total of 13km of data are collected.

[0034] Step 3: Extract the expert trajectory based on the timestamp to construct the expert trajectory library. This includes (a) to (d): (a) Determine multiple groups of start timestamps and end timestamps according to the preset planning time to obtain multiple timestamp intervals to be intercepted. The planning time is used to limit the duration of the expert trajectory segment to be intercepted, for example, the planning time is set to 4s. In one example, the timestamp corresponding to a frame of scene image data can be used as the start timestamp, and the timestamp after 4s can be used as the end timestamp to obtain the timestamp interval; in the subsequent process, the timestamp corresponding to the second frame of scene image data can be used as the new start timestamp.

[0035] (b) Extract the initial expert trajectory segment within the timestamp interval from the expert trajectory.

[0036] (c) Reconstruct the trajectory based on the navigation points contained in the initial expert trajectory segment to obtain a smooth expert trajectory segment. Considering that the number of navigation points contained in the initial expert trajectory segment is affected by the flight speed of the drone, for example, when the drone flies faster, the number of navigation points contained in the initial expert trajectory segment is smaller, and when the drone flies slower, the number of navigation points contained in the initial expert trajectory segment is larger, resulting in the initial expert trajectory segment being non-standard, so it needs to be reconstructed. In one example, fitting can be performed based on the navigation points contained in the initial expert trajectory segment to achieve trajectory reconstruction and obtain a smooth expert trajectory segment, and the fitting algorithm can use a third-order b-spline fitting algorithm.

[0037] (d) Sampling multiple navigation points from the smooth expert trajectory segment, and the sampled navigation points and their corresponding expert speed directions constitute the target expert trajectory segment. For example, 10 navigation position points and their corresponding expert speed directions can be uniformly sampled from the smooth expert trajectory segment to obtain the target expert trajectory segment.

[0038] Step 4: Use the expert trajectory library, the scene image data, drone pose data, and target direction data in the second navigation data, combined with the multimodal loss function, to train the obstacle avoidance model based on imitation learning. Specifically, it includes the following (1) to (4): (1) Outputting a multi-mode second speed direction index according to the scene image data, the UAV posture data and the target direction data in the second navigation data through an obstacle avoidance model based on imitation learning.

[0039] In one embodiment, first feature information of scene image data is extracted, and second feature information of drone posture data and target direction data is extracted; the first feature information and the second feature information are downsampled and fused to obtain target feature information; based on the target feature information, a multi-mode first speed direction index and its corresponding probability prediction value are output.

[0040] The architecture of the obstacle avoidance model based on imitation learning is as follows: In order to reduce the complexity of training and suppress the problem of wandering around obstacles, speed encoding and multi-mode output are adopted. Figure 2 The schematic diagram of the structure of an obstacle avoidance model based on imitation learning is shown, including: a backbone network of an image part and a backbone network of a state part, the backbone network of the image part is used to extract the first feature information of the scene image information, and the backbone network of the state part is used to extract the second feature information of the drone posture data and the target direction data; it also includes multiple convolutional layers (Conv1D) for performing convolution operations on the first feature information and the second feature information after downsampling and fusion processing, and finally outputting the first speed direction indexes and their corresponding probability prediction values ​​of the top three modes with the highest probability prediction values ​​after decoding processing.

[0041] (2) Based on the timestamp in the second navigation data, extract the first target expert trajectory segment that matches the second navigation data from the expert trajectory library. In one example, the expert trajectory executed by the UAV when collecting the second navigation data is known, so multiple target expert trajectory segments intercepted from the expert trajectory can be screened out from the expert trajectory library, and then the first target expert trajectory segment that matches the second navigation data can be further screened out from the screened target expert trajectory segments based on the timestamp in the second navigation data.

[0042] (3) According to the multimodal loss function, based on the multimodal second speed direction index and the first target expert trajectory fragment, the classification loss value and the speed constraint loss value are determined, and the classification loss value and the speed constraint loss value are fused to obtain the multimodal loss value.

[0043] In one example, the process of determining the classification loss is as follows: (31-1) From the expert trajectory library, locate the second target expert trajectory segment corresponding to the second speed direction index of each mode. (31-2) Based on the first target expert trajectory segment, determine the label corresponding to each second target expert trajectory segment, and the label is used to characterize whether the second target expert trajectory segment is the first target expert trajectory segment. For example, if the second target expert trajectory segment is the first target expert trajectory segment, the label of the second target expert trajectory segment can be determined to be 1, otherwise the label of the second target expert trajectory segment can be determined to be 0. (31-3) Determine the classification loss value based on the probability prediction value corresponding to the second speed direction index of each mode and the label corresponding to each second target expert trajectory segment. The classification loss value can be determined according to the following formula: ; in, is the classification loss value, is the number of the second navigation data, is the number of modes, For the The second navigation data output The second speed direction index of the mode, Index for the second speed direction The label of the corresponding second target expert trajectory segment, Index for the second speed direction The probability prediction value of the corresponding second target expert trajectory segment.

[0044] In one example, the process of determining the velocity constraint loss value (Velocity Constraint Loss) is as follows: (32-1) The second speed direction index of each mode is interpreted to obtain the speed direction prediction value corresponding to the second speed direction index of each mode. For example, the second speed direction index can be in numerical form, and the second speed direction index can be interpreted to obtain the corresponding steering angle based on the mapping relationship between the second speed direction index and the steering angle of the drone, and then the corresponding speed direction prediction value is solved according to the steering angle.

[0045] The interpretation formula is as follows: ; in, is the steering angle, For the The second navigation data output The second speed direction index of the mode, is the total number of target expert trajectory segments contained in the expert trajectory library, is the visual range of the monocular fisheye camera, It is half of the visual range of a monocular fisheye camera.

[0046] The process of determining the velocity direction prediction value is as follows: ; in, Index for the second speed direction The corresponding speed direction prediction value.

[0047] (32-2) Determine the matching degree between the speed direction prediction value corresponding to the second speed direction index of each mode and the expert speed direction included in the first target expert trajectory segment. The matching degree can be determined according to the following formula: ; in, Predicted value for velocity direction With expert speed direction The degree of match between is the predicted value of velocity direction With expert speed direction The angle between Index for the second speed direction The corresponding speed direction prediction value, For the The expert velocity direction of the first target expert trajectory segment matched by the second navigation data includes: is a constant, a small value to prevent division by zero.

[0048] (32-3) The speed constraint loss value is determined based on the matching degree. The speed constraint loss value can be determined according to the following formula: ; in, is the speed constraint loss value.

[0049] Based on this, the process of fusing the classification loss value and the speed constraint loss value to obtain the multimodal loss value is as follows: ; in, is the multimodal loss value, is the weight coefficient.

[0050] Based on the above embodiment, the multimodal loss value is determined according to the following formula: ; Through the multimodal loss function, the obstacle avoidance model based on imitation learning can effectively predict the speed direction index and improve the accuracy of obstacle avoidance planning.

[0051] (4) Use multimodal loss values ​​to train the obstacle avoidance model based on imitation learning. In one example, the cosine decay restart strategy can be used to dynamically adjust the learning rate and combined with the Adam optimizer to achieve efficient optimization of the model.

[0052] In summary, see Figure 3The schematic diagram of the architecture of an obstacle avoidance method based on imitation learning is shown. The model training stage involves two parts: data governance and model architecture. Data governance includes processes such as data collection, data cleaning, and training set management; the model architecture includes processes such as model training, model testing, and model iteration. After the model is trained, it can be verified and tested: divided into open-loop testing, closed-loop testing, and actual flight testing. The open-loop test is to divide the data set into a training set and a validation set, and evaluate the correctness of the model on the validation set, which is an open-loop test. The closed-loop test is based on the UE (Unreal Engine) engine and the Airsim drone simulator. The model is used to control the movement of the drone in real time in the simulation environment, and the next step is predicted based on the posture feedback from the environment. The embodiment of the present invention selects multi-scale scenes in a real environment for actual flight testing, and the test results are as follows: The embodiment of the present invention can achieve obstacle avoidance in 0.3m / s-1.6m / s box scenes, 0.3m / s-1m / s forest scenes, 5m / s building scenes, 1m / s road sign scenes, 3m / s lamp poles, and electric tower scenes.

[0053] After the model passes the verification pre-test, it can be put into the actual trajectory planning scenario. After collecting the scene image data, the drone posture data and the target direction data of the drone, the obstacle avoidance model based on imitation learning can be used to output the multi-mode first speed direction index according to the scene image data, the drone posture data and the target direction data, including: extracting the first feature information of the scene image data, and extracting the second feature information of the drone posture data and the target direction data; downsampling and fusing the first feature information and the second feature information to obtain the target feature information; based on the target feature information, outputting the multi-mode first speed direction index and its corresponding probability prediction value. For details, please refer to the above (1), which will not be described in detail in the embodiment of the present invention. The target expert trajectory segment corresponding to the first speed direction index with the highest probability prediction value is located from the expert trajectory library, and sent to the drone to control the drone navigation, and continue to collect new scene image data, drone posture data and target direction data during the navigation process, and repeat this process until the end position is reached.

[0054] As a technology combining expert experience and deep learning, the imitation learning in the embodiment of the present invention has significant advantages. By imitating the decision-making process of humans or algorithm experts, an efficient control strategy can be quickly generated.

[0055] Based on the above embodiments, the present invention provides an obstacle avoidance device based on imitation learning. Figure 4 The schematic diagram of the structure of an obstacle avoidance device based on imitation learning is shown in FIG. The device mainly includes the following parts: The data acquisition module 402 is used to collect the first navigation data of the UAV, the first navigation data includes scene image data, UAV posture data and target direction data, the target direction data is a vector pointing from the current position of the UAV to the end position; The index output module 404 is used to output a multi-mode first speed direction index and its corresponding probability prediction value according to the scene image data, drone posture data and target direction data in the first navigation data through an obstacle avoidance model based on imitation learning; wherein the multi-mode first speed direction index is used to locate the target expert trajectory segment to be executed from the expert trajectory library, and the probability prediction value is used to characterize the possibility of calling the target expert trajectory segment corresponding to the first speed direction index, and the target expert trajectory segment includes multiple navigation points and their corresponding expert speed directions; the obstacle avoidance model is trained by using a multi-modal loss function, and the multi-modal loss function is obtained by fusing the classification loss function and the speed constraint loss function, and the classification loss function is used to describe the deviation between the second target expert trajectory segment corresponding to the multi-mode second speed direction index predicted by the obstacle avoidance model and the first target expert trajectory segment actually executed by the drone, and the speed constraint loss function is used to describe the deviation between the speed direction prediction value corresponding to the second speed direction index and the expert speed direction included in the first target expert trajectory segment; The drone control module 406 is used to control the drone navigation according to the navigation points in the target expert trajectory segment to be executed and their corresponding expert speed directions, and continue to collect new first navigation data during the drone navigation process until the drone reaches the terminal position.

[0056] The obstacle avoidance device based on imitation learning provided in the embodiment of the present invention combines expert experience and deep learning technology, and outputs a multi-mode first speed direction index according to the scene image data, UAV posture data and target direction data in the first navigation data through an imitation learning-based obstacle avoidance model, and then controls the navigation of the UAV according to the target expert trajectory fragment obtained by locating the first speed direction index. The present invention realizes the decision-making process of imitating humans or algorithm experts, and not only has high robustness and flexibility, but also has better obstacle avoidance effect.

[0057] In one embodiment, a model training module is further included, which is used to: Get pre-planned multiple expert trajectories; Controlling the UAV to navigate according to the expert trajectory in scenes of different scales, and collecting second navigation data during the navigation of the UAV, the second navigation data including timestamp, scene image data, UAV position data and target direction data; The expert trajectory is segmented based on the timestamp to construct an expert trajectory library; The expert trajectory library, as well as the scene image data, drone pose data and target direction data in the second navigation data, are used in combination with a multimodal loss function to train the obstacle avoidance model based on imitation learning.

[0058] In one implementation, the expert trajectory library includes a plurality of target expert trajectory segments; and the model training module is specifically used for: Determine multiple groups of start timestamps and end timestamps according to the preset planning time to obtain multiple timestamp intervals to be intercepted; Extracting an initial expert trajectory segment within a timestamp interval from the expert trajectory; Reconstruct the trajectory based on the navigation points contained in the initial expert trajectory segment to obtain a smooth expert trajectory segment; A plurality of navigation points are sampled from the smooth expert trajectory segment, and the sampled navigation points and their corresponding expert speed directions constitute the target expert trajectory segment.

[0059] In one embodiment, the model training module is specifically used to: Outputting a multi-mode second speed direction index according to the scene image data, the UAV posture data and the target direction data in the second navigation data through an obstacle avoidance model based on imitation learning; extracting, from an expert trajectory library, a first target expert trajectory segment matching the second navigation data based on a timestamp in the second navigation data; According to the multimodal loss function, based on the second speed direction index of the multimodal and the first target expert trajectory fragment, a classification loss value and a speed constraint loss value are determined, and the classification loss value and the speed constraint loss value are fused to obtain a multimodal loss value; The obstacle avoidance model based on imitation learning is trained using multimodal loss values.

[0060] In one embodiment, the model training module is specifically used to: From the expert trajectory library, locate the second target expert trajectory segment corresponding to the second speed direction index of each mode; Determine a label corresponding to each second target expert trajectory segment based on the first target expert trajectory segment, where the label is used to indicate whether the second target expert trajectory segment is a first target expert trajectory segment; The classification loss value is determined according to the probability prediction value corresponding to the second speed direction index of each mode and the label corresponding to each second target expert trajectory segment.

[0061] In one embodiment, the model training module is specifically used to: The second speed direction index of each mode is interpreted to obtain a speed direction prediction value corresponding to the second speed direction index of each mode; Determine the matching degree between the speed direction prediction value corresponding to the second speed direction index of each mode and the expert speed direction included in the first target expert trajectory segment; A speed constraint penalty value is determined based on the degree of matching.

[0062] In one embodiment, the model training module is specifically used to: The multimodal loss value is determined according to the following formula: ; in, is the multimodal loss value, is the number of the second navigation data, is the number of modes, For the The second navigation data output The second speed direction index of the mode, Index for the second speed direction The label of the corresponding second target expert trajectory segment, Index for the second speed direction The corresponding probability prediction value of the second target expert trajectory segment, Index for the second speed direction The corresponding speed direction prediction value, For the The expert velocity direction of the first target expert trajectory segment matched by the second navigation data includes: Predicted value for velocity direction With expert speed direction The degree of match between is the weight coefficient, Is a constant.

[0063] In one implementation, the index output module 404 is specifically configured to: Extracting first feature information of scene image data, and extracting second feature information of drone posture data and target direction data; Performing downsampling and fusion processing on the first feature information and the second feature information to obtain target feature information; Based on the target feature information, a multi-mode first speed direction index and its corresponding probability prediction value are output.

[0064] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0065] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.

[0066] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 50, a memory 51, a bus 52 and a communication interface 53, wherein the processor 50, the communication interface 53 and the memory 51 are connected via the bus 52; the processor 50 is used to execute an executable module stored in the memory 51, such as a computer program.

[0067] The memory 51 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0068] The bus 52 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0069] Among them, the memory 51 is used to store programs, and the processor 50 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 50 or implemented by the processor 50.

[0070] The processor 50 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 50. The above processor 50 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 51, and the processor 50 reads the information in the memory 51 and completes the steps of the above method in combination with its hardware.

[0071] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0072] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0073] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An obstacle avoidance method based on imitation learning, characterized in that: include: Collecting first navigation data of the drone, the first navigation data including scene image data, drone posture data and target direction data, the target direction data being a vector pointing from the current position of the drone to the end position; Through an obstacle avoidance model based on imitation learning, a multi-mode first speed direction index and its corresponding probability prediction value are output according to the scene image data, the UAV posture data and the target direction data in the first navigation data; wherein the multi-mode first speed direction index is used to locate the target expert trajectory segment to be executed from the expert trajectory library, and the probability prediction value is used to characterize the possibility of calling the target expert trajectory segment corresponding to the first speed direction index, and the target expert trajectory segment includes multiple navigation points and their corresponding expert speed directions; the obstacle avoidance model is trained using a multi-modal loss function, and the multi-modal loss function is obtained by fusing a classification loss function and a speed constraint loss function, and the classification loss function is used to describe the deviation between the second target expert trajectory segment corresponding to the multi-modal second speed direction index predicted by the obstacle avoidance model and the first target expert trajectory segment actually executed by the UAV, and the speed constraint loss function is used to describe the deviation between the speed direction prediction value corresponding to the second speed direction index and the expert speed direction included in the first target expert trajectory segment; The UAV is controlled to navigate according to the navigation points in the target expert trajectory segment to be executed and the corresponding expert speed directions, and new first navigation data are continuously collected during the UAV navigation process until the UAV reaches the terminal position.

2. The obstacle avoidance method based on imitation learning according to claim 1, characterized in that: Before outputting a multi-mode first speed direction index according to the scene image data, the drone posture data and the target direction data through an obstacle avoidance model based on imitation learning, the method further includes: Get pre-planned multiple expert trajectories; Controlling the UAV to navigate according to the expert trajectory in scenes of different scales, and collecting second navigation data during the navigation of the UAV, wherein the second navigation data includes a timestamp, the scene image data, the UAV posture data, and the target direction data; Segment-cutting the expert trajectory based on the timestamp to construct the expert trajectory library; The expert trajectory library, as well as the scene image data, the drone pose data and the target direction data in the second navigation data, are used in combination with a multimodal loss function to train an obstacle avoidance model based on imitation learning.

3. The obstacle avoidance method based on imitation learning according to claim 2, characterized in that: The expert trajectory library includes a plurality of target expert trajectory segments; The expert trajectory is segmented based on the timestamp to construct the expert trajectory library, including: Determine multiple groups of start timestamps and end timestamps according to the preset planning time to obtain multiple timestamp intervals to be intercepted; Extracting an initial expert trajectory segment located within the timestamp interval from the expert trajectory; Reconstructing the trajectory based on the navigation points included in the initial expert trajectory segment to obtain a smooth expert trajectory segment; A plurality of the navigation points are sampled from the smooth expert trajectory segment, and the sampled navigation points and their corresponding expert speed directions constitute the target expert trajectory segment.

4. The obstacle avoidance method based on imitation learning according to claim 2, characterized in that: Using the expert trajectory library, the scene image data, the drone pose data, and the target direction data in the second navigation data, combined with a multimodal loss function, an obstacle avoidance model based on imitation learning is trained, including: Outputting a multi-mode second speed direction index according to the scene image data, the drone posture data and the target direction data in the second navigation data through an obstacle avoidance model based on imitation learning; extracting, from the expert trajectory library, a first target expert trajectory segment matching the second navigation data based on the timestamp in the second navigation data; According to the multimodal loss function, based on the second speed direction index of the multimodal and the first target expert trajectory segment, a classification loss value and a speed constraint loss value are determined, and the classification loss value and the speed constraint loss value are fused to obtain a multimodal loss value; The multimodal loss value is used to train an obstacle avoidance model based on imitation learning.

5. The obstacle avoidance method based on imitation learning according to claim 4, characterized in that: Determining a classification loss value based on the second speed direction index and the first target expert trajectory segment of the multi-modal according to the multi-modal loss function includes: Locate the second target expert trajectory segment corresponding to the second speed direction index of each mode from the expert trajectory library; Determine a label corresponding to each second target expert trajectory segment based on the first target expert trajectory segment, wherein the label is used to indicate whether the second target expert trajectory segment is the first target expert trajectory segment; A classification loss value is determined according to the probability prediction value corresponding to the second speed direction index of each mode and the label corresponding to each second target expert trajectory segment.

6. The obstacle avoidance method based on imitation learning according to claim 4, characterized in that: Determining a speed constraint loss value based on the second speed direction index of the multi-mode and the first target expert trajectory segment according to the multi-modal loss function includes: Interpreting the second speed direction index of each mode to obtain a speed direction prediction value corresponding to the second speed direction index of each mode; Determine a degree of match between the speed direction prediction value corresponding to the second speed direction index of each mode and the expert speed direction included in the first target expert trajectory segment; A speed constraint penalty value is determined based on the degree of matching.

7. The obstacle avoidance method based on imitation learning according to claim 1, characterized in that: Through the obstacle avoidance model based on imitation learning, according to the scene image data, the drone posture data and the target direction data, a multi-mode first speed direction index and its corresponding probability prediction value are output, including: Extracting first feature information of the scene image data, and extracting second feature information of the drone pose data and the target direction data; Performing downsampling processing and fusion processing on the first feature information and the second feature information to obtain target feature information; Based on the target feature information, a multi-mode first speed direction index and its corresponding probability prediction value are output.

8. An obstacle avoidance device based on imitation learning, characterized in that: include: A data acquisition module, used to collect first navigation data of the UAV, wherein the first navigation data includes scene image data, UAV posture data and target direction data, wherein the target direction data is a vector pointing from the current position of the UAV to the end position; An index output module is used to output a multi-mode first speed direction index and its corresponding probability prediction value according to the scene image data, the UAV posture data and the target direction data in the first navigation data through an obstacle avoidance model based on imitation learning; wherein the multi-mode first speed direction index is used to locate the target expert trajectory segment to be executed from the expert trajectory library, and the probability prediction value is used to characterize the possibility of calling the target expert trajectory segment corresponding to the first speed direction index, and the target expert trajectory segment includes multiple navigation points and their corresponding expert speed directions; the obstacle avoidance model is trained using a multi-modal loss function, and the multi-modal loss function is obtained by fusing a classification loss function and a speed constraint loss function, and the classification loss function is used to describe the deviation between the second target expert trajectory segment corresponding to the multi-mode second speed direction index predicted by the obstacle avoidance model and the first target expert trajectory segment actually executed by the UAV, and the speed constraint loss function is used to describe the deviation between the speed direction prediction value corresponding to the second speed direction index and the expert speed direction included in the first target expert trajectory segment; The drone control module is used to control the drone navigation according to the navigation points and the corresponding expert speed directions in the target expert trajectory segment to be executed, and continue to collect new first navigation data during the drone navigation process until the drone reaches the terminal position.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.

Images