Vision-based autonomous landing method and system for unmanned aerial vehicles

Abstract

The application discloses a kind of unmanned aerial vehicle autonomous landing method and system based on vision, realize the dynamic landing pad autonomous landing of unmanned aerial vehicle under unknown environment;Adopt the feature extraction and state estimation method of unmanned aerial vehicle vision sequence, according to the center position of the dynamic landing pad, motion direction, motion speed and other features extracted from processed visual information, estimate the relative pose of itself and dynamic landing pad, predict dynamic landing pad space-time state and motion behavior;According to the perception result of unmanned aerial vehicle, directly generate the control instruction of the motion direction, lifting height, flight speed and other of unmanned aerial vehicle, realize the dynamic landing pad landing task of unmanned aerial vehicle under GPS denial unknown environment;The method combines the multi-control network cooperative control based on deep reinforcement learning and target space-time state prediction method, so that unmanned aerial vehicle autonomously completes the dynamic landing pad landing task of unmanned aerial vehicle under unknown environment without human intervention, greatly improves the autonomy and intelligence of unmanned aerial vehicle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle navigation, in particular to a vision-based unmanned aerial vehicle autonomous landing method and system. BACKGROUND

[0002] An unmanned aerial vehicle is referred to as "unmanned aerial vehicle" ("UAV"), which is controlled by a wireless remote control device and a self-provided program controller. With the continuous development of science and technology, the application of unmanned aerial vehicles is becoming more and more extensive. Compared with manned aircraft, unmanned aerial vehicles are more suitable for performing tasks in dangerous and harsh environments. Due to its small size, low cost, high cost performance, long distance, unmanned driving and other advantages, it has been widely used in military and civilian fields. In the military field, it is used for tactical reconnaissance, electronic jamming, target identification and other combat support. In the civil field, unmanned aerial vehicles will become the mainstream in the fields of agricultural and forestry plant protection, power inspection, logistics distribution, forest fire prevention, traffic law enforcement, high-altitude safety, and rescue in dangerous environments.

[0003] At present, the autonomous landing navigation technology of unmanned aerial vehicles at home and abroad mainly includes inertial navigation system (INS), GPS navigation, INS / GPS integrated navigation system and vision navigation system. Although the inertial navigation system has the advantages of not relying on any external information, simple implementation and good stability, its biggest problem is that it is very sensitive to initial values, the navigation error increases with time, resulting in a large error in the estimated parameters of the pose, which cannot meet the precision requirements of landing. The GPS navigation system is widely used and relies on the propagation of radio signals. It has the advantages of fast, high efficiency, high positioning accuracy, wide application, all-weather positioning, etc., but it is easy to interfere or lose satellite signals, resulting in inaccurate positioning. Although the INS / GPS integrated navigation system makes up for the shortcomings of each individual navigation system, it is still affected by GPS, such as satellite failure. These systems are not accurate enough to meet the needs of unmanned aerial vehicle landing. Compared with traditional inertial navigation methods, vision navigation error accumulation is not large, and relative to GPS navigation system, it is less affected by the surrounding environment, time and weather conditions. The vision sensor uses visible light, and the external light enters the camera through the reflection of the object to become an image, so it is not affected by electromagnetic interference. The vision sensor has the advantages of saving time and cost, being light and simple in structure. Compared with other sensors, its biggest feature is that it can intelligently identify targets, while other sensors can only provide some sensing data. In addition, the vision navigation system has the advantages of concealment, complete autonomy and anti-interference of on-board cameras, thus improving the performance of the system, so in recent years, the vision-based navigation system has been the focus of research, and the landing process is the most frequent process of unmanned aerial vehicle crashes. Reliable and accurate landing technology is an important trend in unmanned aerial vehicle technology, and the core is how to achieve precise, stable and real-time navigation.

[0004] With the development of computer vision, it is still a technical problem to land on a complex environment and a dynamic platform, and the research using machine vision is increasing, and the algorithm based on the deep learning framework is applied to the visual landing, and the landing mark detection and the complex condition tracking need to be strengthened. Therefore, at present, the research on the autonomous landing of the unmanned aerial vehicle on the moving target is less, and the unmanned aerial vehicle needs to master the target identification, the tracking of the moving target and the behavior state prediction. SUMMARY

[0005] In view of the problems in the prior art, the purpose of the present application is to provide a visual-based unmanned aerial vehicle autonomous landing method, a new algorithm framework and training method based on deep reinforcement learning are proposed, and multiple networks are used to share the fitting task pressure, that is, multiple control networks are used to jointly control the navigation of the unmanned aerial vehicle, so as to solve the problem of dynamic platform landing of the unmanned aerial vehicle, and the method has the characteristics of high robustness and strong portability.

[0006] The present application is realized by the following technical solutions:

[0007] A visual-based unmanned aerial vehicle autonomous landing method, comprising the following steps:

[0008] Step 1, the unmanned aerial vehicle acquires the image of the dynamic parking apron and performs pretreatment;

[0009] Step 2, based on the trained deep learning network, the motion feature, the distance feature and the contour scale change feature of the dynamic parking apron in the pretreated image are extracted, and the direction control action, the height control action and the speed control action of the unmanned aerial vehicle are outputted;

[0010] Step 3, according to the direction control action, the height control action and the speed control action of the unmanned aerial vehicle, the direction control instruction and the height control instruction of the unmanned aerial vehicle are determined;

[0011] Step 4, according to the direction control instruction and the height control instruction, the unmanned aerial vehicle is controlled to land on the dynamic parking apron.

[0012] Preferably, the acquired image is copied into three copies in step 1, and each image is pretreated respectively, and the pretreatment method is to perform dimension reduction, compression and stacking processing on the image.

[0013] Preferably, the deep learning network in step 2 comprises a direction control network, a height control network and a region prediction network;

[0014] The direction control network is used for extracting the motion feature of the parking apron in the pretreated image sequence, and outputting the direction control action according to the motion feature of the parking apron;

[0015] The height control network is configured to extract a distance feature of the apron in the preprocessed depth image sequence, and output a UAV height control action according to the distance feature of the apron.

[0016] The region prediction network is configured to extract a contour scale change feature of the apron in the preprocessed image sequence, and output a UAV speed control action according to the contour scale change feature of the apron.

[0017] Preferably, the method for confirming the direction control instruction and the height control instruction in step 3 is as follows:

[0018] The action selection strategy is adopted to select the optimal action of the direction control action, the height control action and the speed control action, and the direction control instruction and the height control instruction are determined according to the optimal action and the initial speed of the UAV.

[0019] Preferably, the direction control instruction is determined according to the optimal action of the direction control action and the speed control action, and in combination with the initial speed of the UAV.

[0020] Preferably, the height control instruction is determined according to the optimal action of the height control action and the speed control action.

[0021] Preferably, the expression of the direction action control instruction is as follows:

[0022] a uav1 =a i *a0*b i

[0023]

[0024] Wherein, a0 is the initial speed of the UAV, v is the speed, w is the angular velocity, a i is the optimal action of the direction control action, and b i is the optimal action of the speed control action.

[0025] Preferably, the expression of the height action control instruction is as follows:

[0026] a uav2 =a i *b i

[0027] Wherein, A i is the optimal action of the height control action, and b i is the optimal action of the speed control action.

[0028] A system of a visual-based UAV autonomous landing method, comprising,

[0029] An image preprocessing module configured to acquire images of a dynamic apron by the UAV and perform preprocessing.

[0030] a feature extraction module, configured to extract motion features, distance features and contour scale change features of the dynamic apron in the preprocessed image by using the trained deep learning network, and output direction control actions, height control actions and speed control actions of the UAV;

[0031] a control instruction output module, configured to determine direction control instructions and height control instructions of the UAV according to the direction control actions, the height control actions and the speed control actions of the UAV;

[0032] a control module, configured to control the UAV to land on the dynamic apron according to the direction control instructions and the height control instructions.

[0033] Compared with the prior art, the present application has the following beneficial technical effects:

[0034] The application discloses a visual-based autonomous landing method of a UAV, which realizes autonomous landing of a dynamic apron of a UAV in an unknown environment; a feature extraction and state estimation method of a visual sequence of the UAV is adopted to extract features such as a center position, a motion direction and a motion speed of the dynamic apron according to processed visual information, estimate a relative pose of the UAV and the dynamic apron, and predict a spatiotemporal state and a motion behavior of the dynamic apron; according to a perception result of the UAV, control instructions such as a motion direction, a height and a flight speed of the UAV are directly generated, so that the dynamic apron landing task of the UAV in the unknown environment with GPS denial is realized; the method combines a multi-control network cooperative control based on deep reinforcement learning and a target spatiotemporal state prediction method, so that the UAV autonomously completes the dynamic apron landing task in the unknown environment without human intervention, and the autonomy and the intelligence of the UAV are greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a system overall block diagram of the autonomous landing method of the UAV.

[0036] Figure 2 is a network structure block diagram of the autonomous landing method of the UAV.

[0037] Figure 3 is a flow block diagram of the autonomous landing method of the UAV.

[0038] Figure 4 is a landing area prediction display schematic diagram of the autonomous landing method of the UAV.

[0039] Figure 5 is a training protocol algorithm block diagram of the autonomous landing method of the UAV. DETAILED DESCRIPTION

[0040] The application will be further described in detail below with reference to the accompanying drawings, which are provided to explain the application rather than limit it.

[0041] Referring to Figures 1-5 A visual-based unmanned aerial vehicle autonomous landing method, comprising the following steps:

[0042] Step 1, an unmanned aerial vehicle on-board camera acquires images of a landing site, and the acquired images are preprocessed.

[0043] Specifically, the acquired images are images in the first-person perspective of the unmanned aerial vehicle, and the preprocessing of the images is dimension reduction, compression and stacking processing of the images; the acquired images are copied into three copies, and each copy of the images is preprocessed, and the three copies of the preprocessed images are respectively taken as input data of a direction control network, a height control network and a region prediction network.

[0044] The input data of the direction control network, the acquired RGB images are reduced to GRAY images, and the reduced GRAY images are compressed from 640*480 to 110*110, and the compressed continuous multiple frames of images are stacked as the input data of the direction control network.

[0045] The input data of the height control network, the depth images are converted into pseudo-laser radar data, and the RGB images are subjected to the above dimension reduction, compression and stacking processing, and taken as the input data of the height control network.

[0046] The input data of the region prediction network, the acquired RGB images are reduced to GRAY images, and the reduced GRAY images are compressed from 640*480 to 96*96, and the compressed continuous multiple frames of images are stacked as the input data of the direction control network.

[0047] Step 2, a deep learning network is constructed and trained, and the deep learning network comprises a direction control network (D-NET) based on a sequence of GRAY images, a height control network (H-NET) based on a sequence of GRAY-D images, and a region prediction network (F-NET) based on a sequence of GRAY images.

[0048] The preprocessed images are taken as the input of the three control networks, and the three control networks respectively output the motion direction a1 t , the height a2 t , and the speed change a3 t of the unmanned aerial vehicle.

[0049] The three control networks have the same structure, and the network structure is described below taking the direction control network (D-NET) as an example.

[0050] For the D-NET network including three convolutional layers, four 55*55 continuous step images are stacked together as the input of the D-NET, and three convolutional layers activated by the ReLU function are used to generate feature representations from the processed image input. Specifically, the first convolutional layer uses 4 convolutional kernels with a size of 3*3 to extract features with a step of 2; the second convolutional layer uses 8 convolutional kernels with a size of 3*3 to extract features with a step of 2, and the third convolutional layer uses 16 convolutional kernels with a size of 3*3 to extract features with a step of 2. The three optimized convolutional networks can generate targeted feature representations for control networks with different control requirements. In each branch, the first fully connected layer is activated by the ReLU function, the second fully connected layer is activated by the Linear function, and the third fully connected layer is activated by the Softmax function.

[0051] During training, in each step, the training protocol is as shown in the following table. Figure 5 First, the direction control control network, the height control control network, and the speed control control network are initialized. Then, the three control networks obtain images, and the preprocessed images are used as state information (s1, s2, s3). After the control networks obtain the state information, they output the value function, and then select and execute the optimal action. In each step, the UAV detects whether the termination condition is reached, and if so, the loop is stopped; otherwise, the reward information is updated, and the state is converted into a new state. Finally, the new state is used as the network input, and a new action is calculated and selected. The <state, action, reward, new state> is stored in the experience replay pool 1, 2, and 3.

[0052] In the experience replay pool 1, the sample <s1 t , a1 t , r1 t , s1 t+1 > is obtained, where s1 t is the state code obtained by the direction control network at time t, a1 t is the action selected by the UAV, r1 t is the action evaluation of the selected action, and s1 t+1 is the next state. In the experience replay pool 2, the sample <s2 t , a2 t , r2 t , s2 t+1 > is obtained, and in the experience replay pool 3, the sample <s3 t , a3 t , r3 t , s3 t+1 > is obtained. In each iteration process, first, the direction control network obtains s1 t and calculates the state value function Q(s1 t;ω1), where ω is the parameter of the direction control network; the height control network obtains s2t to calculate the state value function Q(s2). t ;ω2), the region prediction network obtains s3 t Calculate the state value function Q(s3) t ;ω3). Then, using the already executed action a1 t a2 t a3 t Choose the state-action value function Q(s1) t a1 t ;ω1),Q(s2) t a2 t ;ω2),Q(s3) t a3 t ;ω3). s1 t+1 The state value function, Q(s1), is obtained by calculating the state at the next moment. t+1 ;ω1),Q(s2) t+1 ;ω2),Q(s3) t+1 ;ω3);for Q(s1) t+1 ;ω1),Q(s2) t+1 ;ω2),Q(s3) t+1 ;ω3) Select the action with the maximum Q value respectively to obtain Q(s1) t+1 a1 t+1 ;ω1),Q(s2) t+1 a2 t+1 ;ω2),Q(s3) t+1 a3 t+1 ;ω3). Finally, the discount function γ and reward r1 are used. t r1 t Calculate the state-action value function y1 at time t, as shown in the equation:

[0053]

[0054] Based on the above formula, the loss function formula for the orientation control network is:

[0055] L1(ω1)=E[(y1-Q(s1 t a1 t ;ω1)) 2 ]

[0056] Using the discount function γ and the reward r2 t The state-action value function y2 at time t is calculated as follows:

[0057]

[0058] Based on the above formula, the loss function formula for the regional prediction network is:

[0059] L2(ω) = E[(y2 - Q(s2 t , a2 t ; ω2)) 2 ]

[0060] The state-action value function y3 at time t is calculated using the discount function γ and the reward r3 t

[0061]

[0062] Based on the above formula, the area prediction network loss function formula is:

[0063] L3(ω) = E[(y3 - Q(s3 t , a3 t ; ω3)) 2 ]

[0064] The anti-tracking network parameters are adjusted using the Adam optimizer.

[0065] Step 3, based on the trained deep learning network, the motion features, distance features and contour scale change features of the dynamic apron in the preprocessed image are obtained.

[0066] The direction control network (D-NET) is used to extract the motion features s1 t of the apron in the preprocessed image sequence, and output the direction control action a dir according to the motion features of the apron.

[0067] The motion features of the apron include the contour features, shape features and position change features of the apron.

[0068] The height control network (H-NET) is used to extract the distance features s2 t of the apron in the preprocessed depth image sequence, and output the height control action a height of the unmanned aerial vehicle according to the distance features of the apron.

[0069] The distance features of the apron include the height features of the unmanned aerial vehicle and the ground, and the distance features of the unmanned aerial vehicle and the apron.

[0070] The area prediction network (F-NET) is used to extract the contour scale change features s3 t of the apron in the preprocessed image sequence, and output the speed control action a height of the unmanned aerial vehicle according to the contour scale change features of the apron.

[0071] The contour scale change features of the apron include the contour features of the apron, and the contour scale change amount.

[0072] ​Step 4, according to the acquired dynamic motion characteristics, distance characteristics and contour scale change characteristics of the parking apron, determine the unmanned aerial vehicle landing direction control instruction and height control instruction.

[0073] Specifically, as shown in Figure 2 , the direction control network, the height control network and the area prediction network work simultaneously.

[0074] The direction control network outputs a state-action value function, and the corresponding direction control action a dir is: a1, a2, a3, a4, a5, a6, a7, a8, a9.

[0075] Wherein a1, a2, a3, a4, a5, a6, a7, a8, a9 represent the directions of forward, backward, left, right, left up, right up, left up, right down and keep the current direction state, and the direction control action is as follows:

[0076] a dir = [a1 a2 a3 a4 a5 a6 a7 a8 a9

[0077] The height control network outputs a state-action value function, and the corresponding height control action a height is: A1, A2, which represent the directions of upward and downward, and the height control action is as follows:

[0078] a height = [A1 A2]

[0079] The area prediction network outputs a state-action value function, and the corresponding speed control action b is: b1, b2, b3, which represent the speed increase, speed unchanged and speed decrease of the unmanned aerial vehicle respectively:

[0080] a fore = [b1 b2 b3]

[0081] An action selection strategy is adopted, and the optimal action a i , A i , b i of the direction control action, the height control action and the speed control action is selected according to the action evaluation estimated by the state-action value function.

[0082] According to the optimal action and the initial speed of the unmanned aerial vehicle, the direction action control instruction a uav1 of the unmanned aerial vehicle is determined.

[0083] a uav1 = a i *a0*b i

[0084]

[0085] Wherein, a0 represents the initial value of the speed of the unmanned aerial vehicle, including the speed v and the angular velocity w.

[0086] The height action control instruction a is determined according to the height control optimal action and the speed control optimal action uav2 , and the expression is as follows:

[0087] a uav2 =A i *b i

[0088] Step 5, according to the direction control instruction and the height control instruction, the unmanned aerial vehicle is controlled to land on the dynamic parking apron.

[0089] According to the D-NET information processing result, the parking apron behavior recognition and the dynamic parking apron space state prediction task are completed, and the movement direction control instruction of the unmanned aerial vehicle is finally output based on the prediction result. According to the H-NET information processing result, the parking apron recognition and search of the unmanned aerial vehicle are completed, and the height control instruction of the unmanned aerial vehicle is finally output based on the detection result. According to the F-NET information processing result, the parking apron movement state prediction is completed, and the speed control instruction of the unmanned aerial vehicle is finally output based on the prediction result. The dynamic parking apron landing task of the unmanned aerial vehicle is completed. As shown in Figure 3 , the F-NET landing area position prediction result can be displayed in real time as shown in Figure 4 .

[0090] The application also provides a visual-based unmanned aerial vehicle autonomous landing system, which comprises an image preprocessing module, a feature extraction module, a control instruction output module and a control module.

[0091] The image preprocessing module is used for acquiring the image of the dynamic parking apron by the unmanned aerial vehicle and pre-processing the image.

[0092] The feature extraction module is used for extracting the motion feature, the distance feature and the contour scale change feature of the dynamic parking apron in the pre-processed image by using the trained deep learning network, and outputting the direction control action, the height control action and the speed control action of the unmanned aerial vehicle.

[0093] The control instruction output module is used for determining the direction control instruction and the height control instruction of the unmanned aerial vehicle according to the direction control action, the height control action and the speed control action of the unmanned aerial vehicle.

[0094] The control module is used for controlling the unmanned aerial vehicle to land on the dynamic parking apron according to the direction control instruction and the height control instruction.

[0095] The above merely illustrates the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical scheme falls within the protection scope of the present application.

Claims

1. A vision-based autonomous landing method for unmanned aerial vehicles, characterized in that, Includes the following steps: Step 1: The drone acquires images of the dynamic helipad and performs preprocessing; Step 2: Extract motion features, distance features, and contour scale change features of the dynamic helipad in the preprocessed image based on the trained deep learning network, and output the direction control action, altitude control action, and speed control action of the UAV. The deep learning network includes an orientation control network, a height control network, and a region prediction network; The direction control network is used to extract the motion features of the helipad in the preprocessed image sequence and output direction control actions based on the motion features of the helipad. The altitude control network is used to extract the distance features of the helipad in the preprocessed depth image sequence, and output the drone altitude control action based on the distance features of the helipad. The region prediction network is used to extract the contour scale change features of the helipad in the preprocessed image sequence, and output the UAV speed control action based on the contour scale change features of the helipad. Step 3: Based on the UAV's directional control actions, altitude control actions, and speed control actions, determine the UAV's directional control commands and altitude control commands; the method for confirming the directional control commands and altitude control commands is as follows: An action selection strategy is adopted to select the optimal actions for directional control, altitude control, and speed control respectively. The directional control command and altitude control command are determined based on the optimal actions and the initial speed of the UAV. The expression for the directional motion control command is as follows: a uav1 =a i *a0*b i Where a0 is the initial value of the UAV's velocity, v is the velocity, w is the angular velocity, and a i For the optimal action of directional control, b i The optimal action for speed control; The expression for the height motion control command is as follows: a uav2 =A i *b i Among them, A i For optimal action with high control, b i The optimal action for speed control; Step 4: Control the UAV to land on the dynamic landing pad according to the direction control command and altitude control command.

2. The vision-based autonomous landing method for unmanned aerial vehicles according to claim 1, characterized in that, In step 1, the acquired image is copied into three copies, and each copy is preprocessed separately. The preprocessing methods are dimensionality reduction, compression, and stacking of the images.

3. The vision-based autonomous landing method for unmanned aerial vehicles according to claim 1, characterized in that, The directional control command is determined based on the optimal directional control action and the speed control action, combined with the initial speed of the UAV.

4. The vision-based autonomous landing method for unmanned aerial vehicles according to claim 1, characterized in that, The altitude control command is determined based on the optimal action of the altitude control action and the speed control action.

5. A system employing the vision-based autonomous landing method for unmanned aerial vehicles as described in any one of claims 1-4, characterized in that, include, The image preprocessing module is used to preprocess images of the dynamic helipad acquired by the UAV. The feature extraction module is used to extract motion features, distance features, and contour scale change features of the dynamic helipad in the preprocessed image using a trained deep learning network, and outputs the direction control actions, altitude control actions, and speed control actions of the UAV. The control command output module is used to determine the direction control command and altitude control command of the UAV based on the UAV's direction control action, altitude control action and speed control action; The control module is used to control the UAV to land on a dynamic landing pad based on directional control commands and altitude control commands.

Images