Intelligent return method of UAV based on UAV mother vehicle

Abstract

The present invention provides a method for intelligent return of drones based on a drone mother vehicle, including: when the drone triggers a return command during flight, after the drone mother vehicle receives the return command, the flight trajectory of the drone is obtained from historical records, the return path of the drone is calculated and sent to the drone, the drone predicts the current position in real time, and adjusts the flight path to match the return path; after the GPS signal fails, the real-time flight position of the drone is predicted by inertial navigation and image matching methods, and the flight route is adjusted along the return path to the parking path mark point, and the parking field mark set on the roof of the drone mother vehicle is used to guide the drone to land. In the technical solution provided by the present invention, it can ensure that the drone intelligent return is successful and the path is the shortest under different conditions including GPS failure.

Description

Technical Field

[0001] The present invention relates to the field of intelligent return of unmanned aerial vehicles (UAVs) by UAV mother vehicles, and in particular to an intelligent return method of unmanned aerial vehicles (UAVs) by UAV mother vehicles. Background Art

[0002] With the rapid development of science and technology, various traditional electronic devices have been applied in new ways. Micro drones can be loaded with relevant monitoring equipment and fly to preset destinations to perform various tasks, making the application scope of drones more and more extensive. However, drones rely on power systems and navigation systems to complete their flights during flight, and the power system relies on batteries. If the power is insufficient, the drone cannot fly. The navigation system relies on GPS positioning. If the GPS fails, the drone will lose its flight direction. At this time, general manufacturers will set up a return system inside the drone to make the drone return. However, the existing autonomous return technology records the trajectory of the drone in flight, and then re-locates the flight trajectory through GPS when returning, and then returns. This method cannot be used when GPS fails, and the return fails at that time. On the other hand, the method of estimating the battery mileage is also inaccurate, because the battery is affected by the environment during use, such as the number of charging times and the current ambient temperature, which will affect the battery power consumption, thereby affecting the drone's cruising range. If the battery power is not enough to support the cruising range, the drone may be lost and unable to return. There are also related improvements in the existing technology, such as patent CN201811541989.1, which estimates the relevant wind speed and direction and accurately estimates the battery life. However, the environment changes very quickly, and the flying environment and the return environment may be quite different, resulting in inaccurate calculation results. Therefore, it is difficult for the existing technology to solve the problem of the correct return of drones. Summary of the invention

[0003] Based on the defects in the prior art, the present invention provides a method for intelligent return of a drone based on a drone mother vehicle, comprising: establishing a communication connection between the drone and the drone mother vehicle and communicating in real time; when the drone triggers a return command during flight, the drone executes the return command and sends a return message to the drone mother vehicle;

[0004] After receiving the return command, the drone mother vehicle obtains the flight trajectory of the drone from the historical records, calculates the return path of the drone and sends it to the drone. The drone predicts its current position in real time and adjusts the flight path to match the return path.

[0005] The drone mother vehicle adjusts its own speed and drives to one of the parking path marking points on the return path. While waiting for the drone mother vehicle to return to the parking path marking point, the drone is guided to land using the parking field signs set by the drone mother vehicle.

[0006] A drone intelligent return method based on a drone mother vehicle, further, when the distance between the drone and the drone mother vehicle is less than a preset first threshold, the drone mother vehicle identifies a parking lot mark on the collected image, and the parking lot mark is set on the roof of the drone mother vehicle;

[0007] If the parking lot sign is identified, the position information obtained by the drone camera is used to track the parking lot sign of the drone mother vehicle; at the same time, PID is used to control the drone to land at the drone mother vehicle parking lot.

[0008] A drone intelligent return method based on a drone mother vehicle, further, when the drone mother vehicle receives a return message after the drone GPS signal disappears, the drone mother vehicle obtains the drone's most recent flight path;

[0009] Match the most recent flight path of the UAV with the historical paths, obtain all historical paths that overlap with the most recent flight path, and combine the historical paths that overlap with the most recent flight path into a path set RoadA;

[0010] The most recent flight path is recombined with the path set RoadA to form a new return path set RoadB. A path with the shortest distance for the drone to travel is selected from RoadB as the return path and the parking path mark points are marked. The drone mother vehicle sends the re-planned return path to the drone.

[0011] A method for intelligent return of a drone based on a drone mother vehicle, further, when the drone receives an instruction to perform a flight mission, the drone checks whether a communication connection is established with the drone mother vehicle, and if a communication connection is established, the drone takes off to perform the flight mission;

[0012] If the communication connection has not been established, the UAV initiates a request to establish a communication connection with the UAV mother vehicle. After the communication connection is established, the UAV takes off to perform the flight mission;

[0013] The UAV collects images and records the GPS information of the sampling points at preset periodic intervals, and sends the images and GPS information to the UAV mother vehicle; the UAV mother vehicle processes the images and GPS information and sends the key information to the UAV.

[0014] A method for intelligent return of drones based on a drone mother vehicle, further, the drone records the flight trajectory and collects environmental information in real time during flight and sends the information to the drone mother vehicle; the drone mother vehicle processes and stores the information sent by the drone; the drone records the flight trajectory and collects environmental information; the environmental information includes environmental images and environmental location information;

[0015] The current environment image and environment location information are collected in real time during the preset sampling period. The location information includes longitude, latitude, altitude, and the roll angle, pitch angle, and yaw angle of the drone.

[0016] A method for intelligent return of a drone based on a drone mother vehicle. Further, the conditions for triggering the intelligent return mode include: failure of GPS signal, low battery, and the user selecting the drone mother vehicle to start the intelligent return command. It can be a conditional trigger or can be triggered according to user needs. The drone flies far away and the user cannot see the behavior of the drone. At this time, for safety reasons, the user starts the return command.

[0017] A method for intelligent return home of a drone based on a drone mother vehicle. Further, when the intelligent return home mode is turned on, it is determined whether the GPS signal is invalid. If the GPS signal is normal, the GPS signal and battery information of the current drone are obtained and sent to the drone mother vehicle. After the drone mother vehicle receives the GPS signal of the drone, the distance between the drone mother vehicle and the drone is calculated.

[0018] A method for intelligent return of a drone based on a drone mother vehicle, further, if the drone battery is sufficient, the positions of the drone and the drone mother vehicle are preset by Kalman filtering, and the drone is controlled by PD to approach the drone mother vehicle until the distance between the drone mother vehicle and the drone meets a first preset distance threshold;

[0019] If the battery of the drone is low and the GPS signal is normal, the drone will remain hovering at the position providing the GPS signal. The drone mother vehicle will calculate the current distance to the drone based on the GPS signal provided by the drone. The drone mother vehicle will drive to the position where the GPS signal matches based on the GPS signal provided by the drone.

[0020] A method for intelligent return of drones based on a drone mother vehicle. Further, if the return path triggering condition is the failure of the GPS signal, the drone mother vehicle selects a mark from the return route as a parking path mark point for the drone mother vehicle to wait for the drone to land based on the return history route and the distance between the GPS position coordinates at time T0 and the drone.

[0021] The UAV uses inertial navigation and visual recognition methods and Kalman filtering to predict the flight position of the UAV. The UAV follows the return route according to the predicted position to the parking path mark point;

[0022] When both the UAV and the UAV mother vehicle arrive at the parking path point, the UAV collects surrounding images and obtains the parking mark of the UAV mother vehicle from the image to park.

[0023] A method for intelligent return of a drone based on a drone mother vehicle, further, the position prediction method of the drone at least includes: an image prediction method, the image prediction method at least includes: obtaining an image imageT at time T, an image imageT0 at time T0;

[0024] Select preset M feature points from image imageT0 as matching feature point arrayT;

[0025] Find the matching preset M feature points arrayT0 in image imageT1

[0026] Using the projection change, the difference in distance between the corresponding feature points in imageT and imageT0 will be calculated respectively.

[0027] A method for intelligent return of drones based on drone mother vehicles. Further, the drone navigation coordinate system and camera coordinate system are established, and the world coordinates of the position of the feature point are Feature:

[0028]

[0029] Among them, R is the change matrix between the camera coordinate system and the drone navigation coordinate system, H is the height of the drone, ax,ay are the pixel coordinates of the feature points, fx,fy are the focal lengths of the camera on the x-axis and y-axis;

[0030] Then the difference between a pair of matching feature points FeatureTi and FeatureT0i in different images when sampling at time T and sampling at time T0 is ΔSi:

[0031] ΔSi=FeatureTi-FeatureTi0i

[0032]

[0033] Where axi, ayi, fx, fy are the pixel coordinates of feature point i in the image collected at time T and the focal length of the camera, Ra and Ha are the transformation matrix of the camera coordinates and navigation coordinates at time T and the height of the drone;

[0034] Where bxi, byi, fx, and fy are the pixel coordinates of feature point i in the image collected at time T0 and the focal length of the camera, respectively. Rb and Hb are the transformation matrices of the camera coordinates and navigation coordinates at time T0 and the height of the drone.

[0035] Then, the average position between a preset number of feature point pairs is obtained as the displacement of the UAV from time T0 to time T;

[0036]

[0037] The position of the drone at time T is ST, M is the number of feature points, and the position of the drone at time T is ST:

[0038] ST=ST0+ΔS

[0039] Among them, ST is the position at time T, and ST0 is the position at time T0.

[0040] A method for intelligent return of drones based on a drone mother vehicle. Further, when judging whether to update the position according to the calculated position of the drone, it is necessary to use the image collected at the current moment and the reference image collected at the same position point of the historical trajectory for contour matching. When the contour match is greater than a preset threshold, the position is updated.

[0041] A method for intelligent return of a drone based on a drone mother vehicle, further comprising:

[0042] The reference image is gray-scaled, then edge detected and cropped according to a preset algorithm before storage; the image collected by the drone is gray-scaled, then median filtered and then edge detected;

[0043] Get the flight altitude of the drone and the resolution of the reference image corresponding to the current point image, then scale the image and align it with the reference image, and perform scaling to convert the current image resolution to the resolution of the reference image;

[0044] According to the contour area obtained by edge detection, fixed contour windows of preset different sizes are selected for matching in the reference image, and the position of the best matching area is obtained from the reference image;

[0045] The position with the largest number of overlapping pixels between the edges of the current image and the corresponding reference image is taken as the matching result.

[0046] A method for intelligent return of a drone based on a drone mother vehicle, further, when the position is updated using the contour matching result, the contour matching result is greater than the confidence of a given contour shape, and the contour confidence acquisition method includes:

[0047] S1: Segment the reference image and assign different contour confidences to different regions;

[0048] S2: Principles for setting contour confidence: image registration in urban areas is greater than that in rural areas, the registration ratio of road intersections is greater than that of other road sections; the registration of feature markers is greater than that of non-feature markers.

[0049] S3: Compare the position where the contour matching occurs with the corresponding contour confidence, and update when it is greater than the corresponding contour confidence.

[0050] A method for intelligent return of a drone based on a drone mother vehicle, further comprising: selecting a preset number of contours in a current image, and matching the selected contours in a reference image; setting a first contour matching threshold to determine whether the contour in the current image matches the contour in the reference image; and setting a second contour matching threshold to determine the proportion of the number of pre-matched contours to the number of matches of the preset contours;

[0051] If the matching result is greater than the first contour matching threshold and greater than the second contour matching threshold, select the first n contours with better matching results and obtain the median of the first n contour matching results, and select the position corresponding to the contour corresponding to the median as the final drone position update.

[0052] Beneficial effects:

[0053] 1. The technical solution provided by the present invention can ensure that the drone can return successfully and with the shortest path under different conditions.

[0054] 2. In the technical solution provided by the present invention, when the GPS signal fails, the inertial navigation and image matching methods can be used to fully and accurately predict and adjust the flight position of the UAV, and the UAV mother vehicle can be used to guide the UAV to land.

[0055] 3. In the technical solution provided by the present invention, when the GPS signal fails, the return path of the UAV can be replanned so that the UAV can return along the optimal path. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The following drawings are only used to schematically illustrate and explain the present invention, and do not limit the scope of the present invention.

[0057] Figure 1 Schematic diagram of the current flight path and flight marking points of a drone in one embodiment of the present invention.

[0058] Figure 2 It is a schematic diagram of the current flight path and flight marking points of the UAV in one embodiment of the present invention and a path diagram of the UAV mother vehicle.

[0059] Figure 3 The diagram is a schematic diagram of return and stop mark path points of the current flight path of the UAV in one embodiment of the present invention.

[0060] Figure 4 Schematic diagram of the current new return flight path and parking mark path points of the drone in one embodiment of the present invention.

[0061] Figure 5 Schematic diagram of the current original historical path return path and stop mark path points of the drone in one embodiment of the present invention.

[0062] Figure 6 It is a schematic diagram of the return and stop marking path points of the UAV along the newly planned historical path in one embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to have a clearer understanding of the technical features, purposes and effects of this invention, the specific implementation methods of the present invention are now described with reference to the accompanying drawings, and the same reference numerals in each figure represent the same parts. In order to make the drawings concise, the parts related to the present invention are schematically shown in each figure, and do not represent the actual structure of the product. In addition, in order to make the drawings concise and easy to understand, in some figures, only one of the parts with the same structure or function is schematically drawn or marked.

[0064] Regarding control systems, functional modules, and application programs (APP), it is well known to those skilled in the art that they can take any appropriate form, either hardware or software, either multiple discrete functional modules or multiple functional units integrated into one hardware. In the simplest form, the control system can be a controller, such as a combinational logic controller, a microprogram controller, etc., as long as the operations described in this application can be implemented. Of course, the control system can also be integrated into a physical device as different modules, which does not deviate from the basic principles and protection scope of the present invention.

[0065] The term “connection” in the present invention may include direct connection, indirect connection, communication connection, and electrical connection, unless otherwise specified.

[0066] The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that when used in the specification, the terms "include" and / or "comprise" refer to the presence of stated features, values, steps, operations, elements and / or components, but do not exclude the presence or additional addition of one or more other features, values, steps, operations, elements, components and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the listed related items.

[0067] It should be understood that the term "vehicle" or "vehicular" or other similar terms used herein generally include motor vehicles, such as passenger cars including sport utility vehicles (SUVs), buses, trucks, various commercial vehicles, ships including various boats, marine vessels, aircraft, etc., and include hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., fuels derived from non-petroleum energy sources). As referred to herein, a hybrid vehicle is a vehicle having two or more power sources, such as both gasoline-powered and electric-powered vehicles.

[0068] In addition, the controller of the present disclosure may be embodied as a non-transitory computer readable medium on a computer readable medium containing executable program instructions executed by a processor, a controller, or the like. Examples of computer readable media include, but are not limited to, ROM, RAM, compact disk (CD)-ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage device. The computer readable recording medium may also be distributed in a computer system coupled via a network so that the computer readable medium is stored and executed in a distributed manner, for example, via a telematics server or a controller area network (CAN).

[0069] The present invention provides an intelligent return method for a UAV based on a UAV mother vehicle, see Figures 1 to 6 , specifically including: when the UAV receives an instruction to perform a flight mission, the UAV checks whether a communication connection is established with the UAV mother vehicle. If a communication connection is established, the UAV takes off to perform the flight mission;

[0070] If the communication connection has not been established, the UAV initiates a request to establish a communication connection with the UAV mother vehicle. After the communication connection is established, the UAV takes off to perform the flight mission;

[0071] The drone mother vehicle communicates with the drone in real time;

[0072] The drone records the flight trajectory and collects environmental information in real time during the flight and sends the information to the drone mother vehicle;

[0073] The drone mother vehicle processes and stores the information sent by the drone;

[0074] When the return command is triggered, the drone executes the return command and sends the return command to the drone mother vehicle;

[0075] After receiving the return command, the drone mother vehicle obtains the flight trajectory of the drone from the historical records. The drone mother vehicle adjusts its own speed and drives to one of the marked points in the flight trajectory. When waiting for the drone mother vehicle to return to the marked point, it uses the parking lot sign set by the drone mother vehicle to guide the drone to land.

[0076] Specifically, see Figures 1 to 6 , Figure 1 The flight trajectory and marking points of the drone are displayed. At this time, the drone is flying normally. A in the figure is the take-off point of the drone, and B in the figure is the location of the drone mother vehicle. Specifically, the drone mother vehicle can also be at point A. A can be a drone station for inspection or other maintenance of the drone. After the drone takes off, when it drives to point E, the return command is triggered, and the drone needs to return. At this time, the return route of the drone needs to be determined. In the existing technology, such as DJI drones, it can only feedback along the original path. The original path is the path recorded when the return mode is turned on. If the path during a certain flight is within the range of the original path, the return will fail.

[0077] Therefore, the flight path of this embodiment is recorded in real time and stored in the UAV mother vehicle, and the UAV can return along the most accurate historical path.

[0078] In addition, if the GPS signal is normal, you can return to the specified location along the custom path, such as Figure 4 As shown in the figure, the drone can return to point F along the new return path. The path EF is not the historical path. In this way, the best path can be selected to return to the drone mother vehicle.

[0079] If the GPS signal is abnormal, follow the most recent historical path and select a stop path marker point suitable for the UAV mother vehicle's driving area, such as Figure 5 Point D in .

[0080] Specifically, the recording of the UAV flight trajectory and the collection of environmental information include: environmental information including environmental images and location information of the environment;

[0081] The current environment image and environment location information are collected in real time during the preset sampling period. The location information includes longitude, latitude, altitude, and the roll angle, pitch angle, and yaw angle of the drone.

[0082] The flight trajectory of the drone is recorded in real time using GPS signals and marked on the flight map according to the GPS update frequency;

[0083] Specifically, when the preset sampling period collects images, its period matches the GPS update frequency. The matching is as follows: when the image is sampled at time t, the GPS position information at time t is updated; but the GPS position is updated at time t1, but sampling is not necessarily required at time t1, that is, when the drone collects environmental images, it is necessary to obtain the GPS position information when the image is collected.

[0084] Specifically, the conditions for triggering the intelligent return mode include: failure of the GPS signal, low battery, and the user's selection of the drone mother vehicle to initiate the intelligent return command. It can be a conditional trigger or it can be triggered according to user needs. The drone flies far away and the user cannot see the drone's behavior. At this time, for safety reasons, the user initiates the return command.

[0085] When the intelligent return mode is turned on, it determines whether the GPS signal is invalid. If the GPS signal is normal, it obtains the current drone's GPS signal and battery information and sends them to the drone mother vehicle. After the drone mother vehicle receives the drone's GPS signal, it calculates the distance between the drone mother vehicle and the drone.

[0086] Determine whether the battery power of the drone is sufficient for the drone to fly to the marked position of the drone mother vehicle. Specifically, since the battery power is affected by many factors on the cruising range, in order to ensure the success of the battery life, a redundant calculation method is adopted in this embodiment, that is, the battery power can at least maintain 2 times the mileage to meet the conditions.

[0087] If the battery of the drone is sufficient, the positions of the drone and the drone mother vehicle are preset through Kalman filtering, and the PD (proportional-differential control method) is used to control the drone to approach the drone mother vehicle;

[0088] When the distance between the drone and the drone mother vehicle is less than a preset first threshold, the drone mother vehicle identifies the parking lot sign on the collected image; the parking lot sign is set on the roof of the drone mother vehicle;

[0089] The preset first threshold is used as a distance measurement for the drone to start searching for the specific location of the drone in the image being collected. The threshold is determined by the receptive field of the visual sensor carried by the drone. By setting the preset first threshold, the drone is prevented from sampling too early. Long-distance sampling may fail to identify the drone mother vehicle.

[0090] If the parking lot sign is identified, the position information obtained by the drone camera is used to track the parking lot sign of the drone mother vehicle; at the same time, PID (proportional-integral-differential control method) is used to control the drone to land at the drone mother vehicle parking lot.

[0091] If the battery of the drone is low and the GPS signal is normal, the drone will remain hovering at the position providing the GPS signal. The drone mother vehicle will calculate the current distance to the drone based on the GPS signal provided by the drone. The drone mother vehicle will drive to the position where the GPS signal matches based on the GPS signal provided by the drone.

[0092] The drone mother vehicle sends an arrival signal to the drone, and the drone collects surrounding images to search for parking field marks. If a parking field mark is found, the PID control drone is used to approach the drone. At the same time, the drone camera is used to collect images, calculate the drone's position and posture, and track the mark until it lands at the drone mother vehicle's parking field.

[0093] If the GPS signal fails and the battery is fully charged, the drone will lose its positioning capability and cannot be determined. In the prior art, after the drone GPS loses the signal, the drone will automatically hover for a period of time to wait for the signal to recover. If it cannot be recovered, after repeating a preset number of times, the drone will land around the GPS signal disappearance and cannot complete the return.

[0094] In order to solve the problem of the drone returning home after the GPS fails or the GPS signal disappears, an embodiment of the present invention provides a better solution, which is as follows:

[0095] The drone collects images and records the GPS information of the sampling points at preset periodic intervals, and sends the images and GPS information to the drone mother vehicle; the drone mother vehicle processes the images and GPS information and sends key information to the drone;

[0096] After the drone samples and obtains GPS signals at time To, if the GPS signal is lost at the next sampling time T, the drone will send the image collected at time T and the return route history to the drone mother vehicle;

[0097] The UAV mother vehicle determines the distance between the UAV and the return route based on the historical return route and the GPS position coordinates at time to, and selects a mark from the historical return route as the parking path point for the UAV mother vehicle to wait for the UAV to land;

[0098] The drone mother vehicle turns on GPS navigation and drives to the parking and landing mark point. The drone uses inertial navigation and visual recognition methods to follow the selected rebate historical route to the parking path point;

[0099] When the drone parking mark point or the drone mother vehicle reaches the parking path point, the drone mother vehicle or the drone sends a message of arrival at the mark point to inform the other party;

[0100] When both vehicles reach the parking path point, the drone collects periodic images and obtains the parking mark of the drone mother vehicle from the image to park;

[0101] Specifically, the method for the drone to use inertial navigation and visual recognition methods to follow the selected rebate historical route to the parking path point is as follows:

[0102] At time T, image imageT is acquired, and preset M feature points are selected from image imageT0 acquired at time T0 as matching landmark points and arrayT = {[ax0, by0, [ax1, ay1], [ax2, ay2]. ... [axM, ayM]};

[0103] Search for the preset M feature points arrayT0 = {[bx0, by0, [bx1, by1], [bx2, by2]. ... [bxM, byM]} in the image imageT;

[0104] Using the projection change, the difference in distance between the corresponding feature points in imageT and imageT0 will be calculated respectively;

[0105] The specific calculation method is as follows:

[0106] Definition: Establish the UAV navigation coordinate system and the camera coordinate system, then the world coordinates of the feature point are Feature:

[0107]

[0108] Among them, R is the change matrix between the camera coordinate system and the drone navigation coordinate system, D is the height of the drone, ax, ay are the pixel coordinates of the feature points, fx, fy are the focal lengths of the camera on the x-axis and y-axis;

[0109] Then the difference between a pair of matching feature points FeatureTi and FeatureT0i in different images when sampling at time T and sampling at time T0 is ΔSi:

[0110] ΔSi=FeatureTi-FatureT0i

[0111]

[0112] Where axi, ayi, fx, and fy are the pixel coordinates in the image collected at time T and the focal length of the camera, respectively; Ra and Ha are the transformation matrix of the camera coordinates and navigation coordinates at time T and the height of the drone;

[0113] Where bxi, byi, fx, and fy are the pixel coordinates in the image collected at time T0 and the focal length of the camera, respectively; Rb and Hb are the transformation matrix of the camera coordinates and navigation coordinates at time T0 and the height of the drone;

[0114] Specifically, in theory, two points can be used to calculate the distance that the drone has flown at time T and T0, but this is subject to various factors, such as: the pairing of feature points may be inaccurate, and there are also errors in the change matrix R and the height H. Therefore, in order to make the result more accurate, in this embodiment, a preset plurality of points, such as at least 40 feature points, are selected for updating, and then the average position between the preset number of feature points is obtained as the displacement of the drone from time T0 to time T.

[0115]

[0116] Specifically, a variety of different preset numbers can be used, the first preset feature point matching pair is M1, the second preset feature point matching pair is M2, if M1 feature point pairs can be obtained, the displacement calculated by the M1 feature points is used as the distance the UAV flies within the ΔT time; if the number of feature points obtained is less than M1 but greater than M2, the position calculated by the actual number of feature points obtained is used as the distance the UAV flies within the ΔT time; if the acquired feature points are less than M2, no displacement update calculation is performed, at this time, re-sampling is performed in the surrounding area to repeat the feature point matching.

[0117] According to the position of the drone at time T as ST, then:

[0118] ST=ST0+ΔS

[0119] Among them, ST is the position at time T, ST0 is the position at time T0, which is obtained by GPS signal positioning, and ΔS is the distance the drone flies from time T0 to time T1 calculated by visual method after the GPS signal disappears.

[0120] The displacement of the UAV from T0 to T1 is predicted again through inertial navigation, and combined with Kalman filtering, the final position of the UAV without anyone at time T1 is obtained.

[0121] Specifically, after the GPS signal is lost, due to the tracking algorithm using inertial navigation and image feature point matching, which is greatly affected by the environment, the error of the drone's return will gradually increase with the increase of distance, resulting in errors in the updated position, causing the drone to lose contact after long-distance flight due to power consumption. Therefore, in order to solve the problems of the prior art, this embodiment provides a solution, which is as follows:

[0122] Compared with the prior art which can only rely on the preset path method for return, this embodiment is improved by recombining and splicing the return path of the drone to obtain the final return path, as follows:

[0123] When the drone mother vehicle receives the return message after the drone GPS signal disappears, the drone mother vehicle obtains the drone's most recent flight path;

[0124] Match the most recent flight path of the UAV with the historical paths, obtain all historical paths that overlap with the most recent flight path, and combine the historical paths that overlap with the most recent flight path into a path set RoadA;

[0125] The most recent flight path is recombined with the path set RoadA to form a new return path set RoadB;

[0126] Select a path with the shortest driving distance from Road B to return and mark the parking point;

[0127] The drone mother vehicle sends the re-planned return path to the drone.

[0128] Specifically, when the shortest path is selected for return, the selection of the parking path point needs to meet the driving conditions of the UAV mother vehicle.

[0129] See also Figure 6 ,In the figure, L2 to L9 are historical paths, L1 is the current path of the UAV, L2, L3, L4, L9 and the current flight path L1 of the UAV have cross paths, so if points V and N in the figure are suitable for the UAV mother vehicle to travel, and paths EK1V and EK2N are suitable for parking, then points V and N are selected as parking path marking points, the distance between paths EK1V and EK2N, and the path with the longest distance is selected for return.

[0130] Specifically, in this embodiment, when the GPS signal is lost, the position of the drone is updated at adjacent sampling intervals. However, due to the matching of the sampled images, when the position update increases at this time, the difference between the inertial navigation and the drone position distance calculation is large. At this time, even if the sampling Kalman filter regresses the position, an accurate result cannot be obtained. In order to solve this technical problem, a solution is provided in this embodiment, which is as follows:

[0131] When calculating the position of the drone and deciding whether to update it, it is necessary to use the image collected at the current moment and the reference image collected at the same position point in the historical trajectory for contour matching. When the contour match is greater than the preset threshold, the position is updated.

[0132] The specific contour matching method is as follows:

[0133] Perform grayscale changes on the reference image, then perform edge detection and crop according to the budget algorithm and store it;

[0134] Edge detection can use the Sobel algorithm or the Canny algorithm;

[0135] Specifically, this process is completed by the drone mother vehicle during the flight of the drone, so that it can be used when contour matching is required after the drone returns. Since this work is actually completed on the drone mother vehicle, it can process a large amount of image data.

[0136] Change the grayscale of the current image of the drone, then perform median filtering and then edge detection;

[0137] Get the flight altitude of the drone and the resolution of the reference image corresponding to the current point, then scale the image and align it with the reference image, and perform scaling to convert the current image to the resolution of the reference image;

[0138] Specifically, in this implementation, the reference image and the current image use different algorithms, and the reference image does not use median filtering. The reason is that the environment will change with time and weather when the drone returns. Due to changes in the environment, the imaging features in the reference image may disappear after a period of time, such as rivers or certain features destroyed due to construction or human activities. If the current image is not median filtered, some smaller detail features will be retained in the current image. These features often differ greatly from the original image after the environment changes. Therefore, the median filter is to remove small details that are visible from the drone camera but not from the reference image.

[0139] The UAV position predicted by the Kalman filter is used as the center of a restricted search area in the reference image, with the goal of ignoring image regions that are too far from the estimated UAV position;

[0140] According to the contour area obtained by edge detection, fixed contour windows of preset different sizes are selected for matching in the reference image, and the position of the best matching area is obtained from the reference image;

[0141] The position with the largest number of overlapping pixels between the edge of the current image and the corresponding reference image is taken as the matching result;

[0142] Specifically, once a match is obtained, the images captured by the drone can be georeferenced and the absolute position of the drone can be calculated;

[0143] However, how to obtain the best matching position becomes a key technical problem that needs to be solved in this embodiment. It is necessary to detect whether the match is an outlier and then reject it or use it to update the drone position. But outlier detection is not an easy task and is affected by many factors. In some areas, outliers dominate. This embodiment provides a matching probability value, which specifically includes:

[0144] S1: Segment the reference image and assign different contour confidences to different regions;

[0145] S2: Principles for setting contour confidence: image registration in urban areas is greater than that in rural areas, the registration ratio of road intersections is greater than that of other road sections; the registration of feature markers is greater than that of non-feature markers.

[0146] S3: Compare the position where the contour matching occurs with the corresponding contour confidence, and update when it is greater than the corresponding contour confidence.

[0147] The above is based on processing the reference image and modeling the image features to build feature matching confidence. Although good matching results can be obtained, a large amount of calculation is required.

[0148] In order to solve the problem of the best match, the applicant conducted a large number of experiments and found that in the area where the match is unreliable, the matching position noise is large; on the contrary, in the area where the match is reliable, the position noise is low. Therefore, the applicant believes that when the algorithm matches the correct position, the noise of the matching position is usually quite constant and the noise level is very low. Therefore, this embodiment provides a method for determining the matching position update, which is as follows:

[0149] Selecting a preset number of contours in the current image and matching the selected contours in the reference image; setting a first contour matching threshold to determine whether the contour in the current image matches the contour in the reference image;

[0150] Setting a second contour matching threshold value for determining the ratio of the number of pre-matched contours to the number of matches of the preset contours;

[0151] If the matching result is greater than the first contour matching threshold and greater than the second contour matching threshold, the first n contours with better matching results are selected and the median of the first n contour matching results is obtained, and the contour corresponding to the median is selected as the final drone position update.

[0152] The above is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiment. It is clear to those skilled in the art that the form in the embodiment is not limited thereto, and the adjustable manner is not limited thereto. It is understood that other improvements and changes directly derived or associated with those skilled in the art without departing from the basic concept of the present invention should be considered to be included in the protection scope of the present invention.

Claims

1. A drone intelligent return method based on a drone mother vehicle, characterized in that: include: The UAV establishes a communication connection with the UAV mother vehicle and communicates in real time; when the UAV triggers a return command during flight, the UAV executes the return command and sends the return message to the UAV mother vehicle; After receiving the return command, the drone mother vehicle obtains the flight trajectory of the drone from the historical records, calculates the return path of the drone and sends it to the drone. The drone predicts its current position in real time and adjusts the flight path to match the return path. The drone mother vehicle adjusts its own speed and drives to one of the parking path mark points on the return path. When waiting for the drone to return to the parking path mark point, the drone is guided to land using the parking field mark set by the drone mother vehicle. The drone collects images and records the GPS information of the sampling points at preset periodic intervals, and sends the images and GPS information to the drone mother vehicle; If the return path triggering condition is the failure of the GPS signal, after the drone samples and obtains the GPS signal at time T0, the GPS signal is lost at the next sampling time T, then the drone sends the image collected at time T and the return route history to the drone mother vehicle; The UAV mother vehicle selects a mark from the return route as the parking path mark point for the UAV mother vehicle to wait for the UAV to land, based on the return historical route and the distance between the GPS position coordinates and the UAV at time T0; The UAV uses inertial navigation and visual recognition methods and Kalman filtering to predict the flight position of the UAV. The UAV follows the return route to the parking path mark point based on the predicted position.

2. A drone intelligent return method based on a drone mother vehicle as claimed in claim 1, characterized in that: When the distance between the drone and the drone mother vehicle is less than a preset first threshold, the drone mother vehicle identifies the parking lot mark on the collected image, and the parking lot mark is set on the roof of the drone mother vehicle; If the parking lot sign is identified, the position information obtained by the drone camera is used to track the parking lot sign of the drone mother vehicle; at the same time, PID is used to control the drone to land at the drone mother vehicle parking lot.

3. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 1, characterized in that: When the drone mother vehicle receives the return message after the drone GPS signal disappears, the drone mother vehicle obtains the drone's most recent flight path; Match the most recent flight path of the UAV with the historical paths, obtain all historical paths that overlap with the most recent flight path, and form a path set RoadA with the historical paths that overlap with the most recent flight path; The most recent flight path is recombined with the path set RoadA to form a new return path set RoadB. A path with the shortest distance for the drone to travel is selected from RoadB as the return path and the parking path mark points are marked. The drone mother vehicle sends the re-planned return path to the drone.

4. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 1, characterized in that: When the UAV receives an instruction to perform a flight mission, the UAV checks whether a communication connection has been established with the UAV mother vehicle. If a communication connection has been established, the UAV takes off to perform the flight mission; If the communication connection has not been established, the UAV initiates a request to establish a communication connection with the UAV mother vehicle. After the communication connection is established, the UAV takes off to perform the flight mission; The drone mother vehicle processes the image and GPS information and sends the key information to the drone.

5. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 1, characterized in that: During the flight, the drone records the flight trajectory and collects environmental information in real time and sends the information to the drone mother vehicle; the drone mother vehicle processes and stores the information sent by the drone; the environmental information includes environmental images and environmental location information; The current environment image and environment location information are collected in real time during the preset sampling period. The location information includes longitude, latitude, altitude, and the roll angle, pitch angle, and yaw angle of the drone.

6. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 1, characterized in that: The conditions for triggering the intelligent return mode include: GPS signal failure, low battery, and the user selecting the drone mother vehicle to start the intelligent return command. It can be a conditional trigger or it can be triggered according to user needs. The drone flies far away and the user cannot see the drone's behavior. At this time, for safety reasons, the user starts the return command.

7. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 1, characterized in that: When the intelligent return mode is turned on, determine whether the GPS signal is invalid. If the GPS signal is normal, obtain the current drone's GPS signal and battery information and send them to the drone mother vehicle; After receiving the GPS signal of the drone, the drone mother vehicle calculates the distance between the drone mother vehicle and the drone.

8. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 7, characterized in that: If the battery power of the drone is sufficient, the positions of the drone and the drone mother vehicle are preset through Kalman filtering, and the drone is controlled by PD to approach the drone mother vehicle until the distance between the drone mother vehicle and the drone meets the first preset distance threshold; If the battery of the drone is low and the GPS signal is normal, the drone will remain hovering at the position providing the GPS signal. The drone mother vehicle will calculate the current distance to the drone based on the GPS signal provided by the drone. The drone mother vehicle will drive to the position where the GPS signal matches based on the GPS signal provided by the drone.

9. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 1, characterized in that: When both the UAV and the UAV mother vehicle arrive at the parking path point, the UAV collects surrounding images, obtains the parking mark of the UAV mother vehicle from the image, and stops.

10. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 9, characterized in that: The position prediction method of the UAV at least includes: an image prediction method, and the image prediction method at least includes: obtaining an image imageT at time T and an image imageT0 at time T0; Select preset M feature points from image imageT0 as matching feature points arrayT0; Search for matching preset M feature points arrayT in image imageT; Using the projection change, the difference in distance between the corresponding feature points in imageT and imageT0 will be calculated respectively.

11. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 10, characterized in that: Establish the UAV navigation coordinate system and camera coordinate system, then the world coordinates of the feature point are Feature: Among them, R is the change matrix between the camera coordinate system and the drone navigation coordinate system, H is the height of the drone, ax,ay are the pixel coordinates of the feature points, fx,fy are the focal lengths of the camera on the x-axis and y-axis; Then the difference between a pair of matching feature points FeatureTi and FeatureT0i in different images when sampling at time T and sampling at time T0 is ΔSi: ΔSi=FeatureTi-FeatureT0i Where axi, ayi, fx, fy are the pixel coordinates of feature point i in the image collected at time T and the focal length of the camera, Ra and Ha are the transformation matrix of the camera coordinates and navigation coordinates at time T and the height of the drone; Where bxi, byi, fx, fy are the pixel coordinates of feature point i in the image collected at time T0 and the focal length of the camera, Rb and Hb are the transformation matrix of the camera coordinates and navigation coordinates at time T0 and the height of the drone; Then, the average position between a preset number of feature point pairs is obtained as the displacement of the UAV from time T0 to time T; M is the number of feature points, and the position of the drone at time T is ST: ST=ST0+ΔS Among them, ST is the position at time T, and ST0 is the position at time T0.

12. The method for intelligent return of a drone based on a drone mother vehicle as claimed in claim 9, characterized in that: When determining whether to update the position based on the calculated position of the drone, it is necessary to use the image collected at the current moment and the reference image collected at the same position point in the historical trajectory for contour matching. When the contour match is greater than the preset threshold, the position is updated.

Images