Control method of vehicle-mounted unmanned aerial vehicle, electronic equipment and storage medium

Abstract

The invention discloses a control method of a vehicle-mounted unmanned aerial vehicle, electronic equipment and a storage medium, and belongs to the technical field of vehicle-mounted unmanned aerial vehicles. The method comprises the steps of determining a real-time horizontal distance between an unmanned aerial vehicle and a vehicle based on multi-source sensing data in response to an unmanned aerial vehicle cabin returning instruction; the multi-source sensing data comprises at least two of relative position data of the unmanned aerial vehicle and the vehicle, airspace environment data and landing area environment data; determining a target cabin returning stage of the unmanned aerial vehicle based on the real-time horizontal distance; and controlling the unmanned aerial vehicle to return to the cabin based on a target cabin returning strategy corresponding to the target cabin returning stage. According to the control method of the vehicle-mounted unmanned aerial vehicle, the technical problem that the vehicle-mounted unmanned aerial vehicle is low in cabin returning success rate is solved.

Description

Technical Field

[0001] This application belongs to the field of vehicle-mounted unmanned aerial vehicle (UAV) technology, and particularly relates to a control method, electronic device and storage medium for a vehicle-mounted UAV. Background Technology

[0002] Traditional vehicle-mounted drone systems primarily control the drone's autonomous takeoff and landing via the vehicle, and perform tasks such as autonomous follow-up flight and aerial photography according to user needs. The drone will land when it receives a return-to-home command from the user or when the battery is low.

[0003] However, because only the relative distance is considered during the drone landing and return process, the success rate of drone return is low. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a control method, electronic equipment, and storage medium for a vehicle-mounted unmanned aerial vehicle (UAV) to solve the technical problem of low return-to-capsule success rate.

[0005] Firstly, this application provides a control method for a vehicle-mounted unmanned aerial vehicle, including: In response to the drone's return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data; the multi-source sensor data includes at least two of the following: relative position data between the drone and the vehicle, airspace environmental data, and landing area environmental data. Determine the target return phase of the UAV based on real-time horizontal distance; The drone's return to its capsule is controlled based on the target return strategy corresponding to the target return phase.

[0006] According to the vehicle-mounted drone control method of this application, in response to the drone's return-to-drone command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as the relative position data between the drone and the vehicle, airspace environmental data, and landing area environmental data; the target return-to-drone stage of the drone is determined based on the real-time horizontal distance; and the drone's return-to-drone is controlled based on the target return-to-drone strategy corresponding to the target return-to-drone stage. By controlling the drone's return-to-drone based on multi-source sensor data such as relative position data, airspace environmental data, and landing area environmental data, the success rate of drone return-to-drone is improved.

[0007] According to one embodiment of this application, determining the target return phase of a UAV based on real-time horizontal distance includes: When the real-time horizontal distance is greater than the distance threshold, the target return phase is determined as the first phase; the first phase is the long-range active distance reduction phase of the UAV. or When the real-time horizontal distance is less than or equal to the distance threshold, the target return phase is defined as the second phase; the second phase is the close-range attitude and position adjustment phase of the UAV.

[0008] According to one embodiment of this application, the relative position data includes real-time altitude difference; controlling the UAV's return to its capsule based on the target return strategy corresponding to the target return phase includes: When the target return phase is in the first stage and the real-time altitude difference is less than the first altitude threshold, control the drone to increase the real-time altitude difference; or When the target return phase is in the first stage and the real-time altitude difference is greater than or equal to the first altitude threshold, control the drone to reduce the real-time horizontal distance at the first speed.

[0009] According to one embodiment of this application, controlling the UAV's return to its capsule based on the target return strategy corresponding to the target return phase further includes: When the target return phase is in the second stage and the real-time altitude difference is less than the second altitude threshold, control the drone to increase the real-time altitude difference; Among them, the first altitude threshold is greater than the second altitude threshold.

[0010] According to one embodiment of this application, after determining that the target return phase is the second phase, the method further includes: When the real-time horizontal distance is less than or equal to the distance threshold, the landing area environmental data is identified; With the landing area environmental data including vehicle images confirmed, the target return phase is defined as the third phase; the third phase is the preparation and execution phase for precise drone landing.

[0011] According to one embodiment of this application, when the landing area environmental data does not include images of the vehicle, the method further includes: Based on airspace environmental data, if it is determined that there is open airspace above the vehicle, the drone is controlled to move at a second speed.

[0012] According to one embodiment of this application, when it is determined that there is no open airspace above the vehicle, the method further includes: Once it is confirmed that there is no open airspace above the vehicle, control the vehicle to move and control the drone to follow the vehicle.

[0013] According to one embodiment of this application, controlling the UAV's return to its capsule based on the target return strategy corresponding to the target return phase further includes: When the target return phase is the third stage, the landing conditions of the UAV are determined based on airspace environmental data and landing area environmental data. If the landing conditions are met, control the drone to land on the vehicle at a third speed.

[0014] According to one embodiment of this application, the airspace environment data includes real-time wind speed; the UAV meets the landing conditions as follows: the landing area environment data contains an image of a land marker, and the real-time wind speed is less than a wind speed threshold. The drone does not meet the landing conditions if: the environmental data of the landing area does not contain an image of a landing marker, or the real-time wind speed is greater than or equal to the wind speed threshold.

[0015] According to one embodiment of this application, when it is determined that the drone does not meet the landing conditions, the method further includes: Control the drone to keep it hovering; A warning message is generated when the drone's hovering duration is greater than or equal to a time threshold.

[0016] According to one embodiment of this application, the method further includes: When the target return phase is in the first or second phase, obstacle detection is performed based on the landing area environmental data, and if the detection results indicate the presence of obstacles, the drone is controlled to perform obstacle avoidance operations.

[0017] Secondly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the vehicle-mounted drone control method described in the first aspect.

[0018] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the vehicle-mounted unmanned aerial vehicle control method described in the first aspect.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts illustrating the control method for a vehicle-mounted unmanned aerial vehicle provided in this application embodiment; Figure 2 This is one of the flowcharts illustrating the vehicle-mounted drone return process provided in this application embodiment; Figure 3 This is the second flowchart illustrating the vehicle-mounted drone return process provided in the embodiments of this application; Figure 4This is the third flowchart illustrating the vehicle-mounted drone return process provided in this application embodiment; Figure 5 This is a second schematic flowchart of the control method for a vehicle-mounted unmanned aerial vehicle provided in this application embodiment; Figure 6 This is a schematic diagram of the control system of the vehicle-mounted drone provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0022] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0023] The control method, electronic equipment, and storage medium of the vehicle-mounted drone provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0024] In some embodiments, the control method for vehicle-mounted drones provided in this application can be executed by electronic devices.

[0025] The control method for vehicle-mounted drones can be applied to electronic devices, which can be deployed on terminals. The control method for vehicle-mounted drones can be specifically executed by the hardware or software in the terminal.

[0026] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0027] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0028] The vehicle-mounted drone control method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can realize the vehicle-mounted drone return method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The following uses an electronic device as the execution subject to illustrate the vehicle-mounted drone control method provided in this application embodiment.

[0029] This application provides a control method for a vehicle-mounted unmanned aerial vehicle.

[0030] like Figure 1 As shown, the control method for the vehicle-mounted drone includes steps 110, 120, and 130.

[0031] Step 110: In response to the UAV return command, determine the real-time horizontal distance between the UAV and the vehicle based on multi-source sensor data; the multi-source sensor data includes at least two of the following: relative position data between the UAV and the vehicle, airspace environmental data, and landing area environmental data.

[0032] In actual operation, the vehicle can be a vehicle that includes a landing bay and a landing pad, with the landing pad used for drone take-off and landing. The landing pad may include landing markers, which are used to indicate the landing position, landing angle, or any theoretically feasible information.

[0033] In some embodiments, the relative position data may include a first real-time coordinate, a second real-time coordinate, and a real-time relative distance. The first real-time coordinate is the real-time horizontal position coordinate of the drone. The second real-time coordinate is the real-time horizontal position coordinate of the vehicle. The real-time relative distance is the distance between the vehicle and the drone.

[0034] In some embodiments, the vehicle may include a first positioning module, which can be used to acquire first real-time coordinates. The first positioning module may integrate a high-precision Global Positioning System (GPS) and an Inertial Navigation System (INS), with positioning accuracy down to the meter level.

[0035] In some embodiments, the drone may include a second positioning module, which can be used to acquire second real-time coordinates.

[0036] In some embodiments, the distance difference between the first real-time coordinates and the second real-time coordinates can be obtained as the real-time horizontal distance.

[0037] In some embodiments, airspace environmental data may include airspace environmental images and real-time wind speed.

[0038] In some embodiments, the vehicle may include an onboard camera that can be used to acquire images of the airspace environment. The onboard camera may be mounted on the roof of the vehicle and may be a camera with a wide-angle or panoramic field of view.

[0039] In some embodiments, the drone may include an onboard camera that can be used to acquire environmental data of the landing area. The onboard camera may include several cameras, each of which can be connected to the drone via a gimbal to adjust its shooting angle. The cameras may be distributed around the drone's fuselage.

[0040] Step 120: Determine the target return phase of the UAV based on real-time horizontal distance.

[0041] In some embodiments, the target return phase of the UAV can be determined based on the magnitude of the real-time horizontal distance.

[0042] Step 130: Control the UAV to return to its capsule based on the target return strategy corresponding to the target return phase.

[0043] In some embodiments, the UAV can be controlled to adjust its flight altitude and shorten the distance between itself and the vehicle based on the target return strategy corresponding to the target return phase, until the UAV lands.

[0044] In some embodiments, different target reentry phases correspond to different cooperative positioning modes with different precision and different flight altitude thresholds.

[0045] In some embodiments, obstacle avoidance detection can be performed based on multi-source sensor data to control the drone to avoid obstacles.

[0046] According to the vehicle-mounted drone control method of this application embodiment, in response to the drone return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as the relative position data between the drone and the vehicle, airspace environment data, and landing area environment data; the target return stage of the drone is determined based on the real-time horizontal distance; and the drone return is controlled based on the target return strategy corresponding to the target return stage. Different target return stages correspond to different precision cooperative positioning modes and different flight altitude thresholds, thereby controlling the drone return based on multi-source sensor data such as relative position data, airspace environment data, and landing area environment data, thus improving the drone return success rate.

[0047] In some embodiments, when the real-time horizontal distance is greater than a distance threshold, the target return phase is determined as the first phase; the first phase is the UAV long-range active distance reduction phase; or when the real-time horizontal distance is less than or equal to the distance threshold, the target return phase is determined as the second phase; the second phase is the UAV close-range attitude and position adjustment phase.

[0048] In actual implementation, the distance threshold can be a pre-set value. For example, the distance threshold can be a value set based on the drone version or the field of view of the vehicle-mounted camera.

[0049] In some embodiments, such as Figure 2 As shown, the target return phase can be determined as the first phase when the real-time horizontal distance between vehicle 210 and drone 222 is greater than the distance threshold; and the target return phase can be determined as the second phase when the real-time horizontal distance between vehicle 210 and drone 221 is less than or equal to the distance threshold.

[0050] In some embodiments, the target reentry stage can be determined as the first stage if the real-time horizontal distance is greater than a distance threshold. For example, if the distance threshold is 50 meters, the target reentry stage can be determined as the first stage if the real-time horizontal distance is 10 meters. If the real-time horizontal distance is less than or equal to the distance threshold, the target reentry stage can be determined as the second stage. For example, if the distance threshold is 50 meters, the target reentry stage can be determined as the second stage if the real-time horizontal distance is 60 meters.

[0051] In some embodiments, after determining the target reentry phase as the second phase, the target reentry phase can be further determined based on landing area environmental data.

[0052] According to the vehicle-mounted drone control method of this application embodiment, in response to the drone return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as relative position data between the drone and the vehicle, airspace environment data, and landing area environment data; the target return stage of the drone is determined based on the real-time horizontal distance; and the drone return is controlled based on the target return strategy corresponding to the target return stage. By controlling the drone return based on multi-source sensor data such as relative position data, airspace environment data, and landing area environment data, the success rate of drone return is improved.

[0053] In some embodiments, the relative position data includes a real-time altitude difference; when the target return phase is in the first phase and the real-time altitude difference is less than a first altitude threshold, the drone is controlled to increase the real-time altitude difference; or when the target return phase is in the first phase and the real-time altitude difference is greater than or equal to the first altitude threshold, the drone is controlled to decrease the real-time horizontal distance at a first speed.

[0054] In some embodiments, when the target return phase is the first phase and the real-time altitude difference is less than the first altitude threshold, the drone is controlled to increase its flight altitude based on the first return strategy to increase the real-time altitude difference until the real-time altitude difference is greater than or equal to the first altitude threshold. In some embodiments, when the target return phase is the first phase and the real-time altitude difference is greater than or equal to a first altitude threshold, the UAV is controlled to fly at a first speed based on a first return strategy to reduce the real-time horizontal distance until the real-time horizontal distance is less than or equal to a distance threshold.

[0055] In actual operation, the drone may include an altitude sensor, which is used to obtain the real-time altitude difference between the drone and the vehicle. The real-time altitude difference represents the real-time vertical distance between the drone and the vehicle.

[0056] In actual execution, the first height threshold can be a preset value.

[0057] In some embodiments, the real-time height difference can be compared with a first height threshold, such as... Figure 3 As shown, when the target return phase is in the first stage and the real-time altitude difference between vehicle 310 and drone 321 is less than the first altitude threshold, drone 321 can be controlled to climb until the real-time altitude difference is greater than or equal to the first altitude threshold; when the target return phase is in the first stage and the real-time altitude difference between vehicle 310 and drone 322 is greater than or equal to the first altitude threshold, drone 322 is controlled to fly at the first speed until the real-time horizontal distance is less than or equal to the distance threshold.

[0058] In actual execution, the first speed can be a pre-set value. For example, the first speed can be a value set based on the drone version.

[0059] In actual implementation, the first return-to-capacity strategy can be the target return-to-capacity strategy corresponding to the first stage.

[0060] In some embodiments, when the target return phase is in the first stage and the real-time altitude difference is less than a first altitude threshold, the drone can be controlled to increase its flight altitude based on a first return strategy until the real-time altitude difference is greater than or equal to the first altitude threshold. For example, if the first altitude threshold is 30 meters, the drone can be controlled to increase its flight altitude when the target return phase is in the first stage and the real-time altitude difference is 25 meters until the real-time altitude difference is greater than or equal to the first altitude threshold.

[0061] In some embodiments, when the target return phase is in the first stage and the real-time altitude difference is greater than or equal to a first altitude threshold, the drone can be controlled to fly at a first speed to shorten the real-time horizontal distance between itself and the vehicle until the real-time horizontal distance is less than or equal to the distance threshold. For example, if the first altitude threshold is 30 meters, when the target return phase is in the first stage and the real-time altitude difference is 60 meters, the drone can be controlled to fly at a first speed to shorten the real-time horizontal distance between itself and the vehicle until the real-time horizontal distance is less than or equal to the distance threshold.

[0062] In some embodiments, when the target return phase is determined to be the first phase, coarse positioning and distance reduction are achieved based on multi-source sensor data. It is only necessary to control the UAV to fly at the first speed to shorten the real-time horizontal distance, without the need to accurately identify vehicle details, which corresponds to a low-precision cooperative positioning mode.

[0063] In some embodiments, when the target return phase is determined to be the first phase, the distance between the drone and the vehicle is relatively far. The first real-time coordinates can be obtained based on the first positioning module in the vehicle, and the second real-time coordinates can be obtained based on the second positioning module in the drone. In this way, a real-time horizontal distance representing the coarse-precision relative position between the vehicle and the drone can be obtained in the first phase, and then the drone can be controlled based on this coarse-precision relative position.

[0064] According to the vehicle-mounted drone control method of this application embodiment, in response to the drone return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as relative position data between the drone and the vehicle, airspace environment data, and landing area environment data; the target return stage of the drone is determined based on the real-time horizontal distance; if the target return stage is the first stage and the real-time altitude difference is less than a first altitude threshold, the drone is controlled to climb until the real-time altitude difference is greater than or equal to the first altitude threshold based on a first return strategy; or if the target return stage is the first stage and the real-time altitude difference is greater than or equal to the first altitude threshold, the drone is controlled to fly at a first speed until the real-time horizontal distance is less than or equal to a distance threshold based on the first return strategy; by controlling the drone return based on multi-source sensor data such as relative position data, airspace environment data, and landing area environment data, the success rate of drone return is improved.

[0065] In some embodiments, when the target return phase is the second phase and the real-time altitude difference is less than the second altitude threshold, the drone is controlled to increase the real-time altitude difference; wherein the first altitude threshold is greater than the second altitude threshold.

[0066] In some embodiments, when the target return phase is the second phase, it can be determined whether the real-time altitude difference is greater than or equal to the second altitude threshold; if the real-time altitude difference is less than the second altitude threshold, the drone is controlled to climb until the real-time altitude difference is greater than or equal to the second altitude threshold based on the second return strategy.

[0067] In actual implementation, the second return-to-capsule strategy is one of the target return-to-capsule strategies corresponding to the second phase.

[0068] In some embodiments, such as Figure 4 As shown, when the real-time altitude difference between vehicle 410 and drone 421 is less than the second altitude threshold, the drone is controlled to increase the real-time altitude difference based on the second return-to-capsule strategy. For example, the drone's flight altitude is increased until the real-time altitude difference is greater than or equal to the second altitude threshold.

[0069] In some embodiments, if the real-time altitude difference is less than a second altitude threshold, the drone can be controlled to climb until the real-time altitude difference is greater than or equal to the second altitude threshold based on a second return-to-base strategy. For example, if the second altitude threshold is 8 meters, the drone can be controlled to climb until the real-time altitude difference is greater than or equal to the second altitude threshold based on a second return-to-base strategy if the real-time altitude difference is 5 meters.

[0070] According to the vehicle-mounted drone control method of this application embodiment, in response to the drone return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as relative position data between the drone and the vehicle, airspace environment data, and landing area environment data; the target return stage of the drone is determined based on the real-time horizontal distance; if the target return stage is the second stage, it is determined whether the real-time altitude difference is greater than or equal to a second altitude threshold; if the real-time altitude difference is less than the second altitude threshold, the drone is controlled to climb until the real-time altitude difference is greater than or equal to the second altitude threshold based on a second return strategy. This method controls the drone's return based on multi-source sensor data such as relative position data, airspace environment data, and landing area environment data, thereby improving the drone's return success rate.

[0071] In some embodiments, after determining the target return phase as the second phase, if the real-time horizontal distance is less than or equal to a distance threshold, the landing area environmental data is identified; if it is determined that the landing area environmental data includes images of the vehicle, the target return phase is determined as the third phase; the third phase is the UAV precision landing preparation and execution phase.

[0072] In some embodiments, after determining the target re-entry stage as the second stage when the real-time horizontal distance is less than or equal to the distance threshold, the landing area environmental data is identified based on the target recognition algorithm when the real-time horizontal distance is less than or equal to the distance threshold; and when the identification result indicates that the landing area environmental data includes images of the vehicle, the target re-entry stage is determined as the third stage.

[0073] In some embodiments, when the real-time horizontal distance is less than or equal to a distance threshold and the real-time height difference is greater than or equal to a second height threshold, the landing area environmental data can be identified based on a target recognition algorithm. For example... Figure 4 As shown, when the real-time altitude difference between vehicle 410 and drone 422 is greater than or equal to the second altitude threshold, the landing area environmental data is identified based on the target recognition algorithm.

[0074] In some embodiments, when the real-time horizontal distance is less than or equal to a distance threshold and the real-time height difference is greater than or equal to a second height threshold, the gimbal adjustment of the airborne camera is controlled so that the airborne camera is adjusted to look downward and acquire environmental data of the landing area.

[0075] In actual operation, landing area environmental data can include images of the space below the drone within the field of view of the airborne camera.

[0076] In actual implementation, the target recognition algorithm can be R-CNN, YOLO, SSD or any theoretically feasible algorithm, and this application does not impose any specific restrictions on it.

[0077] In some embodiments, target identification can be performed on landing area environmental data based on a target recognition algorithm, and the target reentry stage can be determined based on the identification results. If the identification results indicate that the landing area environmental data includes images of the vehicle, the target reentry stage is determined to be the third stage.

[0078] According to the vehicle-mounted drone control method of this application embodiment, in response to the drone return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as the relative position data between the drone and the vehicle, airspace environment data, and landing area environment data; the target return phase of the drone is determined based on the real-time horizontal distance; and the drone return is controlled based on the target return strategy corresponding to the target return phase. The target return strategy is used to control the drone based on a cooperative positioning mode and / or flight altitude threshold corresponding to the accuracy of the target return phase, thereby controlling the drone return based on multi-source sensor data such as relative position data, airspace environment data, and landing area environment data, and improving the drone return success rate.

[0079] In some embodiments, if the landing area environmental data does not include an image of the vehicle, and if it is determined based on the airspace environmental data that there is open airspace above the vehicle, the drone is controlled to move at a second speed.

[0080] In some embodiments, when the real-time horizontal distance is less than or equal to a distance threshold, after identifying the landing area environmental data based on the target recognition algorithm, if the identification result indicates that the landing area environmental data does not include the image of the vehicle, it is determined whether there is open airspace above the vehicle based on the airspace environmental data; if there is open airspace above the vehicle, the UAV is controlled to move at a second speed based on the third return-to-cargo strategy until the landing area environmental data includes the image of the vehicle.

[0081] In some embodiments, the airspace environment data may include airspace environment images, which may be acquired by an onboard camera. The onboard camera may be deployed on the roof of the vehicle. The airspace environment images may include images of the space above the vehicle within the field of view of the onboard camera.

[0082] In some embodiments, when the identification result indicates that the landing area environmental data does not include images of the vehicle, the airspace environment image in the airspace environment data is identified, and when the airspace environment image does not include obstacles (such as mountains, forests, animals, etc.), it is determined that there is open airspace above the vehicle.

[0083] In actual implementation, the third return-to-capsule strategy is one of the target return-to-capsule strategies corresponding to the second phase.

[0084] In actual operation, the second speed can be a pre-set value. For example, the second speed could be a value set based on the drone version. The second speed is lower than the first speed.

[0085] In some embodiments, when there is open airspace above the vehicle, the drone is controlled to move at a second speed based on a third return-to-cargo strategy, while continuously acquiring and identifying environmental data of the landing area. If the identification result indicates that the environmental data of the landing area includes an image of the vehicle, the drone is controlled to stop moving.

[0086] In some embodiments, in the second stage, the UAV position is adjusted based on airspace environmental data and / or landing area environmental data until the vehicle image is captured. This requires higher position control accuracy and corresponds to a medium-precision cooperative positioning mode.

[0087] According to the vehicle-mounted drone control method of this application embodiment, in response to the drone return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as the relative position data between the drone and the vehicle, airspace environment data, and landing area environment data; the target return phase of the drone is determined based on the real-time horizontal distance; and the drone return is controlled based on the target return strategy corresponding to the target return phase. The target return strategy is used to control the drone based on a cooperative positioning mode and / or flight altitude threshold corresponding to the accuracy of the target return phase, thereby controlling the drone return based on multi-source sensor data such as relative position data, airspace environment data, and landing area environment data, and improving the drone return success rate.

[0088] In some embodiments, if it is determined that there is no open airspace above the vehicle, the vehicle is controlled to move, and the drone is controlled to follow the vehicle.

[0089] In some embodiments, after determining whether there is open airspace above the vehicle based on airspace environment data, if there is no open airspace above the vehicle, the vehicle is controlled to move until there is open airspace above the vehicle based on the fourth return-to-carrier strategy, and the drone is controlled to follow the vehicle.

[0090] In actual implementation, the fourth return-to-capsule strategy is one of the target return-to-capsule strategies corresponding to the second phase.

[0091] In some embodiments, if it is determined, based on airspace environment data, that there is no open airspace above the vehicle, the vehicle is controlled to move based on the fourth return-to-cargo strategy, and airspace environment data is continuously acquired. Based on the airspace environment data, it is determined whether there is open airspace above the vehicle. If it is determined that there is open airspace above the vehicle, the vehicle is controlled to stop moving.

[0092] In some embodiments, relative position data may include real-time relative distance. The real-time relative distance is acquired by a vehicle-mounted beacon deployed on the vehicle. If there is no open airspace above the vehicle, based on a fourth return-to-drone strategy, the vehicle is controlled to move until open airspace exists above it, and the drone is controlled to follow the vehicle based on the real-time relative distance.

[0093] In actual operation, the real-time relative distance is the real-time spatial distance between the vehicle and the drone.

[0094] According to the vehicle-mounted drone control method of this application embodiment, in response to the drone return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as the relative position data between the drone and the vehicle, airspace environment data, and landing area environment data; the target return phase of the drone is determined based on the real-time horizontal distance; and the drone return is controlled based on the target return strategy corresponding to the target return phase. The target return strategy is used to control the drone based on a cooperative positioning mode and / or flight altitude threshold corresponding to the accuracy of the target return phase, thereby controlling the drone return based on multi-source sensor data such as relative position data, airspace environment data, and landing area environment data, and improving the drone return success rate.

[0095] In some embodiments, when the target return phase is the third phase, the landing conditions of the UAV are determined based on airspace environmental data and landing area environmental data; if the landing conditions are met, the UAV is controlled to land on the vehicle at a third speed.

[0096] In some embodiments, when the target return phase is the third phase, it is determined whether the UAV meets the landing conditions based on airspace environmental data and landing area environmental data; if the UAV meets the landing conditions, it is controlled to land on the vehicle at a third speed based on the fifth return strategy.

[0097] In actual operation, landing conditions refer to whether the vehicle and the surrounding environment are suitable for the drone to land.

[0098] In some embodiments, environmental data of the landing area can be identified, and real-time wind speed can be compared with wind speed thresholds. Based on the identification results and comparison results, it can be determined whether the landing conditions are met.

[0099] In some embodiments, if landing conditions are met, the drone can be controlled to adjust its attitude and position based on real-time relative distance and a third velocity to land on the vehicle's helipad.

[0100] In actual operation, the third speed can be a pre-set value. For example, the third speed can be a value set based on the drone version. The third speed is lower than the second speed.

[0101] In actual implementation, the fifth return-to-capsule strategy is one of the target return-to-capsule strategies corresponding to the third phase.

[0102] In some embodiments, if landing conditions are met, the return-to-cargo method is determined based on the vehicle's state, and based on the return-to-cargo method, the drone is controlled to land on the vehicle at a third speed. For example, if landing conditions are met and the vehicle is in a dynamic state, the drone is controlled to return dynamically; if landing conditions are met and the vehicle is in a static state, the drone is controlled to return statically.

[0103] In some embodiments, in the third stage, the landing is accurately located based on real-time wind speed and landing markers. The landing conditions are determined by matching the landing markers and real-time wind speed. This stage has the highest requirements for the control accuracy of the UAV's position and attitude, corresponding to the high-precision collaborative positioning mode.

[0104] According to the vehicle-mounted drone control method of this application embodiment, in response to the drone return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as the relative position data between the drone and the vehicle, airspace environment data, and landing area environment data; the target return stage of the drone is determined based on the real-time horizontal distance; if the target return stage is the third stage, it is determined whether the landing conditions are met based on the airspace environment data and landing area environment data; if the landing conditions are met, the drone is controlled to land on the vehicle at a third speed based on the fifth return strategy. By controlling the drone return based on multi-source sensor data such as relative position data, airspace environment data, and landing area environment data, the success rate of drone return is improved.

[0105] In some embodiments, the airspace environment data includes real-time wind speed; the conditions for a UAV to land are: the landing area environmental data contains an image with a land marker, and the real-time wind speed is less than a wind speed threshold; the conditions for a UAV to not land are: the landing area environmental data does not contain an image with a land marker, or the real-time wind speed is greater than or equal to a wind speed threshold.

[0106] In some embodiments, if the landing area environmental data contains an image with a land marker and the real-time wind speed is less than a wind speed threshold, the landing conditions are determined to be met; or if the landing area environmental data does not contain an image with a land marker, or the real-time wind speed is greater than or equal to a wind speed threshold, the landing conditions are determined to be unmet.

[0107] In practice, landing markers can be located in the center of the vehicle parking apron. Landing markers can also serve as directional markers for the drone to identify and distinguish its location.

[0108] In actual implementation, the wind speed threshold can be a preset value. For example, the wind speed threshold can be set based on properties such as the drone's resistance to airflow or wind.

[0109] In some embodiments, landing area environmental data can be identified based on target recognition algorithms, and the identification results indicate whether there is an image of a landing marker in the landing area environmental data.

[0110] In some embodiments, landing conditions can be determined to be met if an image with a land marker is found in the environmental data of the landing area, and the real-time wind speed is less than a wind speed threshold. For example, the wind speed threshold could be 20 m / s, and landing conditions could be determined to be met if an image with a land marker is found in the environmental data of the landing area, and the real-time wind speed is 5 m / s.

[0111] In some embodiments, landing conditions may be deemed not met if the landing area environmental data does not contain an image with a landing marker, or if the real-time wind speed is greater than or equal to a wind speed threshold. For example, the wind speed threshold could be 20 m / s, and landing conditions may be deemed not met if the landing area environmental data does not contain an image with a landing marker, or if the real-time wind speed is 25 m / s.

[0112] According to the vehicle-mounted drone control method of this application embodiment, in response to the drone return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data such as the relative position data between the drone and the vehicle, airspace environment data, and landing area environment data; the target return phase of the drone is determined based on the real-time horizontal distance; and the drone return is controlled based on the target return strategy corresponding to the target return phase. The target return strategy is used to control the drone based on a cooperative positioning mode and / or flight altitude threshold corresponding to the accuracy of the target return phase, thereby controlling the drone return based on multi-source sensor data such as relative position data, airspace environment data, and landing area environment data, and improving the drone return success rate.

[0113] In some embodiments, if it is determined that the drone does not meet the landing conditions, the drone is controlled to remain hovering; if the hovering duration of the drone is greater than or equal to a time threshold, a warning message is generated.

[0114] In some embodiments, when the target return phase is the third phase, after determining whether the landing conditions are met based on airspace environmental data and landing area environmental data, if the landing conditions are not met, the drone is controlled to remain in a hovering state.

[0115] In some embodiments, if it is determined that there is no image of a landing marker in the environmental data of the landing area, or the real-time wind speed is greater than or equal to the wind speed threshold, i.e., the landing conditions are not met, the drone is controlled to remain in a hovering state.

[0116] In some embodiments, the real-time altitude difference between the drone and the vehicle can be maintained at a third altitude threshold even when landing conditions are not met. For example, the second processor can be used to maintain the real-time altitude difference between the drone and the vehicle at 10 meters even when landing conditions are not met.

[0117] In some embodiments, if landing conditions are not met, the drone can be controlled to remain hovering until landing conditions are met, and the duration of hovering can be recorded.

[0118] In some embodiments, while the drone remains hovering, the hovering duration is obtained, and if the hovering duration is greater than or equal to a time threshold, a warning message is generated.

[0119] In actual operation, hovering time is the duration for which the drone remains hovering.

[0120] In actual execution, the time threshold can be a preset value. For example, the time threshold can be set based on the drone's state of charge, remaining battery power, or other feasible parameters, and this application does not impose specific restrictions on this.

[0121] In some embodiments, a warning message is generated when the drone remains hovering for a duration greater than or equal to a time threshold. This warning message prompts the controller to return the drone to its cabin. For example, if the time threshold is 1 minute, a warning message is generated when the drone remains hovering for 1 minute.

[0122] In some embodiments, the drone is controlled to remain in a hovering state if the hovering duration is less than a time threshold. For example, if the time threshold is 1 minute, the drone is controlled to remain in a hovering state if the hovering duration is 50 seconds.

[0123] In some embodiments, when the target return phase is the first or second phase, obstacle detection is performed based on the landing area environmental data, and if the detection results indicate the presence of an obstacle, the drone is controlled to perform obstacle avoidance operations.

[0124] In some embodiments, obstacle detection is performed based on environmental data of the landing area, and if the detection results indicate the presence of an obstacle, the drone is controlled to perform obstacle avoidance operations.

[0125] In some embodiments, landing area environmental data can be identified based on target recognition algorithms to perform obstacle detection based on the landing area environmental data.

[0126] In some embodiments, when the detection results indicate the presence of an obstacle, the drone is controlled to climb, descend, slow down its flight speed, or perform any theoretically feasible operation to perform obstacle avoidance.

[0127] According to the control method of the vehicle-mounted drone in the embodiments of this application, obstacle detection is performed based on the environmental data of the landing area, and when the detection result indicates the presence of an obstacle, the drone is controlled to perform obstacle avoidance operation to achieve a safer and more accurate return to the cabin and improve the user experience.

[0128] To better understand the control method for vehicle-mounted drones provided in the embodiments of this application, further explanation is provided below. It should be understood that the following discussion is merely exemplary.

[0129] This application provides a control method for a vehicle-mounted unmanned aerial vehicle, the specific steps of which are as follows: Figure 5 As shown: Step 510: In response to the UAV return command, acquire multi-source sensor data such as first real-time coordinates, second real-time coordinates, real-time relative distance, airspace environment image, real-time wind speed, and landing area environment.

[0130] In some embodiments, the vehicle may include a first positioning module, which can be used to acquire first real-time coordinates. The first positioning module may integrate a high-precision Global Positioning System (GPS) and an Inertial Navigation System (INS), with positioning accuracy down to the meter level.

[0131] In some embodiments, the drone may include a second positioning module, which can be used to acquire second real-time coordinates.

[0132] In some embodiments, the vehicle may include an onboard camera that can be used to acquire images of the airspace environment. The onboard camera may be mounted on the roof of the vehicle and may be a camera with a wide-angle or panoramic field of view.

[0133] In some embodiments, the drone may include an onboard camera that can be used to acquire environmental data of the landing area. The onboard camera may include several cameras, each of which can be connected to the drone via a gimbal to adjust its shooting angle. The cameras may be distributed around the drone's fuselage.

[0134] Step 513: Obtain the distance difference between the first real-time coordinate and the second real-time coordinate, as the real-time horizontal distance.

[0135] Step 516: If the real-time horizontal distance is greater than the distance threshold, the target return phase will be determined as the first phase.

[0136] In some embodiments, when the target return phase is the first phase and the real-time altitude difference is less than a first altitude threshold, the drone is controlled to climb based on the first return strategy until the real-time altitude difference is greater than or equal to the first altitude threshold.

[0137] When the target return phase is in the first phase and the real-time altitude difference is greater than or equal to the first altitude threshold, the drone is controlled to fly at the first speed based on the first return strategy until the real-time horizontal distance is less than or equal to the distance threshold.

[0138] Step 519: If the real-time horizontal distance is less than or equal to the distance threshold and the landing area environmental data does not include images of the vehicle, the target return phase is determined as the second phase.

[0139] Step 522: Determine whether the real-time altitude difference is greater than or equal to the second altitude threshold; if the real-time altitude difference is less than the second altitude threshold, control the drone to climb until the real-time altitude difference is greater than or equal to the second altitude threshold based on the second return-to-capsule strategy.

[0140] Step 525: Based on airspace environment data, determine whether there is open airspace above the vehicle.

[0141] Step 528: When there is open airspace above the vehicle, based on the third return-to-cargo strategy, control the drone to move at the second speed until the environmental data of the landing area includes images of the vehicle.

[0142] Step 531: If there is no open airspace above the vehicle, based on the fourth return-to-carrier strategy, control the vehicle to move until there is open airspace above the vehicle, and control the drone to follow the vehicle.

[0143] Step 534: If the real-time horizontal distance is less than or equal to the distance threshold, and the identification result indicates that the landing area environmental data includes images of the vehicle, the target return phase is determined to be the third phase.

[0144] Step 537: Based on airspace environmental data and landing area environmental data, determine whether the landing conditions are met.

[0145] Step 540: If the landing conditions are met, based on the fifth return-to-cargo strategy, control the drone to land on the vehicle at the third speed.

[0146] Step 543: If landing conditions are not met, control the drone to remain hovering.

[0147] Step 546: While the drone remains hovering, obtain the hovering duration, and generate a warning message if the hovering duration is greater than or equal to the time threshold.

[0148] This application provides a control system for a vehicle-mounted unmanned aerial vehicle (UAV).

[0149] like Figure 6 As shown, the control system 600 of the vehicle-mounted drone includes: Vehicle 610 and drone 620; vehicle 610 includes a first positioning module 611, vehicle-mounted camera 612, vehicle-mounted beacon 613 and wind speed sensor 614, and drone 620 includes a second positioning module 621, altitude sensor 622 and airborne camera 623.

[0150] In some embodiments, the vehicle camera may be a camera mounted on the roof of the vehicle.

[0151] The control method for the vehicle-mounted drone in this application embodiment can be executed by an electronic device or by a component within the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, vehicle-mounted electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the scope of the device.

[0152] The electronic device in this application embodiment can be a device with an operating system. The operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems. This application embodiment does not specifically limit the specific operating system.

[0153] In some embodiments, such as Figure 7 As shown, this application embodiment also provides an electronic device 700, including a processor 701, a memory 702, and a computer program stored in the memory 702 and executable on the processor 701. When the program is executed by the processor 701, it implements the various processes of the vehicle-mounted drone control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0154] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0155] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the vehicle-mounted drone control method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0156] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0157] This application also provides a computer program product, including a computer program that, when executed by a processor, implements a control method for a vehicle-mounted drone.

[0158] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0159] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement various processes of the vehicle-mounted drone control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0160] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0161] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0163] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0164] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0165] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A control method for a vehicle-mounted unmanned aerial vehicle, characterized in that, include: In response to the drone's return command, the real-time horizontal distance between the drone and the vehicle is determined based on multi-source sensor data; The multi-source sensor data includes at least two of the following: relative position data between the UAV and the vehicle, airspace environment data, and landing area environment data. The target return phase of the UAV is determined based on the real-time horizontal distance. The UAV is controlled to return to its capsule based on the target return strategy corresponding to the target return phase.

2. The control method for a vehicle-mounted unmanned aerial vehicle according to claim 1, characterized in that, The step of determining the target return phase of the UAV based on the real-time horizontal distance includes: If the real-time horizontal distance is greater than a distance threshold, the target return phase is determined to be the first phase; the first phase is the long-range active distance reduction phase of the UAV. or If the real-time horizontal distance is less than or equal to the distance threshold, the target return phase is determined to be the second phase; the second phase is the close-range attitude and position adjustment phase of the UAV.

3. The control method for a vehicle-mounted unmanned aerial vehicle according to claim 2, characterized in that, The relative position data includes real-time altitude difference; the control of the UAV's return to its capsule based on the target return strategy corresponding to the target return phase includes: When the target return phase is in the first phase and the real-time altitude difference is less than the first altitude threshold, the drone is controlled to increase the real-time altitude difference. or When the target return phase is the first phase and the real-time altitude difference is greater than or equal to the first altitude threshold, the drone is controlled to reduce the real-time horizontal distance at a first speed.

4. The control method for a vehicle-mounted unmanned aerial vehicle according to claim 3, characterized in that, The method of controlling the UAV's return to its capsule based on the target return strategy corresponding to the target return phase further includes: When the target return phase is the second phase and the real-time altitude difference is less than the second altitude threshold, the drone is controlled to increase the real-time altitude difference. Wherein, the first height threshold is greater than the second height threshold.

5. The control method for a vehicle-mounted unmanned aerial vehicle according to claim 2, characterized in that, After determining that the target return phase is the second phase, the method further includes: When the real-time horizontal distance is less than or equal to a distance threshold, the environmental data of the landing area is identified; If the landing area environmental data includes images of the vehicle, the target return phase is defined as the third phase; the third phase is the preparation and execution phase for the precise landing of the UAV.

6. The control method for a vehicle-mounted unmanned aerial vehicle according to claim 5, characterized in that, If the landing area environmental data does not include an image of the vehicle, the method further includes: If, based on the airspace environment data, it is determined that there is open airspace above the vehicle, the drone is controlled to move at a second speed.

7. The control method for a vehicle-mounted unmanned aerial vehicle according to claim 6, characterized in that, If it is determined that there is no open airspace above the vehicle, the method further includes: If it is determined that there is no open airspace above the vehicle, control the vehicle to move and control the drone to follow the vehicle.

8. The control method for a vehicle-mounted unmanned aerial vehicle according to claim 5, characterized in that, The method of controlling the UAV's return to its capsule based on the target return strategy corresponding to the target return phase further includes: When the target return phase is the third phase, the landing conditions of the UAV are determined based on the airspace environment data and the landing area environment data. If the landing conditions are met, control the drone to land on the vehicle at a third speed.

9. The control method for a vehicle-mounted unmanned aerial vehicle according to claim 1, characterized in that, The airspace environment data includes real-time wind speed, and the UAV meets the landing conditions as follows: the landing area environment data contains images of land markers, and the real-time wind speed is less than the wind speed threshold. The drone does not meet the landing conditions if: the landing area environmental data does not contain the image of the landing marker, or the real-time wind speed is greater than or equal to the wind speed threshold.

10. The control method for a vehicle-mounted unmanned aerial vehicle according to claim 1, characterized in that, If it is determined that the drone does not meet the landing conditions, the method further includes: Control the drone to keep it hovering; If the hovering duration of the drone is greater than or equal to a time threshold, a warning message is generated.

11. The control method for a vehicle-mounted unmanned aerial vehicle according to any one of claims 1-10, characterized in that, The method further includes: If the target return phase is either the first or second phase, obstacle detection is performed based on the landing area environmental data, and if the detection results indicate the presence of an obstacle, the drone is controlled to perform obstacle avoidance operations.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the control method for the vehicle-mounted unmanned aerial vehicle according to any one of claims 1-11.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the control method for the vehicle-mounted unmanned aerial vehicle as described in any one of claims 1-11.

Images