Flight control method, device and equipment of unmanned aerial vehicle and storage medium

Abstract

The embodiment of the invention discloses a flight control method and device of an unmanned aerial vehicle, equipment and a storage medium, and belongs to the technical field of unmanned aerial vehicles, and the method comprises the steps: obtaining a first position and a second position corresponding to the unmanned aerial vehicle; by taking the first position as a starting point and the second position as an end point, based on a pre-constructed three-dimensional map model, avoiding obstacles and determining a first flight path of the unmanned aerial vehicle; and controlling the unmanned aerial vehicle to fly along the first flight path based on image matching between the real-time environment image and the standard environment image. The first flight path of the unmanned aerial vehicle is determined through the pre-constructed three-dimensional map model, real-time construction of a map model for real-time planning of the flight path of the unmanned aerial vehicle is not needed, the offline three-dimensional map model can effectively reduce the calculation amount required by the unmanned aerial vehicle in the flight process, time is saved, and the flight path planning efficiency is improved. And the real-time environment image is compared with the three-dimensional map model, so that the unmanned aerial vehicle is prevented from yawing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicles, and in particular to a flight control method and device for unmanned aerial vehicles, an apparatus, and a storage medium. BACKGROUND

[0002] An unmanned aerial vehicle is a device that is controlled by using radio remote control equipment and self-provided program control devices. At present, with the development of science and progress of society, the application of unmanned aerial vehicles is gradually increasing.

[0003] In related technologies, during flight, an unmanned aerial vehicle senses the surrounding environment in real time by using a sensing device (such as a visual camera or a laser radar) carried by the unmanned aerial vehicle to construct a map, and then controls the unmanned aerial vehicle to fly based on the real-time constructed map, so that the unmanned aerial vehicle can avoid obstacles and successfully reach a destination.

[0004] However, in the above related technologies, the real-time map construction has a large amount of calculation, which makes the time required for the unmanned aerial vehicle to reach the destination too long. SUMMARY

[0005] Embodiments of the present application provide a flight control method and device for unmanned aerial vehicles, an apparatus, and a storage medium, which can reduce the amount of calculation required for an unmanned aerial vehicle during flight and save time. The technical solution is as follows: On the one hand, the present application provides a flight control method for an unmanned aerial vehicle, which comprises the following steps: obtaining a first position and a second position corresponding to the unmanned aerial vehicle; determining a first flight path of the unmanned aerial vehicle based on a pre-constructed three-dimensional map model, with the first position as a starting point and the second position as an ending point, and avoiding obstacles; controlling the unmanned aerial vehicle to fly along the first flight path based on image matching between a real-time environment image and a standard environment image, wherein the real-time environment image refers to an environment image obtained by the unmanned aerial vehicle in real time during flight, and the standard environment image refers to an image obtained in the three-dimensional map model.

[0006] On the other hand, the present application provides a flight control device for an unmanned aerial vehicle, which is used to implement the flight control method for the unmanned aerial vehicle, and comprises the following modules: a position obtaining module, configured to obtain a first position and a second position corresponding to the unmanned aerial vehicle; a path determining module, configured to determine a first flight path of the unmanned aerial vehicle based on a pre-constructed three-dimensional map model, with the first position as a starting point and the second position as an ending point, and avoiding obstacles; The flight control module is configured to control the UAV to fly along the first flight path based on image matching between a real-time environment image and a standard environment image, wherein the real-time environment image refers to an environment image acquired by the UAV in real time during flight, and the standard environment image refers to an image acquired in the three-dimensional map model.

[0007] In another aspect, an embodiment of the present application provides a computer device, which comprises a processor and a memory, and the memory stores a computer program, which is loaded and executed by the processor to implement the flight control method of the UAV.

[0008] In another aspect, an embodiment of the present application provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the flight control method of the UAV.

[0009] In another aspect, an embodiment of the present application provides a computer program product, which, when executed, causes a computer device to perform the flight control method of the UAV.

[0010] Compared with the prior art, the technical scheme provided by the embodiment of the present application can bring the following beneficial effects: (1) The first flight path of the UAV is determined based on the pre-constructed three-dimensional map model, and the UAV is controlled to fly based on the first flight path, without the need to construct a map model in real time to plan a flight path for the UAV in real time. The offline three-dimensional map model can effectively reduce the amount of calculation required by the UAV during flight, thereby reducing the time required for the UAV to reach the destination and saving time. (2) The UAV is controlled to fly along the first flight path based on image matching between a real-time environment image and a standard environment image, and the standard environment image refers to an image acquired in the three-dimensional map model. That is, the real-time environment image is compared with the three-dimensional map model during flight of the UAV, which is conducive to ensuring that the UAV does not deviate, thereby enabling the UAV to successfully reach the destination, and the amount of calculation required for image matching is less than the amount of calculation required for constructing a three-dimensional map model in real time. (3) The first flight path of the UAV is determined based on the three-dimensional map model, and the three-dimensional map model can effectively contain more environment information, which is conducive to improving the accuracy of the flight path, thereby enabling the UAV to successfully reach the destination. BRIEF DESCRIPTION OF DRAWINGS

[0011] The application will be further described below with reference to the accompanying drawings and embodiments: Figure 1 is a schematic diagram of the flight control system provided by an embodiment of the present application; Figure 2 is a flowchart of a flight control method of a UAV provided by an embodiment of the present application; Figure 3 is a flowchart of a flight control method of a UAV provided by another embodiment of the present application; Figure 4 is a block diagram of a flight control device of a UAV provided by an embodiment of the present application; Figure 5 is a block diagram of a flight control device of a UAV provided by another embodiment of the present application. DETAILED DESCRIPTION

[0012] The present application will be further described in detail below with reference to specific embodiments: Please refer to Figure 1 which shows a schematic diagram of a flight control system provided by an embodiment of the present application. The flight control system can include a UAV 10 and a computer device 20.

[0013] The UAV 10 is a pilotless machine. Exemplarily, the UAV 10 is a fixed-wing UAV, a rotary-wing UAV, a UAV airship, a parafoil UAV or an ornithopter UAV, etc., which are not limited by the embodiments of the present application.

[0014] The computer device 20 is used to control the flight of the UAV 10. Exemplarily, the computer device 20 is an electronic device such as a mobile phone, a tablet computer, a wearable device, a background server, an on-board computer of the UAV, a server cluster or a PC (Personal Computer), etc., which are not limited by the embodiments of the present application.

[0015] In the embodiments of the present application, after determining the starting position and the destination position corresponding to the UAV 10, the computer device 20 determines a flight path for the UAV 10 based on a pre-constructed three-dimensional map model to avoid obstacles, and then controls the UAV 10 to fly along the flight path; and in the process of the UAV 10 flying along the flight path, real-time environment images of the UAV 10 are acquired, and the flight path of the UAV 10 is monitored based on image matching between the real-time environment images and a standard environment image to ensure that the UAV 10 can smoothly reach the destination. The standard environment image is acquired based on the three-dimensional map model. Alternatively, the real-time environment images are acquired based on an on-board image acquisition device (such as an on-board camera, an on-board scanner, etc.) of the UAV 10; or the real-time environment images are acquired based on an image acquisition device (such as a camera, a video camera, a scanner, a depth camera, etc.) other than the UAV 10 in the flight control system.

[0016] Optionally, in the flight control system, the number of unmanned aerial vehicles 10 controlled by the computer device 20 is any value, and the number of unmanned aerial vehicles 10 can be flexibly set and adjusted according to actual conditions, and the embodiments of the present application do not limit this.

[0017] Optionally, the unmanned aerial vehicle 10 and the computer device 20 communicate through a network.

[0018] Please refer to Figure 2 , which shows a flowchart of a flight control method of an unmanned aerial vehicle provided by an embodiment of the present application. The method is applied to Figure 1 the computer device 20 in the flight control system shown. The method can include the following steps (201-203): Step 201, obtaining a first position and a second position corresponding to the unmanned aerial vehicle.

[0019] The first position refers to the departure position of the unmanned aerial vehicle, and the second position refers to the destination position of the unmanned aerial vehicle. In the embodiments of the present application, when the unmanned aerial vehicle is controlled to fly, the computer device obtains the first position and the second position corresponding to the unmanned aerial vehicle.

[0020] Optionally, the unmanned aerial vehicle corresponds to a task to be executed, the computer device obtains the task to be executed, determines the destination of the task to be executed as the destination position of the unmanned aerial vehicle, and thus obtains the second position corresponding to the unmanned aerial vehicle; and the computer device determines the current position of the unmanned aerial vehicle as the departure position, and thus obtains the first position corresponding to the unmanned aerial vehicle. Optionally, the task to be executed is any task. Exemplarily, the task to be executed is an article transportation task.

[0021] Step 202, determining a first flight path of the unmanned aerial vehicle based on a pre-constructed three-dimensional map model, avoiding obstacles, with the first position as the starting point and the second position as the terminal point.

[0022] The three-dimensional map model refers to a map model constructed based on an actual environment. Optionally, before the unmanned aerial vehicle is controlled to fly, the computer device obtains environment data and constructs a three-dimensional map model based on the environment data. Exemplarily, the environment data includes radar data and image data, the computer device calls a movable device (such as an unmanned aerial vehicle) equipped with a laser radar and an image acquisition module, obtains the radar data and the image data by moving the movable device in the actual environment, further constructs a three-dimensional point cloud model based on the radar data, and maps the three-dimensional model based on the image data to obtain a three-dimensional map model. Optionally, the three-dimensional map model is stored in the computer device, or the three-dimensional map model is stored in other devices, and the embodiments of the present application do not limit this.

[0023] In the embodiments of the present application, after the first position and the second position are acquired, the computer device determines a first flight path of the UAV based on the pre-constructed three-dimensional map model, starting from the first position and ending at the second position, while avoiding obstacles. Optionally, the computer device marks the first position and the second position in the three-dimensional map model, connects the first position and the second position with a line segment, removes the part of the line segment that passes through the obstacles to form at least two discontinuous line segments, and then connects the at least two line segments while avoiding the obstacles based on the shape and size of the obstacles in the three-dimensional map model, thereby obtaining the first flight path.

[0024] Optionally, the first flight path corresponds to pose information and speed information. The pose information is used to indicate the flight attitude of the UAV, and the speed information is used to indicate the flight speed of the UAV. Illustratively, when the computer device determines the first flight path of the UAV based on the three-dimensional map model, it determines the flight attitude and flight speed of the UAV at different positions in the first flight path based on the distribution of the obstacles, and then generates the pose information and speed information corresponding to the first flight path.

[0025] Step 203: based on image matching between the real-time environment image and the standard environment image, controlling the UAV to fly along the first flight path.

[0026] In the embodiments of the present application, after the first flight path is acquired, the computer device controls the UAV to fly along the first flight path based on image matching between the real-time environment image and the standard environment image. The real-time environment image refers to an environment image acquired by the UAV in real time during flight, and the standard environment image refers to an image acquired in the three-dimensional map model.

[0027] Optionally, the real-time environment image and the standard environment image correspond to the same coordinates. Illustratively, the computer device controls the UAV to fly along the first flight path, acquires a real-time environment image of the UAV during flight, and acquires real-time coordinates corresponding to the real-time environment image, which are used to indicate the position of the UAV; further, based on the real-time coordinates, a standard environment image is acquired in the three-dimensional map model, and the real-time environment image and the standard environment image are matched; then, in the case that the real-time environment image and the standard environment image are matched, the UAV continues to fly along the first flight path, and in the case that the real-time environment image and the standard environment image are not matched, a new flight path is planned for the UAV.

[0028] Optionally, in order to further ensure the smooth flight of the UAV, in the embodiments of the present application, the computer device performs real-time obstruction detection during the flight of the UAV, and in the case that an obstruction is detected on the first flight path, the computer device controls the UAV to switch to a hovering state at the current position; then, in the case that the obstruction is detected to move out of the first flight path, the computer device controls the UAV to switch from the hovering state to a normal state. The hovering state refers to a state in which the UAV keeps the spatial position substantially unchanged in the air, and the normal state refers to a state in which the UAV normally flies.

[0029] In a possible implementation, the computer device performs obstruction detection by using a laser radar. Optionally, during the flight of the UAV along the first flight path, the computer device performs scanning along the flight direction of the UAV by using the laser radar, and in the case that the scanning detects an obstruction on the first flight path, the computer device controls the UAV to switch to a hovering state at the current position, and continues to perform scanning by using the laser radar, and in the case that the scanning detects that the obstruction moves out of the first flight path, the computer device controls the UAV to switch from the hovering state to a normal state. Exemplarily, the laser radar is a laser radar carried on the UAV, or the laser radar is a laser radar prearranged outside the UAV in the flight control system, and the embodiments of the present application do not make any limitation in this regard.

[0030] In another possible implementation, the computer device performs obstruction detection by using image comparison. Optionally, during the flight of the UAV along the first flight path, the computer device acquires a detection image along the flight direction of the UAV by using an image acquisition device, and performs image comparison on the detection image and the real-time environment image, and in the case that the obstruction on the first flight path is detected based on the image comparison result, the computer device controls the UAV to switch to a hovering state at the current position, and then continues to acquire the detection image and perform image comparison according to the first time interval, and in the case that the obstruction moves out of the first flight path is detected based on the image comparison result, the computer device controls the UAV to switch from the hovering state to a normal state. Exemplarily, the image acquisition device is a device carried on the UAV, or the image acquisition device is a device prearranged outside the UAV in the flight control system, and the embodiments of the present application do not make any limitation in this regard. Optionally, the image acquisition device and the image acquisition device corresponding to the real-time environment image are the same device, or the image acquisition device and the image acquisition device corresponding to the real-time environment image are different devices, and the embodiments of the present application do not make any limitation in this regard.

[0031] In conclusion, in the technical scheme provided by the embodiment of the present application, the first flight path of the unmanned aerial vehicle is determined by the pre-constructed three-dimensional map model, and the unmanned aerial vehicle is controlled to fly based on the first flight path, without the need to construct a map model in real time to plan the flight path of the unmanned aerial vehicle in real time. The offline three-dimensional map model can effectively reduce the amount of calculation required by the unmanned aerial vehicle during flight, thereby reducing the time required for the unmanned aerial vehicle to reach the destination and saving time. Moreover, the unmanned aerial vehicle is controlled to fly along the first flight path based on image matching between the real-time environment image and the standard environment image, and the standard environment image refers to an image obtained in the three-dimensional map model, that is, the real-time environment image is compared with the three-dimensional map model during the flight of the unmanned aerial vehicle, which is conducive to ensuring that the unmanned aerial vehicle does not deviate, thereby enabling the unmanned aerial vehicle to successfully reach the destination, and the amount of calculation required for image matching is less than the amount of calculation required for constructing a three-dimensional map model in real time. In addition, the first flight path of the unmanned aerial vehicle is determined based on the three-dimensional map model, and the three-dimensional map model can effectively contain more environment information, which is conducive to improving the accuracy of the flight path, thereby enabling the unmanned aerial vehicle to successfully reach the destination.

[0032] In addition, during the flight of the unmanned aerial vehicle, in the case where an obstacle is detected on the first flight path, the unmanned aerial vehicle is controlled to switch to a hovering state at the current position, and then in the case where the obstacle is detected to move out of the first flight path, the unmanned aerial vehicle is controlled to switch from the hovering state to the normal state, which is conducive to ensuring the normal flight of the unmanned aerial vehicle and avoiding collision between the unmanned aerial vehicle and the obstacle during flight as much as possible. Moreover, considering that image matching is performed between the real-time environment image and the standard environment image, in the case where the two images match, the detected obstacle is a movable and suddenly appearing obstacle, without the need to start other obstacle avoidance programs, and the obstacle can be waited for to pass in place, thereby effectively reducing the amount of calculation required during the flight of the unmanned aerial vehicle.

[0033] Next, image matching between the real-time environment image and the standard environment image is introduced.

[0034] In the example embodiment, the above step 203 includes the following steps: 1. During the flight of the unmanned aerial vehicle along the first flight path, a real-time environment image of the unmanned aerial vehicle is obtained.

[0035] Optionally, after the first flight path is obtained, the computer device controls the unmanned aerial vehicle to fly along the first flight path. In the embodiment of the present application, during the flight of the unmanned aerial vehicle along the first flight path, a real-time environment image of the unmanned aerial vehicle is obtained.

[0036] In a possible implementation, the computer device invokes an image acquisition device carried on the UAV to acquire the real-time environment image. In another possible implementation, the computer device invokes an image acquisition device other than the UAV in the flight control system to acquire the real-time environment image.

[0037] Optionally, when the real-time environment image is acquired, the computer device acquires an environment acquisition range of the historical environment image. The historical environment image refers to an environment image adjacent to the real-time environment image at an acquisition time, and the environment acquisition range is used to indicate an environment size contained in the historical environment image. Taking the image acquisition device as an on-board camera for example, the computer device determines the environment acquisition range based on an image shooting range of the on-board camera.

[0038] Optionally, after the environment acquisition range is acquired, the computer device determines a flight duration required for the UAV to pass through the environment acquisition range based on the environment acquisition range and a flight speed of the UAV. For example, the speed information corresponding to the first flight path, the computer device determines an intermediate flight path corresponding to the environment acquisition range in the first flight path based on the environment acquisition range, and then acquires a flight speed corresponding to the intermediate flight path from the speed information corresponding to the first flight path. Further, based on the environment size indicated by the environment acquisition range and the flight speed corresponding to the intermediate flight path, the computer device determines the flight duration required for the UAV to pass through the environment acquisition range.

[0039] Optionally, after the flight duration is acquired, the computer device obtains a second time based on the first time and in combination with the flight duration. The first time refers to the acquisition time of the historical environment image. For example, the duration between the first time and the second time is less than or equal to the flight duration.

[0040] Optionally, after the second time is acquired, the computer device acquires the real-time environment image when the current time reaches the second time. For example, if the duration between the first time and the second time is equal to the flight duration, the computer device acquires the real-time environment image when the UAV reaches the environment boundary corresponding to the historical environment image, so as to reduce the environment overlap between the historical environment image and the real-time environment image, and improve the overall image matching efficiency. If the duration between the first time and the second time is less than the flight duration, the computer device acquires the real-time environment image before the UAV reaches the environment boundary corresponding to the historical environment image, so as to improve the timeliness of image matching.

[0041] 2. Based on the historical flight duration and the historical flight speed of the UAV, the real-time coordinate of the UAV in the three-dimensional map model is determined based on the acquisition time of the real-time environment image.

[0042] In the embodiments of the present application, after the real-time environment image is acquired, the computer device determines the real-time coordinate of the UAV in the three-dimensional map model based on the historical flight time length and the historical flight speed of the UAV, with the acquisition time of the real-time environment image as the reference. The historical flight time length refers to the flight time length of the UAV before the acquisition time of the real-time environment image. The historical flight speed refers to the flight speed of the UAV before the acquisition time of the real-time environment image.

[0043] Optionally, since the flight speed of the UAV may change during flight, when the historical flight time length includes at least one historical sub-time length, the historical flight speed includes at least one historical sub-speed, and one historical sub-time length corresponds to one historical sub-speed, the computer device determines the historical flight length of the UAV based on the historical flight time length and the historical flight speed, and further determines the real-time coordinate of the UAV in the three-dimensional map model based on the relationship between the first flight path and the historical flight length, in combination with the relationship between the first flight path and the three-dimensional map model.

[0044] 3. Based on the real-time coordinate, a standard environment image is determined in the three-dimensional map model.

[0045] In the embodiments of the present application, after the real-time coordinate is acquired, the computer device determines a standard environment image in the three-dimensional map model based on the real-time coordinate.

[0046] Optionally, the computer device acquires the standard environment image through a virtual camera. The virtual camera refers to a virtual model of the image acquisition device corresponding to the real-time environment image. For example, the computer device acquires acquisition parameters corresponding to the real-time environment image, and controls the virtual camera to acquire the standard environment image in the three-dimensional map model based on the acquisition parameters. Optionally, the acquisition parameters include but are not limited to at least one of the following: the position of the image acquisition device, the attitude of the image acquisition device, the acquisition range of the image acquisition device, etc.

[0047] 4. When the real-time environment image matches the standard environment image, the UAV is controlled to continue flying along the first flight path.

[0048] Optionally, after the standard environment image is acquired, the computer device performs image matching on the real-time environment image and the standard environment image. In the embodiments of the present application, when the real-time environment image matches the standard environment image, the computer device controls the UAV to continue flying along the first flight path.

[0049] Optionally, the computer device determines that the real-time environment image matches the standard environment image when a similarity between the real-time environment image and the standard environment image is greater than or equal to a target value. The target value is an arbitrary value, which can be flexibly set and adjusted by the staff according to actual conditions, such as 100%, 95%, 90%, and the like, and the embodiments of the present application do not make any limitation on this.

[0050] 5. In a case where the real-time environment image does not match the standard environment image, the computer device re-plans a path for the UAV.

[0051] Optionally, in the embodiments of the present application, in a case where the real-time environment image does not match the standard environment image, the computer device re-plans a path for the UAV.

[0052] Optionally, in a case where the real-time environment image does not match the standard environment image, the computer device determines the collection position of the historical environment image as a third position; further, based on the three-dimensional map model, a second flight path of the UAV is determined by taking the third position as a starting point and the second position as an ending point and avoiding obstacles; then, based on the first flight path, the UAV is controlled to return to the third position; and then, the UAV is controlled to fly along the second flight path. It should be noted that in the process of controlling the UAV to fly along the second flight path as a new flight path, the computer device also needs to perform the image matching and obstacle detection related processes as described above, which will not be repeated here.

[0053] Optionally, before the collection position of the historical environment image is determined as the third position, the computer device acquires a path change frequency of the UAV; further, in a case where the path change frequency is greater than or equal to a threshold value, a warning information for the UAV is sent, and the warning information includes identification information of the UAV; and in a case where the path change frequency is less than the threshold value, the step of determining the collection position of the historical environment image as the third position is performed. The threshold value is an arbitrary value, which can be flexibly set and adjusted by the staff according to actual conditions, such as 10, 50, 100, and the like, and the embodiments of the present application do not make any limitation on this.

[0054] Optionally, the staff performs a return, landing, manual control flight, or the like of the UAV according to specific conditions after acquiring the warning information. Optionally, the warning information further includes at least one set of real-time environment image and standard environment image triggering path change.

[0055] In summary, in the technical scheme provided by the embodiments of the present application, the real-time coordinate corresponding to the real-time environment image is used to determine the standard environment image, and then in the case that the real-time environment image matches the standard environment image, the unmanned aerial vehicle is controlled to continue flying along the first flight path, which is beneficial to ensure that the unmanned aerial vehicle does not deviate, and enables the unmanned aerial vehicle to reach the destination smoothly. Moreover, the real-time coordinate of the unmanned aerial vehicle in the three-dimensional map model is determined by using the historical flight time and the historical flight speed of the unmanned aerial vehicle, which does not need to use more devices (such as positioning satellites) to monitor the position of the unmanned aerial vehicle in real time, effectively utilizes the characteristic that the flight path in the three-dimensional map model is planned in advance, and reduces the real-time flight control cost of the unmanned aerial vehicle.

[0056] In addition, in the case that the real-time environment image does not match the standard environment image, a second flight path is determined for the unmanned aerial vehicle again, so that the unmanned aerial vehicle can continue to fly along the second flight path to reach the destination. The second flight path is still determined based on the three-dimensional map model constructed in advance, and does not need to construct a map model in real time or perform a series of complex operations such as real-time obstacle avoidance of the unmanned aerial vehicle, thereby reducing the calculation amount of the overall flight process of the unmanned aerial vehicle. Moreover, the collection position of the historical environment image is taken as the starting point of the second flight path, considering that the unmanned aerial vehicle has already flown a certain distance, which does not need the unmanned aerial vehicle to return to the starting point (i.e. the first position) at the beginning, retains the correct path that has been flown, and re-plans the flight path at the position where the last image is successfully matched, which is beneficial to reduce the overall flight time of the unmanned aerial vehicle while ensuring the accuracy of the newly acquired flight path, and enables the unmanned aerial vehicle to reach the destination as soon as possible. In addition, in the case that the number of path changes is greater than or equal to the threshold value, a warning information for the unmanned aerial vehicle is sent, that is, even in the case that problems of the three-dimensional map model or the unmanned aerial vehicle cannot be timely repaired, the warning information can be timely sent for notification, which is beneficial to improve the safety of the flight of the unmanned aerial vehicle.

[0057] In addition, the flight time required for the unmanned aerial vehicle to pass through the environment collection range is combined with the acquisition time of the historical environment image to determine the acquisition time of the real-time environment image, which is beneficial to acquire the real-time environment image in time for image matching after the unmanned aerial vehicle passes through the environment range corresponding to the historical environment image, and can improve the timeliness of image matching.

[0058] Please refer to Figure 3 which shows a flowchart of a flight control method of an unmanned aerial vehicle provided by another embodiment of the present application. The method is applied to Figure 1 the computer device 20 in the flight control system shown in the figure. The method can include the following steps (301-306): Step 301, acquiring a first position and a second position corresponding to the unmanned aerial vehicle.

[0059] Step 302, based on the pre-constructed three-dimensional map model, a first flight path of the UAV is determined, starting from the first position and ending at the second position, while avoiding obstacles.

[0060] Step 303, based on image matching between the real-time environment image and the standard environment image, the UAV is controlled to fly along the first flight path.

[0061] The above steps 301-303 are similar to steps 201-203 in the embodiment, which are described in detail in the following. Figure 2 The above steps 301-303 are similar to steps 201-203 in the embodiment, which are described in detail in the following. Figure 2 The above steps 301-303 are similar to steps 201-203 in the embodiment, which are described in detail in the following.

[0062] Step 304, in the case that the flight path of the UAV is changed, the first flight path is determined as a return path of the UAV.

[0063] In the embodiment of the present application, in the case that the flight path of the UAV is changed, the computer device determines the first flight path as a return path of the UAV. The starting point of the return path is the second position and the end point is the first position.

[0064] Step 305, the UAV is controlled to return along the return path, and environment data collected by the UAV on the return path is obtained.

[0065] In the embodiment of the present application, after the return path is obtained, the computer device controls the UAV to return along the return path, and environment data collected by the UAV on the return path is obtained.

[0066] Optionally, the environment data includes radar data and image data. Illustratively, the UAV is equipped with a laser radar and an image acquisition module, and the computer device obtains radar data in the environment data through the laser radar and obtains image data in the environment data through the image acquisition module.

[0067] Step 306, the three-dimensional map model is updated based on the environment data.

[0068] In the embodiment of the present application, after the environment data is obtained, the computer device updates the three-dimensional map model based on the environment data.

[0069] Illustratively, the computer device updates the three-dimensional point cloud model based on the radar data in the environment data, and maps the updated part of the three-dimensional point cloud model based on the image data in the environment data, thereby obtaining an updated three-dimensional map model.

[0070] In summary, the technical scheme provided by the embodiment of the present application considers the possibility of the idle condition of the UAV when returning, determines the first flight path before the change as the return path of the UAV in the case of the change of the flight path of the UAV, collects the environmental data by the UAV when returning, and further updates the three-dimensional map model based on the environmental data, thereby improving the utilization rate of the UAV from the side while maintaining and updating the three-dimensional map model in time.

[0071] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, refer to the method embodiments of the present application.

[0072] Please refer to Figure 4 , which shows a block diagram of a flight control apparatus of a UAV provided by an embodiment of the present application. The apparatus has the function of implementing the flight control method of the UAV as described above, which can be implemented by hardware or corresponding software executed by hardware. The apparatus can be the computer device as described above or be arranged in the computer device. The apparatus can include a position acquisition module 410, a path determination module 420 and a flight control module 430.

[0073] The position acquisition module 410 is configured to acquire a first position and a second position corresponding to the UAV.

[0074] The path determination module 420 is configured to determine a first flight path of the UAV based on a pre-constructed three-dimensional map model, with the first position as the starting point and the second position as the ending point, and avoiding obstacles.

[0075] The flight control module 430 is configured to control the UAV to fly along the first flight path based on image matching between a real-time environmental image and a standard environmental image, wherein the real-time environmental image refers to an environmental image acquired by the UAV in real time during flight, and the standard environmental image refers to an image acquired in the three-dimensional map model.

[0076] In the exemplary embodiment, as Figure 5 shown, the flight control module 430 includes a real-time acquisition unit 431, a coordinate determination unit 432, a standard acquisition unit 433 and a flight control unit 434.

[0077] The real-time acquisition unit 431 is configured to acquire the real-time environmental image of the UAV in the process of the UAV flying along the first flight path.

[0078] The coordinate determination unit 432 is configured to determine a real-time coordinate of the UAV in the three-dimensional map model based on a historical flight time length and a historical flight speed of the UAV, with a time point of acquisition of the real-time environment image as a reference.

[0079] The standard acquisition unit 433 is configured to determine the standard environment image in the three-dimensional map model based on the real-time coordinate.

[0080] The flight control unit 434 is configured to control the UAV to continue flying along the first flight path in a case where the real-time environment image matches the standard environment image.

[0081] In an example embodiment, the position acquisition module 410 is further configured to determine a collection position of a historical environment image as a third position in a case where the real-time environment image does not match the standard environment image, the historical environment image being an environment image adjacent to the real-time environment image in terms of a time point of acquisition. The path determination module 420 is further configured to determine a second flight path of the UAV based on the three-dimensional map model, with the third position as a starting point and the second position as a terminal point, and avoiding obstacles. The flight control module 430 is further configured to control the UAV to return to the third position based on the first flight path, and control the UAV to fly along the second flight path.

[0082] In an example embodiment, as shown in Figure 5 The flight control module 430 further includes a frequency acquisition unit 435 and a warning issuing unit 436.

[0083] The frequency acquisition unit 435 is configured to acquire a path change frequency of the UAV.

[0084] The warning issuing unit 436 is configured to issue a warning information for the UAV in a case where the path change frequency is greater than or equal to a threshold value.

[0085] The position acquisition module 410 is further configured to perform the step of determining the collection position of the historical environment image as the third position in a case where the path change frequency is less than the threshold value.

[0086] In an example embodiment, the real-time acquisition unit 431 is configured to: acquire an environment collection range of the historical environment image; determine a flight time length required for the UAV to pass through the environment collection range based on the environment collection range and a flight speed of the UAV; obtain a second time point based on the first time point and the flight time length, the first time point being a time point of acquisition of the historical environment image. In a case where the current time reaches the second time, the real-time environment image is acquired.

[0087] In an example embodiment, as shown in Figure 5 The apparatus further includes a state switching module 440.

[0088] The state switching module 440 is configured to control the UAV to switch to a hovering state at a current position in a case where the obstruction is detected on the first flight path, and control the UAV to switch from the hovering state to a normal state in a case where the obstruction is detected to move out of the first flight path.

[0089] In an example embodiment, as shown in Figure 5 The apparatus further includes a data acquisition module 450 and a map updating module 460.

[0090] The path determination module 420 is further configured to determine the first flight path as a return path of the UAV in a case where the flight path of the UAV is changed.

[0091] The flight control module 430 is further configured to control the UAV to return along the return path, and the data acquisition module 450 is configured to acquire environment data collected by the UAV on the return path.

[0092] The map updating module 460 is configured to update the three-dimensional map model based on the environment data.

[0093] In summary, in the technical scheme provided by the embodiments of the present application, the first flight path of the UAV is determined based on the pre-constructed three-dimensional map model, and the UAV is controlled to fly based on the first flight path, without the need to construct a map model in real time to plan the flight path of the UAV in real time. The offline three-dimensional map model can effectively reduce the amount of calculation required by the UAV during flight, thereby reducing the time required for the UAV to reach the destination and saving time. Moreover, based on image matching between a real-time environment image and a standard environment image, the UAV is controlled to fly along the first flight path, and the standard environment image refers to an image acquired in the three-dimensional map model, i.e., in the process of UAV flight, the real-time environment image is compared with the three-dimensional map model, which is conducive to ensuring that the UAV does not deviate, thereby enabling the UAV to successfully reach the destination, and the amount of calculation required for image matching is less than the amount of calculation required for constructing a three-dimensional map model in real time. In addition, the first flight path of the UAV is determined based on the three-dimensional map model, and the three-dimensional map model can effectively contain more environment information, which is conducive to improving the accuracy of the flight path, thereby enabling the UAV to successfully reach the destination.

[0094] In an exemplary embodiment, a computer device is also provided, which comprises a processor and a memory, and the memory stores a computer program which is loaded and executed by the processor to implement the flight control method of the unmanned aerial vehicle.

[0095] In an exemplary embodiment, a non-transitory computer readable storage medium is also provided, which stores a computer program which, when executed by a processor, implements the flight control method of the unmanned aerial vehicle.

[0096] In an exemplary embodiment, a computer program product is also provided, which, when executed, causes a computer device to perform the flight control method of the unmanned aerial vehicle.

[0097] The above-described embodiments are only for illustrating the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and cannot limit the protection scope of the present application. It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application, therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, the scope of the present application is defined by the appended claims rather than the above description, therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application.

Claims

1. A flight control method for an unmanned aerial vehicle (UAV), characterized in that, The method includes: Obtain the first and second positions corresponding to the drone; Starting from the first position and ending at the second position, the first flight path of the UAV is determined based on a pre-built 3D map model, avoiding obstacles. Based on image matching between real-time environmental images and standard environmental images, the UAV is controlled to fly along the first flight path; wherein, the real-time environmental image refers to the environmental image acquired by the UAV in real time during flight, and the standard environmental image refers to the image acquired in the three-dimensional map model.

2. The method according to claim 1, characterized in that, The step of controlling the UAV to fly along the first flight path based on image matching between real-time environmental images and standard environmental images includes: During the flight of the drone along the first flight path, the real-time environmental image of the drone is acquired; Based on the acquisition time of the real-time environmental image, and according to the historical flight duration and historical flight speed of the UAV, the real-time coordinates of the UAV in the three-dimensional map model are determined. Based on the real-time coordinates, the standard environment image is determined in the three-dimensional map model; If the real-time environmental image matches the standard environmental image, the drone is controlled to continue flying along the first flight path.

3. The method according to claim 2, characterized in that, After determining the standard environment image in the 3D map model based on the real-time coordinates, the method further includes: If the real-time environmental image does not match the standard environmental image, the acquisition location of the historical environmental image is determined as the third location; wherein, the historical environmental image refers to the environmental image that is adjacent to the real-time environmental image at the acquisition time; Starting from the third position and ending at the second position, the second flight path of the UAV is determined based on the three-dimensional map model, avoiding obstacles. Based on the first flight path, control the drone to return to the third position; Control the drone to fly along the second flight path.

4. The method according to claim 3, characterized in that, Before determining the acquisition location of the historical environment image as the third location, the method further includes: Obtain the number of path changes for the drone; If the number of path changes is greater than or equal to a threshold, a warning message is issued for the drone. If the number of path changes is less than a threshold, the step of determining the acquisition location of the historical environmental image as the third location is performed.

5. The method according to claim 2, characterized in that, The acquisition of the real-time environmental image of the drone includes: The environmental acquisition range for obtaining historical environmental images; Based on the environmental acquisition range and the flight speed of the UAV, determine the flight time required for the UAV to pass through the environmental acquisition range; Based on the first moment and the flight duration, the second moment is obtained; wherein, the first moment refers to the moment when the historical environmental image was acquired; If the second time is reached at the current time, the real-time environmental image is acquired.

6. The method according to claim 1, characterized in that, The method further includes: If an obstacle is detected on the first flight path, the drone is controlled to switch to a hovering state at the current position; If the obstruction is detected to have moved outside the first flight path, the drone is controlled to switch from the hovering state to the normal state.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: If the flight path of the UAV changes, the first flight path will be determined as the UAV's return path. Control the drone to return along the return path; and acquire environmental data collected by the drone along the return path; The 3D map model is updated based on the environmental data.

8. A flight control device for an unmanned aerial vehicle (UAV), characterized in that, The flight control device of the UAV is used to implement the method as described in any one of claims 1 to 7, the device comprising: The location acquisition module is used to acquire the first and second locations of the drone. The path determination module is used to determine the first flight path of the UAV based on a pre-built 3D map model, taking the first position as the starting point and the second position as the ending point, and avoiding obstacles. The flight control module is used to control the UAV to fly along the first flight path based on image matching between real-time environmental images and standard environmental images; wherein, the real-time environmental images refer to environmental images acquired by the UAV in real time during flight, and the standard environmental images refer to images acquired in the three-dimensional map model.

9. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that is loaded and executed by the processor to implement the method as claimed in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images