Unmanned aerial vehicle control method and apparatus, unmanned aerial vehicle, and storage medium

Abstract

Embodiments of the present application disclose a kind of unmanned aerial vehicle control method, device, unmanned aerial vehicle and storage medium, the unmanned aerial vehicle is loaded for shooting different direction multiple dual cameras, unmanned aerial vehicle control method includes: receiving and the flight direction consistent with last time three way video data of the dual camera acquisition of the dual camera;The video data is parsed to obtain the image depth of each direction respectively, and the current flight direction of unmanned aerial vehicle is determined according to the image depth;Control next time receives and the flight direction consistent with current three way video data of the dual camera acquisition of the dual camera.Makes that middle and low end chip can realize the function of high-end chip to realize multiple dual cameras, saves the cost of using high-end chip.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of unmanned aerial vehicle control, and in particular to an unmanned aerial vehicle control method and device, an unmanned aerial vehicle and a storage medium. BACKGROUND

[0002] Current binocular obstacle avoidance technology mostly uses one main control chip to drag several binocular camera modules to achieve several-way obstacle avoidance function.

[0003] The current technology of using one chip to drag several binocular modules for obstacle avoidance is limited by the bandwidth and computing power of the main control chip, and can only drag two-way or three-way binocular cameras. It is very difficult to add binocular modules. There are also binocular camera modules on the market that can drag multiple ways, but such chips have high power consumption and high prices, making it difficult to be widely used in the low-end consumer market. SUMMARY

[0004] Therefore, the present application provides an unmanned aerial vehicle control method, which loads a plurality of binocular cameras for shooting different directions, and the unmanned aerial vehicle control method comprises:

[0005] receiving video data collected by three binocular cameras consistent with the flight direction at the previous time;

[0006] analyzing the video data to obtain image depth in each direction respectively, and determining the current flight direction of the unmanned aerial vehicle according to the image depth;

[0007] controlling the next time to receive video data collected by three binocular cameras consistent with the current flight direction.

[0008] Further, it further comprises:

[0009] The number of the plurality of binocular cameras is six;

[0010] The nose of the unmanned aerial vehicle is the front, wherein the first camera is arranged at the front, the second camera is arranged at the left, the third camera is arranged at the bottom, the fourth camera is arranged at the top, the fifth camera is arranged at the back, and the sixth camera is arranged at the right.

[0011] Further, when the number of the plurality of binocular cameras is six, the receiving video data collected by three binocular cameras consistent with the flight direction at the previous time comprises:

[0012] opening six binocular cameras and collecting data in real time, only opening the transmission of the video data stream of the binocular camera consistent with the flight direction at the previous time, and closing the transmission of the video data stream of the binocular camera inconsistent with the flight direction at the previous time;

[0013] Or, open three binocular cameras consistent with the flight direction of the last time, and close the binocular cameras inconsistent with the flight direction of the last time.

[0014] Further, the flight direction of the unmanned aerial vehicle is determined according to the image depth.

[0015] According to the image depth, a depth difference is calculated, if the depth difference is less than or equal to 0, it is determined that the flight direction of the unmanned aerial vehicle is the same as the shooting direction of the binocular camera, if the depth difference is greater than 0, it is determined that the flight direction of the unmanned aerial vehicle is opposite to the shooting direction of the binocular camera.

[0016] Further, the calculation formula of the depth difference is as follows:

[0017] d i+1 = h i+1 - h i ;

[0018] In the formula, d i+1 is the depth difference, h i+1 is the image depth at i+1 time, and h i is the image depth at i time.

[0019] Further, it further comprises:

[0020] When the unmanned aerial vehicle is just started, the video data collected by the binocular camera in the preset direction is received, the preset direction is the forward direction, the descending direction and the right direction with the head of the unmanned aerial vehicle as the reference.

[0021] Further, the application also provides a unmanned aerial vehicle control device, the unmanned aerial vehicle is loaded with a plurality of binocular cameras for shooting different directions, the unmanned aerial vehicle control device comprises:

[0022] The data transmission module is used for receiving the video data collected by three binocular cameras consistent with the flight direction of the last time;

[0023] The depth analysis module is used for analyzing the video data to obtain the image depth of each direction respectively, and determining the current flight direction of the unmanned aerial vehicle according to the image depth;

[0024] The switching module is used for controlling the next time to receive the video data collected by three binocular cameras consistent with the current flight direction.

[0025] Further, the application also provides a unmanned aerial vehicle, comprising a processor and a memory, the memory stores a computer program, the computer program runs on the processor, and the unmanned aerial vehicle control method in any of the above embodiments.

[0026] Further, the processor comprises a master chip and a switching module connected with the master chip.

[0027] The master chip is configured to control the multi-channel binocular camera to simultaneously send the collected multi-channel video data stream to the switching module.

[0028] The switching module is configured to splice the data according to each channel camera to obtain spliced video data, bind the spliced video data with the camera direction, and then send the video data to the master chip.

[0029] Further, the application also provides a readable storage medium storing a computer program, which performs the unmanned aerial vehicle control method in any of the above embodiments when running on a processor.

[0030] The application provides an unmanned aerial vehicle control method, which receives three-channel video data collected by the binocular camera in the same direction as the flight direction at the previous moment, analyzes the video data to obtain the image depth in each direction, determines the current flight direction of the unmanned aerial vehicle according to the image depth, and controls the next moment to receive three-channel video data collected by the binocular camera in the same direction as the current flight direction. This ensures that at most three-channel data transmitted by the binocular camera is processed each time, and the video data required to receive data is selectively switched according to the flight direction of the unmanned aerial vehicle, thereby reducing the occupation of the bandwidth of the master chip and the consumption of the computing resources, so that the function of the multi-channel binocular camera that can only be realized by high-end chips can be realized by low-end chips, thereby saving the cost of using high-end chips. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope of protection of the present application. In each drawing, similar components are marked with similar reference numerals.

[0032] Figure 1 A flowchart of an unmanned aerial vehicle control method according to an embodiment of the present application is shown;

[0033] Figure 2 A schematic diagram of the external structure of an unmanned aerial vehicle according to an embodiment of the present application is shown;

[0034] Figure 3 A schematic diagram of the internal structure of an unmanned aerial vehicle according to an embodiment of the present application is shown;

[0035] Figure 4 A schematic diagram of an unmanned aerial vehicle control device according to an embodiment of the present application is shown. Detailed Implementation

[0036] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0037] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0038] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0039] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0040] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0041] The technical solution of this application will be explained below with specific embodiments.

[0042] Example 1

[0043] This embodiment provides a drone control method, which is applied to a drone equipped with a multi-channel binocular camera that shoots from different directions.

[0044] like Figure 1 The flowchart is shown below:

[0045] Step S100: Receive video data collected by the three binocular cameras that are in the same flight direction as the previous moment.

[0046] In the embodiment, a UAV is equipped with six binocular cameras, as shown in the figure: Figure 2

[0047] The head of the UAV is taken as the front direction, and the six binocular cameras are arranged at the front, back, left, right, top and bottom directions of the UAV to capture video images of the six directions of the UAV. The first binocular camera is arranged at the front direction, the second binocular camera is arranged at the left direction, the third binocular camera is arranged at the bottom direction, the fourth binocular camera is arranged at the top direction, the fifth binocular camera is arranged at the back direction, and the sixth binocular camera is arranged at the right direction.

[0048] However, for the main control chip of the UAV, simultaneously processing the video data from the six cameras puts very high requirements on the performance of the chip, and a general low-end chip cannot cope with the video data from the six binocular cameras.

[0049] The binocular cameras are mainly used for obstacle avoidance of the UAV. In actual situations, the UAV only needs to pay attention to whether there are obstacles in its own moving direction, and does not need to pay attention to obstacles that are not in its own moving direction, because the distance will not be shortened. In three-dimensional space, the movement of the UAV is decomposed into at most three directions, so the UAV only needs to receive image data from at most three binocular cameras in each direction.

[0050] Therefore, only the video data collected by the three binocular cameras in the same direction as the flight direction of the UAV needs to be obtained to reduce the pressure on the main control chip.

[0051] Further, when the UAV is just started, there is no flight direction, but the cameras still need to be turned on to collect video data, so three directions are selected as preset directions to obtain the video data of the binocular cameras in these three directions. Obviously, in the six directions of the UAV, the top and bottom directions are mutually exclusive, the left and right directions are mutually exclusive, and the front and back directions are mutually exclusive, so the preset directions can be the three mutually exclusive directions of front, bottom and right as the initial preset directions, so the first binocular camera, the second binocular camera and the sixth binocular camera are turned on. Similarly, the front, top and right directions or the front, top and left directions can also be turned on, and the application does not limit the combination of the mutually exclusive directions.

[0052] Specifically, when the UAV moves, the six binocular cameras carried by the UAV can be in an open state to collect images, but only the video data of three of them is received for obstacle avoidance, that is, only the video data transmission of three of the six cameras is enabled, and the data of the remaining three will not be transmitted to the main control. When the flight direction changes, the transmission path is switched. ​

[0053] Similarly, data collection can also be performed by opening only 3 of the 6 binocular cameras, and when the flight direction changes, the binocular cameras in the same direction are opened and the cameras in the opposite direction are closed according to the flight direction.

[0054] In step S200, the video data is parsed to obtain the image depth of each direction respectively, and the current flight direction of the unmanned aerial vehicle is determined according to the image depth.

[0055] After obtaining the video data, the image depth of each frame of image in the video data can be obtained. The image depth at different time is different, so the current flight direction of the unmanned aerial vehicle can be determined according to the image depth.

[0056] Specifically, the depth difference is calculated according to the image depth, if the depth difference is less than or equal to 0, it is determined that the flight direction of the unmanned aerial vehicle is the same as the shooting direction of the binocular camera, if the depth difference is greater than 0, it is determined that the flight direction of the unmanned aerial vehicle is opposite to the shooting direction of the binocular camera.

[0057] The calculation formula of the depth difference is as follows:

[0058] d i+1 = h i+1 - h i ;

[0059] In the formula, d i+1 is the depth difference, h i+1 is the image depth at i+1 moment, and h i is the image depth at i moment.

[0060] Specifically, when the unmanned aerial vehicle flies forward, the image depth returned by the camera in front must be continuously reduced, that is, the image difference is less than 0, so it can be known that the unmanned aerial vehicle is flying forward. If the unmanned aerial vehicle suddenly flies backward at this time, the image depth returned by the camera in front will begin to increase, the image difference is greater than 0, so it can be judged that the unmanned aerial vehicle is flying backward.

[0061] In step S300, the video data collected by the next moment receiving three binocular cameras in the same direction as the current flight direction is controlled.

[0062] According to the judgment of the current flight direction of the unmanned aerial vehicle in step S200, the video data in the shooting direction needed to be received next is switched according to the flight direction.

[0063] Specifically, if the 6-way binocular camera is opened, the video data is received through the video data stream of the camera, for example, if the flight direction changes from forward to backward, the video data stream of the front binocular camera is closed and the video data stream of the rear binocular camera is opened, and if the flight direction does not change, it remains unchanged.

[0064] If only three-way binocular cameras are opened, the video data is received by opening and closing the cameras, for example, if the flight direction changes from forward to backward, the front binocular camera is closed and the rear binocular camera is opened, and if the flight direction does not change, it remains unchanged.

[0065] When the unmanned aerial vehicle is flying in a single direction (i.e. simply forward, backward, left, right, up, down), the image depth collected by the other two cameras will not change in theory, so no switching will be performed.

[0066] The embodiment receives video data in three directions of the unmanned aerial vehicle, confirms the flight direction of the unmanned aerial vehicle by confirming the image depth in three directions of the unmanned aerial vehicle, and controls the video data captured by the binocular camera corresponding to the receiving and flight direction, thereby reducing the occupation of the bandwidth of the main control chip and the consumption of the computing resources, so that a middle-end chip can realize the obstacle avoidance function with 6-way binocular cameras.

[0067] Embodiment 2

[0068] The application also provides an unmanned aerial vehicle, which comprises a processor and a memory, and the memory executes the unmanned aerial vehicle control method in the above embodiments, wherein the processor comprises a main control chip and a switching module.

[0069] According to Figure 3 The internal structure of the unmanned aerial vehicle is shown in the schematic diagram.

[0070] The main control chip is used to control the multi-way binocular camera to send the collected multi-way video data stream to the switching module at the same time, that is, the camera data collected by the binocular camera is not directly sent to the main control chip at the beginning, but is sent to the switching module for processing.

[0071] The switching module is used to splice the data of each camera to obtain the spliced video data, bind the spliced video data with the camera direction, and then send the video data to the main control chip.

[0072] Specifically, the switching module can be a CPLD (complex programmable logic device), which mainly splices the received video data to obtain binocular images meeting requirements in the embodiment, so that the host chip can directly process the images to obtain image depth, and then perform obstacle avoidance and switching of received binocular camera video data and the like.

[0073] The embodiment of the present application also provides a UAV control device, as shown in the figure, the device comprises a data transmission module 10, a depth analysis module 20 and a switching module 30, and specific functions of each module are as follows. Figure 4

[0074] The data transmission module 10 is used for receiving three-way video data collected by the binocular camera in the same direction as the flight direction at the previous moment;

[0075] The depth analysis module 20 is used for analyzing the video data to obtain image depth in each direction respectively, and determining the current flight direction of the UAV according to the image depth;

[0076] The switching module 30 is used for controlling the next moment to receive three-way video data collected by the binocular camera in the same direction as the current flight direction.

[0077] The embodiment of the present application also provides a readable storage medium, which stores a computer program, and the computer program performs the UAV control method in any of the above embodiments when running on a processor.

[0078] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are only schematic, for example, the flowcharts and structural diagrams in the drawings show the possible implementation architectures, functions and operations of the devices, methods and computer program products according to the embodiments of the present application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in alternative implementation ways, the functions noted in the blocks can also occur in different order from that noted in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the structural diagram and / or flowchart, and the combination of blocks in the structural diagram and / or flowchart, can be implemented by a dedicated hardware-based system for executing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0079] ​In addition, each functional module or unit in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0080] If the functions are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0081] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for controlling a drone, the method comprising: The unmanned aerial vehicle is loaded with multi-channel binocular cameras for shooting different directions, and the unmanned aerial vehicle control method comprises: receiving video data collected by three channels of the binocular cameras in the same direction as the flight direction at the previous time; analyzing the video data to obtain image depth in each direction respectively, and calculating a depth difference according to the image depth, if the depth difference is less than or equal to 0, it is determined that the flight direction of the unmanned aerial vehicle is the same as the shooting direction of the binocular cameras, if the depth difference is greater than 0, it is determined that the flight direction of the unmanned aerial vehicle is opposite to the shooting direction of the binocular cameras; controlling the next time to receive video data collected by three channels of the binocular cameras in the same direction as the current flight direction; the number of the multi-channel binocular cameras is six; with the nose of the unmanned aerial vehicle as the front, wherein the first camera is arranged in the front, the second camera is arranged in the left, the third camera is arranged in the lower, the fourth camera is arranged in the upper, the fifth camera is arranged in the rear, and the sixth camera is arranged in the right; when the number of the multi-channel binocular cameras is six, the receiving video data collected by three channels of the binocular cameras in the same direction as the flight direction at the previous time comprises: opening six binocular cameras and collecting data in real time, only opening the transmission of the video data stream of three channels of the binocular cameras in the same direction as the flight direction at the previous time, and closing the transmission of the video data stream of three channels of the binocular cameras in the direction different from the flight direction at the previous time; or, opening three binocular cameras in the same direction as the flight direction at the previous time, and closing the binocular cameras in the direction different from the flight direction at the previous time.

2. The UAV control method of claim 1, wherein, The calculation formula of the depth difference is as follows: ； wherein d i+1 is the depth difference, h i+1 is the image depth at i+1 time, h i is the image depth at i time. 3.The UAV control method of claim 1, wherein, further comprising: when the unmanned aerial vehicle is just started, receiving video data collected by the binocular cameras in a preset direction, the preset direction being three non-mutually exclusive directions with the nose of the unmanned aerial vehicle as the reference.

4. A drone control device, characterized by, The unmanned aerial vehicle is loaded with multi-channel binocular cameras for shooting different directions, and the unmanned aerial vehicle control device comprises: a data transmission module for receiving video data collected by three channels of the binocular cameras in the same direction as the flight direction at the previous time; a depth analysis module for analyzing the video data to obtain image depth in each direction respectively, and calculating a depth difference according to the image depth, if the depth difference is less than or equal to 0, it is determined that the flight direction of the unmanned aerial vehicle is the same as the shooting direction of the binocular cameras, if the depth difference is greater than 0, it is determined that the flight direction of the unmanned aerial vehicle is opposite to the shooting direction of the binocular cameras; a switching module for controlling the next time to receive video data collected by three channels of the binocular cameras in the same direction as the current flight direction; the number of the multi-channel binocular cameras is six; with the nose of the unmanned aerial vehicle as the front, wherein the first camera is arranged in the front, the second camera is arranged in the left, the third camera is arranged in the lower, the fourth camera is arranged in the upper, the fifth camera is arranged in the rear, and the sixth camera is arranged in the right; when the number of the multi-channel binocular cameras is six, the receiving video data collected by three channels of the binocular cameras in the same direction as the flight direction at the previous time comprises: Open six binocular cameras and collect data in real time, only open the transmission of video data stream of three binocular cameras consistent with the flight direction of the last time, and close the transmission of video data stream of three binocular cameras inconsistent with the flight direction of the last time. Or, open three binocular cameras consistent with the flight direction of the last time, and close the binocular cameras inconsistent with the flight direction of the last time.

5. A drone, characterized in that, The unmanned aerial vehicle comprises a processor and a memory, the memory stores a computer program, and the computer program executes the unmanned aerial vehicle control method in any one of claims 1 to 3 when running on the processor.

6. The drone of claim 5, wherein, The processor comprises a main control chip and a switching module connected with the main control chip. The main control chip is used for controlling the multi-channel binocular camera to simultaneously send the collected multi-channel video data stream to the switching module. The switching module is used for performing data splicing on each channel camera to obtain spliced video data, binding the spliced video data with the camera direction, and then sending the video data to the main control chip.

7. A readable storage medium characterized by, The memory stores a computer program, and the computer program executes the unmanned aerial vehicle control method in any one of claims 1 to 3 when running on the processor.

Images