[0020] The present invention will be described in detail below in conjunction with the drawings and embodiments.
[0021] In order to realize the omni-directional obstacle avoidance of the drone based on binocular vision, it is necessary to set up binocular camera devices around the drone, and define the corresponding binocular camera device according to the position of the binocular camera device on the drone. In this embodiment, preferably, binocular camera devices are arranged around the drone, which means that at least one binocular camera is installed on the front, rear, left, right, lower and upper parts of the drone. Camera devices, and all binocular camera devices take real-time shooting, and the binocular camera devices on each side collect left and right images of the space environment on the side of the drone. In this embodiment, preferably, the binocular camera device can be directly combined with two cameras, and the binocular camera device can be an infrared camera. The method of UAV's omnidirectional obstacle avoidance based on binocular vision is as follows, please refer to figure 1 , Methods include:
[0022] Step S201: extracting the left image and the right image currently collected by each binocular camera device;
[0023] Each binocular camera device around the UAV captures the left and right images of the space environment around the UAV in real time.
[0024] Step S202: According to the extracted left image and right image, identify whether there is an obstacle in the orientation corresponding to each binocular camera device, if there is an obstacle, go to step S203, otherwise, keep the current direction of the drone and continue driving;
[0025] Recognizing whether there is an obstacle from the left image and the right image can be recognized by a preset recognition algorithm, which is not specifically limited here.
[0026] Step S203: Determine the distance between the obstacle and the UAV according to the binocular vision ranging algorithm;
[0027] The binocular vision ranging algorithm is based on the parallax principle and uses imaging equipment to obtain two images of the measured object from different positions, and then calculates the position deviation between the corresponding points of the image to obtain the three-dimensional geometric information of the object. Like figure 2 As shown, step S203 includes:
[0028] Step S2031: perform stereo matching on the left image and the right image to obtain a disparity map between the left image and the right image;
[0029] Step S2032: Determine the depth image according to the disparity map;
[0030] Step S2033: Extract depth information in the depth image, and determine the three-dimensional coordinates of the obstacle according to the depth information;
[0031] Step S2034: Determine the distance between the obstacle and the UAV according to the three-dimensional coordinates of the obstacle.
[0032] Step S204: Determine whether the distance between the drone and the obstacle is less than the preset safety distance, if it is less than the preset safety distance, go to step S205, otherwise, keep the UAV in the current direction of movement and continue driving;
[0033] Step S205: Adjust the current movement direction of the drone so that the drone moves in a direction away from the obstacle;
[0034] The step of adjusting the current movement direction of the unmanned aerial vehicle specifically includes: acquiring the orientation corresponding to the binocular camera device where no obstacle is detected, and making the movement direction of the unmanned aerial vehicle face the dual-eye camera where no obstacle is detected. The azimuth deflection corresponding to the eye camera device. By adjusting the current movement direction of the UAV, the UAV can avoid obstacles and avoid the UAV colliding with obstacles to damage the UAV.
[0035] Of course, it can also be set after step S205 to immediately return to step S201 for the next obstacle avoidance process, or after the drone is set to continue to travel for a certain distance according to predetermined conditions, return to step 201 for the next obstacle avoidance process. It can be a time condition, then the UAV will continue to drive along the current direction of motion according to the predetermined conditions as follows: the UAV continues to drive along the current direction of motion for a predetermined time; the predetermined condition can be a distance condition, then the UAV will follow the predetermined conditions along the current direction of motion Continue driving: The drone continues to travel a predetermined distance in the current direction of movement.
[0036] In the embodiment of the present invention, binocular camera devices are arranged around the drone, and the binocular camera devices take real-time shooting, and the left and right images currently collected by each binocular camera device are extracted respectively, according to the extracted Left and right images, identify whether there are obstacles in the azimuth corresponding to each binocular camera device. If there are obstacles in the azimuth corresponding to one or more binocular camera devices, control the drone to face no obstacles detected The direction movement of the binocular camera device realizes the automatic obstacle avoidance of the drone. Since the binocular camera device is arranged around the drone, the drone can avoid obstacles in all directions based on binocular vision.
[0037] The present invention also provides an implementation of an omnidirectional obstacle avoidance system for drones based on binocular vision. See image 3 The omni-directional obstacle avoidance system 30 for drones based on binocular vision includes a processor 302, a direction controller 303, and multiple binocular camera devices 304. The processor 302, the binocular camera device 304 and the direction controller 303 are all fixed on On the drone, the direction controller 303 is connected to the drone to control the movement direction of the drone, and the direction controller 303 and the binocular camera device 304 are both connected to the processor 302. Multiple binocular camera devices 304 are distributed around the drone, and the binocular camera device 304 takes real-time shooting. In this embodiment, preferably, multiple binocular camera devices are distributed on the front and back ends of the drone. , Left, right, bottom and top, each binocular camera device corresponds to an orientation. .
[0038] The processor 302 is configured to: extract the left image and the right image currently collected by each binocular camera device 304; according to the extracted left image and right image, identify whether there is an obstacle in the orientation corresponding to each binocular camera device 304 If there is the obstacle, determine the distance between the obstacle and the UAV according to the binocular vision ranging algorithm; determine whether the distance between the UAV and the obstacle is less than or equal to the preset safety Distance: If it is less than the preset safe distance, the direction controller 303 is used to adjust the current movement direction of the drone to move the drone in a direction away from obstacles to realize the automatic obstacle avoidance function of the drone.
[0039] The step of the processor 302 adjusting the current direction of movement of the drone through the direction controller 303 includes: the processor 302 obtains the position corresponding to the binocular camera device 304 that has not detected obstacles, and controls the direction of the drone through the direction controller 303. The azimuth deflection corresponding to the binocular camera device that does not detect the obstacle. After the movement direction of the drone is deflected, the drone smoothly avoids obstacles.
[0040] Specifically, the processor 302 determines the distance between the obstacle and the drone according to the binocular vision ranging algorithm, including: stereo matching the left image and the right image to obtain a disparity map between the left image and the right image Determine the depth image according to the disparity map; extract the depth information in the depth image, and determine the three-dimensional coordinates of the obstacle according to the depth information; determine the difference between the obstacle and the UAV according to the three-dimensional coordinates of the obstacle The distance between.
[0041] Further, the processor 302 is implemented based on a field programmable gate array FPGA, and multiple dual camera devices are connected to the processor 302 in parallel, so that the processor 302 can simultaneously receive the data collected by the multiple binocular camera devices 304 The processor 302 selects the corresponding binocular camera device 304 to collect the spatial environment image according to the movement direction of the drone, and performs obstacle recognition and obstacle avoidance according to the spatial environment image. Of course, in the drone After adjusting the movement direction to avoid obstacles, the corresponding binocular camera device 304 is selected again according to the new movement direction of the drone to collect the spatial ring mirror image. Because the processor 302 selects the binocular camera device 304 every time it is in line with the movement of the drone. The directions correspond to each other. Therefore, each time the binocular camera device 304 is selected is to collect the spatial environment image in the moving direction of the drone and in front of the drone to ensure that obstacles are avoided in the moving direction of the drone.
[0042] The processor 302 is implemented based on a field programmable gate array FPGA, which can make full use of the FPGA's customizable IO features, simultaneously access the input of the multi-channel binocular camera device 304, and make full use of the FPGA parallel structure to realize multi-channel video capture. In addition, the processor 302 dynamically switches the binocular camera device 304 according to the direction of movement of the drone, so that multiple binocular camera devices 304 can share the obstacle recognition and obstacle avoidance control circuit in the processor 302. The binocular camera device 304 is equipped with corresponding obstacle recognition and obstacle avoidance control circuits. Compared with the traditional single-direction binocular vision solution, the binocular camera device 304 can realize all-round obstacle avoidance with almost the same resources and power consumption. Furthermore, through FPGA built-in pipeline and multi-channel parallel processing mechanism, the processing speed and data throughput are unmatched by traditional CPUs.
[0043] In order to improve the processing capability of the processor 302, the processor 302 also adopts a multi-level cache structure to implement data pipeline processing, and the processor 302 also uses a high-bit-width system bus for parallel operation.
[0044] The omnidirectional obstacle avoidance system of the drone based on binocular vision also includes an alarm device 305, which is fixed to the drone, and the alarm device is connected to the processor 302. When the processor 302 determines that the distance between the drone and the obstacle is less than the preset safe distance, it controls the alarm device to issue an alarm, so as to facilitate the user to obtain the obstacle in the current movement direction of the drone. Of course, when the distance between the drone and the obstacle is greater than or equal to the preset safe distance, the processor 302 controls the alarm device not to issue an alarm.
[0045] In the embodiment of the present invention, binocular camera devices 304 are respectively arranged around the drone, and the processor 302 extracts the left and right images currently collected by each binocular camera device 304 according to the extracted left and right images. The right image identifies whether there are obstacles in the azimuth corresponding to each binocular camera device 304. If there is, the drone is controlled to deflect to the azimuth corresponding to the binocular camera device 304 that does not detect the obstacle to avoid the obstacle and realize unmanned Because the binocular camera device 304 is arranged around the drone, it realizes the function of the drone to avoid obstacles in all directions based on binocular vision; in addition, the processor 302 is based on The programmable gate array FPGA is implemented, and multiple dual camera devices are connected to the processor 302 in parallel. The processor 302 obtains the multiple image resources collected by the multiple binocular camera devices 304 in parallel, and the processor 302 according to the movement of the drone The direction, the binocular camera device 304 is dynamically switched, so that multiple binocular camera devices 304 share the obstacle recognition and obstacle avoidance control circuit in the processor 302, and there is no need to equip each binocular camera device 304 with corresponding obstacle recognition Compared with the obstacle avoidance control circuit, compared with the traditional single-direction binocular vision solution, it can realize all-round obstacle avoidance and lower power consumption with almost the same resources and power consumption.
[0046] The above are only the embodiments of the present invention and do not limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the description and drawings of the present invention, or directly or indirectly applied to other related technologies In the same way, all fields are included in the scope of patent protection of the present invention.
