Method and apparatus for reducing the size of depth maps in a collision avoidance system

Abstract

This document describes methods and apparatuses for reducing a depth map for use in a collision avoidance system. The examples described herein may be implemented in an unmanned aerial vehicle. An example unmanned aerial vehicle includes a depth sensor for generating a first depth map. The first depth map includes a plurality of pixels having respective distance values. The unmanned aerial vehicle further includes a depth map modifier for dividing the plurality of pixels into pixel blocks; and generating a second depth map having fewer pixels than the first depth map based on the distance values of the pixels in the pixel blocks. The unmanned aerial vehicle further includes a collision avoidance system for analyzing the second depth map.

Description

Technical Field

[0001] The present disclosure generally relates to collision avoidance systems and, more particularly, to methods and apparatus for reducing the size of depth maps in collision avoidance systems. Background Art

[0002] Unmanned aerial vehicles (UAVs), often referred to as drones, are becoming more readily available and have developed into a rapidly growing market. UAVs are now widely used in various industries such as agriculture, shipping, forestry management, surveillance, disaster scenarios, gaming, etc. Some UAVs employ collision avoidance systems that assist in controlling the UAV in the event of a detected potential collision. The collision avoidance system analyzes depth maps from one or more depth sensors to determine the positions of objects near the UAV. Brief Description of the Drawings

[0003] Figure 1 Illustrates an example unmanned aerial vehicle (UAV) having an example depth map modifier constructed in accordance with the teachings of the present disclosure.

[0004] Figure 2 Illustrates an example two-dimensional (2D) image representation of an example depth map obtained by an example depth sensor of the example UAV of Figure 1 The example depth map modifier of

[0005] Figure 3 Is a block diagram of an example depth map modifier that can be used to reduce the size of a depth map for processing by an example collision avoidance system of the example UAV of Figure 1 The example depth map modifier of Figure 1 The example depth map modifier of

[0006] Figure 4 Illustrates an example 2D image representation of an example depth map modified using a first example size reduction technique implemented by the example depth map modifier of Figure 1 And 3 The example depth map modifier of

[0007] Figure 5 Illustrates an example 2D image representation of an example depth map modified using a second example size reduction technique implemented by the example depth map modifier of Figure 1 And 3 The example depth map modifier of

[0008] Figure 6 Illustrates an example 2D image representation of an example depth map modified using a third example size reduction technique implemented by the example depth map modifier of Figure 1 And 3 The example depth map modifier of

[0009] Figure 7 Top view of the example UAV of Figure 1 Having two example depth sensors

[0010] Figure 8 Illustrates an example high / low detail region scheme that can be used to modify a depth map using an example concave filter technique implemented by an example depth map modifier when an example UAV 100 travels in a forward direction.

[0011] Figure 9 Illustrates based on Figure 8 An example 2D image representation of an example depth map modified based on the example high / low detail region scheme illustrated in.

[0012] Figure 10 Illustrates another example high / low detail region scheme that can be used when an example UAV travels in a left - forward direction.

[0013] Figure 11 Illustrates another example high / low detail region scheme that can be used when an example UAV travels in the opposite direction.

[0014] Figure 12 Is a flowchart of example machine - readable instructions that can be executed to implement Figure 1 and 3 Of an example depth map modifier.

[0015] Figure 13 Is a flowchart of example machine - readable instructions that can be executed to implement a first example size reduction technique and that correspond to example instructions in an example flowchart of Figure 12 Of.

[0016] Figure 14 Is a flowchart of example machine - readable instructions that can be executed to implement a second example size reduction technique and that correspond to example instructions in an example flowchart of Figure 12 Of.

[0017] Figure 15 Is a flowchart of example machine - readable instructions that can be executed to implement a third example size reduction technique and that correspond to example instructions in an example flowchart of Figure 12 Of.

[0018] Figure 16 Is a flowchart of example machine - readable instructions that can be executed to implement Figure 1 and 3 Of a second example depth map modifier for reducing the size of a depth map using an example concave filter technique.

[0019] Figure 17 Is to build for executing Figure 12 , 13 , 14, 15, and / or 16 of the example instructions to implement Figure 1 and 3A processor platform for an example depth map modifier.

[0020] The figures are not drawn to scale. Instead, the thickness of layers or regions may be exaggerated in the figures. In general, the same reference numerals will be used throughout the figures and the accompanying written description to refer to the same or like components. Detailed Description

[0021] Disclosed herein are example methods, apparatuses, systems, and articles of manufacture for reducing (downsizing) a depth map for analysis by a collision avoidance system. A collision avoidance system, sometimes referred to as an object detection system, is often used on an unmanned aerial vehicle (UAV) to monitor the surroundings of the UAV and prevent the UAV from colliding with objects such as trees, buildings, vehicles, etc. The collision avoidance system analyzes a depth map generated by a depth sensor. The depth map (sometimes referred to as a depth image) is generated at a relatively high resolution such as 848x480 pixels. However, for each depth map, 400,000 pixels need to be processed, which may be done at a frequency of 30 or 60 Hertz (Hz), which can be quite burdensome on the UAV's microprocessor. As such, a larger, heavier, and more robust processor is needed to analyze the depth map, which may not be desirable or feasible in a UAV.

[0022] Disclosed herein is an example depth map modifier that can be used to reduce the size of a depth map provided to a collision avoidance system while still maintaining the relevant data within the depth map. The example depth map modifier disclosed herein can utilize one or more example downsizing techniques to downsize the depth map and reduce the overall data in each depth map. Thus, the downsized depth map requires less computation to be processed by the collision avoidance system, thereby enabling a smaller, lighter, and lower power processor to be used to perform collision avoidance operations, which is desirable in UAVs and other vehicles.

[0023] Also disclosed is an example foveated filter technique for retaining more or fewer pixels in certain regions of a depth map based on the orientation and current direction of travel of the UAV. For example, if the UAV is flying in a forward direction, the position of an object in front of the UAV may be more important than an object to the side and / or rear of the UAV. In some examples, the depth map modifier applies a first sized filter block that can retain relatively more detail in regions of the depth map that have a higher importance, while applying a second filter block to regions of the depth map that have a lower importance where the details are not as important. Such example techniques help further reduce the amount of data included in the depth map and thus reduce the computational load given to the collision avoidance system.

[0024] Although the examples disclosed herein are described in connection with a collision avoidance system implemented by a rotary-wing UAV, the examples disclosed herein may equally be implemented in connection with collision avoidance systems in other types of vehicles (such as fixed-wing UAVs, manned aircraft, road vehicles, off-road vehicles, submarines, etc.). Thus, the examples disclosed herein are not limited to UAVs. The examples disclosed herein may be widely used in other applications to similarly reduce the computational load on a collision avoidance system or other types of systems that use depth maps.

[0025] Figure 1 An example unmanned aerial vehicle (UAV) 100 is illustrated in which the examples disclosed herein may be implemented. UAV 100 is shown Figure 1 twice, once as a UAV and once boxed. In the example illustrated, UAV 100 is a rotary-wing aircraft having eight propeller motors 102a - 102h respectively for driving eight rotor blades 104a - 104h to generate thrust. Motors 102a - 102h are carried by the airframe or fuselage 106. Although in the example illustrated, UAV 100 includes eight rotors and motors, in other examples, the UAV may include more (e.g., nine, ten, etc.) or fewer (e.g., seven, six, etc.) motors and rotors. For example, UAV 100 may be a quadcopter having four rotors.

[0026] The speed of motors 102a - 102h is controlled by one or more motor controllers 108 (sometimes referred to as motor drivers). In the example illustrated, (the) motor controller(s) 108 is implemented by a processor 110 of UAV 100. (The) motor controller(s) 108 applies power (e.g., via a pulse width modulation (PWM) signal) to control the speed of motors 102a - 102h and thereby generate greater or lesser thrust. In Figure 2 the example illustrated, UAV 100 includes a power source for providing power to drive motors 102a - 102h and to power other components of UAV 100 (such as wireless transceiver 122, camera 124, etc.). The power source 112 may be a battery, for example. In other examples, other types of power sources may be implemented. In this example, motors 102a - 102h are electric motors. However, in other examples, UAV 100 may include other types of thrust generators (such as fuel-powered motors, fuel-powered jet engines, etc.) as an addition or alternative to electric motors 102a - 102h.

[0027] In the illustrated example, the UAV 100 includes a sensor system 114, which includes one or more sensors for detecting the orientation, position, and / or one or more other parameters of the UAV 100. In the illustrated example, the sensor system 114 includes an Inertial Measurement Unit (IMU) 116. The IMU 116 may include one or more sensors that measure linear velocity and / or angular velocity as well as acceleration to determine the orientation, position, and / or acceleration of the UAV 100. For example, the IMU 116 may include a solid-state accelerometer, gyroscope, magnetometer, static or dynamic pressure sensors, and / or any other IMU. In the illustrated example, the sensor system 114 includes a Global Positioning System (GPS) sensor 118 for detecting GPS signals and determining the position (location), velocity, and / or acceleration of the UAV 100.

[0028] In the illustrated example, the UAV 100 includes a flight control system 120 implemented by a processor 110. The flight control system 120 is configured to control the flight of the UAV 100. Specifically, the flight control system 120 sends commands or instructions to the motor controller(s) 108 to control the speed of the motors 102a - 102h of the UAV 100 according to a desired flight path. The flight control system 120 uses inputs from the sensor system 114 to maintain the UAV 100 at a desired position or fly the UAV 100 along a desired path. In some examples, the flight control system 120 controls the flight of the UAV 100 based on one or more commands from a manual controller (e.g., a remote control operated by a human pilot). In the illustrated example, the UAV 100 includes a wireless transceiver 122 that operates as a receiver and transmitter for wirelessly communicating with a ground controller and / or another component of an aerial system (e.g., an Unmanned Aerial System (UAS)). The wireless transceiver 122 can be, for example, a low-frequency radio transceiver. In other examples, other types of transceivers can be implemented. Additionally or alternatively, the flight control system 120 can control the flight of the UAV 100 based on one or more autonomous navigation operations. In Figure 1 the illustrated example, the UAV 100 further includes a camera 124 that can be used to record images and / or video. In some examples, the UAV 100 carries the camera 124 via a gimbal to enable the camera 124 to turn relative to the fuselage 106.

[0029] In the illustrated example, the UAV 100 includes a collision avoidance system 126 implemented by a processor 110. The collision avoidance system 126 analyzes depth maps from one or more depth sensors 128 to track objects in the area around the UAV 100. Specifically, the collision avoidance system 126 analyzes the depth maps to monitor for potential collisions with objects (e.g., buildings, trees, vehicles, etc.). If the collision avoidance system 126 detects a potential collision, the collision avoidance system 126 instructs (or overrides) the flight control system 120 to take an appropriate course of action, such as reducing the speed of the UAV 100, changing the trajectory of the UAV 100, changing the altitude of the UAV 100, sending an alert to a remote controller, etc. For example, if the collision avoidance system 126 determines that an object in the field of view is within a threshold distance (e.g., 1 m) of the UAV 100, the collision avoidance system 126 can prohibit the UAV 100 from flying in that direction.

[0030] In the illustrated example, a depth sensor 128 is depicted as being carried on the UAV 100 and adjacent to the camera 124. The depth sensor 128 faces forward and measures the area in front of the UAV 100. In other examples, as further disclosed in detail herein, the UAV 100 can include more than one depth sensor and / or depth sensors that can be positioned to face other directions (e.g., backward, upward, etc.). The depth sensor 128 can include one or more devices, such as one or more cameras (e.g., a stereo vision system including two or more cameras, a time-of-flight camera (which is a sensor that can measure the depth of scene points by illuminating the scene with a controlled laser or LED source and analyzing the reflected light), etc.), infrared lasers, and / or any other device that obtains measurements using any type of detection system (such as vision, sonar, radar, lidar, etc.). The depth sensor 128 generates a depth map (sometimes referred to as a depth image) based on the objects in the field of view. In some examples, the depth sensor 128 generates depth maps at a frequency of 60 Hz (i.e., 60 depth maps per second). However, in other examples, the depth sensor 128 can generate depth maps at a higher or lower frequency. Thus, in some examples, the depth sensor 128 provides a means for obtaining depth maps.

[0031] In some examples, a depth map includes a plurality of pixels (numeric values). Each pixel can be a data element or vector (e.g., a 16-bit string) that defines the position of the corresponding pixel and the distance or depth value associated with the corresponding pixel. For example, a two-dimensional (2D) coordinate system (with coordinates X and Y) can be used to define the position. The distance value represents the distance between the depth sensor 128 and the surface(s) in the field of view corresponding to the corresponding pixel. In some examples, the distance value is defined by a value ranging from 0 to 65535 (corresponding to an unsigned 16-bit integer), where 0 means no depth (or an invalid value or a preset minimum distance from the depth sensor 128 (e.g., 10 cm)), 100 means 10 centimeters (cm), 1000 means 1 m, and 65535 means 65.535 m. In other examples, other numbering schemes can be used to define the distance value (e.g., within the precision of floating-point accuracy, a 32-bit integer can have a floating-point number from 0.0 to a number higher than 65353).

[0032] Figure 2 Example 2D image representation 200 showing an example depth map that can be generated by depth sensor 128. 2D image representation 200 shows an example vehicle. As disclosed above, the depth map includes an array of pixels, and each pixel includes a corresponding position and an associated distance (depth) value. The position can be defined based on a coordinate system. For example, in Figure 2 the zoomed-in portion of 2D image representation 200 showing a plurality of pixels 202 is shown in the callout in. In this example, an XY coordinate system is used to number the pixels 202. For example, the X-axis has been numbered starting from 1, where 1 corresponds to the leftmost column; and the Y-axis has been numbered starting from 1, where 1 corresponds to the top row. However, it is to be understood that the coordinate system can extend in any direction (left, right, up, down) and / or another coordinate system can be used. The pixel in the upper left corner has the position X = 1, Y = 1, the pixel to its right has the position X = 2, Y = 1, and so on. Additionally, as described above, each pixel has an assigned distance value that corresponds to the distance between the depth sensor 128 and the surface depicted by the corresponding pixel.

[0033] In Figure 2In it, pixels with smaller distance values (closer) are colored darker, while pixels with larger distance values (farther) are colored lighter. For example, assume the depth map represents an image of 1000 x 500 pixels. Then the depth map contains 500,000 pixels, and each pixel has an assigned distance value. In some examples, each pixel is a 16-bit data string or vector (defining the position of the pixel (in a 2D coordinate system) and the associated distance value). Thus, the depth map contains a relatively large amount of data. The collision avoidance system 126 uses one or more algorithms or mathematical calculations to analyze the pixels 202 in the depth map to identify and track the positions of one or more objects in the field of view. For example, if an object in the field of view is detected to be within a threshold distance (e.g., 1 m) of the UAV 100, the collision avoidance system 126 can prohibit the UAV 100 from flying in this direction. As described above, in some examples, the depth sensor 128 generates depth maps at a frequency of 60 Hz. Thus, a relatively large amount of data is supplied to the collision avoidance system 126. Such a large amount of data typically requires larger hardware (e.g., a larger, more robust processor) to keep up with the data volume. Otherwise, the collision avoidance system 126 will overheat and / or malfunction.

[0034] To address the above drawbacks, Figure 1 the example UAV 100 employs an example depth map modifier 130 that reduces the amount of data (e.g., pixels) in the depth map while still retaining sufficient detail in the depth map for the collision avoidance system 126 to analyze. In Figure 1 the illustrated example, the depth map modifier 130 is implemented by the processor 100. The depth map modifier 130 receives the depth map from the depth sensor 128, reduces the amount of data in the depth map and / or the generated updated depth map, and transmits the updated or modified depth map to the collision avoidance system 126 for analysis. The depth map modifier 130 can implement one or more example downscaling techniques disclosed herein for reducing the amount of pixels associated with each depth map.

[0035] Figure 3 is Figure 1 a block diagram of an example depth map modifier 130. The example depth map modifier 130 receives a depth map 300 (e.g., a first depth map) from the depth sensor 128 and generates a new depth map or an updated depth map 302 (e.g., a second depth map) with fewer pixels than the depth map 300. Figure 3 The depth map 300 of Figure 2corresponds to the image representation 200. For example, the depth map 300 may have a first plurality of pixels (e.g., 500,000 pixels), while the updated depth map 302 may have a second plurality of pixels that is less than the first plurality of pixels (e.g., 20,000). The updated depth map 302 can be considered a modified version of the original depth map 300. Thus, whenever the generation of the updated depth map 302 is described, it is to be understood that the updated depth map 302 can be a modified version of the original depth map 300 (e.g., having fewer pixels and / or altered pixel information). For example, instead of generating the updated depth map 302, the original depth map 300 can be modified by removing certain pixels and / or altering pixel information (e.g., the distance value associated with the pixel). The updated depth map 302 can then be analyzed by the collision avoidance system 126. Thus, each updated depth map has less data and, as a result, the amount of computing power required for the collision avoidance system 126 to process the updated depth map is reduced. In this way, a smaller, more compact processor can be implemented on the UAV 100, which is beneficial for reducing the overall weight, reducing the overall power consumption, etc. Additionally, by reducing the amount of computing power that the processor 100 uses for the collision avoidance system 126, more processing power can be used for other components of the UAV 100, such as for visual simultaneous localization and mapping (SLAM), object detection, video recording, etc. Reducing the computational load on the processor 100 can also reduce overheating of the processor 110.

[0036] In the illustrated example, the depth map modifier 130 includes an example block limiter 304, an example pixel distance analyzer 306, an example resizer 308, an example pixel distance assigner 310, an example depth map compiler 312, an example high-detail region limiter 314, and an example memory 316. In a first example technique, the depth map modifier 130 can reduce the size or downsample the depth map 300 to a smaller resolution based on the minimum (nearest) distance value within each pixel block. For example, the block limiter 304 applies a filter block to the depth map 300 that divides the pixels into a plurality of blocks. An example filter block size is 5x5 pixels. For example, referring back to Figure 2, four pixel blocks are depicted (each having 5x5 pixels): a first block 204 in the upper left corner, a second block 206 in the upper right corner, a third block 208 in the lower left corner, and a fourth block 210 in the lower right corner. The pixel distance analyzer 306 views the distance values associated with each of the pixels 202 in the depth map 300 and identifies the pixel with the minimum (nearest) distance value in each of the blocks 204-210. The resizer 308 downsizes or subsamples the depth map 300 based on the filter block size. For example, the first block 204 becomes one pixel (e.g., at coordinates X = 1, Y = 1 in the updated depth map 302), the second block 206 becomes one pixel (at coordinates X = 2, Y = 1 in the updated depth map 302), etc. For example, if the original depth map 300 is 1000x500 pixels, the updated (downsized) depth map 302 is 200x100 pixels. Thus, in some examples, the resizer 308 provides means for downsizing each pixel block into a corresponding single pixel. The pixel distance assigner 310 assigns the minimum (nearest) distance value of each of the blocks 204-210 to the new pixel in the updated depth map 302. For example, if pixel X = 2, Y = 2 has the minimum distance value of the pixel 202 in the first block 204, this distance value is assigned to the new pixel represented by the first block 204 (e.g., at coordinates X = 1, Y = 1 in the updated depth map 302). This process is performed for each block, thereby reducing each block to one pixel. The depth map compiler 312 generates the updated depth map 302 using the new pixels and the assigned distance values. Thus, in the illustrated example, the depth map compiler 312 provides means for generating a depth map (e.g., the updated depth map 302) based on the minimum distance values in the pixel blocks. For example, by downsizing the depth map 300 from 1000x500 pixels to 200x100 pixels, the amount of pixels is greatly reduced. Therefore, when the updated depth map 302 is provided to the collision avoidance system 126, less computation is used to analyze the depth map and detect objects. Although the use of filter blocks of size 5x5 pixels is described in the above example, in other examples, filter blocks of other sizes may be used, such as 4x4 pixels, 6x6 pixels, 10x5 pixels, etc. The filter blocks can be square (e.g., 4x4 pixels) or rectangular (e.g., 4x6 pixels). During the example downsizing technique, the depth map 300, the updated depth map 302, and / or information associated with one or more of the pixels may be stored (permanently or temporarily) in the memory 314.

[0037] In some examples, the first size reduction technique can be implemented by preserving the same two-dimensional coordinate system from the depth map 300. For example, after the pixel distance analyzer 306 identifies the minimum distance value in each of the blocks 204-210, the pixel distance assigner 310 assigns the minimum distance value in each of the blocks 204-210 to the central pixel (or coordinate) in each of the blocks 204-210. For example, if pixel X = 2, Y = 2 has the minimum distance value of pixel 202 in the first block 204, the pixel distance assigner 310 assigns this minimum distance value to the central pixel, i.e., pixel X = 3, Y = 3. The depth map compiler 312 generates the updated depth map 302 by saving only the central pixels (e.g., X = 3, Y = 3; X = 8, Y = 3; X = 3, Y = 8; X = 8, Y = X; etc.) from each of the blocks 204-210 and their assigned distance values, and discarding the other pixels in the blocks 204-210. Thus, similar to the example disclosed above, the amount of pixels is reduced from 500,000 pixels to 20,000.

[0038] Figure 4 Example 2D image representation 400 showing the updated depth map 302 modified using this first example size reduction technique. The 2D image representation 400 can correspond to the same portion of the vehicle as circled in Figure 2 As can be seen in the callout box of Figure 4 , the central pixels in each of the blocks 204-210 are colored, indicating that these pixels are saved or used in the updated depth map 302. The other pixels (uncolored) are deleted and / or otherwise removed. As can be seen in the 2D image representation 400 of Figure 4 , the saved pixels still provide a relatively accurate representation of the objects in the field of view while significantly reducing the amount of pixels.

[0039] In some examples, it may be desirable to preserve information regarding which of the pixels contributed to the closest distance value in each of the blocks. Thus, in the second example size reduction technique, the depth map modifier 130 saves the pixels in each of the blocks 204-210 that contributed to the minimum (closest) distance value in the corresponding block 204-210. For example, Figure 5 Example 2D image representation 500 showing the updated depth map 300 modified using the second example size reduction technique. The 2D image representation 500 can correspond to the same portion of the vehicle as circled in Figure 2 In this example, the pixel distance analyzer 306 identifies the pixels 202 in each of the blocks 204-210 that have the minimum (closest) distance value. In Figure 5Among them, the pixels with the minimum distance value in each of blocks 204-210 are colored (as shown in the callout box). The depth map compiler 312 creates the updated depth map 302 by only saving the pixels with the minimum distance value in each of blocks 204-210 (e.g., X = 2, Y = 2; X = 10, Y = 5; etc.) and discarding the other pixels (uncolored) in blocks 204-210. Again, this reduces the amount of pixel information in the depth map from 500,000 to 20,000.

[0040] However, as Figure 5 can be seen, this sometimes results in relatively large gaps between the pixels in adjacent blocks. For example, in the case of a continuous surface, sometimes both of two neighboring pixels are the retained minimum distance values, which results in large gaps and insufficient data between these regions. Therefore, in the third example size reduction technique, some of the pixels corresponding to the minimum distance value are saved, while the remaining pixels are assigned to the central pixel within the block. This achieves high detail in cases where the depth changes, and lower detail in cases where the surface is more continuous or flat. For example, for each pixel block of the original depth map 300, the pixel distance analyzer 306 determines the difference between the minimum and maximum distance values in the corresponding block and compares this difference with a threshold distance. A small difference indicates that the surface corresponding to the block is relatively continuous, while a large difference indicates a change in depth (such as at the edge or corner of two surfaces). If the difference for the block meets the threshold distance (e.g., is less than the threshold distance), the pixel distance assigner 310 assigns the minimum distance value to the central pixel in the corresponding block. For this block, similar to the first example size reduction technique disclosed above, the depth map compiler 312 saves the central pixel with the assigned distance value and discards the other pixels in the block. In other examples, instead of assigning the minimum distance value to the central pixel, the depth map compiler 312 saves the original central pixel with the original depth value. If the difference does not meet the threshold distance (e.g., is greater than the threshold distance), there is a relatively large change in the depth within the block. Thus, the positions of the retained minimum (nearest) distance values may be valuable. Therefore, the depth map compiler 312 saves the pixels with the nearest distance value in the block and discards the other pixels in the block. This example technique is applied to each pixel block.

[0041] Figure 6 shows an example 2D image representation 600 of the updated depth map 302 modified by the third example size reduction technique. The 2D image representation 600 can be associated with as Figure 2corresponds to the same part of the vehicle circled in [the figure]. For example, assume that the difference between the minimum distance value and the maximum distance value in the first block 204 meets a threshold distance (e.g., is below the threshold distance). In such examples, the pixel distance assigner 308 assigns the minimum distance to the center pixel X = 3, Y = 3 of the first block 204. When generating the updated depth map 302, the depth map compiler 312 saves the center pixel in the first block 204 that has the assigned distance value, while the other pixels in the first block 204 are deleted or removed. Looking at the third block 208, for example, assume that the difference between the minimum distance value and the maximum distance value in the third block 208 does not meet the threshold distance (e.g., is greater than the threshold distance). Thus, a relatively large change in depth is represented in the third block 208. In such examples, the depth map compiler 312 saves the pixel X = 4, Y = 10 in the third block 208 that has the minimum distance value and discards or deletes the other pixels in the third block 208. Again, this reduces the amount of pixels in the depth map 300 from 500,000 to 20,000. In other words, in the case of a surface that is substantially continuous or has small changes in distance, the pixel information can be assumed to be the center of the block without loss of accuracy. On the other hand, for surfaces that represent edges or regions of depth change, the positions of the pixels associated with those minimum (nearest) distance values are retained, and thus, the accuracy of these regions can be retained (where the position of the minimum (nearest) distance is important).

[0042] Although in the example size reduction techniques disclosed above, one pixel from each block is saved and / or used to generate the updated depth map 302, in other examples, more than one pixel from each block can be saved and / or used to generate the updated depth map 302. For example, when using a second size reduction technique, two pixels having two minimum distance values can be saved and / or used to generate the updated depth map 302. In another example, the pixel having the minimum distance value and the pixel having the maximum distance value can be saved from each block. In yet another example, more than two pixels from each block can be saved, or different numbers of pixels can be saved from different blocks (e.g., saving two pixels from the first block, including three pixels from the second block, etc.).

[0043] In some examples, the UAV 100 can include two or more depth sensors, and the depth map modifier 130 can perform one or more of the example size reduction techniques (disclosed above) on the depth maps generated by the multiple depth sensors. For example, Figure 7is a top view of an example UAV 100. In the illustrated example, UAV 100 includes two example depth sensors: a first depth sensor 128a facing forward; and a second depth sensor 128b facing left. In this example, the first depth sensor 128a has a field of view of approximately 90° (or a range of 45°) facing forward, and the second depth sensor 128b has a field of view of approximately 90° facing left. In some examples, the fields of view may overlap. In other examples, the fields of view may be aligned with each other, or there may be a gap between the two fields of view. The first depth sensor 128a and the second depth sensor 128b generate depth maps of their respective fields of view. A depth map modifier 128 receives the depth maps and applies one or more of the example techniques disclosed herein to reduce the data size of the depth maps, which reduces the computational load on the collision avoidance system 126.

[0044] In some examples, to help further reduce the computational load on the collision avoidance system 126, an example depth map modifier 130 may apply a concave depth filter technique based on the orientation and current direction of travel of the UAV 100. For example, as Figure 8 illustrated, the UAV 100 is flying in a forward direction (relative to the orientation of the UAV 100). In such examples, detecting (an) obstacle(s) in front of the UAV 100 that may potentially be in the flight path may be more valuable or relevant than obstacles in the left side view of the UAV 100 that are not in the flight path. Referring back to Figure 3 , the example depth map modifier 130 includes a high-detail region limiter 314. The high-detail region limiter 314 defines a region in which a higher resolution is retained based on the orientation and / or direction of travel of the UAV 100. For example, in Figure 8 , the high-detail region is shown in the direction of travel, and in the example of Figure 8 , the direction of travel is forward. In the illustrated example, the high-detail region (labeled HIGH) falls within the field of view of the first depth sensor 128a. The high-detail region can be defined by a virtual cone or camera aperture having a range such as 30° (or a total field of view of 60°) extending from the UAV 100 in the direction of travel. Regions outside the high-detail region (e.g., regions in the periphery) are considered low-detail regions (labeled LOW). As Figure 8 shown, the middle portion of the field of view of the first depth sensor 128a includes the high-detail region, while the regions outside this middle portion (left and right) are low-detail regions. Additionally, the overall field of view of the second depth sensor 128b is considered a low-detail region.

[0045] To further reduce the size or the amount of data in the depth map, in some examples, the block limiter 304 applies a first size filter block to the pixels that fall within the high-detail region and a second size filter block to the pixels outside the high-detail region (i.e., within the low-detail region). For example, for the pixels within the high-detail region of the depth map from the first depth sensor 128a, the block limiter 304 may apply a 5x5 pixel filter block. However, for the pixels within the (one or more) low-detail regions of the depth map from the first depth sensor 128a and for the entire depth map from the second depth sensor 128b, the block limiter 304 may apply a larger size (such as, 10x10 pixels) filter block. Subsequently, one or more of the example reduction techniques disclosed herein (e.g., in combination with Figure 2 and 4 the first size reduction technique disclosed in Figure 5 the second size reduction technique disclosed in Figure 6 the third size reduction technique disclosed in etc.) may be applied to reduce each of the pixel blocks to one pixel. Thus, more pixels are retained in the (one or more) high-detail regions, where detection of obstacles is more important (e.g., because they are in the flight path); while fewer pixels are retained in the (one or more) low-detail regions, where obstacles are less important; thereby further reducing the computational power required to analyze the depth map.

[0046] Figure 9 FIG. 900 shows an example 2D image representation of a depth map modified using an example concave depth filter technique. The circular region in the center of the 2D image representation 900 corresponds to the high-detail region, while the surrounding region corresponds to the low-detail region. Smaller filter blocks are applied to the pixels in the high-detail region, while larger filter blocks are applied to the pixels in the low-detail region. As such, more pixels are retained in the high-detail region, while fewer pixels are retained in the low-detail region. Depending on the orientation and the direction of travel of the UAV 100, the orientation of the high-detail region will change. For example, if the UAV 100 is traveling left, the high-detail region will point to the left, and the high-detail region and the low-detail region will swap relative to Figure 8 each other. In such examples, the entire depth map generated by the first depth sensor 128a is filtered using a larger size filter block, whereas the middle portion of the depth map generated by the second depth sensor 128b is filtered using a smaller size filter block, and the (one or more) regions outside the middle portion are filtered using a larger size filter block.

[0047] Figure 10Shows an example where the UAV 100 is moving forward and to the left. In such examples, the high-detail region includes portions of both fields of view. Thus, the left side of the depth map generated by the first depth sensor 128a will be in the high-detail region and will be processed with smaller filter blocks, while the right side of the depth map generated by the second depth sensor 128b will likewise be in the high-detail region and will be processed with smaller filter blocks. Larger-sized filter blocks can be used to analyze pixels that fall within the low-detail region of the depth map.

[0048] Figure 11 Shows an example where neither field of view includes a high-detail region. For example, in Figure 11 , the UAV 100 is flying backward or in the opposite direction. In this example, the high-detail region will be in the backward or opposite direction. However, since neither of the depth sensors 128a, 128b faces the rear direction, both fields of view are considered low-detail regions. Thus, when downsampling the depth map, the depth map modifier 130 can apply a larger-sized filter block (e.g., 10x10) to the pixels in the depth map.

[0049] In other examples, the UAV 100 can include more depth sensors and / or the depth sensors can be oriented in different directions. In some examples, the UAV 100 can include 6 cameras, one facing each direction, such as forward, backward, left, right, up, and down. In such examples, the depth sensors can provide a full 360° view of all sides of the UAV 100, including above and below. In such examples, one or some of the depth sensors will include a high-detail region, while the remaining depth sensors will include a low-detail region that can be greatly reduced to decrease the computational load on the collision avoidance system. Similarly, although in the examples above, pixels are divided into high-detail and low-detail regions, in other examples, pixels can be divided into more than two region types, and different-sized filter blocks can be used for each region (e.g., a first-sized filter block for the high-detail region, a second-sized filter block (larger than the first-sized filter block) for the intermediate or medium-detail region, and a third-sized filter block (larger than the second-sized filter block) for the low-detail region).

[0050] In some examples, the depth map modifier 130 can change the filter block size based on the future flight path. For example, referring back to Figure 8 , if the flight trajectory includes a plan to eventually fly left, instead of applying a larger filter block to the depth map of the second depth sensor 128b, the block limiter 304 can use a smaller filter block to preserve the details on the left side of the UAV 100 to track potential obstacles that may become a problem soon.

[0051] Although inFigure 1 In the illustrated example, the collision avoidance system 126 and the depth map modifier 130 are implemented by the processor 110 carried by the UAV 100. However, in other examples, the collision avoidance system 126 and / or the depth map modifier 130 may be remote to the UAV 100. For example, the UAV 100 may be part of an unmanned aerial system that communicates with a ground controller. In some such examples, the collision avoidance system 126 and / or the depth map modifier 120 may be implemented in the ground controller. For example, when the depth sensor 128 generates a depth map, the UAV 100 in such examples may transmit the depth map (e.g., via the wireless transceiver 122) to the ground controller. The depth map modifier 130 implemented in the ground controller may implement one or more of the example techniques disclosed herein for reducing the size of the depth map. The ground controller may then transmit the updated depth map back to the UAV 100, where the collision avoidance system 126 analyzes the depth map. In other examples, the collision avoidance system 126 may also be remote and remotely analyze the depth map. If a potential collision is detected, the collision avoidance system 126 may transmit a command (e.g., an instruction) back to the UAV 100 to take an appropriate course of action (e.g., change trajectory, change speed, etc.).

[0052] Although Figure 3 illustrates an example manner of implementing Figure 1 the depth map modifier 130, Figure 1 and 3 one or more of the elements, processes, and / or devices shown Figure 1Example motor controller(s) 108, example flight control system 120, example collision avoidance system 126, example depth map modifier 130 (including example block limiter 304, example pixel distance analyzer 306, example resizer 308, example pixel distance assigner 310, example depth map compiler 312, and / or example high-detail region limiter 314), and / or more generally example processor 110 can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, any one of example motor controller(s) 108, example flight control system 120, example collision avoidance system 126, example depth map modifier 130 (including example block limiter 304, example pixel distance analyzer 306, example resizer 308, example pixel distance assigner 310, example depth map compiler 312, and / or example high-detail region limiter 314), and / or more generally example processor 110 can be implemented by one or more analog or digital circuits, logic circuits, programmable processors, application specific integrated circuits (ASICs), programmable logic devices (PLDs), and / or field programmable logic devices (FPLDs). When reading any one of the apparatus or system claims of this patent that cover a pure software and / or firmware implementation, at least one of example motor controller(s) 108, example flight control system 120, example collision avoidance system 126, example depth map modifier 130 (including example block limiter 304, example pixel distance analyzer 306, example resizer 308, example pixel distance assigner 310, example depth map compiler 312, and / or example high-detail region limiter 314), and / or more generally example processor 110 is hereby expressly defined to include a non-transitory computer-readable storage device or storage disk (such as, a memory, digital versatile disk (DVD), compact disk (CD), Blu-ray disk, etc.) that contains software and / or firmware. Additionally, Figure 1 and 3 example processor 110 and / or example depth map modifier may include, as Figure 1 and 3 one or more elements, processes, and / or devices additional to or alternative to those illustrated in

[0053] In Figures 12 - 16 is shown a representation for implementing Figure 1 and3 Flowchart of example hardware logic or machine-readable instructions for processor 110 and / or depth map modifier. The machine-readable instructions can be a program or portion of a program for execution by a processor (such as processor 1712 shown in example processor platform 1700 discussed below). Although the program can be embodied in software stored on a non-transitory computer-readable storage medium such as a CD-ROM, floppy disk, hard drive, DVD, Blu-ray disc, or memory associated with processor 1712, all or portions of the program can alternatively be executed by a device other than processor 1712 and / or embodied in firmware or dedicated hardware. Additionally, although the example program is described with reference to the flowchart illustrated, many other methods can alternatively be used to implement example processor 110 and / or example depth map modifier 130. For example, the order of execution of the blocks can be changed, and / or some of the blocks described can be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks can be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGA, ASIC, comparator, operational amplifier (op-amp), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware. Figure 17 as shown in example processor platform 1700 discussed below. Although the program can be embodied in software stored on a non-transitory computer-readable storage medium such as a CD-ROM, floppy disk, hard drive, DVD, Blu-ray disc, or memory associated with processor 1712, all or portions of the program can alternatively be executed by a device other than processor 1712 and / or embodied in firmware or dedicated hardware. Additionally, although the example program is described with reference to the flowchart illustrated, many other methods can alternatively be used to implement example processor 110 and / or example depth map modifier 130. For example, the order of execution of the blocks can be changed, and / or some of the blocks described can be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks can be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGA, ASIC, comparator, operational amplifier (op-amp), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware. Figures 12 - 16 the example flowchart, many other methods can alternatively be used to implement example processor 110 and / or example depth map modifier 130. For example, the order of execution of the blocks can be changed, and / or some of the blocks described can be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks can be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGA, ASIC, comparator, operational amplifier (op-amp), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware.

[0054] As mentioned above, the example processes can be implemented using executable instructions (e.g., computer and / or machine-readable instructions) stored on a non-transitory computer and / or machine-readable medium, such as a hard drive, flash memory, read-only memory (ROM), compact disc (CD), digital versatile disc (DVD), cache, random access memory (RAM), and / or any other storage device or storage disk that stores information therein for any duration (e.g., over an extended period of time, permanently, during a brief instance, during temporary buffering and / or information caching). As used herein, the term non-transitory computer-readable medium is expressly defined to include any type of computer-readable storage device and / or storage disk and to exclude propagated signals and to exclude transmission media. Figures 12 - 16 such as a hard drive, flash memory, read-only memory (ROM), compact disc (CD), digital versatile disc (DVD), cache, random access memory (RAM), and / or any other storage device or storage disk that stores information therein for any duration (e.g., over an extended period of time, permanently, during a brief instance, during temporary buffering and / or information caching). As used herein, the term non-transitory computer-readable medium is expressly defined to include any type of computer-readable storage device and / or storage disk and to exclude propagated signals and to exclude transmission media.

[0055] "Comprising" and "including" (and all forms and tenses thereof) are used herein as open-ended terms. Thus, whenever a claim uses any form of "comprising" or "including" (e.g., includes, comprises, etc.) as a preamble or within any kind of claim recitation, it is to be understood that additional elements, items, etc. may exist without falling outside the scope of the corresponding claim or recitation. As used herein, when the phrase "at least" is used as a transitional term, for example, in conjunction with a claim, it is as open-ended as the terms "comprising" and "including". When the term "and / or" is used in forms such as A, B, and / or C, it refers to any combination or subset of A, B, and C, such as (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, and (6) B and C.

[0056] Figure 12 is an example flowchart of machine-readable instructions that may be executed by a processor 110 of the UAV 100 to implement Figure 1 and 3 an example depth map modifier for reducing the size of a depth map. At block 1202, the example depth map modifier 130 receives a depth map 300 (e.g., a first depth map) from a depth sensor 128. At block 1204, the depth map modifier 130 generates an updated depth map 302 (e.g., a second depth map) based on the depth map 300 using one or more of the example size reduction techniques disclosed herein. An example flowchart representing the example size reduction technique that may be implemented as block 1204 is disclosed in conjunction with Figures 13 - 15 below. The updated depth map 302 is a modified version of the depth map 300 and includes fewer pixels than the depth map 300.

[0057] At block 1206, the depth map modifier 130 provides the updated depth map 302, which has less data than the originally generated depth map 300, to a collision avoidance system 126. The collision avoidance system 126 analyzes the updated depth map 302 to track the distances of the object(s) near the UAV 100. If a potential collision is detected, the collision avoidance system 126 may communicate with a flight control system 120 to perform at least one of the following operations: change the trajectory or change the speed of the UAV 100. In some examples, the UAV 100 may transmit an alert back to a remote controller to alert a user of a potential collision.

[0058] At block 1208, the depth map modifier 130 determines whether another depth map is received from the depth sensor 128. If so, the example process begins again. As described above, in some examples, the depth sensor 128 generates depth maps at a relatively high frequency (such as 60 Hz). Each time a depth map is generated, the example depth map modifier 130 may use Figure 12An example process to reduce the size of the depth map. If the depth map is not received (e.g., the UAV 100 is no longer flying), then Figure 12 the example process ends.

[0059] Figure 13 is a flowchart 1300 of example machine-readable instructions that can be executed by the processor 110 of the UAV 100 to implement Figure 12 block 1204. Specifically, flowchart 1300 represents an example process that can be used to reduce the size of the depth map using the first example size reduction technique disclosed above in connection with Figure 4 the disclosure.

[0060] At block 1302, the block limiter 304 applies a filter block to the depth map 300, which divides the pixels into multiple blocks (e.g., Figure 2 and 4 blocks 204-210 shown in Figure 4 . For example, in

[0061] the filter block size is 5x5 pixels. In other examples, the filter block size can be larger or smaller.

[0062] At block 1304, the pixel distance analyzer 306 identifies the pixels in each pixel block (e.g., in each of blocks 204-210) that have the minimum distance value (i.e., representing the closest surface to the UAV 100). At block 1306, the pixel distance assigner 310 assigns the minimum distance value identified (at block 1304) in each block to the center pixel or center coordinates of each block. Thus, in some examples, the pixel distance assigner 310 provides means for assigning the minimum distance value identified in each block to the center pixel or center coordinates of the corresponding block. Figure 12 At block 1308, the depth map compiler 312 generates an updated depth map 302 by saving the center pixel from each block and its assigned distance value and discarding (e.g., deleting) the other pixels in each block. In

[0063] Figure 14 block 1206 of Figure 12 the updated depth map 302 is output to the collision avoidance system 126. Figure 5 is a flowchart 1400 of example machine-readable instructions that can be executed by the processor 110 of the UAV 100 to implement

[0064] block 1204. Specifically, flowchart 1400 represents an example process that can be used to reduce the size of the depth map using the second example size reduction technique disclosed above in connection with Figure 5block 204-210). For example, in Figure 5 the filter block size is 5x5 pixels. In other examples, the filter block size can be larger or smaller. Thus, in some examples, the block limiter 304 provides means for applying the filter block to the depth map.

[0065] At block 1404, the pixel distance analyzer 306 identifies the pixel in each pixel block (e.g., in each of blocks 204-210) having the minimum distance value (i.e., representing the closest surface to the UAV 100). Thus, in some examples, the pixel distance analyzer 306 provides means for identifying the minimum distance value in a pixel block. At block 1406, the depth map compiler 312 generates the updated depth map 302 by saving the pixels having the minimum distance value in each block (identified at block 1404) and discarding the other pixels in each block. In other words, some of the pixels from the depth map 300 (along with their original positions and distance values) are used in the updated depth map 302, while some of the other pixels are discarded. In Figure 12 block 1206, the updated depth map 302 is output to the collision avoidance system 126.

[0066] Figure 15 is a flowchart 1500 of example machine-readable instructions that may be executed by a processor 110 of the UAV 100 to implement Figure 12 block 1204. Specifically, the flowchart 1500 represents an example process that may be used to reduce the size of a depth map using the third example downscaling technique disclosed above in connection with Figure 6 At block 1502, the block limiter 304 applies a filter block to the depth map 300, which divides the pixels into multiple blocks (e.g.,

[0067] block 204-210). For example, in Figure 6 the filter block size is 5x5 pixels. In other examples, the filter block size can be larger or smaller. Figure 6 the filter block size is 5x5 pixels. In other examples, the filter block size can be larger or smaller.

[0068] At block 1504, the pixel distance analyzer 306 identifies the pixels having the minimum distance value (i.e., representing the closest surface to the UAV 100) and the pixels having the maximum distance value in each of the blocks (e.g., each of blocks 204 - 210). At block 1506, for a given pixel block in the original depth map 300, the pixel distance analyzer 306 compares the difference between the minimum distance value and the maximum distance value of the block with a threshold distance. At block 1508, the pixel distance analyzer 306 determines whether the difference meets the threshold distance. If the difference meets the threshold distance (e.g., is below the threshold distance), then at block 1510, the pixel distance assigner 310 assigns the identified minimum distance value in the block to the center pixel (or center coordinates) in the block (e.g., similar to the operation disclosed in Figure 13 ). If the difference does not meet the threshold distance (e.g., exceeds the threshold distance), then at block 1512, the pixel distance assigner 310 saves the pixel associated with the minimum distance value in the block, thereby retaining the position information. Thus, the updated depth map 302 includes the original pixels (and their original position and distance value information), while the other pixels are discarded.

[0069] At block 1514, the exemplary pixel distance analyzer 306 determines whether there is another block to analyze. If so, the process in blocks 1506 - 1512 is repeated. Once all blocks have been analyzed, at block 1516, the depth map compiler 312 generates the updated depth map 302 by saving the center pixels and their assigned distance values for the blocks that meet the threshold distance, saving the pixels having the minimum distance value for the blocks that do not meet the threshold distance, and discarding the other pixels in the blocks. In Figure 12 block 1206, the updated depth map 302 is output to the collision avoidance system 126.

[0070] Figure 16 is a flowchart 1600 of example machine - readable instructions that may be executed by a processor 110 of the UAV 100 to implement Figure 1 and 3 . Specifically, Figure 16 represents an example process using the example concave depth filter technique as described in Figures 7 - 11 .

[0071] At block 1602, example depth map modifier 130 receives depth maps from one or more depth sensors, such as depth sensors 128a, 128b. At block 1604, high-detail region limiter 314 determines the orientation and current direction of travel of UAV 100. In some examples, high-detail region limiter 314 determines the orientation and direction of travel based on inputs from sensor system 114 and / or flight control system 120. Thus, in some examples, high-detail region limiter 314 is by means for determining the direction of travel of UAV 100. At block 1606, high-detail region limiter 314 defines a high-detail region based on the orientation and direction of travel. Specifically, the high-detail region is oriented in the direction of travel (based on the UAV reference frame). The high-detail region may be represented, for example, by a virtual cone extending from UAV 100 in the direction of travel.

[0072] At block 1608, high-detail region limiter 314 identifies or determines the regions within the depth maps that fall within the high-detail region as high-detail regions, while identifying or determining the other regions (falling outside the high-detail region) as low-detail regions. Thus, in some examples, high-detail region limiter 314 is by means for identifying high-detail regions and / or low-detail regions within pixels of a depth map. At block 1610, block limiter 304 applies a first size filter block, such as a 5x5 filter block, to the high-detail regions and a second size filter block, such as a 10x10 filter block, to the low-detail regions. At block 1612, depth map compiler 312 uses one or more of the example downscaling techniques disclosed herein (e.g., as disclosed in connection with Figures 13 - 15 to generate one or more updated depth maps. Thus, less information is preserved in the regions with lower detail.

[0073] At block 1614, depth map modifier 130 provides the updated depth maps to collision avoidance system 126. Collision avoidance system 126 analyzes the updated depth maps to track the distances of objects near UAV 100. If a potential collision is detected, collision avoidance system 126 may communicate with flight control system 120 to perform at least one of the following: change the trajectory, or change the speed of UAV 100. In some examples, UAV 100 may transmit an alert back to the remote controller to alert the user of a potential collision.

[0074] At block 1616, the depth map modifier 130 determines whether one or more additional depth maps are received from the depth sensor(s). If so, the example process begins again. As noted above, in some examples, the depth sensor generates depth maps at a relatively high frequency (such as, 60 Hz). Each time a depth map is generated, the example depth map modifier can use the example techniques of Figure 16 to reduce the size of the depth map. If no depth map is received (e.g., the UAV 100 is no longer flying), then Figure 16 the example process ends.

[0075] Figure 17 is constructed to execute the instructions of Figures 12 - 16 to implement Figure 1 and 3 is a block diagram of an example processor platform 1700 of the processor 110 and / or the depth map modifier 130. The processor platform 1700 can be, for example, a printed circuit board, an aircraft (e.g., the UAV 100), or any other type of computing device that can be implemented by a server, a personal computer, a workstation, a mobile device (e.g., a cellular phone, a smart phone, a tablet such as an iPad TM and the like).

[0076] The illustrated example of the processor platform 1700 includes a processor 1712. The illustrated example of the processor 1712 is hardware. For example, the processor 1712 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor can be a semiconductor-based (e.g., silicon-based) device. In this example, the processor 1712 can implement the example motor controller 108, the example flight control system 120, the example collision avoidance system 126, the example depth map modifier 130 (including the example block limiter 304, the example pixel distance analyzer 306, the example resizer 308, the example pixel distance assigner 310, the example depth map compiler 312, and / or the example high-detail region limiter 314), and / or more generally the example processor 110.

[0077] The illustrated example of the processor 1712 includes local memory 1713 (e.g., a cache). The illustrated example of the processor 1712 communicates via a bus 1718 with a main memory that includes volatile memory 1714 and non-volatile memory 1716. The volatile memory 1714 can be synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), dynamic random access memory and / or any other type of random access memory device. The non-volatile memory 1716 can be implemented by flash memory and / or any other desired type of memory device. Access to the main memories 1714, 1716 is controlled by a memory controller.

[0078] The processor platform 1700 of the illustrated example also includes interface circuitry 1720. The interface circuitry 1720 can be implemented by any type of interface standard, such as an Ethernet interface, Universal Serial Bus (USB), Bluetooth interface, Near Field Communication (NFC) interface, and / or a PCI Express interface.

[0079] In the illustrated example, one or more input devices 1722 are connected to the interface circuitry 1720. In this example, the input device(s) 1722 can include the example sensor system 114, including the example IMU 116 and / or the example GPS sensor 118, the example camera 124, and / or the example depth sensor(s) 128. Additionally or alternatively, the input device(s) 1722 allow a user to input data and / or commands into the processor 1712.

[0080] One or more output devices 1724 are also connected to the interface circuitry 1720 of the illustrated example. In this example, the output device(s) 1024 can include the motors 102a - 102h. Additionally or alternatively, the output device(s) 1024 can be implemented, for example, by a display device (e.g., a light-emitting diode (LED), an organic light-emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-plane switching (IPS) display, a touch screen, etc.), a haptic output device, a printer, and / or a speaker. Thus, in some examples, the interface circuitry 1720 can include a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0081] The interface circuitry 1720 of the illustrated example also includes communication devices such as a transmitter, a receiver, a transceiver (e.g., the wireless transceiver 122), a modem, a residential gateway, a wireless access point, and / or a network interface to facilitate the exchange of data with external machines (e.g., any type of computing device) via a network 1726. The communication can be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-sight wireless system, a cellular telephone system, etc.

[0082] The processor platform 1700 of the illustrated example also includes one or more mass storage devices 1728 for storing software and / or data. Examples of such mass storage devices 1728 include floppy disk drives, hard disk drives, compact disc drives, Blu-ray disc drives, redundant array of independent disks (RAID) systems, and digital versatile disc (DVD) drives. The mass storage device may include, for example, the memory 316.

[0083] The machine-executable instructions 1732 of FIG. 1216 may be stored in the mass storage device 1728, stored in the volatile memory 1714, stored in the non-volatile memory 1716, and / or stored on a removable non-transitory computer-readable storage medium such as a CD or DVD.

[0084] From the foregoing, it will be appreciated that example methods, apparatuses, systems, and articles have been disclosed for reducing or scaling down a depth map to reduce the computational load imposed on a collision avoidance system while still ensuring that high accuracy can be achieved. Thus, less computational power is required to analyze the depth map, enabling a smaller, lighter processor to be used to implement the collision avoidance system. UAVs desire smaller, lighter processors to reduce weight and power consumption (which is limited in UAVs). Additionally, by reducing the computational load on the processor, the examples disclosed herein enable the processor power to be alternatively used for other tasks. The examples disclosed herein can be implemented using collision avoidance systems used in any industry, such as collision avoidance systems on UAVs (fixed-wing or rotary-wing), manned vehicles, autonomous vehicles (e.g., cars, trucks, etc.), and robots.

[0085] Example methods, apparatuses, systems, and articles for reducing the size of a depth map are disclosed herein. Further examples and combinations thereof include the following:

[0086] Example 1 includes an unmanned aerial vehicle including a depth sensor for generating a first depth map. The first depth map includes a plurality of pixels having respective distance values. The unmanned aerial vehicle of Example 1 also includes a depth map modifier for dividing the plurality of pixels into pixel blocks; and generating a second depth map having fewer pixels than the first depth map based on the distance values of the pixels in the pixel blocks. The unmanned aerial vehicle of Example 1 further includes a collision avoidance system for analyzing the second depth map.

[0087] Example 2 includes the unmanned aerial vehicle of Example 1, wherein the depth map modifier is for generating the second depth map by reducing a respective block size to a single pixel, the single pixel having an assigned distance value corresponding to a respective minimum distance value within the respective block.

[0088] Example 3 includes the unmanned aerial vehicle of Example 1, wherein, in order to generate a second depth map, a depth map modifier is used to assign the minimum distance value in each block to the central pixel of the corresponding block, save the central pixel having the assigned minimum distance value, and discard the other pixels in the block.

[0089] Example 4 includes the unmanned aerial vehicle of Example 1, wherein the depth map modifier generates a second depth map by saving the corresponding pixels having the minimum distance value from the corresponding block in the block and discarding the other pixels in the block.

[0090] Example 5 includes the unmanned aerial vehicle of Example 1, wherein the depth map modifier is used to generate a second depth map by comparing the difference between the minimum distance value and the maximum distance value in each block with a threshold distance.

[0091] Example 6 includes the unmanned aerial vehicle of Example 5, wherein, if the difference of a block meets the threshold distance, the depth map modifier is used to assign the minimum distance value in the block to the central pixel in the block, save the central pixel, and discard the other pixels in the block; and if the difference of a block does not meet the threshold distance, the depth map modifier is used to save the pixel having the minimum distance value in the block and discard the other pixels in the block.

[0092] Example 7 includes the unmanned aerial vehicle of any one of Examples 1-6, wherein the depth map modifier is used to determine the traveling direction of the unmanned aerial vehicle; determine a high-detail region and a low-detail region within a plurality of pixels of the first depth map based on the traveling direction; and divide the pixels in the high-detail region into pixel blocks by using a first size filter block, while dividing the pixels in the low-detail region into pixel blocks by using a second size filter block, the second size filter block having a larger filter block than the first size filter block.

[0093] Example 8 includes the unmanned aerial vehicle of Example 7, wherein the depth map has a field of view, and the high-detail region corresponds to a virtual cone projected from the unmanned aerial vehicle in the traveling direction, the virtual cone being smaller than the field of view.

[0094] Example 9 includes the unmanned aerial vehicle of any one of 1-8, wherein the depth sensor is a first depth sensor, and further includes a second depth sensor, the second depth sensor facing a different direction from the first depth sensor.

[0095] Example 10 includes the unmanned aerial vehicle of Example 9, wherein the second depth sensor is used to generate a third depth map, the depth map modifier is used to generate a fourth depth map based on the third depth map, the fourth depth map having fewer pixels than the third depth map, and the collision avoidance system is used to analyze the fourth depth map.

[0096] Example 11 includes an unmanned aerial vehicle of any one of Examples 1-10, wherein the collision avoidance system is configured to modify at least one of the following in the event of a detected potential collision: the trajectory or speed of the unmanned aerial vehicle.

[0097] Example 12 includes a method for reducing the size of a depth map, the method comprising: applying a filter block to a plurality of pixels of a first depth map to divide the plurality of pixels into pixel blocks by executing instructions using at least one processor, wherein the pixels of the first depth map have corresponding distance values; identifying a corresponding minimum distance value associated with the pixels in a corresponding block by executing instructions using at least one processor; and generating a second depth map based on the minimum distance values in the pixel blocks by executing instructions using at least one processor, wherein the second depth map has fewer pixels than the first depth map.

[0098] Example 13 includes the method of Example 12, wherein generating the second depth map includes reducing the size of each pixel block to one pixel, which one pixel has an assigned distance value corresponding to the minimum distance value within the pixel block.

[0099] Example 14 includes the method of Example 12, wherein generating the second depth map includes: assigning the minimum distance value in each corresponding pixel block to the central pixel of the corresponding block; saving the central pixel having the assigned minimum distance value; and discarding the other pixels in the block.

[0100] Example 15 includes the method of Example 12, wherein generating the second depth map includes: saving the pixels having the minimum distance value from the corresponding pixel blocks; and discarding the other pixels in the blocks.

[0101] Example 16 includes the method of Example 12, wherein generating the second depth map includes: for each corresponding block, determining a difference between the minimum distance value and the maximum distance value of the pixels in the corresponding block to determine a difference for the corresponding pixel block; and comparing the difference with a threshold distance. If the difference of the corresponding block meets the threshold distance, the method includes assigning the minimum distance value in the corresponding block to the central pixel in the corresponding block, saving the central pixel of the corresponding block, and discarding the other pixels in the corresponding block. If the corresponding difference of the corresponding block does not meet the threshold distance, the method includes saving the pixels having the minimum distance value in the corresponding block and discarding the other pixels in the corresponding block.

[0102] Example 17 includes the method of any one of Examples 12-16, further comprising obtaining the first depth map using a depth sensor carried on a vehicle.

[0103] Example 18 includes the method of Example 17, further comprising: determining a direction of travel of a vehicle by executing instructions using at least one processor; and identifying a high-detail region and a low-detail region within a first plurality of pixels based on the direction of travel by executing instructions using at least one processor, and wherein applying the filter blocks includes applying a first-size filter block to pixels within the high-detail region and applying a second-size filter block to pixels within the low-detail region, the second-size filter block being larger than the first-size filter block.

[0104] Example 19 includes the method of Example 17, wherein the vehicle is an unmanned aerial vehicle.

[0105] Example 20 includes the method of Example 19, further comprising analyzing a second depth map via a collision avoidance system to monitor the position of an object near the unmanned aerial vehicle.

[0106] Example 21 includes a non-transitory machine-readable storage medium including instructions that, when executed, cause one or more processors to at least: apply filter blocks to a first plurality of pixels of a first depth map to divide the first plurality of pixels into pixel blocks, the first plurality of pixels having corresponding distance values; identify a corresponding minimum distance value associated with pixels in a corresponding block; and generate a second depth map based on the minimum distance values in the pixel blocks, wherein the second depth map has a second plurality of pixels that is less than the first plurality of pixels.

[0107] Example 22 includes the non-transitory machine-readable storage medium of Example 21, wherein the instructions, when executed, cause one or more processors to generate the second depth map by: reducing the size of each pixel block to one pixel having an assigned distance value corresponding to the minimum distance value within the pixel block.

[0108] Example 23 includes the non-transitory machine-readable storage medium of Example 21, wherein the instructions, when executed, cause one or more processors to generate the second depth map by: assigning the minimum distance value in each corresponding pixel block to the center pixel of the corresponding block; saving the center pixel having the assigned minimum distance value; and discarding the other pixels in the block, the second plurality of pixels corresponding to the saved center pixels having the assigned minimum distance value.

[0109] Example 24 includes the non-transitory machine-readable storage medium of Example 21, wherein the instructions, when executed, cause one or more processors to generate the second depth map by: saving the pixels having the minimum distance value from the corresponding pixel blocks; and discarding the other pixels in the block, the second plurality of pixels corresponding to the saved pixels.

[0110] Example 25 includes the non-transitory machine-readable storage medium of Example 21, wherein when the instructions are executed, one or more processors generate a second depth map by: for each respective block, determining a difference between a minimum distance value and a maximum distance value of pixels in the respective block to determine a difference for the respective pixel block; comparing the difference with a threshold distance, and if the difference of the respective block meets the threshold distance, assigning the minimum distance value in the respective block to a central pixel in the respective block, saving the central pixel of the respective block, and discarding other pixels in the respective block, and if the respective difference of the respective block does not meet the threshold distance, saving the pixel with the minimum distance value in the respective block and discarding other pixels in the respective block..

[0111] Example 26 includes the non-transitory machine-readable storage medium of Examples 21-25, wherein the first depth map is obtained by a depth sensor carried on a vehicle.

[0112] Example 27 includes the non-transitory machine-readable storage medium of Example 26, wherein when the instructions are executed, one or more processors: determine a traveling direction of the vehicle; determine a high-detail region and a low-detail region within a plurality of pixels of the first depth map based on the traveling direction; and apply filter blocks by applying a first-size filter block to pixels within the high-detail region and applying a second-size filter block to pixels within the low-detail region, wherein the second-size filter block is larger than the first-size filter block.

[0113] Example 28 includes the non-transitory machine-readable storage medium of Example 26, wherein the vehicle is an unmanned aerial vehicle.

[0114] Example 29 includes an apparatus, comprising: means for applying filter blocks to a first plurality of pixels of a first depth map to divide the first plurality of pixels into pixel blocks, the first plurality of pixels having respective distance values; means for identifying a respective minimum distance value associated with pixels in the respective block; and means for generating a second depth map based on the minimum distance value in the pixel blocks, wherein the second depth map has a second plurality of pixels, and the second plurality of pixels is less than the first plurality of pixels.

[0115] Example 30 includes the apparatus of Example 29, further comprising means for reducing the size of each pixel block to one pixel, the one pixel having an assigned distance value corresponding to the minimum distance value within the pixel block; and wherein the means for generating the second depth map is for generating the second depth map based on the reduced pixel blocks.

[0116] Example 31 includes the apparatus of Example 29, further comprising means for assigning the minimum distance value in each respective pixel block to the central pixel of the respective block, and wherein the means for generating the second depth map is for saving the central pixel having the assigned minimum distance value and discarding the other pixels in the block, wherein the second plurality of pixels corresponds to the saved central pixels having the assigned minimum distance value.

[0117] Example 32 includes the apparatus of Example 29, wherein the means for generating the second depth map is for saving the pixels having the minimum distance value from the respective pixel block and discarding the other pixels in the block, wherein the second plurality of pixels corresponds to the saved pixels.

[0118] Example 33 includes the apparatus of Example 29, wherein the means for identifying is for: for each respective block, determining the difference between the minimum distance value and the maximum distance value of the pixels in the respective block to determine the difference of the respective pixel block; and comparing the difference with a threshold distance. If the respective difference of the respective block meets the threshold distance, the means for assigning is for assigning the minimum distance value in the respective block to the central pixel in the respective block, and the means for generating the second depth map is for saving the central pixel of the respective block and discarding the other pixels in the respective block. If the respective difference of the respective block does not meet the threshold distance, the means for generating the second depth map is for saving the pixels having the minimum distance value in the respective block and discarding the other pixels in the respective block.

[0119] Example 34 includes the apparatus of any one of Examples 29 - 33, further comprising means for obtaining a first depth map carried on a vehicle.

[0120] Example 35 includes the apparatus of Example 34, further comprising: means for determining the traveling direction of the vehicle; and means for determining a high - detail area and a low - detail area within the plurality of pixels of the first depth map based on the traveling direction, wherein the means for applying applies a first - sized filter block to the pixels within the high - detail area and a second - sized filter block to the pixels within the low - detail area, wherein the second - sized filter block is larger than the first - sized filter block.

[0121] Example 36 includes the apparatus of Example 34, wherein the vehicle is an unmanned aerial vehicle.

[0122] Example 37 includes an unmanned aerial vehicle system, comprising an unmanned aerial vehicle having: a depth sensor for obtaining a first depth map, the first depth map including a plurality of pixels having respective distance values; a depth map modifier for dividing the plurality of pixels into pixel blocks and generating a second depth map having fewer pixels than the first depth map based on the distance values of the pixels in the pixel blocks; and a collision avoidance system for analyzing the second depth map to track the position of an object relative to the unmanned aerial vehicle.

[0123] Example 38 includes the unmanned aerial vehicle system of Example 37, wherein the depth map modifier is implemented by a processor carried on the unmanned aerial vehicle.

[0124] Example 39 includes the unmanned aerial vehicle system of Example 37 or 38, wherein the collision avoidance system is implemented by a processor carried on the unmanned aerial vehicle system.

[0125] Although certain example methods, devices, systems, and articles have been disclosed herein, the scope covered by this patent is not limited thereto. Instead, this patent covers all methods, devices, systems, and articles that fall within the scope of the claims of this patent.

Claims

1. An unmanned aerial vehicle, comprising: A depth sensor for generating a first depth map, the first depth map including a plurality of pixels having respective distance values; A depth map modifier for: Dividing the plurality of pixels into pixel blocks; And Generating a second depth map having fewer pixels than the first depth map based on the distance values of the pixels in the pixel blocks; And A collision avoidance system for analyzing the second depth map; Wherein, the depth map modifier is used to generate the second depth map by comparing the difference between the minimum distance value and the maximum distance value in each block with a threshold distance.

2. The unmanned aerial vehicle according to claim 1, characterized in that: If the difference for a certain block satisfies the threshold distance, the depth map modifier is used to assign the minimum distance value in the block to the central pixel in the block, save the central pixel, and discard the other pixels in the block; And If the difference for a certain block does not satisfy the threshold distance, the depth map modifier is used to save the pixel having the minimum distance value in the block and discard the other pixels in the block.

3. The unmanned aerial vehicle according to any one of claims 1-2, characterized in that, The depth map modifier is used to: Determine the traveling direction of the unmanned aerial vehicle; Determine a high-detail area and a low-detail area within the plurality of pixels of the first depth map based on the traveling direction; and Divide the pixels in the high-detail area into pixel blocks by using a first-size filter block, and divide the pixels in the low-detail area into pixel blocks by using a second-size filter block, the second-size filter block having a larger filter block than the first-size filter block.

4. The unmanned aerial vehicle according to claim 3, characterized in that, The depth sensor has a field of view, and the high-detail area corresponds to a virtual cone projected from the unmanned aerial vehicle in the traveling direction, the virtual cone being smaller than the field of view.

5. The unmanned aerial vehicle according to any one of claims 1-2, characterized in that, The depth sensor is a first depth sensor, and further includes a second depth sensor, the second depth sensor facing a different direction from the first depth sensor.

6. The unmanned aerial vehicle according to claim 5, characterized in that, The second depth sensor is used to generate a third depth map, the depth map modifier is used to generate a fourth depth map based on the third depth map, the fourth depth map having fewer pixels than the third depth map, and the collision avoidance system is used to analyze the fourth depth map.

7. The unmanned aerial vehicle according to any one of claims 1-2, characterized in that, The collision avoidance system is used to modify at least one of the following in the case of detecting a potential collision: the trajectory or speed of the unmanned aerial vehicle.

8. A method for reducing the size of a depth map, the method comprising: Applying a filter block to a plurality of pixels of a first depth map by executing instructions using at least one processor to divide the plurality of pixels into pixel blocks, the pixels of the first depth map having respective distance values; Identifying a respective minimum distance value associated with the pixels in the respective block by executing instructions using the at least one processor in the respective block; And Generating a second depth map having fewer pixels than the first depth map by executing instructions using the at least one processor based on the distance values of the pixels in the pixel blocks; Among them, generating the second depth map includes: For each corresponding block, determining the difference between the minimum distance value and the maximum distance value of the pixels in the corresponding block to determine the difference of the corresponding pixel block; Comparing the difference with a threshold distance to generate the second depth map.

9. The method according to claim 8, wherein Generating the second depth map includes: If the corresponding difference of the corresponding block satisfies the threshold distance, then: Assigning the minimum distance value in the corresponding block to the central pixel in the corresponding block; Saving the central pixel of the corresponding block; and Discarding the other pixels in the corresponding block; and If the corresponding difference of the corresponding block does not satisfy the threshold distance, then: Saving the pixel with the minimum distance value in the corresponding block; and Discarding the other pixels in the corresponding block.

10. The method according to any one of claims 8-9, characterized in that, Further includes obtaining the first depth map by using a depth sensor carried on a vehicle.

11. The method according to claim 10, further comprising: Determining the traveling direction of the vehicle by executing instructions by using the at least one processor; And Identifying a high-detail region and a low-detail region within the first plurality of pixels based on the traveling direction by executing instructions by using the at least one processor, and wherein applying the filter block includes applying a first-size filter block to the pixels within the high-detail region and applying a second-size filter block to the pixels within the low-detail region, the second-size filter block being larger than the first-size filter block.

12. The method according to claim 10, wherein The vehicle is an unmanned aerial vehicle.

13. The method according to claim 12, further comprising: Analyzing the second depth map via a collision avoidance system to monitor the positions of objects near the unmanned aerial vehicle.

14. A non-transitory machine-readable storage medium including instructions that, when executed, cause one or more processors to at least: Apply a filter block to a first plurality of pixels of a first depth map to divide the first plurality of pixels into pixel blocks, the first plurality of pixels having corresponding distance values; Identify a corresponding minimum distance value associated with the pixels in the corresponding block; And Generate a second depth map based on the distance values of the pixels in the pixel blocks, the second depth map having a second plurality of pixels, the second plurality of pixels being fewer than the first plurality of pixels; Wherein, when the instructions are executed, the one or more processors generate the second depth map by: For each corresponding block, determining the difference between the minimum distance value and the maximum distance value of the pixels in the corresponding block to determine the difference of the corresponding pixel block; Comparing the difference with a threshold distance to generate the second depth map.

15. The non-transitory machine-readable storage medium according to claim 14, wherein When the instructions are executed, the one or more processors generate the second depth map by: If the corresponding difference of the corresponding block satisfies the threshold distance, then: Assigning the minimum distance value in the corresponding block to the central pixel in the corresponding block; Saving the central pixel of the corresponding block; and Discarding the other pixels in the corresponding block; And If the corresponding difference of the corresponding block does not satisfy the threshold distance, then: Save the pixels in the corresponding block that have the minimum distance value; and Discard the other pixels in the corresponding block.

16. The non-transitory machine-readable storage medium according to claim 14 or 15, characterized in that The first depth map is obtained by a depth sensor carried on a vehicle.

17. An apparatus for reducing the size of a depth map, comprising: Means for applying filter blocks to a first plurality of pixels of a first depth map to partition the first plurality of pixels into pixel blocks, the first plurality of pixels having corresponding distance values; Means for identifying a corresponding minimum distance value associated with the pixels in the corresponding block; And Means for generating a second depth map based on the distance values of the pixels in the pixel blocks, the second depth map having a second plurality of pixels, the second plurality of pixels being fewer than the first plurality of pixels; Wherein the means for identifying is configured to: For each corresponding block, determine a difference between the minimum distance value and the maximum distance value of the pixels in the corresponding block to determine a difference of the corresponding pixel block; and Compare the difference with a threshold distance to generate the second depth map.

18. The device according to claim 17, characterized in that, The means for identifying is configured to: If the corresponding difference of the corresponding block satisfies the threshold distance, the means for assignment is configured to assign the minimum distance value in the corresponding block to a central pixel in the corresponding block, and the means for generating the second depth map is configured to save the central pixel of the corresponding block and discard the other pixels in the corresponding block; And If the corresponding difference of the corresponding block does not satisfy the threshold distance, the means for generating the second depth map is configured to save the pixels in the corresponding block that have the minimum distance value and discard the other pixels in the corresponding block.

19. The device according to claim 17 or 18, characterized in that, Further comprising means for obtaining the first depth map carried on a vehicle.

Images