Multi-mode stereo matching for determining depth information

Abstract

Systems and techniques for determining disparity information are described. For example, a method for determining disparity information is provided. The method comprises the following steps: obtaining a plurality of cost functions; determining a first parallax for a center pixel of the first pixel region at least in part by comparing one or more minimum values of a cost function of the center pixel with one or more minimum values of cost functions of other pixels in the first region; determining, at least in part, a second parallax for a center pixel of a second region comprising the center pixel of the first region by comparing one or more minimum values of a cost function of the center pixel with one or more minimum values of cost functions of other pixels in the second region; and determining a third parallax for the center pixel of the first region based on the first parallax and the second parallax.

Description

Technical Field

[0001] The present disclosure generally relates to determining depth information. In some examples, aspects of the present disclosure relate to systems and techniques for determining depth information using multimodal stereo matching, such as by tracking multiple disparity hypotheses when performing stereo matching. Background Art

[0002] Stereoscopic images of a scene can be used to determine depth information relative to the scene. A stereoscopic image may include two images captured substantially simultaneously by two cameras with slightly different views of the scene. Stereoscopic images simulate the slightly different perspectives of a scene captured by a person's two eyes. In addition to providing depth information relative to a scene, stereoscopic images can also be used to generate a three-dimensional model of the scene. When stereoscopic images are captured by two cameras, pixels in each of the two images typically correspond to the same object within the scene, and in many cases, pixels in one image can be correlated with pixels in the second image. Summary of the Invention

[0003] In some examples, systems and techniques for determining depth information using multimodal stereo matching are described. According to at least one example, a method for determining disparity information is provided. The method includes: obtaining a plurality of cost functions, the plurality of cost functions including a corresponding cost function for each of a plurality of pixels of a first image, wherein the corresponding cost function for each of the plurality of pixels includes an indication of similarity between a window including the pixel and a corresponding window of a second image as a function of disparity along an epipolar line in the second image; determining a first disparity for a center pixel of a first pixel region at least in part by comparing one or more minima of the cost function for the center pixel of the first pixel region with one or more minima of cost functions for other pixels in the first region, the plurality of cost functions including the corresponding cost function for the center pixel of the first pixel region. cost function and the cost functions of the other pixels in the first area; determining a second disparity for the center pixel of the first area at least in part by comparing one or more minimum values ​​of the cost function of the center pixel of the second area with one or more minimum values ​​of the cost functions of the other pixels in the second area, the second pixel area including the center pixel of the first area, the multiple cost functions including the cost function of the center pixel of the second area and the cost functions of the other pixels in the second area; and determining a third disparity for the center pixel of the first area based on the first disparity of the center pixel of the first area and the second disparity of the center pixel of the first area.

[0004] In another example, an apparatus for determining disparity information is provided, the apparatus comprising: at least one memory; and at least one processor (e.g., configured in a circuit) coupled to the at least one memory. The at least one processor is configured to: obtain a plurality of cost functions, the plurality of cost functions comprising a corresponding cost function for each of a plurality of pixels of a first image, wherein the corresponding cost function for each of the plurality of pixels comprises an indication of similarity between a window including the pixel and a corresponding window of a second image, the similarity being a function of disparity along an epipolar line in the second image; determine a first disparity for a center pixel of a first pixel region at least in part by comparing one or more minima of the cost function for the center pixel of the first pixel region with one or more minima of cost functions for other pixels in the first region, the plurality of cost functions comprising the center pixel of the first pixel region; The method comprises the steps of: determining a first disparity for the central pixel of the first region at least in part by comparing one or more minimum values ​​of the cost function of the central pixel of the second region with one or more minimum values ​​of the cost function of the other pixels in the second region, the second pixel region including the central pixel of the first region, the multiple cost functions including the cost function of the central pixel of the second region and the cost function of the other pixels in the second region; and determining a third disparity for the central pixel of the first region based on the first disparity of the central pixel of the first region and the second disparity of the central pixel of the first region.

[0005] In another example, a non-transitory computer-readable medium having instructions stored thereon is provided that, when executed by one or more processors, cause the one or more processors to: obtain a plurality of cost functions, the plurality of cost functions comprising a respective cost function for each of a plurality of pixels of a first image, wherein the respective cost function for each of the plurality of pixels comprises an indication of similarity between a window including the pixel and a corresponding window of a second image as a function of disparity along an epipolar line in the second image; determine a center pixel of a first pixel region for the first pixel region at least in part by comparing the center pixel of the first pixel region with one or more minima of the cost functions of other pixels in the first region; a first disparity for the central pixel of the first area, the multiple cost functions including the cost function of the central pixel of the first area and the cost functions of the other pixels in the first area; determining a second disparity for the central pixel of the first area at least in part by comparing one or more minimum values ​​of the cost function of the central pixel of the second area with one or more minimum values ​​of the cost functions of the other pixels in the second area, the second pixel area including the central pixel of the first area, the multiple cost functions including the cost function of the central pixel of the second area and the cost function of the other pixels in the second area; and determining a third disparity for the central pixel of the first area based on the first disparity of the central pixel of the first area and the second disparity of the central pixel of the first area.

[0006] In another example, an apparatus for determining disparity information is provided. The apparatus includes: means for obtaining a plurality of cost functions, the plurality of cost functions including a respective cost function for each of a plurality of pixels of a first image, wherein the respective cost function for each of the plurality of pixels includes an indication of similarity between a window including the pixel and a corresponding window of a second image as a function of disparity along an epipolar line in the second image; and means for determining a first disparity for a center pixel of a first pixel region at least in part by comparing one or more minima of the cost function for the center pixel of the first pixel region with one or more minima of cost functions for other pixels in the first region, the plurality of cost functions including the respective cost function for the center pixel of the first pixel region. A cost function and the cost function of the other pixels in the first area; a component for determining a second disparity for the center pixel of the first area at least in part by comparing one or more minimum values ​​of the cost function of the center pixel of the second area with one or more minimum values ​​of the cost function of the other pixels in the second area, the second pixel area including the center pixel of the first area, the multiple cost functions including the cost function of the center pixel of the second area and the cost function of the other pixels in the second area; and a component for determining a third disparity for the center pixel of the first area based on the first disparity of the center pixel of the first area and the second disparity of the center pixel of the first area.

[0007] In some aspects, one or more of the apparatuses described herein is, is part of, and / or includes a robot, a drone, a vehicle, or a computing system, device, or component of a vehicle, a mobile device (e.g., a mobile phone and / or mobile handset and / or so-called "smartphone" or other mobile device), an extended reality (XR) device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a head-mounted device (HMD) device, a wearable device (e.g., a web-connected watch or other wearable device), a wireless communication device, a camera, a personal computer, a laptop computer, a server computer, another device, or a combination thereof. In some aspects, the apparatus includes a camera or cameras for capturing one or more images. In some aspects, the apparatus also includes a display for displaying one or more images, notifications, and / or other displayable data. In some aspects, the apparatus described above may include one or more sensors (e.g., one or more inertial measurement units (IMUs), such as one or more gyroscopes, one or more gyrometers, one or more accelerometers, any combination thereof, and / or other sensors).

[0008] This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation to determine the scope of the claimed subject matter. This subject matter should be understood by reference to appropriate portions of the entire specification of this patent, any or all of the drawings, and each claim.

[0009] The foregoing and other features and aspects will become more apparent upon reference to the following description, claims and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Illustrative aspects of the present application are described in detail below with reference to the following drawings:

[0011] Figure 1 Two images of a single scene captured from different camera positioning are illustrated;

[0012] Figure 2 Two images and the associated cost function are illustrated;

[0013] Figure 3 including a graph illustrating a cost function;

[0014] Figure 4 Three example cost functions for an image, a pixel region of the image, and three corresponding pixels of the pixel region are illustrated according to various aspects of the present disclosure;

[0015] Figure 5 illustrates a first region centered on a pixel and twenty-four additional five-by-five pixel regions including the pixel according to various aspects of the present disclosure;

[0016] Figure 6 A system for determining disparity according to various aspects of the present disclosure is illustrated;

[0017] Figure 7 A system for determining depth according to various aspects of the present disclosure is illustrated;

[0018] Figure 8 is a flow chart illustrating another example method for determining depth according to various aspects of the present disclosure;

[0019] Figure 9 is a diagram illustrating an example of a system for implementing certain aspects of the present disclosure. DETAILED DESCRIPTION

[0020] Provided below are certain aspects of the present disclosure. Some of these aspects can be applied independently, and some of them can be applied in combination, which will be apparent to those skilled in the art. In the following description, specific details are set forth for explanation purposes to provide a thorough understanding of various aspects of the application. However, it is apparent that various aspects can be practiced without these specific details. Each drawing and description is not intended to be restrictive.

[0021] The following description provides only exemplary aspects and is not intended to limit the scope, applicability, or configuration of the present disclosure. On the contrary, the following description of exemplary aspects will provide those skilled in the art with a description that can be used to implement the exemplary aspects. It should be understood that various changes may be made to the function and arrangement of elements without departing from the spirit and scope of the present application as set forth in the appended claims.

[0022] The terms "exemplary" and / or "example" are used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" and / or "example" is not necessarily to be construed as preferred or advantageous over other aspects. Likewise, the term "aspects of the disclosure" does not require that all aspects of the disclosure include the discussed feature, advantage, or mode of operation.

[0023] Two cameras can be positioned with different perspectives of the same scene, and each camera can capture images of the scene substantially simultaneously. The system can determine depth information (e.g., a depth map of the scene) of the scene based on the images captured by the cameras (which can be referred to as stereo images). The depth information can include the depth of objects in the scene (e.g., the distance between the camera (or a point relative to the camera) and the object).

[0024] For example, if a scene captured in stereoscopic images includes an object, a pixel representing a point on the object in an image from one camera may have a corresponding pixel representing the same point on the same object in an image from a second camera. However, because the images are captured by cameras with different perspectives of the same scene, the location of the pixel corresponding to the point on the object in the first image may be different from the location of the pixel corresponding to the same point on the object in the second image. By matching corresponding pixels in the two images and calculating the distance between these corresponding pixels, the relative depth of a point on the object within the scene can be determined. For example, in some cases, the closer the object is to the camera, the greater the distance between corresponding pixels within the image.

[0025] Figure 1 Illustrated are two images 106 and 108 of a single scene 102 captured from different camera positions (in Figure 1 Also represented as image I L and image I R ). Different camera positions are marked as left “origin” O Land the right "origin" O R , they are offset by a distance T x Due to the offset T x , the same point P of the object 104 appears in both images 106 (I L ) and 108(I R ) at different pixel positions p L and p R As can be seen, with the image 108 (I R ) in the image 108 (I R ) in the x-axis coordinate x R Along the epipolar line 110 from coordinate x L Offset by the parallax d, where the coordinate x L Corresponding to image 106 (I L ). This disparity (also known as phase difference) of pixel positions can be used to determine the approximate distance from the camera to point P on object 104 in scene 102. By understanding the stereo camera geometry and applying this analysis to each point in the image, a depth map of the scene can be generated.

[0026] To determine the disparity d, the system may determine the image 108 (I R ) in the pixel position p R Corresponding to image 106 (I L ) in the pixel position p L , for example, by including L The pixel window of the pixels at and around the image 108 (I R ) in multiple pixel windows for comparison. Figure 2 , describes an example of such a window-based comparison technique. For example, the system may determine that image 108 (I R ) in the epipolar line 110. The epipolar line 110 can be obtained by L The ray projected to point P is defined as shown in Figure 108 (I R ) is observed in the pixel position p. L A pixel window of pixels at and around φ is compared to a similarly sized window along epipolar line 110 .

[0027] Figure 2Two images are illustrated, including image 202 (which may be a "right image" or "reference image") and image 204 (which may be a "left image"), along with an associated cost function 214. To compare windows between the images, a pixel window 206 from image 202 may be selected. Pixel window 206 from image 202 may be compared to one or more pixel windows from image 204. In some cases, window 206 may be compared to similarly sized windows (e.g., all similarly sized windows) along epipolar lines 212 of image 204.

[0028] Figure 2 Cost function 214 is shown representing the similarity between window 206 and similarly sized windows along epipolar lines 212 of image 204 as a function of disparity. The similarity between windows can be based on the similarity between the corresponding red, green, blue, and / or intensity (or brightness or luminance) values ​​of the pixels included in the corresponding windows. The lower the value of cost function 214 for a particular disparity, the higher the similarity between window 206 and the window of image 204 at the corresponding disparity. For example, cost function 214 includes two minima c1 and c2. Minimum c1 corresponds to disparity d1, which corresponds to the comparison between window 206 and candidate window 208 of image 204. Minimum c2 corresponds to disparity d2, which corresponds to the comparison between window 206 and candidate window 210 of image 204.

[0029] The disparity map may be a two-dimensional map of disparity. The two-dimensional map may be compared to an image (e.g., Figure 1 For example, a two-dimensional disparity map may include the same (or in some cases substantially the same) resolution as the corresponding image, where each pixel of the image has a corresponding disparity value. In one illustrative example, the disparity map may be generated by determining a corresponding disparity for each of a plurality of pixels (e.g., all or most of the pixels) of the image (e.g., by scanning a window across the epipolar lines of the stereo pair of images and determining the disparity for each of the plurality of pixels). Each value of the disparity map may represent a disparity (e.g., Figure 1 The depth map can be based on the scene (e.g., Figure 1 The three-dimensional geometry of the scene 102) including the distances between the cameras capturing the images (e.g., Figure 1 Distance T X ).

[0030] A depth map may be a representation of three-dimensional information (e.g., depth information). For example, a depth map may be a two-dimensional map of values ​​(e.g., pixel values) representing depth. The values ​​of the depth map may correspond to the corresponding image (e.g., Figure 1For example, the depth map may have the same or substantially the same resolution as the corresponding image, where each depth value of the depth map represents a pixel at an origin (e.g., Figure 1 Origin O L ) and points (e.g., Figure 1 In some cases, each pixel in the depth map may have a depth value. Because the depth map is based on the disparity map, in some cases, each pixel in the disparity map may have a disparity.

[0031] Figure 3 A graph illustrating a cost function 302 corresponding to a pixel 310 of an image 312 is included. Cost function 302 is illustrated as cost as a function of disparity (where a low cost indicates a high similarity between a window and a reference window). Cost function 302 includes several minima (which may also be referred to as valleys), including minimum 304, minimum 306, and minimum 308. Minimum 304 may be referred to as the "lowest minimum" or the "first minimum." Minimum 306 may be referred to as the "second-lowest minimum" or the "second minimum." Minimum 308 may be referred to as the "third-lowest minimum" or the "third minimum." In the present disclosure, ordinal references to minima may relate to the order of the values ​​of the minima. For example, the lowest minimum may be referred to as the "first minimum," while the second-lowest minimum may be referred to as the "second minimum," and so on.

[0032] One example of a method for determining disparity for a pixel of an image is to determine that the lowest minimum value of a cost function for the pixel corresponds to the disparity for the pixel. In an illustrative example of using this method to determine disparity, the system may select the disparity corresponding to minimum value 304 as the disparity for pixel 310 because minimum value 304 is the lowest minimum value among minimum values ​​304, 306, and 308 of cost function 302 for pixel 310.

[0033] However, in some cases, the method described above may not be able to select the most appropriate minimum value and corresponding disparity for each pixel in the image, resulting in incorrect depth information. For example, this method may frequently fail ... Figure 2 The width of window 206 (smaller objects) may result in incorrect disparity selection. For example, a window including pixels representing a thin object may include fewer pixels representing the thin object than pixels not representing the object (e.g., background pixels). Therefore, when comparing windows, such a window may be the same as or more similar to a window including background pixels than a window including thin objects. Therefore, the method described above using the lowest minimum value may result in determining a disparity corresponding to the background rather than the thin object.

[0034] This disclosure describes systems, apparatuses, methods (also referred to herein as processes), and computer-readable media (collectively, "systems and techniques") for determining depth information using multimodal stereo matching. Compared to existing methods, these systems and techniques may further analyze the cost function before selecting a minimum value (e.g., the lowest minimum value or another minimum value) of the cost function to determine the disparity for the pixel. In some cases, the further analysis may result in selecting a minimum value that is not the lowest minimum value to determine the disparity for the pixel. For example, in some cases, these systems and techniques may select Figure 3 Instead of selecting only the minimum value 304 to determine the disparity of the pixel, the minimum value 306 or the minimum value 308 of the cost function 302 is selected to determine the disparity of the pixel.

[0035] These systems and techniques can compare each cost function for each pixel in a region of pixels to the cost function of the center pixel of the region. Figure 4 An image 402, a pixel region 404 of image 402, and three example cost functions 412, 422, and 432 for three corresponding pixels 410, 420, and 430 of pixel region 404 are illustrated. These systems and techniques can segment a region (e.g., pixel region 404) by determining whether each pixel in the region belongs to the foreground or background relative to the region. For example, these systems and techniques can compare the value of one or more corresponding minima (e.g., second minima, such as minimum 426 of cost function 422 and minimum 436 of cost function 432) of each cost function for each pixel in the region with one or more minima (e.g., second minimum 416 of cost function 412) of the cost function for a center pixel (e.g., pixel 410) of the region. Based on the comparison, these systems and techniques can associate each pixel in the region with the minimum value of the cost function for the center pixel. For example, based on the relationship between second minimum 426 of cost function 422 and first minimum 414 of cost function 412, these systems and techniques can associate pixel 420 with first minimum 414. Relating the pixels of the region to the minimum of a cost function of the center pixel of the region effectively separates the region based on disparity (which is related to distance), which may include determining whether each pixel in the region belongs to the foreground or background relative to the region.

[0036] Additionally, these systems and techniques can compare each cost function of each pixel of other regions (the other regions also include the center pixel of the region) with the corresponding center pixel of the other regions. For example, a second pixel region 444 centered on pixel 430 can be identified. Pixel 410 can be in second pixel region 444. The cost function of each pixel of second pixel region 444 can be compared with cost function 432 of pixel 430. As another example, Figure 55. A first region 504 centered on a pixel 502 and twenty-four additional five-by-five pixel regions 506 (e.g., based on regions 506 centered on other pixels) are illustrated. Each region 506 is labeled with a corresponding offset value corresponding to the offset of the center location of the corresponding region 506 from the pixel 502 (e.g., the number of pixels in the x-direction and the number of pixels in the y-direction). The corresponding cost function of the pixel 502 in each of the regions 506 can be compared to the corresponding cost function of the center pixel of each of the regions 506. Based on the comparison, the systems and techniques can associate the pixel 502 in each of the regions 506 with the corresponding minimum value of the cost function of the center pixel of the corresponding region 506.

[0037] Because region 506 includes pixel 502, the systems and techniques may associate pixel 502 with a minimum of the cost function for the center pixel of region 506. For example, pixel 502 may be associated with a minimum of each cost function for the corresponding center pixel of each region in region 506. In this manner, pixel 502 may be associated with multiple minima (e.g., one minimum for region 504 and a corresponding minimum for each region in region 506). Minimum 510 illustrates multiple minima associated with pixel 502. Minimum 510 includes one minimum from each region in region 506 and a corresponding minimum from each region in region 504. Ordered minima 512 illustrates minima 510 arranged in a one-dimensional format (e.g., from maximum to minimum or from minimum to maximum). The systems and techniques may select minimum 516 from minima 510 (or from ordered minima 512) as the minimum for pixel 502 based on test statistic 514. The test statistic 514 can be, for example, the mean or average of the minimum values ​​510, the median of the ordered minimum values ​​512, or other statistics based on the minimum values ​​510 and / or the ordered minimum values ​​512. The minimum value 516 can be related to the disparity (e.g., based on a cost function from which the minimum value 516 is derived). The minimum value 516 (and corresponding disparity) for the pixel 502 selected based on the analysis of the plurality of minimum values ​​may be more appropriate than other minimum values ​​(and other corresponding disparities) determined by prior art techniques, which may select the lowest minimum value of the cost function for the pixel as the minimum value for the pixel.

[0038] In some cases, the first and second images may be captured passively (e.g., without the scene being illuminated by the device capturing the images). In other cases, the scene may be illuminated by the device capturing the images. In some cases, the device capturing the images may illuminate the scene using patterned lighting (e.g., applying different intensities of lighting to different parts of the scene, such as in a checkerboard pattern) to improve contrast between objects close to the device and objects farther away. In some cases, the device may illuminate the scene by emitting electromagnetic radiation (e.g., light, infrared radiation, etc.) having a particular carrier frequency. In such cases, the captured images may be bandpass filtered (based on a passband of the particular carrier frequency). These systems and techniques may be applied to images captured according to any of these or other situations to improve disparity detection when comparing pixels between the images. When the images are bandpass filtered, the bandpass filtering may affect edge detection (e.g., because the bandpass filtering may remove high-frequency content of the images). In such cases, these systems and techniques may be used to improve disparity selection, which may improve edge detection.

[0039] Figure 6 Illustrated is a system 600 for determining disparity according to various aspects of the present disclosure. The system 600 may include a classifier 604, a segmenter 606, and an accumulator 608.

[0040] The system 600 may receive as input a cost quantity 602. The cost quantity 602 may include a respective cost function for each of a plurality of pixels of an image. Each cost function may be for a pixel and may be or may include an indication of similarity between a window including the pixel and a corresponding window of another image as a function of disparity along epipolar lines in the other image. For example, the cost quantity 602 may be or may include a cost function for an image (e.g., Figure 1 A cost function (e.g., Figure 2 Cost function 214, Figure 3 Cost function 302, Figure 4 Cost function 412, Figure 4 The cost function 422 or Figure 4 Each of the cost functions may have been computed by dividing the image window (e.g., Figure 2 206) with the epipolar lines of another image along the epipolar lines of the other image (e.g., Figure 1 The polar line 110 or Figure 2 of the epipolar lines 212) (e.g., Figure 2 The cost amount 602 may be or may include a four-dimensional matrix having dimensions including: image height, image width, disparity, and cost.

[0041] The classifier 604 may receive the cost volume 602 and may identify pixels of the image from which the cost volume 602 is derived that have a blurry cost function (which may be referred to herein as "blurry pixels"). For example, the classifier 604 may identify a cost function of the cost volume 602 that may result in selecting an incorrect disparity.

[0042] For example, the classifier 604 may identify a cost function that includes multiple minima, one of which may be a correct disparity, while other minima in the multiple minima may be incorrect. The classifier 604 may identify the cost function based on the following: the value of one or more minima relative to the value of the lowest minimum, the difference in cost between the local minimum and the first minimum, the second minimum, the third minimum, etc., the significance (e.g., how significant / obvious the minimum is, such as based on the cost value on either side of the minimum), any combination thereof, and / or other factors. In some cases, when identifying the cost function, there may be a bias towards proximity (e.g., larger disparity). The classifier 604 may detect multiple local minima by analyzing the relative values ​​of the cost function (e.g., by thresholding the difference or ratio, calculating a peak significance metric (which can determine the quality of the match at a given shift), and various heuristically determined clues, such as the spacing between local minima, the total variability of the waveform relative to its mean, and a bidirectional search around the global minimum).

[0043] In some cases, the classifier 604 may be optional in the system 600. For example, where the system 600 includes the classifier 604, the classifier 604 may select pixels for further analysis by the segmenter 606. In other cases, the system 600 may not include the classifier 604, in which case the segmenter 606 may analyze pixels of the image (e.g., all pixels).

[0044] Segmenter 606 may receive cost volume 602 (or a subset of cost volume 602, e.g., as selected by classifier 604) and may determine a plurality of minimum values ​​(and / or a plurality of corresponding disparities) for pixels of the image. For example, segmenter 606 may apply each cost function (e.g., Figure 4 The cost function 422 and / or Figure 4 The corresponding second minimum (and / or third minimum, and / or fourth minimum, etc.) of the cost function 432 of Figure 4 The second minimum value 426, the second minimum value 436, the third minimum value 428 and / or the third minimum value 438 in the region are related to the central pixel of the region (e.g., Figure 4The second minimum value (and / or the third minimum value, and / or the fourth minimum value, etc.) of the pixel 410) (for example, all of which are Figure 4 Based on the comparison, segmenter 606 may associate each pixel of the region (e.g., each pixel of pixel region 404) with the minimum value of the cost function of the center pixel (e.g., first minimum value 414, second minimum value 416, and / or third minimum value 418, etc. of cost function 412).

[0045] Additionally, the segmenter 606 may segment other regions (the other regions also include the central pixel of the region) (e.g., Figure 4 Pixel area 444 or Figure 5 Each cost function of each pixel of the region 506) is compared with the corresponding center pixel of the other region (e.g., Figure 4 Based on the comparison, segmenter 606 may associate each of the pixels of the corresponding other region (e.g., pixel region 444 or region 506) with a corresponding minimum value of the cost function of the center pixel of the corresponding other region (e.g., pixel 430). Because the other region (e.g., pixel region 444 or region 506) includes the center pixel of the region (e.g., pixel 410 or pixel 502), segmenter 606 may associate the center pixel of the region (e.g., pixel 410 or pixel 502) with the minimum value of the other cost function of the center pixel (e.g., pixel 430) of the other region (e.g., pixel region 444 or region 506). In this way, the center pixel of the region (e.g., pixel 410 or pixel 502) may be associated with multiple minimum values.

[0046] Segmenter 606 may map a surrounding neighborhood (e.g., a region) of each blurry pixel (e.g., a pixel identified by classifier 604, which may have a blur cost function) into foreground (which will contain small objects) and background (which may be the dominant but erroneous disparity of the window).

[0047] In some cases, the mapping performed by the segmenter 606 can be an iterative process because once segmentation is performed, knowledge of which pixels in the window belong to each minimum can be obtained. The cost function for the pixel can then be recalculated based on this knowledge of which pixels in the window belong to each minimum (which can represent a more accurate region association). For example, initially, the segmenter 606 can map each pixel of the window to a disparity. Based on these initial results, the segmenter 606 can recalculate the cost using a subset of pixels when recalculating the cost. For example, the disparity for a pixel can be recalculated based on pixels that have the same disparity as the pixel.

[0048] In some cases, segmenter 606 may operate on an unfiltered cost volume that is not affected by low-pass filtering.

[0049] The segmenter 606 can generate a test statistic based on a comparison between the first lowest minimum and the second lowest minimum (or any arbitrary number of minima) in the local neighborhood. The cost value can be compared to the test statistic to separate the local pixels into foreground labels or background labels. This process can be iterated to improve and refine the segmentation results by recalculating the cost based on updated knowledge of the segmentation.

[0050] The accumulator 608 may receive a plurality of minimum values ​​for one or more pixels (e.g., as determined by the segmenter 606) and / or the cost volume 602. The accumulator 608 may determine a minimum value (and, in some cases, a corresponding disparity) for each pixel. The accumulator 608 may select the minimum value (and corresponding disparity) for each pixel based on a test statistic, such as, for example, a mean of the plurality of minimum values, a median of the plurality of minimum values, any combination thereof, and / or other test statistics.

[0051] The accumulator 608 can determine the correct local minimum (but not necessarily the global minimum) and output the correct disparity value. The accumulator 608 can shift and aggregate adjacent overlapping local windows relative to a reference window including the center pixel to produce a more accurate and / or less noisy second minimum disparity map. Because each shifted window is relative to the reference window, this information can be used to detect and eliminate incorrect segmentation results. The accumulator 608 can generate a disparity map 610.

[0052] Figure 7 Illustrated is a system 700 for determining depth according to various aspects of the present disclosure. The system 700 may include a filter 704, a scanline optimizer 708, a winner-take-all algorithm 712 (WTA 712), a classifier 716, a segmenter 720, and an accumulator 724.

[0053] System 700 can receive a cost amount 702. Cost amount 702 can be Figure 6 The cost amounts 602 are the same or substantially similar.

[0054] Filter 704 can filter cost amount 702 to produce filtered cost amount 706. Filter 704 can be a 5×5 low-pass filter. Cost values ​​calculated based on individual pixels can be noisy. Filtering at filter 704 (and / or aggregation, which can be part of the filtering) can reduce noise by calculating the cost over many pixels and then averaging.

[0055] Scanline optimizer 708 can optimize filtered cost volume 706 to produce optimized cost volume 710. Scanline optimization can be used as a constraint to guide the selection of the correct disparity. The constraint can penalize large disparity changes, such as based on many images consisting of smooth regions with some large jumps at object boundaries.

[0056] The WTA 712 may select a disparity for each pixel of the optimized cost volume 710. The WTA 712 may select the lowest minimum value of the cost function for each pixel to determine the disparity of the pixel. The WTA 712 (or another element not illustrated in the system 700) may calculate a depth based on each of the disparities for each of the corresponding pixels to generate a depth map 714 (alternatively, the depth map 714 may be a disparity map).

[0057] Classifier 716 can be used with Figure 6 702. The classifier 716 may be the same as, substantially similar to, and / or perform the same operations or some of the same operations as the classifier 604 of the system 600. For example, the classifier 716 may receive the optimized cost volume 710 and the depth map 714 as inputs and may identify pixels of the optimized cost volume 710 (or the depth map 714) that may have a blur cost. The classifier 716 may provide an identifier 718 indicating the pixels of the cost volume 702 that have a blur cost. Like the classifier 604 of the system 600, the classifier 716 is optional in the system 700.

[0058] Segmenter 720 can be used with Figure 6 702. The segmenter 720 may be the same as, substantially similar to, and / or perform the same operations or some of the same operations as the segmenter 606 of the cost volume 702. For example, the segmenter 720 may identify a plurality of candidate minimum values ​​722 for each of one or more pixels of the cost volume 702. In some cases, the segmenter 720 may identify a plurality of candidate minimum values ​​722 for pixels identified by the classifier 716 as having a blurry cost (e.g., as indicated by the identifier 718). The segmenter 720 may identify the candidate minimum values ​​722 based on the cost volume 702 rather than based on the optimized cost volume 710. The segmenter 720 may provide the candidate minimum values ​​722 to the accumulator 724.

[0059] The accumulator 724 can be used with Figure 6 The accumulator 608 of the cost volume 702 may be the same as, substantially similar to, and / or perform the same operations or some of the same operations as the accumulator 608. For example, the accumulator 608 may determine a minimum value for each pixel of the cost volume 702. The accumulator 724 may determine the minimum value based on the cost volume 702 rather than the optimized cost volume 710, for example, because low-pass filtering (e.g., by the filter 704) may reduce the fidelity of the data (e.g., through an averaging (smoothing) process).

[0060] Based on the identified minimum value for each of the pixels, accumulator 724 (or another element not illustrated in system 700 ) may calculate depth based on each of the disparities for each of the corresponding pixels to produce depth map 726 .

[0061] Figure 8 is a flow chart illustrating another example method 800 for determining depth information according to various aspects of the present disclosure. Process 800 may be performed by a computing device (or apparatus) or a component of a computing device (e.g., a chipset, one or more processors, one or more memories, any combination thereof, or other components). The computing device may be a mobile device (e.g., a mobile phone), a network-connected wearable device such as a watch, an extended reality (XR) device (e.g., a virtual reality (VR) device or an augmented reality (AR) device), a vehicle or a component or system of a vehicle, a camera device, or other type of computing device. In some cases, process 800 may be performed by a device that implements Figure 6 System 600 and / or Figure 7 The operations of process 800 may be implemented as a single processor ( Figure 9 Furthermore, the sending and receiving of signals by the computing device in process 800 may be performed by, for example, one or more antennas, one or more transceivers (e.g., wireless transceivers), and / or other communication components (e.g., Figure 9 This may be implemented using the communication interface 926, or other antennas, transceivers and / or components).

[0062] In some aspects, process 800 may include: illuminating a scene; capturing a first image of the scene at a first image sensor; and capturing a second image of the scene at a second image sensor. There may be a known offset between the first image sensor and the second image sensor (e.g., Figure 1 In some aspects, illuminating the scene may include emitting electromagnetic radiation having a carrier frequency. The first image may be filtered using a filter having a passband based on the carrier frequency, and the second image may be filtered using the filter.

[0063] At block 802, a computing device (or one or more components thereof) may obtain a plurality of cost functions, the plurality of cost functions comprising a respective cost function for each of a plurality of pixels of a first image, wherein the respective cost function for each of the plurality of pixels comprises an indication of similarity between a window including the pixel and a corresponding window of a second image as a function of disparity along epipolar lines in the second image. For example, Figure 6The system 600 can obtain Figure 6 The cost amount 602 and / or Figure 7 The system 700 may obtain a cost amount 702 . Figure 2 Cost function 214, Figure 3 Cost function 302, Figure 4 Cost function 412, Figure 4 The cost function 422 and Figure 4 Cost function 432 is an example of a cost function among the plurality of cost functions obtained at block 802. For example, cost function 214 indicates similarity between window 206 of image 202 and a plurality of windows along epipolar line 212, including candidate window 208 and candidate window 210.

[0064] In some aspects, the computing device (or one or more components thereof) may determine the first region based on associating a determination of a center pixel of the first region with a blur cost function based on one or more factors. In some aspects, the one or more factors include at least one of a cost difference between the lowest minima of the cost function or a disparity difference between the lowest minima of the cost function. For example, Figure 6 The classifier 604 and / or Figure 7 The classifier 716 may select the first region based on a cost function of a center pixel of the center region. For example, the classifier 604 and / or the classifier 716 may select the region surrounding and including the pixel 310 based on the cost function 302 (e.g., based on a cost difference between one or more of the minimum value 304, the minimum value 306, and / or the minimum value 308 and / or based on a disparity difference between the minimum value 304, the minimum value 306, and / or the minimum value 308).

[0065] At block 804, the computing device (or one or more components thereof) may determine a first disparity for a central pixel of a first pixel region at least in part by comparing one or more minimum values ​​of a cost function of the central pixel of the first pixel region with one or more minimum values ​​of cost functions of other pixels in the first region, the plurality of cost functions including the cost function of the central pixel of the first region and the cost functions of the other pixels in the first region. For example, Figure 6 Segmenter 606 and / or Figure 7 The segmenter 720 may determine a first disparity for the pixel 410 (e.g., the center pixel of the region 404) by comparing one or more minimum values ​​of the cost function 412 with one or more minimum values ​​of the cost functions of other pixels of the pixel region 404 (e.g., pixel 420, pixel 430, etc.). Figure 6 Segmenter 606 and / or Figure 7The segmenter 720 may compare one or more minimum values ​​of the cost function 412 (e.g., the first minimum value 414, the second minimum value 416, the third minimum value 418, etc.) with one or more minimum values ​​of the cost function 422 (e.g., the first minimum value 424, the second minimum value 426, the third minimum value 428, etc.) and / or with one or more minimum values ​​of the cost function 432 (e.g., the first minimum value 434, the second minimum value 436, the third minimum value 438, etc.).

[0066] In some aspects, block 804 may include comparing the second minimum value of the cost function for the center pixel of the first region with the second minimum value of each of the cost functions for the other pixels in the first region. Figure 6 Segmenter 606 and / or Figure 7 The segmenter 720 may compare the second minimum value 416 of the cost function 412 of the pixel 410 with the corresponding second minimum values ​​of the corresponding cost functions of other pixels of the pixel area 404 (e.g., the second minimum value 426 of the cost function 422 of the pixel 420 and the second minimum value 436 of the cost function 432 of the pixel 430).

[0067] In some aspects, block 804 may include identifying the first disparity for the center pixel of the first pixel region based on determining whether a second minimum value of each of the cost functions for the other pixels in the first region is less than a second minimum value of the cost function for the center pixel of the first region. For example, Figure 6 Segmenter 606 and / or Figure 7 The segmenter 720 can determine whether each of the corresponding second minimum values ​​of the corresponding cost functions of other pixels in the pixel area 404 (for example, the second minimum value 426 of the cost function 422 of pixel 420 and the second minimum value 436 of the cost function 432 of pixel 430) is less than the second minimum value 416 of the cost function 412 of pixel 410.

[0068] At block 806, the computing device (or one or more components thereof) may determine a second disparity for the center pixel of the first region at least in part by comparing one or more minima of a cost function for the center pixel of the second region with one or more minima of cost functions for other pixels in the second region, the second pixel region including the center pixel of the first region, the plurality of cost functions including the cost function for the center pixel of the second region and the cost functions for the other pixels in the second region. For example, the segmenter 606 and / or the segmenter 720 may determine the second disparity for the pixel 410 by comparing one or more minima of the cost function 432 (the cost function for the pixel 430, the center pixel of the region 444) with one or more minima of cost functions for other pixels of the pixel region 444 (including the other pixels of the region 444 of the pixel 410). For example, Figure 6 Segmenter 606 and / or Figure 7 The segmenter 720 may compare one or more minimum values ​​(e.g., the first minimum value 434, the second minimum value 436, the third minimum value 438, etc.) of the cost function 432 with one or more minimum values ​​(e.g., the first minimum value 414, the second minimum value 416, the third minimum value 418, etc.) of the cost function 412.

[0069] At block 808, the computing device (or one or more components thereof) may determine a third disparity for the central pixel of the first region based on the first disparity of the central pixel of the first region and the second disparity of the central pixel of the first region. For example, Figure 6 The accumulator 608 and / or Figure 7 The accumulator 724 may determine the disparity based on the disparities determined at blocks 804 and 806. For example, the accumulator 608 and / or the accumulator 724 may select one of the disparities determined at blocks 804 and 808 as the third disparity. Figure 5 The minimum value 510 of can be an example of a minimum value corresponding to the first disparity and the second disparity (and other disparities) determined for pixel 410 based on a comparison between the cost function for pixel 410 and the cost function for pixel region 506. For example, block 806 can be repeated for any number of regions (e.g., region 506) that include pixel 502. The minimum value 516 can be an example of a minimum value corresponding to the disparity selected at block 808. In some aspects, block 808 can include determining a median of the determined number of disparities for the center pixel of the first region as the third disparity for the center pixel of the first region. For example, the test statistic 514 can select the median of the ordered minimum values ​​512 as the third disparity.

[0070] In some aspects, the computing device (or one or more components thereof) may determine the number of disparities for the center pixel of the first region at least in part by comparing one or more minimum values ​​of a corresponding cost function for the center pixel of a corresponding number of regions with one or more minimum values ​​of other corresponding cost functions for other corresponding pixels in the number of regions, each of the number of regions including the center pixel of the first region. Furthermore, block 808 may include determining the third disparity for the center pixel of the first region based on the determined number of disparities for the center pixel of the first region. For example, accumulator 608 and / or accumulator 724 may identify region 506 and its center pixel. Furthermore, accumulator 608 and / or accumulator 724 may compare the one or more minimum values ​​of the cost function for the center pixel with the cost functions for the other pixels of the corresponding region 506. Because pixel 502 is included in each region of region 506, such a process may determine multiple disparities corresponding to pixel 502. In such cases, block 808 may include selecting one of the determined multiple disparities as the third disparity.

[0071] In some aspects, the computing device (or one or more components thereof) may determine depth information for the center pixel of the first region based on the third disparity for the center pixel of the first region. For example, the accumulator 724 may determine a depth map 726 including a depth corresponding to the pixel 410 based on the selected third disparity (e.g., based on the three-dimensional geometry of the image capture device and the disparity). For example, the accumulator 724 may determine a depth map 726 based on the selected third disparity (e.g., based on the three-dimensional geometry of the image capture device and the disparity). Figure 1 The offset Tx uses the determined third disparity as Figure 1 The parallax d is used to determine Figure 1 The depth of point P.

[0072] In some examples, the methods described herein (e.g., method 800 and / or other methods described herein) may be performed by a computing device or apparatus. In one example, one or more of the methods may be performed by Figure 6 System 600, Figure 6 Classifier 604, Figure 6 Segmenter 606, Figure 6 The accumulator 608, Figure 7 System 700, Figure 7 Classifier 716, Figure 7 Segmenter 720 and / or Figure 7 In another example, one or more of these methods may be performed by Figure 9 The computing system 900 shown executes. For example, Figure 9 The computing devices of the illustrated computing system 900 may include Figure 6Components of system 600, and / or Figure 7 The components of the system 700 may be implemented Figure 8 The operations of method 800 and / or other processes described herein.

[0073] The computing device may include any suitable device, such as a vehicle or a computing device of a vehicle, a mobile device (e.g., a mobile phone), a desktop computing device, a tablet computing device, a wearable device (e.g., a VR headset, an AR headset, AR glasses, a web-connected watch or smartwatch, or other wearable device), a server computer, a robotic device, a television, and / or any other computing device with the resource capacity to perform the processes described herein (including method 800 and / or other processes described herein). In some cases, the computing device or apparatus may include various components, such as one or more input devices, one or more output devices, one or more processors, one or more microprocessors, one or more microcomputers, one or more cameras, one or more sensors, and / or other components configured to perform the steps of the processes described herein. In some examples, the computing device may include a display, a network interface configured to communicate and / or receive data, any combination thereof, and / or other components. The network interface may be configured to communicate and / or receive data based on an Internet Protocol (IP) or other types of data.

[0074] Components of a computing device may be implemented in circuitry. For example, a component may include and / or be implemented using electronic circuitry or other electronic hardware that may include one or more programmable electronic circuits (e.g., a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a central processing unit (CPU), and / or other suitable electronic circuitry) and / or may include and / or be implemented using computer software, firmware, or any combination thereof for performing the various operations described herein.

[0075] Method 800 and other processes described herein are illustrated as a logical flow diagram, the operations of which represent a sequence of operations that can be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, each operation represents a computer-executable instruction stored on one or more computer-readable storage media that, when executed by one or more processors, performs the described operation. Generally speaking, computer-executable instructions include routines, programs, objects, components, and data structures that perform specific functions or implement specific data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement the process.

[0076] Additionally, method 800 and / or other processes described herein may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that executes together on one or more processors, implemented by hardware, or implemented by a combination thereof. As noted above, the code may be stored on a computer-readable or machine-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions that can be executed by one or more processors. The computer-readable storage medium or machine-readable storage medium may be non-transitory.

[0077] Figure 9 is a diagram illustrating an example of a system for implementing certain aspects of the present disclosure. Specifically, Figure 9 An example of a computing system 900 is illustrated, which can be any computing device, for example, constituting an internal computing system, a remote computing system, a camera, or any component thereof, wherein the components of the system communicate with each other using connection 912. Connection 912 can be a physical connection using a bus, or a direct connection into processor 902, such as in a chipset architecture. Connection 912 can also be a virtual connection, a networked connection, or a logical connection.

[0078] In some aspects, computing system 900 is a distributed system in which the functionality described in this disclosure may be distributed within a data center, multiple data centers, a peer-to-peer network, etc. In some aspects, one or more of the described system components may represent a plurality of such components, each of which performs some or all of the functionality of the described component. In some aspects, these components may be physical or virtual devices.

[0079] The example computing system 900 includes at least one processing unit (CPU or processor) 902 and connections 912 that couple various system components including system memory 910, such as read-only memory (ROM) 908 and random access memory (RAM) 906, to the processor 902. The computing system 900 may include a cache 904 of high-speed memory directly connected to, proximate to, or integrated as part of the processor 902.

[0080] The processor 902 may include any general-purpose processor and hardware or software services (such as services 916, 918, and 920 configured to control the processor 902 and stored in the storage device 914), as well as a special-purpose processor in which the software instructions are incorporated into the actual processor design. The processor 902 may essentially be a completely independent computing system containing multiple cores or processors, a bus, a memory controller, a cache, etc. Multi-core processors may be symmetric or asymmetric.

[0081] To enable user interaction, the computing system 900 includes an input device 922, which may represent any number of input mechanisms, such as a microphone for voice, a touch-sensitive screen for gesture or graphical input, a keyboard, a mouse, motion input, voice, and the like. The computing system 900 may also include an output device 924, which may be one or more of a plurality of output mechanisms. In some cases, a multimodal system may enable a user to provide multiple types of input / output to communicate with the computing system 900. The computing system 900 may include a communication interface 926, which may generally govern and manage user input and system output. The communication interface 926 may perform or facilitate the use of wired and / or wireless transceivers to receive and / or send wired or wireless communications, including utilizing an audio jack / plug, a microphone jack / plug, a universal serial bus (USB) port / plug, Ports / plugs, Ethernet ports / plugs, fiber optic ports / plugs, dedicated wired ports / plugs, Wireless signal transmission, Low energy (BLE) wireless signal transmission, The communication interface 1540 may also include one or more global navigation satellite system (GNSS) receivers or transceivers for determining the location of the computing system 1500 based on receiving one or more signals from one or more satellites associated with one or more GNSS systems. GNSS systems include, but are not limited to, the United States' Global Positioning System (GPS), Russia's Global Navigation Satellite System (GLONASS), China's BeiDou Navigation Satellite System (BDS), and Europe's Galileo GNSS. There is no restriction on operating on any particular hardware arrangement, and thus the base features herein may be readily substituted for improved hardware or firmware arrangements as they are developed.

[0082] The storage device 914 may be a non-volatile and / or non-transitory and / or computer-readable memory device and may be a hard disk or other type of computer-readable medium that can store data that can be accessed by a computer, such as a magnetic tape cartridge, a flash memory card, a solid-state memory device, a digital versatile disk, a magnetic cassette, a floppy disk, a flexible disk, a hard disk, a magnetic tape, a magnetic stripe / strip, any other magnetic storage medium, a flash memory, a memristor memory, any other solid-state memory, a compact disc read-only memory (CD-ROM) optical disc, a rewritable compact disc (CD) optical disc, a digital video disc (DVD) optical disc, a Blu-ray disc (BDD) optical disc, a holographic optical disc, another optical medium, a secure digital (SD) card, a micro secure digital (microSD) card, a memory card, a smart card chip, an EMV chip, a subscriber identity module (SIM) card, a mini / micro / nano / pico SIM card, another integrated circuit (IC) chip / card, a random access memory (RAM), a static RAM (SRAM), a dynamic RAM (DRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash EPROM (FLASH EPROM), a cache memory (L1 / L2 / L3 / L4 / L5 / L#), a resistive random access memory (RRAM / ReRAM), a phase change memory (PCM), a spin-transfer torque RAM (STT-RAM), another memory chip or cartridge, and / or a combination thereof.

[0083] Storage devices 914 may include software services, servers, services, etc. that, when code defining such software is executed by processor 902, cause the system to perform functions. In some aspects, hardware services that perform specific functions may include software components for performing functions stored in a computer-readable medium connected to the necessary hardware components (such as processor 902, connections 912, output devices 924, etc.).

[0084] As used herein, the term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instructions and / or data. A computer-readable medium may include a non-transient medium in which data can be stored and does not include carrier waves and / or transient electronic signals that are propagated wirelessly or on a wired connection. Examples of non-transient media may include, but are not limited to, disks or tapes, optical storage media (such as compact discs (CDs) or digital versatile discs (DVDs)), flash memory, memory, or memory devices. A computer-readable medium may store thereon code and / or machine-executable instructions that may represent any combination of a process, function, subroutine, program, routine, subroutine, module, software package, class, or instruction, data structure, or program statement. A code segment may be coupled to another code segment or hardware circuit by transmitting and / or receiving information, data, independent variables, parameters, or memory contents. Information, independent variables, parameters, data, etc. may be transmitted, forwarded, or sent using any suitable means, including memory sharing, message passing, token passing, or network transmission.

[0085] In some aspects, computer-readable storage devices, media, and memories may include wired or wireless signals containing bit streams, etc. However, when referred to, non-transitory computer-readable storage media expressly excludes media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0086] Specific details are provided in the above description to provide a detailed understanding of the aspects and examples provided herein. However, it will be understood by those skilled in the art that these aspects can be practiced without these specific details. For clarity of explanation, in some cases, the present technology can be presented as comprising separate functional blocks, including functional blocks comprising devices, device components, steps in the method embodied in software or a combination of hardware and software or routines. Additional components other than those components shown in the accompanying drawings and / or described herein can be used. For example, circuits, systems, networks, processes and other components can be shown as components in block diagram form to avoid confusing these aspects in unnecessary details. In other cases, known circuits, processes, algorithms, structures and techniques can be shown without unnecessary details to avoid confusing various aspects.

[0087] Various aspects may be described above as processes or methods, which may be depicted as flow charts, flowcharts, data flow diagrams, structure diagrams, or block diagrams. Although a flow chart may describe operations as a sequential process, many of the operations may be performed in parallel or concurrently. Furthermore, the order of the operations may be rearranged. A process is terminated when its operations are completed, but a process may have additional steps not included in the accompanying figures. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. When a process corresponds to a function, termination of the process may correspond to the function returning to the calling function or the main function.

[0088] The processes and methods according to the examples described above can be implemented using stored computer-executable instructions or computer-executable instructions otherwise obtained from a computer-readable medium. Such instructions may include, for example, instructions and data that configure a general-purpose computer, a special-purpose computer, or a processing device to perform a certain function or group of functions. Portions of the computer resources used may be accessible over a network. Computer-executable instructions may be, for example, binary, intermediate format instructions such as assembly language, firmware, source code, etc. Examples of computer-readable media that can be used to store instructions, information used, and / or information created during the methods according to the described examples include magnetic or optical disks, flash memory, USB devices with non-volatile memory, networked storage devices, etc.

[0089] Devices implementing the processes and methods according to these disclosures may include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and may take any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., computer program products) for performing the necessary tasks may be stored in a computer-readable or machine-readable medium. A processor may perform the necessary tasks. Typical examples of form factors include laptop computers, smartphones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rack-mounted devices, stand-alone devices, etc. The functionality described herein may also be embodied in peripheral devices or add-in cards. By way of further example, this functionality may also be implemented on circuit boards among different chips or different processes executed in a single device.

[0090] Instructions, media for conveying such instructions, computing resources for executing them, and other structures for supporting such computing resources are example means for providing the functionality described in this disclosure.

[0091] In the foregoing description, various aspects of the present application are described with reference to specific aspects thereof, but those skilled in the art will recognize that the present application is not limited thereto. Therefore, although the illustrative aspects of the present application have been described in detail herein, it is to be understood that each inventive concept can be implemented and adopted in various other ways, and the appended claims are not intended to be interpreted as including these variations, unless limited by the prior art. The various features and aspects of the application described above can be used individually or in combination. In addition, without departing from the scope of this specification, the various aspects can be used in any number of environments and applications beyond the environment and application described herein. Therefore, the description and the accompanying drawings should be considered as illustrative rather than restrictive. For illustrative purposes, each method is described in a specific order. It should be understood that, in alternative aspects, each method can be performed in an order different from the order described.

[0092] One of ordinary skill will understand that the less than (“<”) and greater than (“>”) symbols or terms used herein may be replaced with less than or equal to (“≤”) and greater than or equal to (“≥”) symbols without departing from the scope of this specification.

[0093] Where a component is described as being “configured to” perform certain operations, such configuration may be achieved, for example, by designing electronic circuits or other hardware to perform the operations, by programming programmable electronic circuits (e.g., a microprocessor or other suitable electronic circuits) to perform the operations, or any combination thereof.

[0094] The phrase “coupled to” refers to any component being physically connected directly or indirectly to another component, and / or any component being in communication directly or indirectly with another component (e.g., connected to another component via a wired or wireless connection and / or other suitable communication interface).

[0095] Claim language or other language reciting "at least one of" a set and / or "one or more of" a set indicates that one member of the set or multiple members of the set (in any combination) satisfies the claim. For example, claim language reciting "at least one of A and B" or "at least one of A or B" means A, B, or A and B. In another example, claim language reciting "at least one of A, B, and C" or "at least one of A, B, or C" means A, B, C, or A and B, or A and C, or B and C, or A, B, and C. The language "at least one of" a set and / or "one or more of" a set does not limit the set to the items listed in the set. For example, claim language reciting "at least one of A and B" or "at least one of A or B" may mean A, B, or A and B, and may additionally include items not listed in the set of A and B.

[0096] The various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the various aspects disclosed herein can be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. A skilled person may implement the described functionality in different ways for each specific application, but such specific implementation decisions should not be interpreted as departing from the scope of this application.

[0097] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as general-purpose computers, wireless communication devices, or integrated circuit devices with multiple uses, including applications in wireless communication devices and other devices. Any features described as modules or components may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, these techniques may be implemented at least in part by a computer-readable data storage medium comprising program code, the program code comprising instructions that, when executed, perform one or more of the methods described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may include memory or data storage media, such as random access memory (RAM) (such as synchronous dynamic random access memory (SDRAM)), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, and magnetic or optical data storage media. Additionally or alternatively, the techniques may be implemented at least in part by a computer-readable communication medium that carries or communicates program code in the form of instructions or data structures and that can be accessed, read, and / or executed by a computer, such as a propagated signal or wave.

[0098] The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Such a processor may be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor; however, in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Thus, the term "processor," as used herein, may refer to any of the foregoing structures, any combination of the foregoing structures, or any other structure or device suitable for implementing the techniques described herein.

[0099] Illustrative aspects of the present disclosure include:

[0100] Aspect 1. An apparatus for determining disparity information, the apparatus comprising: at least one memory; and at least one processor, the at least one processor being coupled to the at least one memory and configured to: obtain a plurality of cost functions, the plurality of cost functions comprising a respective cost function for each of a plurality of pixels of a first image, wherein the respective cost function for each of the plurality of pixels comprises an indication of similarity between a window including the pixel and a corresponding window of a second image, the similarity being a function of disparity along an epipolar line in the second image; determine a first disparity for a center pixel of a first pixel region at least in part by comparing one or more minima of the cost function for the center pixel of the first pixel region with one or more minima of the cost functions for other pixels in the first region, The multiple cost functions include the cost function of the central pixel of the first area and the cost functions of the other pixels in the first area; determining the second disparity for the central pixel of the first area at least in part by comparing one or more minimum values ​​of the cost function of the central pixel of the second area with one or more minimum values ​​of the cost function of the other pixels in the second area, the second pixel area including the central pixel of the first area, the multiple cost functions include the cost function of the central pixel of the second area and the cost functions of the other pixels in the second area; and determining the third disparity for the central pixel of the first area based on the first disparity of the central pixel of the first area and the second disparity of the central pixel of the first area.

[0101] Aspect 2. An apparatus according to Aspect 1, wherein, when comparing the one or more minimum values ​​of the cost function of the central pixel of the first area with the one or more minimum values ​​of the cost function of the other pixels in the first area, the at least one processor is configured to compare the second minimum value of the cost function of the central pixel of the first area with the second minimum value of each of the cost functions of the other pixels in the first area.

[0102] Aspect 3. An apparatus according to any one of Aspects 1 or 2, wherein, when identifying the first disparity for the central pixel of the first pixel area, the at least one processor is configured to identify the first disparity for the central pixel of the first pixel area based on determining whether the second minimum value of each of the cost functions of the other pixels in the first area is less than the second minimum value of the cost function of the central pixel of the first area.

[0103] Aspect 4. An apparatus according to any one of Aspects 1 to 3, wherein the at least one processor is further configured to determine the number of disparities of the central pixel of the first area at least in part by comparing one or more minimum values ​​of the corresponding cost function of the corresponding central pixels of a corresponding number of areas with one or more minimum values ​​of other corresponding cost functions of other corresponding pixels in the said number of areas, each area of ​​the said number of areas including the central pixel of the first area; wherein, when determining the third disparity for the central pixel of the first area, the at least one processor is configured to determine the third disparity for the central pixel of the first area based on the determined number of disparities of the central pixel of the first area.

[0104] Aspect 5. An apparatus according to Aspect 4, wherein, when determining the third disparity for the central pixel of the first area based on the determined number of disparities of the central pixel of the first area, the at least one processor is configured to determine the median of the determined number of disparities of the central pixel of the first area as the third disparity for the central pixel of the first area.

[0105] Aspect 6. An apparatus according to any one of Aspects 1 to 5, wherein the at least one processor is further configured to determine the first area based on associating the center pixel of the first area with a blur cost function based on one or more factors.

[0106] Aspect 7. The apparatus according to aspect 6, wherein the one or more factors include at least one of a cost difference between the lowest minima of the cost function or a disparity difference between the lowest minima of the cost function.

[0107] Clause 8. The apparatus according to any one of clauses 1 to 7, wherein the at least one processor is further configured to determine depth information for the central pixel of the first region based on the third disparity for the central pixel of the first region.

[0108] Aspect 9. An apparatus according to any one of Aspects 1 to 8, wherein the apparatus further comprises: an illuminator configured to illuminate a scene; a first image sensor configured to capture the first image of the scene; and a second image sensor configured to capture the second image of the scene.

[0109] Aspect 10. An apparatus according to Aspect 9, wherein: the illuminator is configured to illuminate the scene by emitting electromagnetic radiation having a carrier frequency; the at least one processor is further configured to filter the first image using a filter having a passband based on the carrier frequency; and the at least one processor is further configured to filter the second image using the filter.

[0110] Aspect 11. A method for determining disparity information, the method comprising: obtaining a plurality of cost functions, the plurality of cost functions comprising a corresponding cost function for each of a plurality of pixels of a first image, wherein the corresponding cost function for each of the plurality of pixels comprises an indication of similarity between a window comprising the pixel and a corresponding window of a second image, the similarity being a function of disparity along an epipolar line in the second image; determining a first disparity for a center pixel of a first pixel region at least in part by comparing one or more minima of the cost functions of the center pixel of the first pixel region with one or more minima of cost functions of other pixels in the first region, the plurality of cost functions comprising the center pixel of the first pixel region The method comprises the steps of: determining a first disparity for the central pixel of the first region at least in part by comparing one or more minimum values ​​of the cost function of the central pixel of the second region with one or more minimum values ​​of the cost function of the other pixels in the second region, wherein the second pixel region includes the central pixel of the first region, and the multiple cost functions include the cost function of the central pixel of the second region and the cost function of the other pixels in the second region; and determining a third disparity for the central pixel of the first region based on the first disparity of the central pixel of the first region and the second disparity of the central pixel of the first region.

[0111] Aspect 12. A method according to Aspect 11, wherein comparing the one or more minimum values ​​of the cost function of the central pixel of the first area with the one or more minimum values ​​of the cost function of the other pixels in the first area includes: comparing the second minimum value of the cost function of the central pixel of the first area with the second minimum value of each cost function in the cost functions of the other pixels in the first area.

[0112] Aspect 13. A method according to any one of Aspects 11 or 12, wherein identifying the first disparity for the center pixel of the first pixel area includes: identifying the first disparity for the center pixel of the first pixel area based on determining whether the second minimum value of each cost function in the cost functions of the other pixels in the first area is less than the second minimum value of the cost function of the center pixel of the first area.

[0113] Aspect 14. A method according to any one of Aspects 11 to 13, the method further comprising: determining the number of disparities of the central pixel of the first area at least in part by comparing one or more minimum values ​​of the corresponding cost function of the corresponding central pixels of a corresponding number of areas with one or more minimum values ​​of other corresponding cost functions of other corresponding pixels in the said number of areas, each area of ​​the said number of areas including the central pixel of the first area; wherein determining the third disparity for the central pixel of the first area comprises: determining the third disparity for the central pixel of the first area based on the determined number of disparities of the central pixel of the first area.

[0114] Aspect 15. A method according to aspect 14, wherein determining the third disparity for the central pixel of the first area based on the determined number of disparities of the central pixel of the first area includes: determining the median of the determined number of disparities of the central pixel of the first area as the third disparity for the central pixel of the first area.

[0115] Aspect 16. The method according to any one of aspects 11 to 15, further comprising: determining the first region based on associating the central pixel of the first region with a blur cost function based on one or more factors.

[0116] Aspect 17. The method according to aspect 16, wherein the one or more factors include at least one of a cost difference between the lowest minima of the cost function or a disparity difference between the lowest minima of the cost function.

[0117] Clause 18. The method according to any one of clauses 11 to 17, further comprising: determining depth information for the central pixel of the first region based on the third disparity for the central pixel of the first region.

[0118] Aspect 19. The method according to any one of aspects 11 to 18, further comprising: illuminating a scene; capturing the first image of the scene at a first image sensor; and capturing the second image of the scene at a second image sensor.

[0119] Aspect 20. A method according to aspect 19, wherein: illuminating the scene includes emitting electromagnetic radiation having a carrier frequency; the first image is filtered using a filter having a passband based on the carrier frequency; and the second image is filtered using the filter.

[0120] Aspect 21. A non-transitory computer-readable storage medium having instructions stored thereon, the instructions, when executed by at least one processor, causing the at least one processor to perform the operations according to any one of aspects 11 to 20.

[0121] Clause 22. An apparatus for determining disparity information, the apparatus comprising one or more means for performing the operations of any one of clauses 11 to 20.

Claims

1. A device for determining disparity information, the device comprising: at least one memory; and at least one processor coupled to the at least one memory and configured to: obtaining a plurality of cost functions, the plurality of cost functions comprising a respective cost function for each of a plurality of pixels of the first image, wherein the respective cost function for each of the plurality of pixels comprises an indication of similarity between a window including the pixel and a corresponding window of the second image as a function of disparity along an epipolar line in the second image; determining a first disparity for a center pixel of a first pixel region at least in part by comparing one or more minima of a cost function of the center pixel of the first pixel region with one or more minima of cost functions of other pixels in the first region, the plurality of cost functions including the cost function of the center pixel of the first region and the cost functions of the other pixels in the first region; determining a second disparity for the central pixel of the first region at least in part by comparing one or more minima of a cost function of the central pixel of a second region with one or more minima of cost functions of other pixels in the second region, the second pixel region including the central pixel of the first region, the plurality of cost functions including the cost function of the central pixel of the second region and the cost functions of the other pixels in the second region; as well as A third disparity for the central pixel of the first area is determined based on the first disparity of the central pixel of the first area and the second disparity of the central pixel of the first area.

2. The device according to claim 1, wherein In order to compare the one or more minimum values ​​of the cost function of the central pixel of the first area with the one or more minimum values ​​of the cost functions of the other pixels in the first area, the at least one processor is configured to compare the second minimum value of the cost function of the central pixel of the first area with the second minimum value of each of the cost functions of the other pixels in the first area.

3. The device according to claim 1, wherein In order to identify the first disparity for the central pixel of the first pixel area, the at least one processor is configured to identify the first disparity for the central pixel of the first pixel area based on determining whether the second minimum value of each of the cost functions of the other pixels in the first area is less than the second minimum value of the cost function of the central pixel of the first area.

4. The apparatus of claim 1 , wherein the at least one processor is further configured to determine the number of disparities of the central pixel of the first region at least in part by comparing one or more minimum values ​​of corresponding cost functions of corresponding central pixels of a corresponding number of regions with one or more minimum values ​​of other corresponding cost functions of other corresponding pixels in the number of regions, each region of the number of regions including the central pixel of the first region; in, To determine the third disparity for the central pixel of the first area, the at least one processor is configured to determine the third disparity for the central pixel of the first area based on the determined amount of disparity for the central pixel of the first area.

5. The device according to claim 4, wherein In order to determine the third disparity for the central pixel of the first area based on the determined number of disparities of the central pixel of the first area, the at least one processor is configured to determine a median of the determined number of disparities of the central pixel of the first area as the third disparity for the central pixel of the first area. 6 . The apparatus of claim 1 , wherein the at least one processor is further configured to determine the first region based on associating the center pixel of the first region with a blur cost function based on one or more factors. 7 . The apparatus of claim 6 , wherein the one or more factors comprise at least one of a cost difference between lowest minima of the cost function or a disparity difference between the lowest minima of the cost function. 8 . The apparatus of claim 1 , wherein the at least one processor is further configured to determine depth information for the central pixel of the first region based on the third disparity for the central pixel of the first region.

9. The apparatus according to claim 1, further comprising: an illuminator configured to illuminate a scene; a first image sensor configured to capture the first image of the scene; and A second image sensor is configured to capture the second image of the scene.

10. The apparatus according to claim 9, wherein: The illuminator is configured to illuminate the scene by emitting electromagnetic radiation having a carrier frequency; The at least one processor is further configured to filter the first image using a filter having a passband based on the carrier frequency; and The at least one processor is further configured to filter the second image using the filter.

11. A method for determining disparity information, the method comprising: obtaining a plurality of cost functions, the plurality of cost functions comprising a respective cost function for each of a plurality of pixels of the first image, wherein the respective cost function for each of the plurality of pixels comprises an indication of similarity between a window including the pixel and a corresponding window of the second image as a function of disparity along an epipolar line in the second image; determining a first disparity for a center pixel of a first pixel region at least in part by comparing one or more minima of a cost function of the center pixel of the first pixel region with one or more minima of cost functions of other pixels in the first region, the plurality of cost functions including the cost function of the center pixel of the first region and the cost functions of the other pixels in the first region; determining a second disparity for the central pixel of the first region at least in part by comparing one or more minima of a cost function of the central pixel of a second region with one or more minima of cost functions of other pixels in the second region, the second pixel region including the central pixel of the first region, the plurality of cost functions including the cost function of the central pixel of the second region and the cost functions of the other pixels in the second region; as well as A third disparity for the central pixel of the first area is determined based on the first disparity of the central pixel of the first area and the second disparity of the central pixel of the first area.

12. The method of claim 11 , wherein comparing the one or more minimum values ​​of the cost function of the central pixel of the first region with the one or more minimum values ​​of the cost function of the other pixels in the first region comprises: A second minimum value of the cost function of the central pixel of the first region is compared with a second minimum value of each of the cost functions of the other pixels in the first region.

13. The method of claim 11 , wherein identifying the first disparity for the center pixel of the first pixel region comprises: The first disparity for the center pixel of the first pixel region is identified based on determining whether a second minimum value of each of the cost functions for the other pixels in the first region is less than a second minimum value of the cost function for the center pixel of the first region.

14. The method according to claim 11, further comprising: determining a number of disparities for the center pixel of the first region at least in part by comparing one or more minima of a corresponding cost function for corresponding center pixels of a corresponding number of regions with one or more minima of other corresponding cost functions for other corresponding pixels in the number of regions, each region of the number of regions including the center pixel of the first region; Wherein determining the third disparity for the central pixel of the first region includes determining the third disparity for the central pixel of the first region based on the determined disparity amount of the central pixel of the first region.

15. The method of claim 14 , wherein determining the third disparity for the central pixel of the first region based on the determined number of disparities of the central pixel of the first region comprises: A median of the determined number of disparities of the central pixel of the first area is determined as the third disparity for the central pixel of the first area.

16. The method according to claim 11, further comprising: The first region is determined based on associating the center pixel of the first region with a blur cost function based on one or more factors. 17 . The method of claim 16 , wherein the one or more factors include at least one of a cost difference between lowest minima of the cost function or a disparity difference between the lowest minima of the cost function.

18. The method according to claim 11, further comprising: Depth information for the central pixel of the first area is determined based on the third disparity for the central pixel of the first area.

19. The method according to claim 11, further comprising: Illuminate the scene; capturing the first image of the scene at a first image sensor; as well as The second image of the scene is captured at a second image sensor.

20. The method of claim 19, wherein: illuminating the scene includes emitting electromagnetic radiation having a carrier frequency; The first image is filtered using a filter having a passband based on the carrier frequency; and The second image is filtered using the filter.

21. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to: obtaining a plurality of cost functions, the plurality of cost functions comprising a respective cost function for each of a plurality of pixels of the first image, wherein the respective cost function for each of the plurality of pixels comprises an indication of similarity between a window including the pixel and a corresponding window of the second image as a function of disparity along an epipolar line in the second image; determining a first disparity for a center pixel of a first pixel region at least in part by comparing one or more minima of a cost function of the center pixel of the first pixel region with one or more minima of cost functions of other pixels in the first region, the plurality of cost functions including the cost function of the center pixel of the first region and the cost functions of the other pixels in the first region; determining a second disparity for the central pixel of the first region at least in part by comparing one or more minima of a cost function of the central pixel of a second region with one or more minima of cost functions of other pixels in the second region, the second pixel region including the central pixel of the first region, the plurality of cost functions including the cost function of the central pixel of the second region and the cost functions of the other pixels in the second region; as well as A third disparity for the central pixel of the first area is determined based on the first disparity of the central pixel of the first area and the second disparity of the central pixel of the first area.

22. A non-transitory computer-readable storage medium according to claim 21, wherein the instructions, when executed by the at least one processor, cause the at least one processor to: when comparing the one or more minimum values ​​of the cost function of the center pixel of the first area with the one or more minimum values ​​of the cost function of the other pixels in the first area, compare the second minimum value of the cost function of the center pixel of the first area with the second minimum value of each of the cost functions of the other pixels in the first area.

23. The non-transitory computer-readable storage medium of claim 21, wherein: In order to identify the first disparity for the central pixel of the first pixel area, the instructions, when executed by the at least one processor, cause the at least one processor to: identify the first disparity for the central pixel of the first pixel area based on determining whether the second minimum value of each of the cost functions of the other pixels in the first area is less than the second minimum value of the cost function of the central pixel of the first area.

24. The non-transitory computer-readable storage medium of claim 21, wherein the instructions, when executed by the at least one processor, cause the at least one processor to: determine a number of disparities for the center pixel of the first region by, at least in part, comparing one or more minimum values ​​of a corresponding cost function for corresponding center pixels of a corresponding number of regions with one or more minimum values ​​of other corresponding cost functions for other corresponding pixels in the number of regions, each region of the number of regions including the center pixel of the first region; Wherein the instructions, when executed by the at least one processor, cause the at least one processor to: when determining the third disparity for the central pixel of the first area, determine the third disparity for the central pixel of the first area based on the determined disparity amount of the central pixel of the first area.

25. The non-transitory computer-readable storage medium of claim 24, wherein: In order to determine the third disparity for the central pixel of the first area based on the determined number of disparities of the central pixel of the first area, the instructions, when executed by the at least one processor, cause the at least one processor to: determine a median value of the determined number of disparities of the central pixel of the first area as the third disparity for the central pixel of the first area.

26. The non-transitory computer-readable storage medium of claim 21, wherein the instructions, when executed by the at least one processor, cause the at least one processor to: determine the first region based on associating the center pixel of the first region with a blur cost function based on one or more factors.

27. The non-transitory computer-readable storage medium of claim 26, wherein the one or more factors include at least one of a cost difference between lowest minima of the cost function or a disparity difference between the lowest minima of the cost function.

28. The non-transitory computer-readable storage medium of claim 21, wherein the instructions, when executed by the at least one processor, cause the at least one processor to: determine depth information for the center pixel of the first region based on the third disparity for the center pixel of the first region.

29. An apparatus for determining disparity information, the apparatus comprising: means for obtaining a plurality of cost functions, the plurality of cost functions comprising a respective cost function for each of a plurality of pixels of a first image, wherein the respective cost function for each of the plurality of pixels comprises an indication of similarity between a window including the pixel and a corresponding window of a second image as a function of disparity along an epipolar line in the second image; means for determining a first disparity for a center pixel of a first region of pixels at least in part by comparing one or more minima of a cost function for the center pixel of the first region with one or more minima of cost functions for other pixels in the first region, the plurality of cost functions including the cost function for the center pixel of the first region and the cost functions for the other pixels in the first region; means for determining a second disparity for the central pixel of the first region at least in part by comparing one or more minima of a cost function of the central pixel of a second region with one or more minima of cost functions of other pixels in the second region, a second pixel region including the central pixel of the first region, the plurality of cost functions comprising the cost function of the central pixel of the second region and the cost functions of the other pixels in the second region; and Means for determining a third disparity for the central pixel of the first region based on the first disparity of the central pixel of the first region and the second disparity of the central pixel of the first region.

30. The apparatus of claim 29, wherein comparing the one or more minimum values ​​of the cost function of the central pixel of the first region with the one or more minimum values ​​of the cost function of the other pixels in the first region comprises: A second minimum value of the cost function of the central pixel of the first region is compared with a second minimum value of each of the cost functions of the other pixels in the first region.

Images