Binocular parallax estimation method and system based on Asic chip, and storage medium

Abstract

The embodiment of the invention discloses a binocular parallax estimation method and system based on an As ic chip, and a storage medium, and the method comprises the steps: obtaining a gray-scale map of a binocular image, and the binocular image comprises a first image and a second image; performing census transformation on the grey-scale map of the first image and the grey-scale map of the second image; calculating a matching cost value between each pixel point in the first census transformation graph and each pixel point in the second census transformation graph, wherein the matching cost values form an initial cost matrix; a first parallax value and a second parallax value of each pixel point in the initial cost matrix are calculated, the first aggregation direction group comprises at least two of horizontal left, vertical upper, left upper diagonal and right upper diagonal directions, and the second aggregation direction group comprises at least two of horizontal right, vertical lower, left lower diagonal and right lower diagonal directions; and fusing the first disparity map and the second disparity map to obtain the disparity map. According to the technical scheme of the invention, the real-time requirement of the method is met, and the processing speed of a chip is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a binocular disparity estimation method and system based on an ASIC chip, and a storage medium. Background Art

[0002] Binocular stereo matching is an important branch of computer vision and is widely used in many fields such as autonomous driving, remote sensing, and robotics. The goal of binocular stereo matching is to estimate the binocular disparity based on the correspondence between the two images taken by the left and right cameras to obtain a disparity map.

[0003] Traditional methods for binocular disparity estimation include local algorithms, semi-global algorithms, and global algorithms. Although local algorithms are fast, they have poor accuracy; global algorithms have high accuracy but high computational complexity; and semi-global algorithms take into account both speed and accuracy. Therefore, semi-global algorithms are widely used.

[0004] However, the semi-global algorithm relies on the calculation results of the previous and subsequent orders, consumes a lot of chip resources, and cannot be implemented by Asic chips. Therefore, it is urgent to improve the semi-global algorithm to achieve the purpose of ensuring the real-time requirements of the algorithm under limited Asic chip resources. Summary of the invention

[0005] The purpose of the present application is to provide a binocular disparity estimation method and system based on an Asic chip, and a storage medium to solve at least one of the problems existing in the prior art.

[0006] In order to achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a binocular disparity estimation method, the method comprising:

[0008] Acquire a grayscale image of a binocular image, wherein the binocular image includes a first image and a second image;

[0009] Performing census transformation on the grayscale image of the first image and the grayscale image of the second image respectively to obtain a first census transformation image corresponding to the first image and a second census transformation image corresponding to the second image;

[0010] Calculating matching cost values ​​between each pixel point in the first census transformation image and its corresponding pixel point in the second census transformation image, wherein the matching cost values ​​constitute an initial cost matrix;

[0011] The first disparity value of each pixel point in the initial cost matrix corresponding to the first aggregation direction group and the second disparity value corresponding to the second aggregation direction group are calculated respectively, the first disparity value constitutes a first disparity map, and the second disparity value constitutes a second disparity map,

[0012] The first group of aggregation directions includes at least two of the horizontal left, vertical upper, upper left diagonal and upper right diagonal directions, and the second group of aggregation directions includes at least two of the horizontal right, vertical lower, lower left diagonal and lower right diagonal directions;

[0013] The first disparity map and the second disparity map are fused to obtain a disparity map.

[0014] The binocular disparity estimation method provided by the present invention divides the aggregation direction into a first aggregation direction group and a second aggregation direction group when performing cost aggregation, wherein the first aggregation direction group includes at least two of the horizontal left, vertical upper, upper left diagonal and upper right diagonal directions, and the second aggregation direction group includes at least two of the horizontal right, vertical lower, lower left diagonal and lower right diagonal directions, so that when performing cost aggregation, it is not necessary to require the Asic chip to scan from two directions at the same time and the real-time requirement of the method is met, thereby effectively improving the processing speed of the Asic chip.

[0015] In a possible implementation, respectively calculating the first disparity value of each pixel point in the initial cost matrix corresponding to the first aggregation direction group and the second disparity value corresponding to the second aggregation direction group includes:

[0016] Scanning each pixel point in the initial cost matrix one by one along a first scanning direction, and calculating a first disparity value of the scanned pixel point corresponding to a first aggregation direction group;

[0017] Scanning each pixel point in the initial cost matrix one by one along a second scanning direction, and calculating a second disparity value of the scanned pixel point corresponding to a second aggregation direction group;

[0018] The first scanning direction is scanning from left to right and from top to bottom line by line, and the second scanning direction is scanning from right to left and from bottom to top line by line.

[0019] In a possible implementation manner, calculating the first disparity value of each pixel point corresponding to the first aggregation direction group includes:

[0020] Based on the matching cost value of the target pixel point and the aggregation cost value of the pixel point required for calculating the first disparity value of the target pixel point in each aggregation direction in the first aggregation direction group, the aggregation cost value of the target pixel point in each aggregation direction in the first aggregation direction group is calculated, and the target pixel point is any pixel point in the initial cost matrix;

[0021] Calculating a first disparity value of the target pixel based on an aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group;

[0022] The calculating the second disparity value of each pixel point corresponding to the second aggregation direction group includes:

[0023] Calculate the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group based on the matching cost value of the target pixel and the aggregation cost value of the pixel in each aggregation direction in the second aggregation direction group required for calculating the second disparity value of the target pixel;

[0024] A second disparity value of the target pixel is calculated based on the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group.

[0025] In a possible implementation manner, calculating the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group includes:

[0026] Determining whether a pixel point required for calculating a first disparity value of the target pixel point is within a range of the initial cost matrix;

[0027] If the pixel points required for calculating the first disparity value of the target pixel point are within the range of the initial cost matrix, then according to the energy function, the matching cost value of the target pixel point, and the aggregation cost values ​​of the pixel points required for calculating the first disparity value of the target pixel point in each aggregation direction in the first aggregation direction group, the aggregation cost values ​​of the target pixel point in each aggregation direction in the first aggregation direction group are calculated;

[0028] If there is a pixel point required to calculate the first disparity value of the target pixel point that is not within the range of the initial cost matrix, the aggregation cost value of the target pixel point in the aggregation direction corresponding to the pixel point beyond the range of the initial cost matrix is ​​calculated by the following formula (1):

[0029]

[0030] Among them, C r (x, y, d) represents the aggregation cost value of the target pixel point (x, y) in the aggregation direction r when the disparity is d, d = 0, 1, ..., D-1, D; D represents the maximum disparity search range; X represents the horizontal coordinate of the pixel point corresponding to the path r, W represents the number of pixels in the horizontal direction of the initial cost matrix, min_disp represents the minimum disparity value within the disparity range; Cosy(x, y, d) represents the matching cost value of the target pixel point (x, y) when the disparity is d; default valRepresents the maximum cost value;

[0031] The calculating the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group includes:

[0032] Determining whether the pixel points required for calculating the second disparity value of the target pixel point are within the range of the initial cost matrix;

[0033] If the pixel points required for calculating the second disparity value of the target pixel point are within the range of the initial cost matrix, then according to the energy function, the matching cost value of the target pixel point, and the aggregation cost values ​​of the pixel points required for calculating the second disparity value of the target pixel point in each aggregation direction in the second aggregation direction group, the aggregation cost value of the target pixel point in each aggregation direction in the second aggregation direction group is calculated;

[0034] If there is a pixel point required to calculate the second disparity value of the target pixel point that is not within the range of the initial cost matrix, the aggregation cost value of the target pixel point in the aggregation direction corresponding to the pixel point beyond the range of the initial cost matrix is ​​calculated by formula (1).

[0035] In one possible implementation,

[0036] The calculating the first disparity value of the target pixel point comprises:

[0037] Obtaining the weight of the target pixel in each aggregation direction in the first aggregation direction group based on the Sobel operator and a gray value matrix within a 3*3 matrix range centered on the target pixel;

[0038] Performing weighted processing on the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group based on the weight of the target pixel in each aggregation direction in the first aggregation direction group, to obtain a first disparity value of the target pixel;

[0039] The calculating the second disparity value of the target pixel point includes:

[0040] Obtaining the weight of the target pixel in each aggregation direction in the second aggregation direction group based on the Sobel operator and a gray value matrix within a 3*3 matrix range centered on the target pixel;

[0041] Based on the weight of the target pixel in each aggregation direction in the second aggregation direction group, the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group is weighted to obtain a second disparity value of the target pixel.

[0042] In a possible implementation manner, fusing the first disparity map and the second disparity map to obtain the disparity map includes:

[0043] The first disparity map and the second disparity map are fused using the WTA strategy to obtain a disparity map.

[0044] In a possible implementation, fusing the first disparity map and the second disparity map to obtain a disparity map includes:

[0045] The first disparity map and the second disparity map are fused by a weighted fusion method to obtain a disparity map.

[0046] In a second aspect, an embodiment of the present application provides a binocular disparity estimation system, comprising:

[0047] An image acquisition device, used to acquire a binocular image, wherein the binocular image includes a first image and a second image;

[0048] Asic chip, the Asic chip includes a grayscale processing unit, a census transformation unit, a matching cost calculation unit, a cost aggregation unit and a fusion unit;

[0049] The grayscale processing unit is configured to receive the binocular image sent by the image acquisition device and perform grayscale processing on it to obtain a grayscale image of the binocular image;

[0050] The census transformation unit is configured to perform census transformation on the grayscale image of the first image and the grayscale image of the second image respectively, to obtain a first census transformation image of the first image and a second census transformation image of the second image;

[0051] The matching cost calculation unit is configured to calculate a matching cost value between each pixel point in the first census transformation image and its corresponding pixel point in the second census transformation image, wherein the matching cost values ​​constitute an initial cost matrix;

[0052] The cost aggregation unit is configured to divide the aggregation directions into a first aggregation direction group and a second aggregation direction group, and respectively calculate a first disparity value of each pixel point in the initial cost matrix corresponding to the first aggregation direction group and a second disparity value corresponding to the second aggregation direction group, wherein the first disparity values ​​constitute a first disparity map, and the second disparity values ​​constitute a second disparity map.

[0053] The first group of aggregation directions includes at least two of the horizontal left, vertical upper, upper left diagonal and upper right diagonal directions, and the second group of aggregation directions includes at least two of the horizontal right, vertical lower, lower left diagonal and lower right diagonal directions;

[0054] The fusion unit is configured to fuse the first disparity map and the second disparity map to obtain a disparity map.

[0055] In a possible implementation, the cost aggregation unit is configured to scan each pixel point in the initial cost matrix one by one along a first scanning direction, and calculate a first disparity value of the scanned pixel point corresponding to a first aggregation direction group; scan each pixel point in the initial cost matrix one by one along a second scanning direction, and calculate a second disparity value of the scanned pixel point corresponding to a second aggregation direction group; wherein the first scanning direction is scanning row by row from left to right and from top to bottom, and the second scanning direction is scanning row by row from right to left and from bottom to top.

[0056] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program code. When the computer program code runs on a computer, the computer executes the method provided in the first aspect of the present application.

[0057] Beneficial Effects

[0058] The binocular disparity estimation method provided by the present application divides the aggregation direction into a first aggregation direction group and a second aggregation direction group when performing cost aggregation, the first aggregation direction group includes at least two of the horizontal left, vertical upper, upper left diagonal and upper right diagonal directions, and the second aggregation direction group includes at least two of the horizontal right, vertical lower, lower left diagonal and lower right diagonal directions, and respectively calculates the first disparity value of each pixel point in the initial cost matrix corresponding to the first aggregation direction group and the second disparity value corresponding to the second aggregation direction group. When calculating the first disparity value and the second disparity value of each pixel point, only the data of the pixel point that has been scanned by the Asic chip is relied on, so that the Asic chip can meet the real-time requirements of the binocular disparity estimation method, and effectively improves the processing speed of the Asic chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of eight-path aggregation provided by the prior art;

[0060] Figure 2 A schematic diagram of a process flow of a binocular disparity estimation method provided in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of matching cost calculation based on a census transformation graph provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of a calculation principle of a first disparity map value corresponding to a first aggregation direction group provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of a calculation principle of a second disparity map value corresponding to a second aggregation direction group provided in an embodiment of the present application;

[0064] Figure 6-9 A schematic diagram of the Sobel operator corresponding to each aggregation direction provided in an embodiment of the present application.

[0065] Fig.10 A schematic diagram of the structure of a binocular disparity estimation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0068] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0069] When a vehicle with an autonomous driving function identifies a drivable area, it needs to estimate the binocular disparity based on the binocular images obtained by the binocular camera to obtain a disparity map, and then use the disparity map to determine the drivable area in the image; Asic chips, as an application-specific integrated circuit, have both high efficiency and low power consumption, and are suitable for real-time processing of complex data. Therefore, the vehicle's autonomous driving system often uses Asic chips for binocular disparity estimation. In addition, since the semi-global algorithm for binocular disparity estimation has higher accuracy than the local algorithm and is faster than the global algorithm, the current binocular disparity estimation usually uses a semi-global algorithm.

[0070] However, ASIC chips are optimized for specific applications. While achieving high performance, they have a small cache and cannot meet the real-time requirements of the semi-global algorithm for the disparity map.

[0071] Specifically, when estimating the binocular disparity of a binocular image based on a semi-global algorithm, it is necessary to perform cost aggregation on each pixel to obtain the aggregated cost value of each pixel and establish the connection between adjacent pixels, so as to more accurately reflect the correlation between pixels. Taking the classic eight-path aggregation as an example, Figure 1As shown, when calculating the aggregation cost value of pixel 101, it is necessary to obtain the pixel data of each pixel in the eight-neighborhood range 102 of pixel 101, and the pixel data of each pixel in the eight-neighborhood range 102 needs to be iteratively calculated from the pixel data of each pixel in the first row at the top and the bottom row in the cost matrix. When processing images, Asic chips can only process from a single scanning direction A, such as from top to bottom, from left to right; or from right to left, from bottom to top, and cannot process data in two directions at the same time, therefore, Asic chips cannot implement semi-global algorithms.

[0072] Based on the above considerations, the first embodiment of the present application provides a binocular disparity estimation method based on an Asic chip, such as Figure 2 Shown, including

[0073] Step S100, obtaining a grayscale image of a binocular image, wherein the binocular image includes a first image and a second image;

[0074] Specifically, the binocular image can be obtained through a binocular camera, which includes a first camera and a second camera. The first camera is one of a left camera and a right camera, and the second camera is the other of the left camera and the right camera. The first camera and the second camera usually have the same specifications.

[0075] The image of the object in the three-dimensional space is captured by a binocular camera, wherein the image captured by the first camera is the first image, and the image captured by the second camera is the second image.

[0076] Step S200, performing census transformation on the grayscale image of the first image to obtain a first census transformation image;

[0077] Specifically, a census transformation is performed on the grayscale image of the first image to obtain a first census transformation image, and the value of each pixel in the first census transformation image is a binary string representing a local neighborhood feature of the pixel.

[0078] Performing a census transformation on the grayscale image of the second image to obtain a second census transformation image;

[0079] Specifically, a census transformation is performed on the grayscale image of the second image to obtain a second census transformation image, and the value of each pixel in the second census transformation image is a binary string representing a local neighborhood feature of the pixel.

[0080] Step S300, calculating the matching cost value between each pixel point in the first census transformation image and the corresponding pixel point in the second census transformation image, the matching cost value forming an initial cost matrix;

[0081] It should be noted that the initial cost matrix may be the initial cost matrix of the grayscale image of the left image relative to the grayscale image of the right image, or may be the initial cost matrix of the grayscale image of the right image relative to the grayscale image of the left image.

[0082] Step S400, respectively calculating the first disparity value of each pixel point in the initial cost matrix corresponding to the first aggregation direction group and the second disparity value corresponding to the second aggregation direction group, the first disparity value constitutes a first disparity map, and the second disparity value constitutes a second disparity map,

[0083] The first aggregation direction group is composed of at least two of the horizontal left, vertical upper, upper left diagonal and upper right diagonal directions, and the second aggregation direction group is composed of at least two of the horizontal right, vertical lower, lower left diagonal and lower right diagonal directions;

[0084] Step S500: fusing the first disparity map and the second disparity map to obtain a disparity map.

[0085] The binocular disparity estimation method provided by the present application divides the aggregation direction into a first aggregation direction group and a second aggregation direction group when performing cost aggregation. The first aggregation direction group includes at least two of the horizontal left, vertical upper, upper left diagonal and upper right diagonal directions, and the second aggregation direction group includes at least two of the horizontal right, vertical lower, lower left diagonal and lower right diagonal directions. The first disparity value corresponding to the first aggregation direction group and the second disparity value corresponding to the second aggregation direction group of each pixel point in the initial cost matrix are calculated respectively. When calculating the first disparity value and the second disparity value of each pixel point, only the data of the pixel point that has been scanned before is relied on, so that the Asic chip can meet the real-time requirements of the binocular disparity estimation method, and the processing speed of the Asic chip is effectively improved.

[0086] In a possible implementation, calculating the matching cost between each pixel in the first census transformation image and its corresponding pixel in the second census transformation image includes the following steps:

[0087] According to the preset maximum disparity search range D, the values ​​of each pixel in the first census transformation image and the second census transformation image are traversed, that is, a binary string representing the local neighborhood features of the pixel, and the matching cost value within the disparity search range [0, D] is calculated to generate a three-dimensional initial cost matrix of size W*H*D, as shown in Figure 3 As shown, the first census transformation image c1 and the second census transformation image c2 have the same size, both of which are W*H, where W is the number of pixels in the horizontal direction and H is the number of pixels in the vertical direction.

[0088] Exemplarily, the value of the maximum disparity search range D is, for example, 3, 4, 5 or 6, which is not specifically limited herein.

[0089] The set of matching cost values ​​of each pixel point at each disparity value within the disparity search range is the matching cost value of the pixel point.

[0090] Exemplarily, the matching cost value of the pixel point P with coordinates (x, y) is [cost(x, y, 0), ..., cost(x, y, d), ..., cost(x, y, D)], where

[0091] cost(x,y,0)=hanming_dis(L(x,y),R(x,y))

[0092] …

[0093] cost(x,y,d)=haming-dis(L(x,y),R(xd,y))

[0094] …

[0095] cost(x,y,D)=haming_dis(L(x,y),R(xD,y))

[0096] Among them, cost(x, y, d) represents the matching cost of the pixel point P when the disparity is d, d = 0, 1, ..., D-1, D; D represents the maximum disparity search range; L(x, y) is the value of the pixel point with coordinates (x, y) in the first census transformation image; R(x, y) is the value of the pixel point with coordinates (x, y) in the second census transformation image; hanming-dis() is the Hamming distance calculation function between two points, hanming-dis(L(x, y), R(xd, y)) is the Hamming distance between the value of the pixel point with coordinates (x, y) in the first census transformation image and the value of the pixel point with coordinates (xd, y) in the second census transformation image.

[0097] In one possible implementation, Figure 4 , 5 As shown, each pixel point in the initial cost matrix is ​​scanned one by one along the first scanning direction A1, and the first disparity value of the scanned pixel point corresponding to the first aggregation direction group is calculated, and each pixel point in the initial cost matrix is ​​scanned one by one along the second scanning direction A2, and the second disparity value of the scanned pixel point corresponding to the second aggregation direction group is calculated.

[0098] Among them, the first disparity value of each pixel point is obtained based on the aggregation cost value of each pixel point in each aggregation direction in the first aggregation direction group; the second disparity value of each pixel point is obtained based on the aggregation cost value of each pixel point in each aggregation direction in the second aggregation direction group.

[0099] It should be noted that when the chip performs disparity calculation, the initial cost matrix is ​​copied twice to obtain a first initial cost matrix and a second initial cost matrix.

[0100] Specifically, each pixel point in the first initial cost matrix is ​​scanned one by one along the first scanning direction A1, and a first disparity value of the scanned pixel point corresponding to the first aggregation direction group is calculated;

[0101] Scan each pixel in the second initial cost matrix one by one along the second scanning direction A2, and calculate the second disparity value of the scanned pixel corresponding to the second aggregation direction group.

[0102] Specifically, Figure 4 As shown, starting from the first pixel point at the upper left corner of the initial cost matrix, the first disparity value of each pixel point corresponding to the first aggregation direction group is calculated one by one along the first scanning direction A1;

[0103] like Figure 5 As shown, starting from the first pixel point at the lower right corner of the initial cost matrix, the second disparity value of each pixel point corresponding to the second aggregation direction group is calculated one by one along the second scanning direction A2.

[0104] It should be noted that Figure 4 , 5 The first aggregation direction group shown in the figure consists of horizontal left, vertical top, upper left diagonal and upper right diagonal directions; the second aggregation direction group consists of horizontal right, vertical bottom, lower left diagonal and lower right diagonal directions; correspondingly, the first auxiliary pixel point 202 of the target pixel point 201, that is, the pixel point required for calculating the first disparity value of the target pixel point 201, is located in the horizontal left direction, vertical top direction, upper left diagonal direction and upper right diagonal direction of the target pixel point 201; the second auxiliary pixel point 203 of the target pixel point 201, that is, the pixel point required for calculating the second disparity value of the target pixel point 201, is located in the horizontal right direction, vertical bottom direction, lower left diagonal direction and lower right diagonal direction of the target pixel point 201.

[0105] Based on the above-mentioned grouping of eight aggregation directions and the limitation of the scanning direction, the present invention only relies on the data of the pixel points that have been scanned by the Asic chip when calculating the first disparity value and the second disparity value of each pixel point, so that the Asic chip can meet the real-time requirements of the binocular disparity estimation method, and further improves the speed of binocular disparity estimation.

[0106] In a possible implementation, calculating the first disparity value of each pixel corresponding to the first aggregation direction group includes:

[0107] Based on the matching cost value of the target pixel point and the aggregation cost value of the first auxiliary pixel point 202 of the target pixel point 201 in each aggregation direction in the first aggregation direction group, the aggregation cost value of the target pixel point in each aggregation direction in the first aggregation direction group is calculated, wherein the first auxiliary pixel point 202 of the target pixel point represents the pixel point required to calculate the first disparity value of the target pixel point;

[0108] A first disparity value of the target pixel is calculated based on the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group.

[0109] It should be noted that, according to the above content, the aggregation cost values ​​of the first auxiliary pixel point of the target pixel point in each aggregation direction in the first aggregation direction group are obtained before calculating the aggregation cost values ​​of the target pixel point in each aggregation direction in the first aggregation direction group, and the first auxiliary pixel points of the target pixel point are all preceding pixel points of the target pixel point. Before scanning the target pixel point, the Asic chip has calculated the first disparity value of the first auxiliary pixel point of the target pixel point, and stored the aggregation cost values ​​of the first auxiliary point of the target pixel point in each aggregation direction in the first aggregation direction group obtained during the calculation process.

[0110] like Figure 4 As shown, the first auxiliary pixel points 202 of the target pixel point 201 are at least two of the four pixel points located on the left side, the upper side, the upper left side and the upper right side of the target pixel point 201 .

[0111] In a possible implementation, calculating the second disparity value of each pixel corresponding to the second aggregation direction group includes:

[0112] Based on the matching cost value of the target pixel point and the aggregation cost value of the second auxiliary pixel point of the target pixel point in each aggregation direction in the second aggregation direction group, the aggregation cost value of the target pixel point in each aggregation direction in the second aggregation direction group is calculated, and the second disparity value of the target pixel point is obtained based on the aggregation cost value of the target pixel point in each aggregation direction in the second aggregation direction group.

[0113] like Figure 5 As shown, the second auxiliary pixel points 203 of the target pixel point 201 are at least two of the four pixel points located on the right side, the bottom side, the lower left side and the lower right side of the target pixel point 201 .

[0114] It should be noted that, according to the above content, the aggregation cost values ​​of the second auxiliary pixel point 203 of the target pixel point in each aggregation direction in the second aggregation direction group are all obtained before calculating the aggregation cost values ​​of the target pixel point in each aggregation direction in the second aggregation direction group.

[0115] Since there are pixels located at the edge of the image, when calculating the aggregation cost value of each pixel, it is necessary to determine whether the target pixel is an edge point. The edge point is located in the edge area of ​​the image. There are cases where the pixel points required to calculate its aggregation cost value are not within the range of the initial cost matrix. Therefore, it is necessary to calculate the aggregation cost based on the judgment result.

[0116] In a possible implementation, calculating the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group includes:

[0117] Determine whether the first auxiliary pixel of the target pixel is within the range of the initial cost matrix;

[0118] If the first auxiliary pixel of the target pixel is within the range of the initial cost matrix, the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group is calculated according to the energy function, the matching cost value of the target pixel and the aggregation cost value of the first auxiliary pixel of the target pixel in each aggregation direction in the first aggregation direction group;

[0119] If there is a pixel point required to calculate the first disparity value of the target pixel point that is not within the range of the initial cost matrix, the aggregation cost value of the target pixel point in the aggregation direction corresponding to the pixel point beyond the range of the initial cost matrix is ​​calculated by the following formula (1):

[0120]

[0121] Among them, Cr(x, y, d) represents the aggregation cost value of the target pixel point (x, y) in the aggregation direction r when the disparity is d, d = 0, 1, ..., D-1, D; D represents the maximum disparity search range; X represents the horizontal coordinate of the pixel point corresponding to the path r, W represents the number of pixels in the initial cost matrix in the horizontal direction, min_disp represents the minimum disparity value within the disparity range; Cost(x, y, d) represents the matching cost value of the target pixel point (x, y) when the disparity is d; default val Represents the maximum cost value.

[0122] Specifically, the maximum disparity search range, the minimum disparity value within the disparity range, and the maximum cost value are preset; the value of the maximum cost value is positively correlated with the number of pixels in the sampling window of the census transformation, and taking the sampling window of the census transformation as 7*9 as an example, the maximum cost value here is 31; the minimum disparity value is 0; the maximum disparity search range is less than or equal to 255, for example, 50, 100, 200 or 250. It should be noted that due to the improvement of the semi-global algorithm in this implementation, the semi-global algorithm reduces the resource consumption of the Asic chip, and the maximum value of the disparity search range can reach 255 when performing matching cost calculation and cost aggregation.

[0123] In a possible implementation, calculating the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group includes:

[0124] Determine whether a second auxiliary pixel of the target pixel is within the range of the initial cost matrix, the second auxiliary pixel of the target pixel representing a pixel required to calculate a second disparity value of the target pixel;

[0125] If the second auxiliary pixel of the target pixel is within the range of the initial cost matrix, the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group is calculated according to the energy function, the matching cost value of the target pixel, and the aggregation cost value of the second auxiliary pixel of the target pixel in each aggregation direction in the second aggregation direction group;

[0126] If the second auxiliary pixel of the target pixel is not within the range of the initial cost matrix, the aggregation cost value of the target pixel in the aggregation direction corresponding to the pixel beyond the range of the initial cost matrix is ​​calculated by formula (1).

[0127] The aggregation cost value of the target pixel in each aggregation direction is calculated in combination with the energy function. In a possible implementation, the energy function is, for example, an SGM energy function. The SGM energy function, the matching cost value of the target pixel, and the aggregation cost value of the pixel required to calculate the first disparity value of the target pixel in each aggregation direction in the first aggregation direction group are combined. Calculating the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group includes:

[0128] The matching cost value of the target pixel is obtained, and the matching cost values ​​of the pixels required for calculating the first disparity value of the target pixel in each aggregation direction in the first aggregation direction group are obtained, that is, the aggregation cost values ​​of each pixel located in the previous row of the target pixel and each pixel located on the left side of the target pixel in the row where the target pixel is located in each aggregation direction in the first aggregation direction group obtained previously are calculated, and the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group is calculated according to formula (2):

[0129]

[0130] Among them, L r (p, d) is the aggregation cost value of the target pixel p in the aggregation direction r when the disparity is d, d = 0, 1, ..., D-1, D; C (p, d) is the matching cost value of the target pixel p when the disparity is d, L r (pr, d) is the aggregation cost value when the disparity of the pixel point corresponding to the pixel point p in the aggregation direction r is d, P 1 is the first penalty term, is the minimum aggregation cost of the pixel point p corresponding to the pixel point in the aggregation direction r under each disparity i, P 2 is the second penalty item.

[0131] Specifically, the first penalty item and the second penalty item are preset according to specific circumstances, and the value of the second penalty item is greater than the value of the first penalty item.

[0132] Specifically, r=1, 2, 3, 4, 5, 6, 7, 8, when r is 1, it indicates the horizontal left direction, when r is 2, it indicates the upper left diagonal direction, when r is 3, it indicates the vertical upper direction, when r is 4, it indicates the upper right diagonal direction, when r is 5, it indicates the horizontal right direction, when r is 6, it indicates the lower right diagonal direction, when r is 7, it indicates the vertical lower direction, and when r is 8, it indicates the lower left diagonal direction.

[0133] This implementation method divides the aggregation direction into a first aggregation direction group and a second aggregation direction group, and sets a first scanning direction corresponding to the first aggregation direction group and a second scanning direction corresponding to the second aggregation direction group, so that the Asic chip can meet the real-time requirements of the semi-global algorithm and improve the efficiency of the binocular disparity estimation method; and calculates the aggregation results of the target pixel points in each aggregation direction through the energy function. One cost aggregation calculation only needs to load 2 rows of pixel data, which greatly reduces the resource consumption of the Asic chip and improves the overall computing efficiency.

[0134] Specifically, taking the first aggregation direction group consisting of vertical up, horizontal left and upper left diagonal directions as an example, when solving the aggregation cost value of the target pixel point in each aggregation direction in the first aggregation direction group, it is necessary to obtain the aggregation cost value of each pixel point located on the upper side, left side and upper left side of the target pixel point and the matching cost value of the target pixel point, and calculate the aggregation cost value of the target pixel point in each aggregation direction by formula (2).

[0135] It should be understood that since the present application calculates the first disparity value of each pixel in the initial cost matrix in sequence according to the scanning order of the first scanning direction, the Asic chip has completed the calculation of the first disparity value of the first auxiliary pixel of the target pixel before calculating the target pixel. When calculating the first disparity value of the first auxiliary pixel of the target pixel, it is necessary to obtain its aggregation cost value in each aggregation direction in the first aggregation group. Therefore, the aggregation cost value of any pixel located on the upper side, left side, upper left side and upper right side of the target pixel in each aggregation direction in the first aggregation direction group has been stored in the Asic chip.

[0136] In a possible implementation, calculating the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group according to the energy function, the matching cost value of the target pixel, and the aggregation cost value of the pixel points required to calculate the target pixel in each aggregation direction in the second aggregation direction group includes:

[0137] The matching cost value of the target pixel is obtained, and the aggregation cost value of the pixel points required to calculate the target pixel point in each aggregation direction in the second aggregation direction group is obtained, and the aggregation cost value of the target pixel point in each aggregation direction in the second aggregation direction group is calculated according to formula (2).

[0138] Obtaining the aggregation cost values ​​of the pixels required to calculate the target pixel in each aggregation direction in the second aggregation direction group includes obtaining the previously calculated aggregation cost values ​​of each pixel located in the next row of the target pixel and the aggregation cost values ​​of each pixel located to the right of the target pixel in the same row.

[0139] It should be understood that since the present application calculates the second disparity value of each pixel in the initial cost matrix in sequence according to the scanning order of the second scanning direction, the Asic chip has completed the calculation of the second disparity value of the second auxiliary pixel of the target pixel before calculating the target pixel. When calculating the second disparity value of the second auxiliary pixel of the target pixel, it is necessary to obtain its aggregation cost value in each aggregation direction in the second aggregation group. Therefore, the aggregation cost value of any pixel located on the right side, bottom side, lower right side and lower left side of the target pixel in each aggregation direction group has been stored in the Asic chip, wherein the second auxiliary pixel of the target pixel represents the pixel required to calculate the target pixel.

[0140] It should be noted that the composition of the aggregation directions of the first aggregation direction group and the second aggregation direction group is determined according to the user's selection, and the number of aggregation directions in the first aggregation direction group and the second aggregation direction group may be the same or different, which is not specifically limited here.

[0141] For example, the first aggregation direction group consists of horizontal left and upper left diagonal directions, and the second aggregation direction group consists of lower left diagonal, vertical lower and lower right diagonal directions; or the first aggregation direction group consists of vertical upper and upper right diagonal directions, and the second aggregation direction group consists of horizontal right, vertical lower and lower left diagonal directions.

[0142] The following takes the example that the first aggregation direction group consists of horizontal left and vertical upper directions, and the second aggregation direction group consists of horizontal right and vertical lower directions to explain the specific steps of cost aggregation.

[0143] First, each pixel point in the initial cost matrix is ​​scanned one by one along a first scanning direction, and a first disparity value of the scanned pixel point is calculated; and a second disparity value of each pixel point in the initial cost matrix is ​​calculated one by one along a second scanning direction, including:

[0144] Obtain matching cost values ​​of pixels required to calculate the first disparity value of the target pixel, wherein if the coordinates of the target pixel are (x, y), the pixels required to calculate the first disparity value of the target pixel are the first pixel corresponding to the horizontal left direction and the second pixel corresponding to the vertical upward direction, the coordinates of the first pixel are (x-1, y), and the coordinates of the second pixel are (x, y+1);

[0145] It is determined whether the pixel points (x-1, y) and (x, y+1) required for calculating the first disparity value of the target pixel point are within the initial cost matrix.

[0146] If the coordinate (x-1, y) is beyond the range of the initial cost matrix, and (x, y+1) is within the range of the initial cost matrix, the aggregation cost value of the target pixel in the horizontal left direction is calculated using the following formula (1):

[0147]

[0148] Among them, C 1 (x, y, d) represents the aggregate cost value of the target pixel (x, y) on path 1 when the disparity is d, X represents the horizontal coordinate of the pixel (x-1, y), W represents the number of pixels in the initial cost matrix in the horizontal direction, min_disp represents the minimum disparity value within the disparity range, Cost, y, d) represents the matching cost value of the target pixel (x, y) when the disparity is d; default val Represents the maximum cost value.

[0149] The aggregation cost of the target pixel in the vertical direction is calculated by the following formula (2):

[0150]

[0151] Among them, L 3 (p, d) is the aggregation cost value of the target pixel p in the vertical direction when the disparity is d, C(p, d) is the matching cost value of the target pixel p when the disparity is d, L 3 (p-3, d) is the aggregation cost value of the pixel point (x, y+1) in the vertical direction when the disparity is d, that is, the aggregation cost value of the pixel point (x, y+1) in the vertical direction when the disparity is d, d = 0, 1, ..., D-1, D; D represents the maximum disparity search range; P 1 is the first penalty term, is the minimum aggregation cost value of the pixel points in the vertical direction of pixel point p under each parallax in the vertical direction, that is, the minimum aggregation cost value of the pixel point (x, y+1) under each parallax in the vertical direction, P 2 is the second penalty item.

[0152] The first disparity value of the target pixel is calculated based on the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group, that is, the first disparity value of the target pixel is calculated according to the aggregation cost value of the target pixel in the horizontal left and vertical upper directions.

[0153] It should be noted that the step of calculating the second disparity value of each pixel point may refer to the above-mentioned step of calculating the first disparity value of each pixel point, which will not be described in detail here.

[0154] In order to better reflect the actual changes in the image, the inventors added a directional supervision mechanism to control the importance of different aggregation paths, and adjusted the weight of each path in the cost calculation according to the direction of the path and the noise it brought.

[0155] In a possible implementation, calculating the first disparity value and the second disparity value of the target pixel includes:

[0156] Based on the Sobel operator and the gray value matrix within the 3*3 matrix range centered on the target pixel, the weight of the target pixel in each aggregation direction is obtained;

[0157] Based on the weight of the target pixel in each aggregation direction, the aggregation cost value of the target pixel in each aggregation direction is weighted to obtain the first disparity value and the second disparity value of the target pixel.

[0158] This implementation adds a directional supervision mechanism, which can more accurately reflect the actual cost of the current pixel by assigning different weights to different aggregation directions, improves the ability to resist path noise, and further improves the estimation accuracy and robustness of the binocular disparity estimation method.

[0159] Specifically, Figure 6 As shown, it is the Sobel operator in the horizontal left direction and the horizontal right direction, such as Figure 7 As shown, it is the Sobel operator in the upper left diagonal direction and the lower right diagonal direction, such as Figure 8 As shown, it is the Sobel operator in the vertical upward direction and the vertical downward direction, such as Fig. 9 As shown, the Sobel operator is in the upper right diagonal direction and the lower left diagonal direction.

[0160] The Sobel operator is used to calculate the gradients corresponding to the four sets of directions:

[0161]

[0162] Among them, G1&5 Indicates the gradient values ​​corresponding to the horizontal left and right directions. Represents the gray value matrix within the 3*3 neighborhood centered on the target pixel, P 1 Represents the gray value of the pixel located in the upper left diagonal direction of the target pixel, P 2 Represents the grayscale value of the pixel located vertically above the target pixel, P 3 Represents the gray value of the pixel located in the upper right diagonal direction of the target pixel, P 4 Represents the grayscale value of the pixel located to the horizontal left of the target pixel, P 5 Represents the gray value of the target pixel, P 6 Represents the grayscale value of the pixel located to the horizontal right of the target pixel, P 7 Represents the gray value of the pixel located in the lower left diagonal direction of the target pixel, P 8 Represents the grayscale value of the pixel located vertically below the target pixel, P 9 Represents the gray value of the pixel located in the lower right diagonal direction of the target pixel, G 2&6 Indicates the gradient value corresponding to the upper left diagonal direction and the lower right diagonal direction, G 3&7 Indicates the gradient value corresponding to the vertical upward direction and the vertical downward direction, G 4&8 Indicates the gradient values ​​corresponding to the upper right diagonal direction and the lower left diagonal direction.

[0163] The weight value corresponding to each aggregation direction is equal to the ratio of the sum of the gradient values ​​corresponding to other aggregation directions in the aggregation direction group except the target aggregation direction to the sum of the gradient values ​​corresponding to each aggregation direction in the aggregation direction group.

[0164] When calculating the first disparity value and the second disparity value, this implementation method assigns a weight to the aggregation cost value corresponding to each aggregation direction, adds a directional supervision mechanism for controlling the importance of different aggregation paths, and adjusts the weight of each path in the cost calculation according to the direction of the path and the noise it brings, thereby improving the accuracy of the first disparity value and the second disparity value, so that the disparity map can better reflect the actual changes in the image.

[0165] In a possible implementation, after the first disparity map and the second disparity map are obtained, a WTA (Winner Takes ALL) strategy is adopted to fuse the first disparity map and the second disparity map to obtain a disparity map.

[0166] This implementation performs WTA fusion on the first disparity map and the second disparity map obtained by group calculation, so as to better handle blur and noise in the disparity map and improve the clarity of the disparity map.

[0167] In a possible implementation manner, after the first disparity map and the second disparity map are obtained, a weighted fusion method is used to fuse the first disparity map and the second disparity map to obtain the disparity map.

[0168] Specifically, take the case where the weights corresponding to the first disparity map and the second disparity map are the same as an example:

[0169] L=0.5*L g1 +0.5*L g2

[0170] Among them, L represents the disparity map, L g1 represents the first disparity map, L g2 represents the second disparity map.

[0171] This implementation method performs weighted fusion on the first disparity map and the second disparity map obtained by group calculation, thereby improving the accuracy and detail expression of the depth map and improving the adaptability of the algorithm.

[0172] The second embodiment of the present application provides a binocular disparity estimation system, such as Fig.10 As shown, it includes an image acquisition device 30 for acquiring a binocular image, wherein the binocular image includes a first image and a second image;

[0173] Asic chip 31, Asic chip 31 includes a grayscale processing unit 311, a census transformation unit 312, a matching cost calculation unit 313, a cost aggregation unit 314 and a fusion unit 315;

[0174] The grayscale processing unit 311 is configured to receive the binocular image sent by the image acquisition device and perform grayscale processing on it to obtain a grayscale image of the binocular image;

[0175] The census transformation unit 312 is configured to perform census transformation on the grayscale image of the first image and the grayscale image of the second image respectively to obtain a first census transformation image of the first image and a second census transformation image of the second image;

[0176] The matching cost calculation unit 313 is configured to calculate the matching cost value between each pixel point in the first census transformation image and its corresponding pixel point in the second census transformation image, and the matching cost value constitutes an initial cost matrix;

[0177] The cost aggregation unit 314 is configured to divide the aggregation direction into a first aggregation direction group and a second aggregation direction group, and respectively calculate the first disparity value of each pixel point in the initial cost matrix corresponding to the first aggregation direction group and the second disparity value corresponding to the second aggregation direction group, the first disparity value constitutes a first disparity map, and the second disparity value constitutes a second disparity map.

[0178] The first aggregation direction group includes at least two of the horizontal left, vertical upper, upper left diagonal and upper right diagonal directions, and the second aggregation direction group includes at least two of the horizontal right, vertical lower, lower left diagonal and lower right diagonal directions;

[0179] The fusion unit 315 is configured to fuse the first disparity map and the second disparity map to obtain a disparity map.

[0180] In a possible implementation, the cost aggregation unit is configured to scan each pixel point in the initial cost matrix one by one along a first scanning direction, and calculate a first disparity value of the scanned pixel point corresponding to a first aggregation direction group; scan each pixel point in the initial cost matrix one by one along a second scanning direction, and calculate a second disparity value of the scanned pixel point corresponding to a second aggregation direction group; wherein the first scanning direction is scanning row by row from left to right and from top to bottom, and the second scanning direction is scanning row by row from right to left and from bottom to top.

[0181] The specific embodiments of the binocular disparity estimation system provided in this embodiment can refer to the relevant embodiments of the binocular disparity estimation method provided in the first embodiment of the present application, which will not be repeated here. The beneficial effects of the binocular disparity estimation system provided in this embodiment can refer to the beneficial effects of the binocular disparity estimation method provided above, which will not be repeated here.

[0182] A third embodiment of the present application provides a computer-readable storage medium. The embodiment of the present application provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is run on an electronic device, the electronic device executes any binocular disparity estimation method in the above embodiments.

[0183] The executable instructions may be specifically used to enable the electronic device to execute the binocular disparity estimation method in the above embodiment.

[0184] For example, the computer readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0185] The beneficial effects that can be achieved by the computer-readable storage medium provided in the embodiment of the present application can refer to the beneficial effects of the corresponding binocular disparity estimation method based on Asic chip provided above, which will not be repeated here.

[0186] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. A binocular disparity estimation method based on ASIC chip, characterized in that: include Acquire a grayscale image of a binocular image, wherein the binocular image includes a first image and a second image; Performing census transformation on the grayscale image of the first image and the grayscale image of the second image respectively to obtain a first census transformation image corresponding to the first image and a second census transformation image corresponding to the second image; Calculating matching cost values ​​between each pixel point in the first census transformation image and its corresponding pixel point in the second census transformation image, wherein the matching cost values ​​constitute an initial cost matrix; The first disparity value of each pixel point in the initial cost matrix corresponding to the first aggregation direction group and the second disparity value corresponding to the second aggregation direction group are calculated respectively, the first disparity value constitutes a first disparity map, and the second disparity value constitutes a second disparity map, The first group of aggregation directions includes at least two of the horizontal left, vertical upper, upper left diagonal and upper right diagonal directions, and the second group of aggregation directions includes at least two of the horizontal right, vertical lower, lower left diagonal and lower right diagonal directions; The first disparity map and the second disparity map are fused to obtain a disparity map.

2. The binocular disparity estimation method according to claim 1, characterized in that: The respectively calculating the first disparity value of each pixel point in the initial cost matrix corresponding to the first aggregation direction group and the second disparity value corresponding to the second aggregation direction group includes: Scanning each pixel point in the initial cost matrix one by one along a first scanning direction, and calculating a first disparity value of the scanned pixel point corresponding to a first aggregation direction group; Scanning each pixel point in the initial cost matrix one by one along a second scanning direction, and calculating a second disparity value of the scanned pixel point corresponding to a second aggregation direction group; The first scanning direction is scanning from left to right and from top to bottom line by line, and the second scanning direction is scanning from right to left and from bottom to top line by line.

3. The binocular disparity estimation method according to claim 2, characterized in that: The first disparity value of the first aggregation direction group corresponding to the pixel point of the calculation scan includes: Based on the matching cost value of the target pixel point and the aggregation cost value of the pixel point required for calculating the first disparity value of the target pixel point in each aggregation direction in the first aggregation direction group, the aggregation cost value of the target pixel point in each aggregation direction in the first aggregation direction group is calculated, and the target pixel point is any pixel point in the initial cost matrix; Calculating a first disparity value of the target pixel based on an aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group; The second disparity value of the pixel point of the calculation scan corresponding to the second aggregation direction group includes: Calculate the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group based on the matching cost value of the target pixel and the aggregation cost value of the pixel in each aggregation direction in the second aggregation direction group required for calculating the second disparity value of the target pixel; A second disparity value of the target pixel is calculated based on the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group.

4. The binocular disparity estimation method according to claim 3, characterized in that: The calculating the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group includes: Determining whether a pixel point required for calculating a first disparity value of the target pixel point is within a range of the initial cost matrix; If the pixel points required for calculating the first disparity value of the target pixel point are within the range of the initial cost matrix, then according to the energy function, the matching cost value of the target pixel point, and the aggregation cost values ​​of the pixel points required for calculating the first disparity value of the target pixel point in each aggregation direction in the first aggregation direction group, the aggregation cost values ​​of the target pixel point in each aggregation direction in the first aggregation direction group are calculated; If there is a pixel point required to calculate the first disparity value of the target pixel point that is not within the range of the initial cost matrix, the aggregation cost value of the target pixel point in the aggregation direction corresponding to the pixel point beyond the range of the initial cost matrix is ​​calculated by the following formula (1): Among them, C r (x, y, d) represents the aggregation cost of the target pixel (x, y) in the aggregation direction r when the disparity is d, d = 0, 1, ..., D-1, D; D represents the maximum disparity search range; X represents the horizontal coordinate of the pixel corresponding to the path r, W represents the number of pixels in the horizontal direction of the initial cost matrix, min_disp represents the minimum disparity value within the disparity range; Cost(x, y, d) represents the matching cost of the target pixel (x, y) when the disparity is d; default val Represents the maximum cost value; The calculating the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group includes: Determining whether the pixel points required for calculating the second disparity value of the target pixel point are within the range of the initial cost matrix; If the pixel points required for calculating the second disparity value of the target pixel point are within the range of the initial cost matrix, then according to the energy function, the matching cost value of the target pixel point, and the aggregation cost values ​​of the pixel points required for calculating the second disparity value of the target pixel point in each aggregation direction in the second aggregation direction group, the aggregation cost value of the target pixel point in each aggregation direction in the second aggregation direction group is calculated; If there is a pixel point required to calculate the second disparity value of the target pixel point that is not within the range of the initial cost matrix, the aggregation cost value of the target pixel point in the aggregation direction corresponding to the pixel point beyond the range of the initial cost matrix is ​​calculated by formula (1).

5. The binocular disparity estimation method according to claim 3, characterized in that: The calculating the first disparity value of the target pixel point comprises: Obtaining the weight of the target pixel in each aggregation direction in the first aggregation direction group based on the Sobel operator and a gray value matrix within a 3*3 matrix range centered on the target pixel; Performing weighted processing on the aggregation cost value of the target pixel in each aggregation direction in the first aggregation direction group based on the weight of the target pixel in each aggregation direction in the first aggregation direction group, to obtain a first disparity value of the target pixel; The calculating the second disparity value of the target pixel point includes: Obtaining the weight of the target pixel in each aggregation direction in the second aggregation direction group based on the Sobel operator and a gray value matrix within a 3*3 matrix range centered on the target pixel; Based on the weight of the target pixel in each aggregation direction in the second aggregation direction group, the aggregation cost value of the target pixel in each aggregation direction in the second aggregation direction group is weighted to obtain a second disparity value of the target pixel.

6. The binocular disparity estimation method according to claim 1, characterized in that: The fusing the first disparity map and the second disparity map to obtain a disparity map comprises: The first disparity map and the second disparity map are fused using the WTA strategy to obtain a disparity map.

7. The binocular disparity estimation method according to claim 1, characterized in that: The fusing the first disparity map and the second disparity map to obtain a disparity map comprises: The first disparity map and the second disparity map are fused by a weighted fusion method to obtain a disparity map.

8. A binocular disparity estimation system, characterized in that: include An image acquisition device, used to acquire a binocular image, wherein the binocular image includes a first image and a second image; Asic chip, the Asic chip includes a grayscale processing unit, a census transformation unit, a matching cost calculation unit, a cost aggregation unit and a fusion unit; The grayscale processing unit is configured to receive the binocular image sent by the image acquisition device and perform grayscale processing on it to obtain a grayscale image of the binocular image; The census transformation unit is configured to perform census transformation on the grayscale image of the first image and the grayscale image of the second image respectively, to obtain a first census transformation image of the first image and a second census transformation image of the second image; The matching cost calculation unit is configured to calculate a matching cost value between each pixel point in the first census transformation image and its corresponding pixel point in the second census transformation image, wherein the matching cost values ​​constitute an initial cost matrix; The cost aggregation unit is configured to divide the aggregation directions into a first aggregation direction group and a second aggregation direction group, and respectively calculate a first disparity value of each pixel point in the initial cost matrix corresponding to the first aggregation direction group and a second disparity value corresponding to the second aggregation direction group, wherein the first disparity values ​​constitute a first disparity map, and the second disparity values ​​constitute a second disparity map. The first group of aggregation directions includes at least two of the horizontal left, vertical upper, upper left diagonal and upper right diagonal directions, and the second group of aggregation directions includes at least two of the horizontal right, vertical lower, lower left diagonal and lower right diagonal directions; The fusion unit is configured to fuse the first disparity map and the second disparity map to obtain a disparity map.

9. The system according to claim 8, characterized in that The cost aggregation unit is configured to scan each pixel point in the initial cost matrix one by one along a first scanning direction, and calculate a first disparity value of the scanned pixel point corresponding to a first aggregation direction group; scan each pixel point in the initial cost matrix one by one along a second scanning direction, and calculate a second disparity value of the scanned pixel point corresponding to a second aggregation direction group; wherein the first scanning direction is scanning row by row from left to right and from top to bottom, and the second scanning direction is scanning row by row from right to left and from bottom to top.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code, and when the computer program code is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images