Stereo matching method and device, computer device and storage medium

Abstract

The application relates to a stereo matching method and device, computer equipment and a storage medium. The method comprises the following steps: performing grid processing on a to-be-matched image to obtain image blocks of the to-be-matched image, and for each image block, performing inverse search optimization operation on the gray values of each pixel point on the image block and the gray values of first candidate matching points corresponding to the pixel points on a reference image by using a pre-constructed inverse search optimization model to determine the first inverse depth value of the image block, and then determining the second inverse depth value corresponding to each pixel point according to the first inverse depth value of each image block, obtaining the depth map of the to-be-matched image by using a pre-constructed energy minimization model according to the second inverse depth value corresponding to each pixel point, and improving the calculation efficiency.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a stereo matching method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Stereo matching, also known as disparity estimation or binocular depth estimation, is a research hotspot in acquiring structural information from 3D scenes. It plays a crucial role in applications such as binocular vision systems, 3D tracking, and 3D navigation. The basic principle of stereo matching is to observe objects within the same field of view from two viewpoints, match corresponding pixels, calculate the disparity, and obtain 3D information.

[0003] Currently, dense stereo matching is commonly used, which involves four steps: pixel feature description, matching cost calculation, matching cost aggregation, and disparity calculation. However, dense stereo matching suffers from low computational efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a stereo matching method, apparatus, computer equipment, and storage medium that can improve the computational efficiency of stereo matching in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a stereo matching method. The method includes:

[0006] The image to be matched is processed into a grid to obtain the image blocks of the image to be matched;

[0007] For each of the image blocks, based on the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching points corresponding to the pixels in the reference image, an inverse search optimization operation is performed using a pre-built inverse search optimization model to determine the first inverse depth value of the image block, wherein the first candidate matching point is determined based on the inverse depth value obtained after the current inverse search optimization operation.

[0008] Based on the first inverse depth value of each image block, determine the second inverse depth value corresponding to each pixel point;

[0009] The depth map of the image to be matched is obtained by using a pre-built energy minimization model based on the second inverse depth value corresponding to each pixel.

[0010] Secondly, this application also provides a stereo matching device. The device includes:

[0011] The first obtaining module is used to perform gridding processing on the image to be matched to obtain each image block of the image to be matched;

[0012] The first determining module is used to perform an inverse search optimization operation on each of the image blocks, based on the gray values ​​of each pixel on the image block and the gray values ​​of the first candidate matching points on the reference image corresponding to the pixel, using a pre-built inverse search optimization model to determine the first inverse depth value of the image block, wherein the first candidate matching point is determined based on the inverse depth value obtained after the current inverse search optimization operation.

[0013] The second determining module is used to determine the second inverse depth value corresponding to each pixel point based on the first inverse depth value of each image block.

[0014] The second obtaining module is used to obtain the depth map of the image to be matched by using a pre-built energy minimization model based on the second inverse depth value corresponding to each pixel.

[0015] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of any of the methods described above.

[0016] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0017] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0018] The aforementioned stereo matching method, apparatus, computer equipment, and storage medium obtain image blocks of the image to be matched by meshing the image. For each image block, based on the gray values ​​of each pixel and the gray values ​​of the corresponding first candidate matching points in the reference image, an inverse search optimization operation is performed using a pre-built inverse search optimization model to determine the first inverse depth value of the image block. Then, based on the first inverse depth values ​​of each image block, the second inverse depth value corresponding to each pixel is determined. Finally, based on the second inverse depth values ​​corresponding to each pixel, a pre-built energy minimization model is used to obtain the depth map of the image to be matched. Traditional dense stereo matching methods suffer from high computational costs due to matching cost aggregation, resulting in decreased speed and efficiency. This method, however, eliminates the need for matching cost aggregation and obtains the depth map of the image to be matched based solely on the first inverse depth values ​​of each image block, thus improving computational efficiency compared to dense stereo matching methods. Attached Figure Description

[0019] Figure 1This is a flowchart illustrating a stereo matching method provided in an embodiment of this application;

[0020] Figure 2 This is a flowchart illustrating a method for determining a first inverse depth value provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of a stereo matching process provided in an embodiment of this application;

[0022] Figure 4 This is a flowchart illustrating a method for determining the inverse depth value corresponding to the current inverse search optimization operation provided in an embodiment of this application.

[0023] Figure 5 This is a flowchart illustrating a method for determining a second parameter provided in an embodiment of this application;

[0024] Figure 6 This is a schematic flowchart of a depth map acquisition method provided in an embodiment of this application;

[0025] Figure 7 This is a flowchart illustrating a method for determining the inverse depth value of a target provided in an embodiment of this application;

[0026] Figure 8 This is a schematic diagram of the structure of a stereo matching device provided in an embodiment of this application;

[0027] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] Reference Figure 1 , Figure 1 This is a flowchart illustrating a stereo matching method provided in an embodiment of this application. The method is applied to electronic devices, including smartphones, tablets, laptops, wearable mobile terminals, etc. The electronic device is equipped with a first camera and a second camera. The first camera, for example, is used to capture a left image, and the second camera, for example, is used to capture a right image. Either the left image or the right image can be used as the image to be matched. The following embodiments use the left image as the image to be matched as an example. The method includes the following steps:

[0030] S101. Perform gridding processing on the image to be matched to obtain each image block of the image to be matched.

[0031] Meshing is one of the main steps in DIS optical flow, which stands for Dense Inverse Search. Meshing the image to be matched yields image patches, each representing a single patch. Multiple patches are obtained after meshing the image.

[0032] S102. For each image block, based on the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching points corresponding to the pixels in the reference image, an inverse search optimization operation is performed using a pre-built inverse search optimization model to determine the first inverse depth value of the image block. The first candidate matching point is determined based on the inverse depth value obtained after the current inverse search optimization operation.

[0033] In this step, the inverse search optimization model can be any model provided by Gauss_Newton, the Newton method, gradient descent, or the Levenberg-Marquardt method. Based on the gray values ​​of each pixel in the image patch and the gray values ​​of the first candidate matching points corresponding to the pixels in the reference image, the inverse search optimization operation is performed using the pre-built inverse search optimization model to determine the first inverse depth value of each patch.

[0034] This step is the inverse search step of DIS optical flow. Traditional DIS optical flow inverse search utilizes a two-dimensional inverse search model, calculating the offset in two-dimensional space. The two-dimensional search range is larger than the one-dimensional search, resulting in lower computational efficiency for traditional DIS optical flow inverse search. However, in this embodiment, a pre-constructed inverse search optimization model is used to perform inverse search optimization operations to determine the first inverse depth value of the image patch. That is, only this first inverse depth value needs to be determined during the inverse search process, reducing the search range to one-dimensional space and thus improving computational efficiency.

[0035] S103. Determine the second inverse depth value corresponding to each pixel point based on the first inverse depth value of each image block.

[0036] Each pixel in the image patch corresponds to a target inverse depth value, which is equal to the reciprocal of the target inverse depth. This step is a DIS optical flow densification step, through which the second inverse depth value of each pixel on each patch can be determined.

[0037] S104. Based on the second inverse depth value corresponding to each pixel, obtain the depth map of the image to be matched using a pre-built energy minimization model.

[0038] This step is the energy minimization step of DIS optical flow, through which the depth map of the image to be matched can be obtained.

[0039] A depth map is a map determined based on the depth values ​​corresponding to each pixel in the image to be matched. After determining the target inverse depth value corresponding to each pixel, the depth value corresponding to each pixel can be determined based on the target inverse depth value, and the depth map of the image to be matched can be obtained based on the depth values ​​corresponding to each pixel.

[0040] The method provided in this embodiment obtains image blocks of the image to be matched by meshing the image. For each image block, based on the gray values ​​of each pixel in the image block and the gray values ​​of the corresponding first candidate matching points in the reference image, an inverse search optimization operation is performed using a pre-built inverse search optimization model to determine the first inverse depth value of the image block. Then, based on the first inverse depth values ​​of each image block, the second inverse depth value corresponding to each pixel is determined. Based on the second inverse depth values ​​corresponding to each pixel, a pre-built energy minimization model is used to obtain the depth map of the image to be matched. Traditional dense stereo matching methods suffer from high computational cost aggregation, leading to a decrease in the speed and efficiency of stereo matching. This method, however, eliminates the need for matching cost aggregation and obtains the depth map of the image to be matched based on the first inverse depth values ​​of each image block, thus improving computational efficiency compared to dense stereo matching methods.

[0041] It should be noted that the stereo matching method provided in this embodiment only needs to calculate the target inverse depth value corresponding to the pixel point, that is, solve for one unknown quantity. The depth map of the image to be matched can be obtained based on this parameter. That is, the target inverse depth value is calculated first, and then the depth map of the reference image is obtained based on the target inverse depth value. Then, the target matching point that matches the pixel point on the image to be matched is determined on the reference image based on the depth map, thus realizing stereo matching. Since only one unknown quantity needs to be solved, the amount of calculation is reduced, and the calculation efficiency of stereo matching is further improved.

[0042] Reference Figure 2 , Figure 2 This is a flowchart illustrating a method for determining a first inverse depth value according to an embodiment of this application. This embodiment relates to an optional implementation of determining the first inverse depth value of an image block by performing an inverse search optimization operation using a pre-built inverse search optimization model based on the grayscale values ​​of each pixel in the image block and the grayscale values ​​of first candidate matching points corresponding to those pixels in a reference image. Based on the above embodiment, S102 includes the following steps:

[0043] S201. Determine the result of the current inverse search optimization operation based on the sum of the squares of the first differences between the gray values ​​of each pixel on the image block and the gray values ​​of the first candidate matching point.

[0044] The determination of the first candidate matching point can be achieved in the following way:

[0045] For each image patch, an inverse search optimization operation is performed based on the directional derivative of each pixel in the image patch to obtain the inverse depth value corresponding to the current inverse search optimization operation.

[0046] Based on the inverse depth value corresponding to the current inverse search optimization operation, determine the corresponding first candidate matching point on the reference image.

[0047] Wherein, if the epipolar unit tangent vector of the pixel in the image to be matched is denoted by t(p), then t(p) = (t x (p)t y (p)) T , and t x (p) 2 +t y (p) 2 =1,t x (p) represents the component of the epipolar unit tangent vector along the x-axis in a two-dimensional coordinate system, t y (p) represents the component of the epipolar unit tangent vector in the y-axis direction of the two-dimensional coordinate system.

[0048] If the gradient of a pixel in the image to be matched is represented by g(p), then g(p) = (g x (p) g y (p)) T , where g x (p) represents the gradient component along the x-axis in a two-dimensional coordinate system, g y (p) represents the component of the epipolar unit tangent vector in the y-axis direction of the two-dimensional coordinate system.

[0049] Once the epipolar unit tangent vector and gradient of each pixel in the image to be matched are determined, the directional derivative of that pixel can be determined based on these vectors. If the directional derivative is denoted by der(p), then der(p) = g(p). T t(p)=g x (p)t x (p)+g y (p)t y (p).

[0050] In this embodiment, taking Gauss_Newton as an example, we will introduce how to determine the first inverse depth value of each image block of the image to be matched based on the directional derivative of each pixel. For example, the first inverse depth value of the image block can be determined according to the following formulas (1)-(4).

[0051] s k =s k-1 -JtJ -1 Jtb k-1(1)

[0052]

[0053]

[0054]

[0055] For example, s can be determined according to the above formulas (1), (2), and (3). k The value of k is within the range of 1 greater than or equal to and less than or equal to the preset inverse search iteration number threshold. For example, if the preset inverse search iteration number threshold is 10, then the value of k is within the range of 1 greater than or equal to and less than or equal to 10. p is the pixel point on the image block P in the image to be matched. The inverse depth value corresponding to the current inverse search optimization operation can be determined by the above formulas (1), (2), and (3), that is, the value of s is determined. k .

[0056] Among them, T(x) in the above formula (3) p (s k-1 ) y p (s k-1 )) indicates that s = s k-1 The grayscale value of a candidate matching point for pixel p, where the candidate matching point is a pixel in the reference image. For example, the reference... Figure 3 , Figure 3 This is a schematic diagram of a stereo matching process provided in an embodiment of this application. Figure 3 In this diagram, the left image L is used as the image to be matched, and the right image R is used as the reference image. The image to be matched is denoted by I, and the reference image is denoted by T. l O r The baseline, P, and the baseline form a polar plane PO. l O r The epipolar line is the intersection of the epipolar plane, the image to be matched, and the reference image. For example, epipolar line 1 is the intersection of the epipolar plane and the image to be matched, and epipolar line 2 is the intersection of the epipolar plane and the reference image. Therefore, each pixel p in the image to be matched corresponds to an epipolar line passing through point p: (x p (s) y p (s)) T .

[0057] After determining the corresponding first candidate matching point on the reference image, the gray value T(T(x) of the first candidate matching point can be determined. p (s) y p (s)), then calculate the sum of squares of the differences in gray values ​​between pixel p and the first candidate matching point (I(p)-T(T(x)). p (s) y p (s)))2 Similarly, the sum of squares of the differences corresponding to other pixels on image block P can be calculated. Based on the sum of squares of the differences corresponding to each pixel on image block P, the result of the current inverse search optimization operation can be solved using formula (4). The result of the current inverse search optimization operation can be equal to ∑ p∈P (I(p)-T(T(x p (s) y p (s))) 2 , can also be equal to

[0058] It should be noted that when the value of 's' in the above formula changes, the position of the candidate matching point of pixel p changes accordingly, and the gray value of the candidate matching point of pixel p may change. However, regardless of how the value of 's' changes, the candidate matching point of pixel p will always fall on the epipolar line of pixel p in the reference image. This epipolar line indicates... Figure 3 The epipolar line on the right side of the image. Since all candidate matching points of pixel p can fall on the epipolar line, the accuracy of DIS optical flow can be improved compared to traditional DIS optical flow.

[0059] S202. If the result of the current inverse search optimization operation meets the preset inverse search optimization conditions, then the inverse search optimization model is used to perform inverse search optimization operation to obtain the first inverse depth value of the image patch.

[0060] Wherein, the result of each inverse search optimization operation is equal to ∑ p∈P (I(p)-T(T(x p (s) y p (s))) 2 For example, if the sum of squares corresponding to the current inverse search optimization operation is less than the sum of squares corresponding to the previous inverse search optimization operation, and the current number of iterations is less than the preset threshold for the number of inverse search iterations, then the inverse search optimization model is used to perform an inverse search optimization operation to obtain the first inverse depth value of the image patch.

[0061] The inverse search optimization model can include a model composed of the above formulas (1), (2), (3) and (4). The result of the next inverse search optimization operation can be obtained through the inverse search optimization model until the result of the inverse search optimization operation satisfies the preset iteration termination condition. Under the condition of satisfying the iteration termination condition, the first inverse depth value of the image block is determined.

[0062] In this step, s is determined. k Then, replace s in equation (4) with s k And calculate the result of the inverse search optimization operation ∑ p∈P (I(p)-T(T(x p (s) y p(s))) 2 Assuming the result is represented by A k This indicates that the result of the previous inverse search optimization operation refers to s. k-1 Substitute into ∑ p∈P (I(p)-T(T(x p (s) y p (s))) 2 The result was calculated later, assuming the result is represented by A. k-1 This means that if A k Less than A k-1 If the current number of iterations is less than the preset threshold for the number of inverse search iterations, then the inverse search optimization model is used to perform inverse search optimization to obtain the first inverse depth value of the image patch, that is, to continue calculating the result of the next inverse search optimization operation, that is, to continue using formulas (1)-(3) to obtain s. k+1 s k+1 This refers to the new inverse depth value corresponding to the next inverse search optimization operation, and s k+1 Substituting ∑ into formula (4) p∈P (I(p)-T(T(x p (s) y p (s))) 2 The result A was then calculated. k+1 If A k+1 Less than A k If k+1 at this point equals the preset threshold for the number of inverse search iterations, then the preset iteration termination condition is met, and the inverse search optimization operation is stopped. Therefore, s can be... k+1 As the first inverse depth value of the image patch, where s k+1 This is the inverse depth value corresponding to the next inverse search optimization operation. If A k+1 Not less than A k Then s k As the first inverse depth value of the image patch, s k This is the inverse depth value corresponding to the current inverse search optimization operation.

[0063] For example, if the threshold for the number of inverse search iterations is 10, after calculating A... 10 Afterwards, A 10 For the result of the last inverse search optimization operation, if A 10 If it is less than A9, then s 10 As the first inverse depth value of the image patch. If A 10 If the value is not less than A9, then s9 will be used as the first inverse depth value of the image block.

[0064] Optionally, if the result of the current inverse search optimization operation does not meet the preset inverse search optimization conditions, the first inverse depth value of the image patch is determined based on the result of the previous inverse search optimization operation.

[0065] If the result of the current inverse search optimization operation is not less than the result of the previous inverse search optimization operation, it means that the result of the current inverse search optimization operation does not meet the preset inverse search optimization conditions. In this case, it is determined that the inverse search optimization operation meets the iteration termination condition, and the inverse depth value corresponding to the previous inverse search optimization operation is taken as the first inverse depth value of the image patch.

[0066] Based on the examples above, if the sum of squares corresponding to the current inverse search optimization operation is A5, and A5 is not less than A4, then the iteration termination condition can be determined, and s4 can be used as the first inverse depth value of image block P.

[0067] Reference Figure 4 , Figure 4 This is a flowchart illustrating a method for determining the inverse depth value corresponding to the current inverse search optimization operation, provided in an embodiment of this application. Based on the above embodiment, the method includes the following steps:

[0068] S401. For each image block, sum the squares of the directional derivatives of each pixel in the image block to obtain the first parameter.

[0069] Referring to the above formula (2), the first parameter JTj is obtained by summing the squares of the directional derivatives of each pixel in the image block.

[0070] S402. Determine the second parameter based on the directional derivative of each pixel in the image block, the gray value of each pixel in the image block, and the gray value of the second candidate matching point, wherein the second candidate matching point is a pixel on the reference image determined based on the inverse depth value obtained from the previous inverse search optimization operation.

[0071] Referring to the above formula (3), based on the directional derivative of each pixel in the image block, the gray value of each pixel in the image block, and the gray value of the second candidate matching point, the second parameter JTb can be obtained using formula (3). k-1 The second candidate matching point is a pixel on the reference image determined based on the inverse depth value corresponding to the previous inverse search optimization operation. Here, the inverse depth value corresponding to the previous inverse search optimization operation refers to s. k-1 .

[0072] S403. Based on the first parameter, the second parameter, and the inverse depth value obtained from the previous inverse search optimization operation, obtain the inverse depth value of the current inverse search optimization operation.

[0073] S403 can be implemented using the following steps:

[0074] A. Determine the product of the reciprocal of the first parameter and the second parameter, i.e., the reciprocal of the first parameter equals JTJ. -1The second parameter is equal to JTb k-1 .

[0075] B. Determine the difference between the inverse depth value corresponding to the previous inverse search optimization operation and the result of the second product, where the inverse depth value corresponding to the previous inverse search optimization operation refers to s. k-1 Calculate s k-1 -JTJ -1 ×JTb k-1 The difference.

[0076] C. Determine the inverse depth value corresponding to the current inverse search optimization operation based on the difference. For example, the difference can be used as the inverse depth value corresponding to the current inverse search optimization operation, that is, the difference can be used as s. k Alternatively, the result of multiplying the difference by a preset coefficient can be used as s. k .

[0077] Reference Figure 5 , Figure 5 This is a flowchart illustrating a method for determining a second parameter according to an embodiment of this application. This embodiment relates to an optional implementation of how to determine the second parameter based on the directional derivative of each pixel in the image block, the grayscale value of each pixel in the image block, and the grayscale value of the second candidate matching point.

[0078] Based on the above embodiments, S402 may include the following steps:

[0079] S501. Determine the sum of squares of the directional derivatives of each pixel in the image block.

[0080] This step can be referred to the above formula (2) and the corresponding introduction.

[0081] S502. Determine the second difference between the gray value of each pixel in the image block and the gray value of the corresponding second candidate matching point.

[0082] This step can refer to the information about I(p) and T(x) in formula (3) above. p (s k-1 ) y p (s k-1 I(p) is the gray value of pixel p in the image patch, and T(x) is the gray value of pixel p. p (s k-1 ) y p (s k-1 The grayscale value of the second candidate matching point corresponding to pixel p. It can be understood that each pixel in this image block corresponds to a second candidate matching point.

[0083] S503. Obtain the first product result of the directional derivative of each pixel in the image block and the corresponding second difference, and sum the first product results corresponding to each pixel in the image block to obtain the second parameter.

[0084] In this step, we can use the above formula (3): The second parameter is obtained.

[0085] Reference Figure 6 , Figure 6 This is a schematic flowchart of a depth map acquisition method provided in an embodiment of this application. This embodiment relates to an optional implementation of how to obtain the depth map of an image to be matched using a pre-built energy minimization model based on the second inverse depth value corresponding to each pixel. Based on the above embodiment, S104 may include the following steps:

[0086] S601. Based on the second inverse depth value corresponding to each pixel, determine the offset between the pixel value of each pixel and the pixel value of the third candidate matching point on the reference image, wherein the third candidate matching point is determined based on the second inverse depth value corresponding to the pixel on the image to be matched.

[0087] In this step, such as Figure 3 As shown, for example, if the second inverse depth value corresponding to pixel p in the image to be matched is equal to s, then s can be used to determine the depth value in O. l A point P is defined on the ray p, and this point P is oriented to point O. l The distance between point P and point O is equal to 1 / s. r The intersection point p′ of the line connecting the two points and the epipolar line 2 is the third candidate matching point. Once the third candidate matching point is determined, the offset between the pixel value of pixel p and the pixel value of the third candidate matching point on the reference image can be determined. Similarly, for other pixels on the image to be matched, the corresponding third candidate matching points can be determined based on the second inverse depth value of the other pixels, and then the corresponding offset can be calculated.

[0088] S602. Based on the second inverse depth value and offset corresponding to each pixel, determine the target inverse depth value corresponding to each pixel using the energy minimization model.

[0089] After determining the offset corresponding to each pixel, the gray value of a virtual image I′ can be calculated based on the offset and the gray value of the pixel in the reference image. Assume there is a virtual image I′, and the gray value coordinates of the pixels in the virtual image I′ are denoted as (x′ y′). If the gray value coordinates of the pixels in the reference image T corresponding to (x′ y′) are denoted as (xy), and the offset is denoted as (Δx Δy), since x′+Δx=x and y′+Δy=y, and since the offset and (xy) are known, (x′ y′) can be calculated. That is, the gray value of each pixel in the virtual image I′ can be determined, and a virtual image I′ can be determined. Then, based on the virtual image I′ and the image I to be matched, the inverse depth value correction amount of each pixel can be determined using the energy minimization model. The sum of the inverse depth value correction amount and the corresponding second inverse depth value can be used as the target inverse depth value corresponding to the pixel.

[0090] The inverse depth value correction amount corresponding to each second inverse depth value can be determined according to the following energy minimization model. In the following equation, s(p) represents the inverse depth value correction amount corresponding to the second inverse depth value of pixel point p in the image to be matched.

[0091] Energy minimization model: E = σE I +γE G +αE S

[0092] Where E1 represents the photometric uniformity energy:

[0093] E G Represents gradient consistency energy:

[0094]

[0095] E S Energy representing the smoothing term:

[0096] Ω represents the set of pixels in the image to be matched, and p represents the number of pixels in that set. x (p) is equal to the average of the first component and the second component. The first component is equal to the x-axis component of the gray value of pixel p in the image to be matched I, and the second component is equal to the x-axis component of the gray value of the pixel corresponding to pixel p in the virtual image I′; I y (p) is equal to the average of the third and fourth components. The third component is equal to the y-axis component of the gray value of pixel p in the image to be matched I, and the fourth component is equal to the y-axis component of the gray value of the pixel corresponding to pixel p in the virtual image I′; I t(p) represents the value obtained by differentiating the gray value of pixel p in the image to be matched I with respect to time from the gray value of the virtual image I′.

[0097] I x (p), I y (p), I t (p) Once the parameters are determined, they can be based on I x (p) obtain I xx (p), that is It can be based on I y (p) obtain I yy (p), that is Based on I x (p) or I y (p) obtain I xy (p), that is Based on I t (p) can obtain I xt (p) and I yt (p), that is

[0098] I x (p), I y (p), I t (p), I xx (p), I yy (p), I xy (p), I xt (p) and I yt Once (p) is determined, the inverse depth value correction s(p) can be calculated based on the energy minimization model described above.

[0099] in, It is an polar line (u p (s) v p (s)) T The first derivative with respect to s at s = 0 is (u′(0) v′(0)) T In this context, 0 refers to s = 0. (u p (s) v p (s)) T This represents the epipolar line of pixel p in the image I to be matched on the reference frame T.

[0100] S603. Obtain the depth map of the image to be matched based on the target inverse depth value corresponding to each pixel.

[0101] In this step, given that the target inverse depth value corresponding to each pixel is known, the depth map of the image to be matched can be determined based on the depth value of each pixel, since the reciprocal of the target inverse depth value corresponding to each pixel is the depth value of the corresponding pixel.

[0102] Reference Figure 7 , Figure 7 This is a flowchart illustrating a method for determining the inverse depth value of a target according to an embodiment of this application. This embodiment relates to an optional implementation of how to determine the target inverse depth value corresponding to each pixel based on the second inverse depth value and offset corresponding to each pixel. Based on the above embodiment, S602 includes the following steps:

[0103] S701. Determine the grayscale value of each pixel in the virtual image based on the offset corresponding to each pixel and the grayscale value of each pixel in the reference image.

[0104] S702. Based on the gray values ​​of each pixel in the virtual image and the gray values ​​of each pixel in the image to be matched, determine the inverse depth value correction amount corresponding to each second inverse depth value using the energy minimization model.

[0105] S703. Correct the corresponding second inverse depth value according to the correction amount of each inverse depth value to obtain the target inverse depth value corresponding to each pixel in the image to be matched.

[0106] Optionally, the above-mentioned S703 can be implemented in the following way:

[0107] The second inverse depth value and the corresponding inverse depth value correction amount corresponding to each pixel in the image to be matched are summed to correct the corresponding second inverse depth value, and the target inverse depth value corresponding to each pixel in the image to be matched is obtained.

[0108] For example, the sum of the second inverse depth value and the corresponding inverse depth value correction amount corresponding to a pixel can be used as the target inverse depth value corresponding to that pixel. Alternatively, the sum of the second inverse depth value and the corresponding inverse depth value correction amount corresponding to that pixel can be multiplied by a preset coefficient as the target inverse depth value corresponding to that pixel. In this embodiment, by correcting the second inverse depth value, the matching accuracy can be further improved.

[0109] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0110] Based on the same inventive concept, this application also provides a stereo matching apparatus for implementing the stereo matching method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more stereo matching apparatus embodiments provided below can be found in the limitations of the stereo matching method described above, and will not be repeated here.

[0111] In one embodiment, such as Figure 8 As shown, Figure 8 This is a schematic diagram of a stereo matching device provided in an embodiment of this application. The device 800 includes:

[0112] The first obtaining module 801 is used to perform gridding processing on the image to be matched to obtain each image block of the image to be matched;

[0113] The first determining module 802 is used to perform an inverse search optimization operation on each image block based on the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching points corresponding to the pixels in the reference image, using a pre-built inverse search optimization model to determine the first inverse depth value of the image block, wherein the first candidate matching point is determined based on the inverse depth value obtained after the current inverse search optimization operation.

[0114] The second determining module 803 is used to determine the second inverse depth value corresponding to each pixel based on the first inverse depth value of each image block.

[0115] The second acquisition module 804 is used to obtain the depth map of the image to be matched by using a pre-built energy minimization model based on the second inverse depth value corresponding to each pixel.

[0116] In one embodiment, the first determining module 802 is specifically used to determine the result of the current inverse search optimization operation based on the sum of the squares of the first differences between the gray values ​​of each pixel on the image block and the gray values ​​of the first candidate matching point; if the result of the current inverse search optimization operation satisfies the preset inverse search optimization conditions, then the inverse search optimization operation is performed using the inverse search optimization model to obtain the first inverse depth value of the image block.

[0117] In one embodiment, the first determining module 802 is further configured to determine the first inverse depth value of the image block based on the inverse search optimization operation result obtained from the previous inverse search optimization operation if the result of the current inverse search optimization operation does not meet the preset inverse search optimization conditions.

[0118] In one embodiment, the device 800 may further include:

[0119] The third acquisition module is used to sum the squares of the directional derivatives of each pixel in each image block to obtain the first parameter;

[0120] The third determining module is used to determine the second parameter based on the directional derivative of each pixel in the image block, the gray value of each pixel in the image block, and the gray value of the second candidate matching point. The second candidate matching point is a pixel in the reference image determined based on the inverse depth value obtained from the previous inverse search optimization operation.

[0121] The fourth obtaining module is used to obtain the inverse depth value of the current inverse search optimization operation based on the first parameter, the second parameter, and the inverse depth value obtained from the previous inverse search optimization operation.

[0122] In one embodiment, the third determining module is specifically used to determine the sum of squares of the directional derivatives of each pixel in the image block; determine the second difference between the gray value of each pixel in the image block and the gray value of the corresponding second candidate matching point; obtain the first product result of the directional derivative of each pixel in the image block and the corresponding second difference, and sum the first product results corresponding to each pixel in the image block to obtain the second parameter.

[0123] In one embodiment, the second determining module 804 includes:

[0124] The first determining unit is used to determine the offset between the pixel value of each pixel and the pixel value of the third candidate matching point on the reference image based on the second inverse depth value corresponding to each pixel, wherein the third candidate matching point is determined based on the second inverse depth value corresponding to the pixel on the image to be matched.

[0125] The second determining unit is used to determine the target inverse depth value corresponding to each pixel by using an energy minimization model based on the second inverse depth value and offset corresponding to each pixel.

[0126] The acquisition unit is used to obtain the depth map of the image to be matched based on the target inverse depth value corresponding to each pixel.

[0127] In one embodiment, the second determining unit includes:

[0128] The first determining subunit is used to determine the grayscale value of each pixel in the virtual image based on the offset corresponding to each pixel and the grayscale value of each pixel in the reference image.

[0129] The second determining subunit is used to determine the inverse depth value correction amount corresponding to each second inverse depth value based on the gray values ​​of each pixel in the virtual image and the gray values ​​of each pixel in the image to be matched, using an energy minimization model.

[0130] A subunit is obtained to correct the corresponding second inverse depth value according to the correction amount of each inverse depth value, so as to obtain the target inverse depth value corresponding to each pixel on the image to be matched.

[0131] In one embodiment, the subunit is specifically used to sum the second inverse depth value corresponding to each pixel in the image to be matched and the corresponding inverse depth value correction amount, so as to correct the corresponding second inverse depth value and obtain the target inverse depth value corresponding to each pixel in the image to be matched.

[0132] Each module in the aforementioned stereo matching device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0133] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a stereo matching method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0134] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0135] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0136] The image to be matched is processed into a grid to obtain the image blocks of the image to be matched;

[0137] For each image block, based on the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching points corresponding to the pixels in the reference image, an inverse search optimization operation is performed using a pre-built inverse search optimization model to determine the first inverse depth value of the image block. The first candidate matching point is determined based on the inverse depth value obtained after the current inverse search optimization operation.

[0138] The second inverse depth value corresponding to each pixel is determined based on the first inverse depth value of each image block.

[0139] Based on the second inverse depth value corresponding to each pixel, the depth map of the image to be matched is obtained using a pre-built energy minimization model.

[0140] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0141] The result of the current inverse search optimization operation is determined by the sum of the squares of the first differences between the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching point.

[0142] If the result of the current inverse search optimization operation meets the preset inverse search optimization conditions, then the inverse search optimization model is used to perform an inverse search optimization operation to obtain the first inverse depth value of the image patch.

[0143] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0144] If the result of the current inverse search optimization operation does not meet the preset inverse search optimization conditions, then the first inverse depth value of the image patch is determined based on the result of the previous inverse search optimization operation.

[0145] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0146] For each image patch, the first parameter is obtained by summing the squares of the directional derivatives of each pixel in the image patch;

[0147] The second parameter is determined based on the directional derivative of each pixel in the image block, the gray value of each pixel in the image block, and the gray value of the second candidate matching point. The second candidate matching point is a pixel in the reference image determined based on the inverse depth value obtained from the previous inverse search optimization operation.

[0148] Based on the first parameter, the second parameter, and the inverse depth value obtained from the previous inverse search optimization operation, the inverse depth value of the current inverse search optimization operation is obtained.

[0149] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0150] Determine the sum of squares of the directional derivatives of each pixel in the image patch;

[0151] Determine the second difference between the gray value of each pixel in the image block and the gray value of the corresponding second candidate matching point;

[0152] Obtain the first product of the directional derivative of each pixel in the image block and the corresponding second difference, and sum the first product results corresponding to each pixel in the image block to obtain the second parameter.

[0153] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0154] Based on the second inverse depth value corresponding to each pixel, the offset between the pixel value of each pixel and the pixel value of the third candidate matching point on the reference image is determined, wherein the third candidate matching point is determined based on the second inverse depth value corresponding to the pixel on the image to be matched.

[0155] Based on the second inverse depth value and offset of each pixel, the target inverse depth value of each pixel is determined using the energy minimization model.

[0156] The depth map of the image to be matched is obtained based on the target inverse depth value corresponding to each pixel.

[0157] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0158] The grayscale value of each pixel in the virtual image is determined based on the offset corresponding to each pixel and the grayscale value of each pixel in the reference image.

[0159] Based on the gray values ​​of each pixel in the virtual image and the gray values ​​of each pixel in the image to be matched, the energy minimization model is used to determine the inverse depth value correction amount corresponding to each second inverse depth value.

[0160] The corresponding second inverse depth value is corrected according to the correction amount of each inverse depth value to obtain the target inverse depth value corresponding to each pixel in the image to be matched.

[0161] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0162] The second inverse depth value and the corresponding inverse depth value correction amount corresponding to each pixel in the image to be matched are summed to correct the corresponding second inverse depth value, and the target inverse depth value corresponding to each pixel in the image to be matched is obtained.

[0163] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0164] The image to be matched is processed into a grid to obtain the image blocks of the image to be matched;

[0165] For each image block, based on the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching points corresponding to the pixels in the reference image, an inverse search optimization operation is performed using a pre-built inverse search optimization model to determine the first inverse depth value of the image block. The first candidate matching point is determined based on the inverse depth value obtained after the current inverse search optimization operation.

[0166] The second inverse depth value corresponding to each pixel is determined based on the first inverse depth value of each image block.

[0167] Based on the second inverse depth value corresponding to each pixel, the depth map of the image to be matched is obtained using a pre-built energy minimization model.

[0168] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0169] The result of the current inverse search optimization operation is determined by the sum of the squares of the first differences between the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching point.

[0170] If the result of the current inverse search optimization operation meets the preset inverse search optimization conditions, then the inverse search optimization model is used to perform an inverse search optimization operation to obtain the first inverse depth value of the image patch.

[0171] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0172] If the result of the current inverse search optimization operation does not meet the preset inverse search optimization conditions, then the first inverse depth value of the image patch is determined based on the result of the previous inverse search optimization operation.

[0173] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0174] For each image patch, the first parameter is obtained by summing the squares of the directional derivatives of each pixel in the image patch;

[0175] The second parameter is determined based on the directional derivative of each pixel in the image block, the gray value of each pixel in the image block, and the gray value of the second candidate matching point. The second candidate matching point is a pixel in the reference image determined based on the inverse depth value obtained from the previous inverse search optimization operation.

[0176] Based on the first parameter, the second parameter, and the inverse depth value obtained from the previous inverse search optimization operation, the inverse depth value of the current inverse search optimization operation is obtained.

[0177] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0178] Determine the sum of squares of the directional derivatives of each pixel in the image patch;

[0179] Determine the second difference between the gray value of each pixel in the image block and the gray value of the corresponding second candidate matching point;

[0180] Obtain the first product of the directional derivative of each pixel in the image block and the corresponding second difference, and sum the first product results corresponding to each pixel in the image block to obtain the second parameter.

[0181] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0182] Based on the second inverse depth value corresponding to each pixel, the offset between the pixel value of each pixel and the pixel value of the third candidate matching point on the reference image is determined, wherein the third candidate matching point is determined based on the second inverse depth value corresponding to the pixel on the image to be matched.

[0183] Based on the second inverse depth value and offset of each pixel, the target inverse depth value of each pixel is determined using the energy minimization model.

[0184] The depth map of the image to be matched is obtained based on the target inverse depth value corresponding to each pixel.

[0185] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0186] The grayscale value of each pixel in the virtual image is determined based on the offset corresponding to each pixel and the grayscale value of each pixel in the reference image.

[0187] Based on the gray values ​​of each pixel in the virtual image and the gray values ​​of each pixel in the image to be matched, the energy minimization model is used to determine the inverse depth value correction amount corresponding to each second inverse depth value.

[0188] The corresponding second inverse depth value is corrected according to the correction amount of each inverse depth value to obtain the target inverse depth value corresponding to each pixel in the image to be matched.

[0189] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0190] The second inverse depth value and the corresponding inverse depth value correction amount corresponding to each pixel in the image to be matched are summed to correct the corresponding second inverse depth value, and the target inverse depth value corresponding to each pixel in the image to be matched is obtained.

[0191] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps.

[0192] The image to be matched is processed into a grid to obtain the image blocks of the image to be matched;

[0193] For each image block, based on the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching points corresponding to the pixels in the reference image, an inverse search optimization operation is performed using a pre-built inverse search optimization model to determine the first inverse depth value of the image block. The first candidate matching point is determined based on the inverse depth value obtained after the current inverse search optimization operation.

[0194] The second inverse depth value corresponding to each pixel is determined based on the first inverse depth value of each image block.

[0195] Based on the second inverse depth value corresponding to each pixel, the depth map of the image to be matched is obtained using a pre-built energy minimization model.

[0196] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0197] The result of the current inverse search optimization operation is determined by the sum of the squares of the first differences between the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching point.

[0198] If the result of the current inverse search optimization operation meets the preset inverse search optimization conditions, then the inverse search optimization model is used to perform an inverse search optimization operation to obtain the first inverse depth value of the image patch.

[0199] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0200] If the result of the current inverse search optimization operation does not meet the preset inverse search optimization conditions, then the first inverse depth value of the image patch is determined based on the result of the previous inverse search optimization operation.

[0201] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0202] For each image patch, the first parameter is obtained by summing the squares of the directional derivatives of each pixel in the image patch;

[0203] The second parameter is determined based on the directional derivative of each pixel in the image block, the gray value of each pixel in the image block, and the gray value of the second candidate matching point. The second candidate matching point is a pixel in the reference image determined based on the inverse depth value obtained from the previous inverse search optimization operation.

[0204] Based on the first parameter, the second parameter, and the inverse depth value obtained from the previous inverse search optimization operation, the inverse depth value of the current inverse search optimization operation is obtained.

[0205] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0206] Determine the sum of squares of the directional derivatives of each pixel in the image patch;

[0207] Determine the second difference between the gray value of each pixel in the image block and the gray value of the corresponding second candidate matching point;

[0208] Obtain the first product of the directional derivative of each pixel in the image block and the corresponding second difference, and sum the first product results corresponding to each pixel in the image block to obtain the second parameter.

[0209] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0210] Based on the second inverse depth value corresponding to each pixel, the offset between the pixel value of each pixel and the pixel value of the third candidate matching point on the reference image is determined, wherein the third candidate matching point is determined based on the second inverse depth value corresponding to the pixel on the image to be matched.

[0211] Based on the second inverse depth value and offset of each pixel, the target inverse depth value of each pixel is determined using the energy minimization model.

[0212] The depth map of the image to be matched is obtained based on the target inverse depth value corresponding to each pixel.

[0213] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0214] The grayscale value of each pixel in the virtual image is determined based on the offset corresponding to each pixel and the grayscale value of each pixel in the reference image.

[0215] Based on the gray values ​​of each pixel in the virtual image and the gray values ​​of each pixel in the image to be matched, the energy minimization model is used to determine the inverse depth value correction amount corresponding to each second inverse depth value.

[0216] The corresponding second inverse depth value is corrected according to the correction amount of each inverse depth value to obtain the target inverse depth value corresponding to each pixel in the image to be matched.

[0217] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0218] The second inverse depth value and the corresponding inverse depth value correction amount corresponding to each pixel in the image to be matched are summed to correct the corresponding second inverse depth value, and the target inverse depth value corresponding to each pixel in the image to be matched is obtained.

[0219] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0220] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0221] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0222] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A stereo matching method, characterized in that, The method includes: The gridding process in the dense inverse search is performed on the image to be matched to obtain each image block of the image to be matched; For each of the image blocks, based on the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching points corresponding to the pixels in the reference image, an inverse search optimization operation is performed using a pre-built inverse search optimization model to determine the first inverse depth value of the image block, wherein the first candidate matching point is determined based on the inverse depth value obtained after the current inverse search optimization operation. Based on the first inverse depth value of each image block, determine the second inverse depth value corresponding to each pixel point; The depth map of the image to be matched is obtained by using a pre-built energy minimization model based on the second inverse depth value corresponding to each pixel.

2. The method according to claim 1, characterized in that, The step of determining the first inverse depth value of the image block by performing an inverse search optimization operation using a pre-built inverse search optimization model based on the grayscale values ​​of each pixel in the image block and the grayscale values ​​of the first candidate matching points corresponding to the pixels in the reference image includes: The result of the current inverse search optimization operation is determined based on the sum of the squares of the first differences between the gray values ​​of each pixel in the image block and the gray values ​​of the first candidate matching point. If the result of the current inverse search optimization operation meets the preset inverse search optimization conditions, then the inverse search optimization model is used to perform an inverse search optimization operation to obtain the first inverse depth value of the image patch.

3. The method according to claim 2, characterized in that, The method further includes: If the result of the current inverse search optimization operation does not meet the preset inverse search optimization conditions, then the first inverse depth value of the image block is determined based on the result of the previous inverse search optimization operation.

4. The method according to claim 1, characterized in that, The method further includes: For each of the aforementioned image blocks, the squares of the directional derivatives of each pixel in the image block are summed to obtain the first parameter; The second parameter is determined based on the directional derivative of each pixel in the image block, the gray value of each pixel in the image block, and the gray value of the second candidate matching point, wherein the second candidate matching point is a pixel in the reference image determined based on the inverse depth value obtained from the previous inverse search optimization operation; The inverse depth value of the current inverse search optimization operation is obtained based on the first parameter, the second parameter, and the inverse depth value obtained from the previous inverse search optimization operation.

5. The method according to claim 4, characterized in that, The step of determining the second parameter based on the directional derivative of each pixel in the image block, the grayscale value of each pixel in the image block, and the grayscale value of the second candidate matching point includes: Determine the sum of squares of the directional derivatives of each pixel in the image block; Determine the second difference between the gray value of each pixel in the image block and the gray value of the corresponding second candidate matching point; The first product of the directional derivative of each pixel in the image block and the corresponding second difference is obtained, and the second parameter is obtained by summing the first product of each pixel in the image block.

6. The method according to claim 1, characterized in that, The step of obtaining the depth map of the image to be matched using a pre-built energy minimization model based on the second inverse depth value corresponding to each pixel includes: Based on the second inverse depth value corresponding to each pixel, the offset between the pixel value of each pixel and the pixel value of the third candidate matching point on the reference image is determined, wherein the third candidate matching point is determined based on the second inverse depth value corresponding to the pixel on the image to be matched; Based on the second inverse depth value corresponding to each pixel and the offset, the target inverse depth value corresponding to each pixel is determined using the energy minimization model. The depth map of the image to be matched is obtained based on the target inverse depth value corresponding to each pixel.

7. The method according to claim 6, characterized in that, The step of determining the target inverse depth value corresponding to each pixel using the energy minimization model based on the second inverse depth value corresponding to each pixel and the offset includes: The grayscale value of each pixel in the virtual image is determined based on the offset corresponding to each pixel and the grayscale value of each pixel in the reference image. Based on the gray values ​​of each pixel in the virtual image and the gray values ​​of each pixel in the image to be matched, the energy minimization model is used to determine the inverse depth value correction amount corresponding to each second inverse depth value. The corresponding second inverse depth value is corrected according to the inverse depth value correction amount to obtain the target inverse depth value corresponding to each pixel point on the image to be matched.

8. The method according to claim 7, characterized in that, The step of correcting the corresponding second inverse depth value according to each of the inverse depth value correction amounts to obtain the target inverse depth value corresponding to each of the pixels in the image to be matched includes: The second inverse depth value and the corresponding inverse depth value correction amount corresponding to each pixel in the image to be matched are summed to correct the corresponding second inverse depth value, and the target inverse depth value corresponding to each pixel in the image to be matched is obtained.

9. A stereo matching device, characterized in that, The device includes: The first obtaining module is used to perform gridding processing in the dense inverse search of the image to be matched to obtain each image block of the image to be matched; The first determining module is used to perform an inverse search optimization operation on each of the image blocks, based on the gray values ​​of each pixel on the image block and the gray values ​​of the first candidate matching points on the reference image corresponding to the pixel, using a pre-built inverse search optimization model to determine the first inverse depth value of the image block, wherein the first candidate matching point is determined based on the inverse depth value obtained after the current inverse search optimization operation. The second determining module is used to determine the second inverse depth value corresponding to each pixel point based on the first inverse depth value of each image block. The second obtaining module is used to obtain the depth map of the image to be matched by using a pre-built energy minimization model based on the second inverse depth value corresponding to each pixel.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images