Method and apparatus for determining parallax image, integrated circuit chip and computer device

Abstract

The application relates to a disparity image determination method and device, an integrated circuit chip, a computer device, a storage medium and a computer program product. The method comprises the following steps: performing stereo matching on left and right camera images to obtain a to-be-processed disparity image, and slidingly traversing the to-be-processed disparity image by using a first sliding window; in each local disparity image corresponding to the first sliding window, determining whether the absolute value of the difference between each disparity value in the local disparity image and a center disparity value corresponding to the center position of the local disparity image is less than or equal to a first threshold value; counting the statistical quantity of the disparity values whose absolute value of the difference is less than or equal to the first threshold value; if the statistical quantity is less than or equal to a second threshold value, the center disparity value is reserved; if the statistical quantity is greater than the second threshold value, the center disparity value is set to 0; after traversing the to-be-processed disparity image by using the first sliding window, a target disparity image is determined. The method can improve the convenience and efficiency of determining the disparity image.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method, apparatus, integrated circuit chip, computer equipment, storage medium, and computer program product for determining a disparity image. Background Art

[0002] Binocular stereo matching, also known as disparity estimation or binocular depth estimation, involves using a binocular camera to capture left and right viewpoint images of the same scene (i.e., left and right camera images). A stereo matching algorithm is then applied to the left and right camera images to calculate a disparity image. Disparity is the pixel-level difference between the positions of a point in a three-dimensional scene and its corresponding point in the left and right camera images. A disparity image is a disparity map constructed by calculating the disparity value for each pixel in a reference image (usually the left image). Given the baseline distance and focal length of the cameras, the disparity image can be used to calculate the corresponding depth image. This depth image can then be used to implement technologies in fields such as depth perception, autonomous driving, and security monitoring. Binocular stereo matching primarily consists of four steps: initial cost calculation, cost aggregation, disparity calculation, and disparity post-processing. It is computationally complex and requires high precision, making its deployment on embedded processing platforms particularly challenging.

[0003] Among them, the disparity post-processing stage includes a denoising operation on the disparity image obtained by disparity calculation. In the traditional technical solution, for each disparity value in the disparity image, it is generally determined that there is at least one disparity value above, below, left, and right (i.e., surrounding) that meets the disparity connectivity condition, and then they are used as starting points (called propagation) to detect whether the disparity values ​​around them meet the disparity connectivity condition; each time a new connected point is detected, the flag position of its corresponding point is 1, and the counter is incremented by one until, for each new connected point, the points around it do not meet the disparity connectivity condition, and the counting stops; then it is determined whether the count value (i.e., the number of pixels in the connected area with the currently processed disparity value) is greater than a preset threshold; if it is, the currently processed disparity value is considered valid and retained; otherwise, the currently processed disparity value is considered to be a noise point, and the currently processed disparity value is removed. However, this method of determining the disparity image requires judgment and calculation for each disparity value, and the processing process is complex, requiring a large amount of logical resources and storage resources.

[0004] Therefore, how to improve the convenience and efficiency of determining the parallax image is a technical problem that those skilled in the art currently need to solve. Summary of the Invention

[0005] Based on this, it is necessary to provide a method, device, integrated circuit chip, computer equipment, computer-readable storage medium and computer program product for determining a disparity image that can improve the convenience and efficiency of determining the disparity image in response to the above technical problems.

[0006] In a first aspect, the present application provides a method for determining a disparity image. The method comprises:

[0007] Perform stereo matching on the left and right camera images to obtain a disparity image to be processed, and use a first sliding window to slide through the disparity image to be processed;

[0008] determining, in each local disparity image corresponding to the first sliding window, whether an absolute value of a difference between each disparity value in the local disparity image and a central disparity value corresponding to a central position of the local disparity image is less than or equal to a first threshold;

[0009] Counting the number of disparity values ​​whose absolute value of the difference is less than or equal to the first threshold;

[0010] If the statistical number is less than or equal to a second threshold, retaining the central disparity value;

[0011] If the statistical number is greater than the second threshold, setting the central disparity value to a target value;

[0012] After traversing the disparity image to be processed using the first sliding window, a target disparity image is determined.

[0013] In one embodiment, performing stereo matching on the left and right camera images to obtain the disparity image to be processed includes:

[0014] Acquire the left and right camera images;

[0015] Calculating an initial cost based on the left and right camera images;

[0016] Performing a cost aggregation operation based on the initial cost to obtain a final cost;

[0017] Performing disparity calculation on the final cost to determine the disparity image to be processed.

[0018] In one embodiment, calculating the initial cost based on the left and right camera images includes:

[0019] traversing the first image using a second sliding window, determining each first local pixel image corresponding to the second sliding window, and determining a first transformation vector corresponding to each first local pixel image; wherein the left and right camera images include the first image and the second image;

[0020] determining a second transformation vector of a second local pixel image corresponding to the first local pixel image in the second image;

[0021] determining a local initial cost corresponding to the first local pixel image according to the first transformation vector and the second transformation vector;

[0022] After traversing the first image using the second sliding window, the initial cost corresponding to the first image is obtained; the initial cost includes the local initial cost.

[0023] In one embodiment, determining a second transformation vector of a second local pixel image corresponding to the first local pixel image in the second image includes:

[0024] Determining a preset parallax range, and calculating second transformation vectors to be confirmed corresponding to second local pixel images in the second image corresponding to the first local pixel image within the preset parallax range; the second transformation vectors to be confirmed include the second transformation vector;

[0025] The determining, according to the first transformation vector and the second transformation vector, a local initial cost corresponding to the first local pixel image includes:

[0026] Determine the corresponding to-be-confirmed local initial cost according to the first transformation vector and each of the to-be-confirmed second transformation vectors;

[0027] The minimum value among the to-be-confirmed local initial costs is determined as the local initial cost.

[0028] In one embodiment, after calculating the initial costs based on the left and right camera images and before performing a cost aggregation operation based on the initial costs to obtain a final cost, the method further includes:

[0029] A saturation operation is performed on the initial cost to obtain the updated initial cost.

[0030] In one embodiment, performing a saturation operation on the initial cost to obtain the updated initial cost includes:

[0031] If the initial cost is greater than or equal to a third threshold, setting the initial cost to a first preset value;

[0032] Otherwise, the lowest bit of the binary value corresponding to the initial cost is deleted, and the initial cost is updated using the binary value after deleting the lowest bit.

[0033] In one embodiment, performing a cost aggregation operation based on the initial cost to obtain a final cost includes:

[0034] A four-directional cost aggregation operation is performed according to the initial cost to obtain the final cost.

[0035] In one embodiment, the method further comprises:

[0036] Grouping the to-be-confirmed local initial costs and the local initial costs according to a preset rule;

[0037] The grouped local initial costs to be confirmed and the local initial costs are spliced ​​together, and the local initial costs to be confirmed and the local initial costs are stored in a preset storage location according to the splicing result.

[0038] In one embodiment, the left and right camera images include:

[0039] Acquire two sets of left and right camera images using a dual-channel binocular camera module;

[0040] The method for determining the disparity image is performed on the two groups of left and right camera images respectively in a time-division multiplexing manner.

[0041] In a second aspect, the present application further provides a device for determining a disparity image. The device comprises:

[0042] A sliding window traversal module is used to perform stereo matching on the left and right camera images to obtain a disparity image to be processed, and to slide and traverse the disparity image to be processed using a first sliding window;

[0043] a connectivity determination module configured to determine, in each local disparity image corresponding to the first sliding window, whether an absolute value of a difference between each disparity value in the local disparity image and a central disparity value corresponding to a central position of the local disparity image is less than a first threshold;

[0044] A statistics module, configured to count the number of disparity values ​​whose absolute value of the difference is less than the first threshold;

[0045] an execution module, configured to retain the central disparity value if the statistical quantity is less than or equal to a second threshold; and set the central disparity value to a target value if the statistical quantity is greater than the second threshold;

[0046] The image determination module is configured to determine a target disparity image after traversing the disparity image to be processed using the first sliding window.

[0047] In a third aspect, the present application also provides an integrated circuit chip, which is used to implement the steps of the above method.

[0048] In a fourth aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0049] In a fifth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above method.

[0050] In a sixth aspect, the present application further provides a computer program product, which includes a computer program that implements the steps of the above method when executed by a processor.

[0051] The above-mentioned method, apparatus, integrated circuit chip, computer device, storage medium, and computer program product for determining a disparity image employ a first sliding window to traverse the disparity images to be processed. For each local disparity image corresponding to the first sliding window, the method determines whether the absolute difference between each disparity value in the local disparity image and the central disparity value corresponding to the center position of the local disparity image is less than or equal to a first threshold. The method then counts the number of disparity values ​​whose absolute difference is less than or equal to the first threshold, and based on the statistical results, removes noise to obtain a target disparity image. Compared to existing technologies, this method performs judgment calculations on the local disparity images corresponding to the first sliding window, reducing the complexity of the judgment and calculation process and thereby improving the convenience and efficiency of determining the disparity image. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a schematic flow chart of a method for determining a disparity image in one embodiment;

[0053] Figure 2 A schematic diagram of a process for calculating an initial cost in one embodiment;

[0054] Figure 3 Schematic diagram of a process for calculating a transformation vector in one embodiment;

[0055] Figure 4 A schematic diagram of a process for calculating an initial cost in one embodiment;

[0056] Figure 5 is a schematic diagram of a first image or a second image in left and right camera images in one embodiment;

[0057] Figure 6 is a schematic diagram of a disparity image to be processed corresponding to an initial cost without saturation processing;

[0058] Figure 7is a schematic diagram of a disparity image to be processed corresponding to the initial cost after saturation processing;

[0059] Figure 8 Schematic diagram of the disparity map corresponding to the eight-directional aggregation cost operation in the traditional technical solution;

[0060] Figure 9 A schematic diagram of a disparity map corresponding to a four-directional aggregation cost operation in one embodiment;

[0061] Figure 10 A timing diagram of time division multiplexing in one embodiment;

[0062] Figure 11 is a structural block diagram of a device for determining a disparity image in one embodiment;

[0063] Figure 12 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0065] The method for determining a disparity image provided in an embodiment of the present application can be applied to a terminal device equipped with a binocular camera. The terminal device uses the binocular camera to obtain left and right camera images, performs stereo matching on the left and right camera images, obtains a disparity image to be processed, and uses a first sliding window to slide through the disparity images to be processed. In each local disparity image corresponding to the first sliding window, it is determined whether the absolute value of the difference between each disparity value in the local disparity image and the central disparity value corresponding to the center position of the local disparity image is less than or equal to a first threshold. The number of disparity values ​​whose absolute value of the difference is less than or equal to the first threshold is counted. If the counted number is less than or equal to a second threshold, the central disparity value is retained. If the counted number is greater than the second threshold, the central disparity value is set to a target value. After traversing the disparity images to be processed using the first sliding window, a target disparity image is determined. The terminal device can be, but is not limited to, various intelligent driving devices and portable wearable devices. The intelligent driving device can be an unmanned vehicle, a sweeping robot, etc., and the portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. It is understood that this method can also be applied to a system including a binocular camera and a server. The binocular camera captures left and right camera images and sends them to the server. The server then executes the disparity image determination method to determine the target disparity image. The server can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0066] In one embodiment, a method for determining a disparity image is provided. The method is described by applying the method to a terminal device as an example, and includes the following steps:

[0067] Step 102 : stereo matching is performed on the left and right camera images to obtain a disparity image to be processed, and the disparity image to be processed is traversed by sliding a first sliding window.

[0068] In this embodiment, the left and right camera images refer to images acquired by a binocular camera, generally including a first image and a second image. In a specific embodiment, the first image corresponds to the left image, the second image corresponds to the right image, and the left image is generally used as the reference image. The disparity image to be processed refers to the disparity image that requires sliding window denoising; the disparity image refers to a disparity map constructed based on the disparity values ​​corresponding to each pixel in the reference image; and the disparity is the pixel-level difference between the corresponding points in the left and right camera images at a certain point in a three-dimensional scene. In actual operation, the disparity image to be processed is obtained by stereo matching the left and right camera images.

[0069] The first sliding window refers to a sliding window of a first preset format. After stereo matching is performed on the left and right camera images to obtain the disparity image to be processed, the first sliding window is used to slide through the disparity image to be processed at a first sliding step size. In this embodiment, the window size of the first sliding window is generally m*n, where m and n are both odd numbers and can be equal. The first sliding step size corresponding to the first sliding window is 1.

[0070] Step 104: determining, in each local disparity image corresponding to the first sliding window, whether an absolute value of a difference between each disparity value in the local disparity image and a central disparity value corresponding to a central position of the local disparity image is less than or equal to a first threshold;

[0071] Step 106 , counting the number of disparity values ​​whose absolute difference value is less than or equal to a first threshold;

[0072] Step 108 : If the statistical number is less than or equal to the second threshold, retain the central disparity value; if the statistical number is greater than the second threshold, set the central disparity value as the target value.

[0073] Specifically, in the process of sliding the first sliding window through the disparity image to be processed, after each sliding of the first sliding window, the local image in the disparity image to be processed that corresponds to the first sliding window is the local disparity image. The following processing is performed on each local disparity image corresponding to the sliding of the first sliding window through the disparity image to be processed:

[0074] Based on the first sliding window and the current local disparity image, the center position of the local disparity image is determined, and the central disparity value corresponding to the center position is obtained; the disparity values ​​corresponding to each other position in the local disparity image are determined; the absolute difference between each disparity value in the local disparity image and the central disparity value is calculated; each absolute difference value is compared with a first threshold (threshold 0) to determine whether the absolute difference value is less than or equal to the first threshold (threshold 0); the number of disparity values ​​whose absolute difference value is less than or equal to the first threshold (threshold 0) is counted; and whether the counted number is less than or equal to a second threshold (threshold 1) is determined; if the counted number is less than or equal to the second threshold (threshold 1), the central disparity value is retained; if the counted number is greater than the second threshold (threshold 1), the central disparity value is set to a target value; wherein the target value may be 0. At this point, sliding window denoising of the current local disparity image is completed.

[0075] Then, the first sliding window continues to slide according to the first sliding step size, and the local disparity image corresponding to the sliding is used as the current local disparity image and continues to be processed.

[0076] It should be noted that the first threshold and the second threshold in this embodiment are set based on actual experience, and this embodiment does not limit the specific values ​​of the first threshold and the second threshold.

[0077] Step 110 : After traversing the disparity images to be processed using the first sliding window, a target disparity image is determined.

[0078] In this step, the target disparity image is determined after the first sliding window is used to traverse the disparity image to be processed and the sliding window denoising operation is performed on each local disparity image corresponding to the first sliding window. In other words, the target disparity image is the image obtained by traversing the disparity image to be processed using the first sliding window and performing the sliding window denoising operation.

[0079] It should be noted that after determining the target disparity image, the terminal device can perform operations such as obstacle avoidance, three-dimensional modeling, detection and recognition based on the target disparity image.

[0080] The above-mentioned method for determining a disparity image utilizes a first sliding window to traverse the disparity images to be processed. For each local disparity image corresponding to the first sliding window, the method determines whether the absolute difference between each disparity value in the local disparity image and the central disparity value corresponding to the center position of the local disparity image is less than or equal to a first threshold. The method then counts the number of disparity values ​​whose absolute difference is less than or equal to the first threshold, and based on the statistical results, removes noise to obtain a target disparity image. Compared to existing techniques, this method performs judgment calculations on the local disparity images corresponding to the first sliding window, reducing the complexity of the judgment and calculation process, thereby improving the convenience and efficiency of determining the disparity image.

[0081] As a preferred embodiment, the statistical number of disparity values ​​whose absolute difference values ​​are less than a first threshold value includes: setting the disparity values ​​in the local disparity image whose absolute difference values ​​are less than or equal to the first threshold value to 1; setting the disparity values ​​in the local disparity image whose absolute difference values ​​are greater than the first threshold value to 0; and calculating the cumulative value of each updated disparity value in the local disparity image to obtain the statistical number.

[0082] Specifically, in this embodiment, after comparing the absolute value of each difference with a first threshold (threshold0) to determine whether the absolute value of each difference is less than or equal to the first threshold (threshold0), the disparity values ​​in the local disparity image whose absolute value of the difference is less than or equal to the first threshold (threshold0) are set to 1, and the disparity values ​​in the local disparity image whose absolute value of the difference is greater than the first threshold (threshold0) are set to 0. That is, a disparity value of 1 is used to indicate that the pixel corresponding to the position is connected to the pixel corresponding to the central disparity value, and a disparity value of 0 is used to indicate that the pixel corresponding to the position is not connected to the pixel corresponding to the central disparity value. Then, the cumulative value of each updated disparity value in the local disparity image is calculated to obtain a statistical number, that is, the number of 1s in the current local disparity image is calculated to obtain a statistical number of disparity values ​​whose absolute value of the difference is less than or equal to the first threshold. It should be noted that after determining the statistical number according to the method of this embodiment, the disparity value needs to be further restored to the disparity value before adjustment so that the disparity value can be used for subsequent processing and calculation.

[0083] As another preferred embodiment, the number of disparity values ​​whose absolute difference values ​​are less than a first threshold is counted, a statistical variable is preset, and the statistical variable is initialized to 0; when judging in turn whether the calculated absolute value of each difference is less than or equal to the first threshold, if it is judged that the absolute value of the difference is less than or equal to the first threshold, the statistical variable is increased by 1, otherwise the statistical variable remains unchanged; and continue to judge whether the next absolute value of the difference is less than or equal to the first threshold until all the corresponding absolute values ​​of the difference in the local disparity image are judged, and the value corresponding to the statistical variable at this time is the statistical number.

[0084] It can be seen that according to the method of this embodiment, the statistical number of disparity values ​​whose absolute value of the difference is less than or equal to the first threshold can be determined conveniently and accurately.

[0085] Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, stereo matching is performed on the left and right camera images to obtain a parallax image to be processed, including:

[0086] Get left and right camera images;

[0087] Calculate the initial cost based on the left and right camera images;

[0088] Perform cost aggregation based on the initial cost to obtain the final cost;

[0089] The disparity is calculated for the final cost to determine the disparity image to be processed.

[0090] Specifically, after obtaining the left and right camera images using a binocular camera, the corresponding initial costs are first calculated based on the left and right camera images. The initial costs corresponding to the left and right camera images can be calculated using a Census transformation method.

[0091] Cost aggregation operation refers to an operation based on similar regions having similar disparity, which is used to reduce the influence of matching blur and image noise in the initial cost. Cost aggregation operation includes methods such as image segmentation, adaptive weights, and adaptive windows. After the cost aggregation operation is performed on the initial cost, the final cost is obtained.

[0092] After determining the final cost, a disparity calculation is performed on the final cost to obtain a disparity image to be processed. More specifically, the corresponding disparity can be calculated using a WTA (winner takes all) disparity calculation method; this embodiment does not limit the specific method of disparity calculation.

[0093] It can be seen that according to the method of this embodiment, the parallax image to be processed can be determined quickly and conveniently.

[0094] It should be noted that in actual operation, after disparity calculation, further disparity optimization operations can be performed. Disparity optimization operations include removing false matches, removing small connected areas (peaks), uniqueness checks, sub-pixel interpolation, and left-right consistency checks. By further performing disparity optimization on the disparity image to be processed, disparity accuracy is improved, making the processed disparity image more reliable and accurate. This can also improve the accuracy of subsequent sliding window denoising operations based on the processed disparity image.

[0095] It should also be noted that in actual operation, after acquiring the left and right camera images, they can be further image-corrected, and the initial cost is subsequently calculated for the left and right cameras after the image correction operation. The image correction operation includes preprocessing steps such as lens shading correction, bilateral filtering denoising, downsampling, and automatic exposure. After preprocessing, the left and right camera image data needs to be dedistorted and parallelized to obtain the corrected left and right camera images.

[0096] It can be seen that this embodiment can enhance the effective information in the left and right camera images by performing image correction operations on the acquired left and right camera images, improve the image quality of the to-be-processed disparity images determined using the left and right camera images, and further improve the image quality of the determined target disparity image.

[0097] like Figure 2 FIG. 1 is a schematic diagram of a process for calculating the initial cost provided by this embodiment. Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the initial cost is calculated based on the left and right camera images, including:

[0098] Step 202: traverse the first image using a second sliding window, determine each first local pixel image corresponding to the second sliding window, and determine a first transformation vector corresponding to each first local pixel image; wherein the left and right camera images include the first image and the second image.

[0099] The left and right camera images include a first image (left image) and a second image (right image), with the first image (left image) serving as a reference image. The second sliding window is a sliding window in a second preset format. The second sliding window is traversed over the first image using a second sliding step size. During the traversal of the first image by the second sliding window, the local image in the first image corresponding to the second sliding window is a first local pixel image. For each local pixel image, a first transformation vector corresponding to the first local pixel image is determined.

[0100] like Figure 3 FIG. 1 is a schematic diagram of a process for calculating a transformation vector; assuming that the second sliding window is a 3*3 sliding window, each pixel value of the second sliding window is compared with the pixel value at the center of the sliding window, and positions where the pixel value is less than or equal to the pixel value at the center of the sliding window are set to 0, and positions where the pixel value is greater than the pixel value at the center of the sliding window are set to 1, thereby determining a first transformation vector (Cencus vector).

[0101] Step 204: Determine a second transformation vector of a second local pixel image corresponding to the first local pixel image in the second image.

[0102] According to the position of the second sliding window in the first image, a second local pixel image is determined in the second image according to the same sliding window position; and in the same manner as above, a second transformation vector corresponding to the second local pixel image is determined.

[0103] Step 206: Determine a local initial cost corresponding to the first local pixel image according to the first transformation vector and the second transformation vector.

[0104] After determining the corresponding first and second transformation vectors for the first and second local pixel images corresponding to the second sliding window, respectively, the Hamming distance between the first and second transformation vectors is calculated. Specifically, an exclusive OR operation is performed bit by bit on the sequence corresponding to the first and second transformation vectors, and the number of bits in the exclusive OR operation result that are 1 is counted to obtain the corresponding Hamming distance. This determines the local initial cost corresponding to a sliding window position. The Hamming distance represents the similarity between the first and second transformation vectors; a smaller Hamming distance indicates a greater similarity.

[0105] Step 208: After traversing the first image using the second sliding window, an initial cost corresponding to the first image is obtained; the initial cost includes a local initial cost.

[0106] Specifically, the second sliding window is used to traverse the first image, and for each sliding window position of the second sliding window, the local initial cost of the first local pixel image corresponding to each sliding window position is determined respectively; after the second sliding window is used to traverse the first image, the local initial costs corresponding to each first local pixel image of the first image are obtained, and then the initial cost corresponding to the first image is obtained through sorting.

[0107] Specifically, such as Figure 4 A schematic diagram of a process for calculating the initial cost is shown. Specifically, the Hamming distances corresponding to the first local pixel images in a 3*3 format are obtained, and the Hamming distances in the 3*3 format are summed to obtain the corresponding initial cost.

[0108] It can be seen that the method for determining the initial cost in this embodiment retains the position characteristics of the pixels in the second sliding window, is relatively robust to brightness deviation, and can reduce mismatching caused by illumination differences.

[0109] As a preferred embodiment, step 204: determining a second transformation vector of a second local pixel image corresponding to the first local pixel image in the second image includes:

[0110] Determining a preset parallax range, calculating the second transformation vectors to be confirmed corresponding to the second local pixel images corresponding to the first local pixel image in the second image within the preset parallax range; the second transformation vectors to be confirmed include the second transformation vector;

[0111] In the embodiment, first, a preset disparity range is determined, after the second local pixel image corresponding to the same sliding window position in the first image is determined, for each candidate disparity value in the preset disparity range, a second transformation vector corresponding to the second local pixel image when the assumed disparity value is the candidate disparity value is calculated respectively, that is, the second transformation vector to be confirmed is obtained.

[0112] Correspondingly, step 206: determining a local initial cost corresponding to the first local pixel image according to the first transformation vector and the second transformation vector, comprising:

[0113] determining a corresponding to-be-confirmed local initial cost according to the first transformation vector and each to-be-confirmed second transformation vector respectively;

[0114] determining the minimum value in the to-be-confirmed local initial cost as the local initial cost.

[0115] Correspondingly, after the multiple to-be-confirmed second transformation vectors corresponding to the second local pixel image are determined according to the preset disparity range, the Hamming distance corresponding to the first transformation vector and each to-be-confirmed second transformation vector is calculated, that is, the multiple to-be-confirmed local initial costs corresponding to each first local pixel image are obtained. Then, the sizes of the multiple to-be-confirmed local initial costs corresponding to the first local pixel image are compared, and the minimum value in the multiple to-be-confirmed local initial costs is determined as the local initial cost.

[0116] It can be seen that the local initial cost is determined according to the method of the embodiment, so that the initial cost determined by each minimum local initial cost is the minimum initial cost, and the logic resources required by the initial cost can be reduced.

[0117] On the basis of the above-mentioned embodiment, the technical solution is further described and optimized in the embodiment, specifically, after the initial cost is calculated according to the left and right camera images and the cost aggregation operation is performed according to the initial cost to obtain the final cost, the method further comprises:

[0118] performing saturation operation on the initial cost to obtain an updated initial cost.

[0119] Specifically, the saturation operation refers to adjusting the initial cost so that the subsequent operation using the adjusted initial cost can achieve better processing effect. In actual operation, the initial cost can be directly calculated, and the value corresponding to half or other proportion of the initial cost is taken as the updated initial cost; or the initial cost is directly subtracted by a preset value, and the calculation result is taken as the updated initial cost; the specific process of the saturation operation is not limited in the embodiment.

[0120] It should be noted that the saturation operation is further performed on the initial cost, so that the adjusted initial cost can reduce the occupation of logical resources without affecting the accuracy of the determined target disparity image.

[0121] As a preferred embodiment, the saturation operation is performed on the initial cost to obtain an updated initial cost, including:

[0122] If the initial cost is greater than or equal to a third threshold value, the initial cost is set to a first preset value;

[0123] Otherwise, the lowest bit of the binary corresponding to the initial cost is deleted, and the initial cost is updated using the binary value after deleting the lowest bit.

[0124] The third threshold value is a judgment value for determining which way to adjust the initial cost; the actual value of the third threshold value is not limited in this embodiment, and can be set according to actual needs. In one actual operation, in order to reduce the adjusted initial cost to below 4 bits, the maximum value 31 of 5-bit binary is set as the third threshold value. According to the maximum value 1111 of 4-bit binary, the first preset value is set to 15.

[0125] In one specific embodiment, assuming that the third threshold value is 31 and the first preset value is 15, the saturation operation is performed on the initial cost, and the operation principle is as follows:

[0126]

[0127] According to the above operation principle, if the initial cost (cost_block) is greater than or equal to the third threshold value 31, the initial cost is directly set to the first preset value 15; if the initial cost (cost_block) is less than the third threshold value 31, the lowest bit of the binary corresponding to the initial cost is deleted, and the initial cost is updated using the binary value after deleting the lowest bit. For example, if the initial cost is 35 (100011), the initial cost is directly set to the first preset value 15; if the initial cost is 30 (11110), the lowest bit corresponding to the initial cost is deleted to obtain binary 1111, and the updated initial cost is determined to be 15 according to the binary 1111 obtained by deleting the lowest bit; if the initial cost is 12 (1100), the lowest bit corresponding to the initial cost is deleted to obtain binary 110, and the updated initial cost is determined to be 6 according to the binary 110 obtained by deleting the lowest bit.

[0128] It can be understood that the saturation operation on the initial cost can reduce the logical resources required by the initial cost. Moreover, since the minimum initial cost is used for calculation in the subsequent processing process, the influence of the disparity accuracy caused by the saturation processing of the initial cost can be ignored. For example, Figure 5is a schematic diagram of the first image or the second image in the left and right camera images, and the corresponding simulation algorithm effect is set according to whether the initial cost is saturated to obtain the corresponding disparity image to be processed; Figure 6 is a schematic diagram of the disparity image to be processed corresponding to the initial cost without saturation processing, Figure 7 Schematic diagram of the disparity image to be processed corresponding to the initial cost after saturation processing, comparison Figure 6 and Figure 7 It can be seen that if the initial cost is saturated, the edges of the obtained disparity image to be processed will have a fuzzy phenomenon, but there will be no obvious error in the depth information. Most of the disparity information of the image is retained, and the error in the edge depth information can be ignored. Therefore, it can be explained that saturating the initial cost not only retains the accuracy of the depth information, but also greatly reduces the consumption of logical resources.

[0129] Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, a cost aggregation operation is performed based on the initial cost to obtain the final cost, including:

[0130] Perform a four-way aggregation cost operation based on the initial cost to obtain the final cost.

[0131] Specifically, in this embodiment, a four-directional cost aggregation operation is performed on the initial cost after saturation processing. The cost aggregation principle is as follows:

[0132]

[0133] Among them, L r (p, d) represents the cost aggregation value of the matching point p after aggregation in the r direction;

[0134] Cost(p,d) represents the matching cost of matching point p;

[0135] L r (pr,d) represents the cost aggregation value of the previous matching point of matching point p under the same disparity;

[0136] L r (pr,d-1) represents the aggregated cost value of the disparity of the previous matching point of the matching point p minus one;

[0137] L r (pr,d+1) represents the aggregated value of the matching cost plus the disparity of the previous matching point of the matching point p;

[0138] min i (L r (pr,i)) represents the minimum cost aggregation value of all disparities of the previous matching point p;

[0139] P1 and P2 are adjustable parameters used to compensate for parallax and fine-tune the algorithm; P1 represents the penalty coefficient for a parallax difference of 1, and P2 represents the penalty coefficient for the minimum initial cost in that direction.

[0140] The multi-directional aggregation cost is:

[0141]

[0142] In this embodiment, the cost aggregation operation is performed on the four directions of W, NW, N, and NE, which greatly reduces the amount of data that needs to be cached, reduces the computational complexity, and improves the efficiency of cost aggregation. Moreover, according to the test, the cost aggregation method of this embodiment can effectively retain the accuracy of the aggregated cost. Figure 8 and Figure 9 As shown; Figure 8 This is a schematic diagram of the disparity map corresponding to the eight-directional aggregation cost operation in the traditional technical solution. Figure 9 Schematic diagram of the disparity map corresponding to the four-directional aggregation cost operation in this embodiment; Figure 8 and Figure 9 It can be seen that the disparity map information does cause information loss, but most depth information is not lost, and most applications such as obstacle avoidance, detection and recognition are not affected.

[0143] Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the method further includes:

[0144] Grouping the local initial costs to be confirmed and the local initial costs according to preset rules;

[0145] The grouped local initial costs to be confirmed and the local initial costs are spliced ​​together, and the local initial costs to be confirmed and the local initial costs are stored in a preset storage location according to the splicing result.

[0146] Among them, the local initial costs to be confirmed and the local initial costs may be grouped according to the aggregation direction of the cost aggregation to obtain data corresponding to each aggregation direction, and the data includes the local initial costs to be confirmed and the local initial costs.

[0147] After grouping the data, the grouped local initial costs to be confirmed and the local initial costs are spliced ​​to obtain spliced ​​information; then the local initial costs to be confirmed and the local initial costs are stored in a preset storage location according to the splicing results.

[0148] It is understandable that when performing parallel cost aggregation operations, it is necessary to cache the pending local initial costs and local initial costs corresponding to each aggregation direction. If a parallax independent cache method is used, a large amount of on-chip storage resources will be required.

[0149] The stitching cache method can be used in both the cost aggregation and disparity calculation processes. In a specific operation, when the image resolution is 800x500 and the disparity range is 74, the memory usage of using independent disparity cache and stitching cache is compared as follows:

[0150] Table 1 Comparison of memory resource usage of caching methods

[0151] Cache Methods Independent cache Stitching Cache Occupies bram resources (36Kb) 185 63

[0152] It can be seen from the above table that the splicing cache method according to this embodiment can greatly reduce the occupation of on-chip storage resources.

[0153] Based on the above embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically, in this embodiment, the left and right camera images include:

[0154] Use a dual-channel binocular camera module to obtain two sets of left and right camera images;

[0155] A method for determining disparity images is performed on two groups of left and right camera images respectively in a time-division multiplexing manner.

[0156] A binocular camera module refers to a camera that can capture two images of a subject, one for the left and one for the right. Common binocular camera modules vary depending on actual needs, and this embodiment does not limit the specific type of binocular camera module. A two-way binocular camera module refers to a camera that uses two binocular camera modules to capture the same subject.

[0157] Assume that a dual-channel binocular camera module includes binocular camera module 0 and binocular camera module 1, and that binocular camera module 0 and binocular camera module 1 each acquire corresponding left and right camera images. The method for determining disparity images is then executed on each of the two sets of left and right camera images using time-sharing multiplexing. Specifically, binocular camera module 0 and binocular camera module 1 generate image frame headers, valid image signals, and image data in a time-sharing manner. The processing channel proc_chanel is switched based on the frame header count, and the two target disparity images corresponding to binocular camera module 0 and binocular camera module 1 are output in a time-sharing manner.

[0158] like Figure 10A timing diagram of time-division multiplexing is shown in the figure; wherein, clk is the clock signal and rst_n is the reset signal; img_frm_vld1 and img_frm_vld2 represent the image frame header signals corresponding to the binocular camera module 0 and the binocular camera module 1 respectively; img_row_vld1 and img_row_vld2 represent the image row valid signals corresponding to the binocular camera module 0 and the binocular camera module 1 respectively; imgl_dat1 and imgr_dat1 represent the left and right images of the binocular camera module 0; imgl_dat2 and imgr_dat2 represent the left and right images of the binocular camera module 1; proc_chanel represents the channel being processed. By switching the channel to the binocular camera module 0 or the binocular camera module 1, it means processing the left and right camera images corresponding to the binocular camera module 0 or the binocular camera module 1; disp_frm_vld represents the depth map data frame header; disp_row_vld represents the depth row valid signal; disp_dat represents the obtained target disparity image.

[0159] According to the method of this embodiment, the processing mechanism of time-sharing multiplexing of the two sets of left and right camera images obtained by the dual-channel binocular camera module can not only ensure the processing speed of the dual-channel binocular camera module, but also reduce the exposure frequency of a single module, significantly reducing the operating power consumption of the binocular camera module.

[0160] In order to enable those skilled in the art to better understand the technical solutions in this application, the following describes the technical solutions in the embodiments of this application in detail in combination with actual application scenarios. The embodiment of this application provides a method for determining a parallax image, and the specific steps are as follows:

[0161] Use a dual-channel binocular camera module to obtain two sets of left and right camera images;

[0162] A method for determining a disparity image is performed on two sets of left and right camera images respectively in a time-division multiplexing manner; wherein the left and right camera images acquired by each binocular camera module include a first image and a second image;

[0163] traversing the first image using the second sliding window, determining each first local pixel image corresponding to the second sliding window, and determining a first transformation vector corresponding to each first local pixel image;

[0164] Determining a preset parallax range, calculating the second transformation vectors to be confirmed corresponding to the second local pixel images corresponding to the first local pixel image in the second image within the preset parallax range; the second transformation vectors to be confirmed include the second transformation vector;

[0165] Determine the corresponding to-be-confirmed local initial costs according to the first transformation vector and each to-be-confirmed second transformation vector;

[0166] The minimum value among the local initial costs to be confirmed is determined as the local initial cost;

[0167] Grouping the local initial costs to be confirmed and the local initial costs according to preset rules;

[0168] Splicing the grouped local initial costs and local initial costs to be confirmed, and storing the local initial costs and local initial costs to be confirmed in a preset storage location according to the splicing result;

[0169] After traversing the first image using the second sliding window, an initial cost corresponding to the first image is obtained; the initial cost includes a local initial cost;

[0170] Perform a saturation operation on the initial cost to obtain an updated initial cost; perform a saturation operation on the initial cost to obtain an updated initial cost, including:

[0171] If the initial cost is greater than or equal to the third threshold, setting the initial cost to a first preset value;

[0172] Otherwise, delete the lowest bit of the binary corresponding to the initial cost, and use the binary value after deleting the lowest bit to update the initial cost;

[0173] Perform a four-way aggregation cost operation based on the updated initial cost to obtain the final cost;

[0174] Perform disparity calculation on the final cost to determine the disparity image to be processed;

[0175] traversing the disparity image to be processed by sliding the first sliding window;

[0176] In each local disparity image corresponding to the first sliding window, determining whether an absolute value of a difference between each disparity value in the local disparity image and a central disparity value corresponding to a central position of the local disparity image is less than or equal to a first threshold;

[0177] Setting the disparity values ​​in the local disparity image whose absolute values ​​of the differences are less than or equal to the first threshold to 1;

[0178] Setting the disparity values ​​in the local disparity image whose absolute values ​​of the differences are greater than a first threshold to 0;

[0179] Calculating the cumulative value of each updated disparity value in the local disparity image to obtain a statistical quantity;

[0180] If the statistical number is less than or equal to the second threshold, retain the central disparity value;

[0181] If the statistical number is greater than the second threshold, the central disparity value is set to 0;

[0182] After traversing the disparity image to be processed using the first sliding window, a target disparity image is determined.

[0183] The method for determining a disparity image, in the method, a first sliding window is used to traverse a to-be-processed disparity image, in each local disparity image corresponding to the first sliding window, it is determined whether an absolute value of a difference between each disparity value in the local disparity image and a center disparity value corresponding to a center position of the local disparity image is less than or equal to a first threshold value; then, a statistical quantity of the disparity values with the absolute value of the difference less than or equal to the first threshold value is counted, and noise points are deleted according to a statistical result, to obtain a target disparity image. Compared with the prior art, the method is to determine and calculate the local disparity image corresponding to the first sliding window, so that the complexity of the determination and calculation process can be reduced, and therefore the convenience and efficiency of determining the disparity image can be improved.

[0184] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the order of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0185] Based on the same inventive concept, the embodiments of the present application also provide a disparity image determination apparatus for implementing the above-mentioned disparity image determination method. The problem-solving implementation scheme provided by the apparatus is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more disparity image determination apparatus embodiments provided below can refer to the limitations of the disparity image determination method in the above text, which will not be described here again.

[0186] In one embodiment, as shown in Figure 11 A disparity image determination apparatus is provided, comprising a sliding window traversal module 1102, a connectivity determination module 1104, a statistical module 1106, an execution module 1108 and an image determination module 1110, wherein:

[0187] The sliding window traversal module 1102 is configured to perform stereo matching on left and right camera images to obtain a to-be-processed disparity image, and traverse the to-be-processed disparity image by using a first sliding window;

[0188] a connectivity determination module 1104 configured to determine, in each local disparity image corresponding to the first sliding window, whether an absolute value of a difference between each disparity value in the local disparity image and a central disparity value corresponding to a central position of the local disparity image is less than a first threshold;

[0189] A statistics module 1106 is configured to count the number of disparity values ​​whose absolute difference values ​​are less than a first threshold;

[0190] An execution module 1108 is configured to retain the central disparity value if the statistical number is less than or equal to a second threshold; and set the central disparity value to a target value if the statistical number is greater than the second threshold;

[0191] The image determination module 1110 is configured to determine a target disparity image after traversing the disparity images to be processed using a first sliding window.

[0192] An apparatus for determining a disparity image provided by an embodiment of the present invention has the same beneficial effects as the above-mentioned method for determining a disparity image.

[0193] In one embodiment, the sliding window traversal module includes:

[0194] Acquisition submodule, used to acquire left and right camera images;

[0195] The initial cost calculation submodule is used to calculate the initial cost based on the left and right camera images;

[0196] The cost aggregation submodule is used to perform cost aggregation operations based on the initial cost to obtain the final cost;

[0197] The disparity calculation submodule is used to perform disparity calculation on the final cost and determine the disparity image to be processed.

[0198] In one embodiment, the initial cost calculation submodule includes:

[0199] A first determining unit is configured to traverse the first image using a second sliding window, determine each first local pixel image corresponding to the second sliding window, and determine a first transformation vector corresponding to each first local pixel image; wherein the left and right camera images include the first image and the second image;

[0200] a second determining unit, configured to determine a second transformation vector of a second local pixel image corresponding to the first local pixel image in the second image;

[0201] a third determining unit, configured to determine a local initial cost corresponding to the first local pixel image according to the first transformation vector and the second transformation vector;

[0202] The fourth determining unit is configured to obtain an initial cost corresponding to the first image after traversing the first image using the second sliding window; the initial cost includes a local initial cost.

[0203] In one of the embodiments, the second determining unit comprises:

[0204] The first determining sub-unit is configured to determine a preset disparity range, and calculate second local pixel images corresponding to the first local pixel images in the second image and corresponding to the second transform vectors in the preset disparity range; the second transform vectors include the second transform vectors;

[0205] The third determining unit comprises:

[0206] The second determining sub-unit is configured to determine the corresponding to-be-confirmed local initial cost according to the first transform vectors and the to-be-confirmed second transform vectors respectively.

[0207] The third determining sub-unit is configured to determine the minimum value of the to-be-confirmed local initial costs as the local initial cost.

[0208] In one of the embodiments, the disparity image determining apparatus further comprises:

[0209] The saturation operation module is configured to perform a saturation operation on the initial cost to obtain an updated initial cost.

[0210] In one of the embodiments, the saturation operation module comprises:

[0211] The execution sub-module is configured to set the initial cost as a first preset value if the initial cost is greater than or equal to a third threshold value; otherwise, delete the lowest bit of the binary value corresponding to the initial cost, and update the initial cost by using the binary value after the deletion of the lowest bit.

[0212] In one of the embodiments, the cost aggregation sub-module comprises:

[0213] The cost aggregation unit is configured to perform a four-direction cost aggregation operation on the initial cost to obtain a final cost.

[0214] In one of the embodiments, the disparity image determining apparatus further comprises:

[0215] The grouping module is configured to group the to-be-confirmed local initial costs and the local initial costs according to a preset rule.

[0216] The storage module is configured to splice information of the grouped to-be-confirmed local initial costs and the local initial costs, and store the to-be-confirmed local initial costs and the local initial costs in a preset storage position according to the splicing result.

[0217] The modules in the above disparity image determination apparatus can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be invoked and executed by the processor to perform operations corresponding to the modules.

[0218] In an embodiment, an integrated circuit chip is provided, which implements the steps of the above method when running.

[0219] The integrated circuit chip can be an FPGA (Field-Programmable Gate Array) or a CPLD (Complex Programmable Logic Device), and the like. The specific type of the integrated circuit chip is not limited in the embodiment.

[0220] The integrated circuit chip provided in the embodiment has the same beneficial effects as the above disparity image determination method. The integrated circuit chip is used to execute the above method, and the processing speed is fast.

[0221] In an embodiment, a computer device is provided, which can be a terminal device. The internal structure diagram of the computer device can be as shown in FIG. 1. Figure 12 The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program is executed by the processor to implement a disparity image determination method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the shell of the computer device. The input device can also be an external keyboard, touchpad, or mouse, and the like.

[0222] Those skilled in the art can understand that, Figure 12The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0223] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0224] A computer device provided in an embodiment of the present application has the same beneficial effects as the above-mentioned method for determining a disparity image.

[0225] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0226] A computer-readable storage medium provided in an embodiment of the present application has the same beneficial effects as the above-mentioned method for determining a disparity image.

[0227] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0228] A computer program product provided in an embodiment of the present application has the same beneficial effects as the above-mentioned method for determining a disparity image.

[0229] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0230] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0231] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0232] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for determining a disparity image, characterized in that: The method comprises: Get left and right camera images; traversing the first image using a second sliding window, determining each first local pixel image corresponding to the second sliding window, and determining a first transformation vector corresponding to each first local pixel image, wherein the left and right camera images include a first image and a second image; determining a second transformation vector of a second local pixel image corresponding to the first local pixel image in the second image; determining a local initial cost corresponding to the first local pixel image according to the first transformation vector and the second transformation vector; After traversing the first image using the second sliding window, obtaining an initial cost corresponding to the first image, the initial cost including the local initial cost; Performing a saturation operation on the initial cost to obtain an updated initial cost; Performing a cost aggregation operation based on the updated initial cost to obtain a final cost; Performing disparity calculation on the final cost to determine a disparity image to be processed, and slidingly traversing the disparity image to be processed using a first sliding window; determining, in each local disparity image corresponding to the first sliding window, whether an absolute value of a difference between each disparity value in the local disparity image and a central disparity value corresponding to a central position of the local disparity image is less than or equal to a first threshold; Counting the number of disparity values ​​whose absolute value of the difference is less than or equal to the first threshold; If the statistical number is less than or equal to a second threshold, retaining the central disparity value; If the statistical number is greater than the second threshold, setting the central disparity value to a target value; After traversing the disparity image to be processed using the first sliding window, a target disparity image is determined.

2. The method according to claim 1, characterized in that The determining a second transformation vector of a second local pixel image corresponding to the first local pixel image in the second image includes: Determining a preset parallax range, and calculating second transformation vectors to be confirmed corresponding to second local pixel images in the second image corresponding to the first local pixel image within the preset parallax range; the second transformation vectors to be confirmed include the second transformation vector; The determining, according to the first transformation vector and the second transformation vector, a local initial cost corresponding to the first local pixel image includes: Determine the corresponding to-be-confirmed local initial cost according to the first transformation vector and each of the to-be-confirmed second transformation vectors; The minimum value among the to-be-confirmed local initial costs is determined as the local initial cost.

3. The method according to claim 1, characterized in that The performing a saturation operation on the initial cost to obtain the updated initial cost includes: If the initial cost is greater than or equal to a third threshold, setting the initial cost to a first preset value; Otherwise, the lowest bit of the binary value corresponding to the initial cost is deleted, and the initial cost is updated using the binary value after deleting the lowest bit.

4. The method according to claim 1, wherein The performing a cost aggregation operation according to the initial cost to obtain a final cost includes: A four-directional cost aggregation operation is performed according to the initial cost to obtain the final cost.

5. The method according to claim 2, characterized in that The method further comprises: Grouping the to-be-confirmed local initial costs and the local initial costs according to a preset rule; The grouped local initial costs to be confirmed and the local initial costs are spliced ​​together, and the local initial costs to be confirmed and the local initial costs are stored in a preset storage location according to the splicing result.

6. The method according to any one of claims 1 to 5, characterized in that The left and right camera images include: Acquire two sets of left and right camera images using a dual-channel binocular camera module; The method for determining the disparity image is performed on the two groups of left and right camera images respectively in a time-division multiplexing manner.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images