Image processing methods, apparatus, and readable storage media

Abstract

This application provides an image processing method, apparatus, and readable storage medium. The method includes: a processing apparatus acquiring two images of the same scene acquired by two image acquisition devices positioned at different locations; and performing image processing based on the acquired two images using stereo vision technology to obtain the disparity of a target object in the images. This improves image processing efficiency and reduces image processing storage overhead.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method, apparatus, and readable storage medium. Background Technology

[0002] Stereo vision is an important component of computer vision, aiming to reconstruct the three-dimensional geometric information of a scene. It can be applied to artificial intelligence fields such as robotics, autonomous driving, and industrial automation. Binocular stereo vision is an important branch of stereo vision. By simulating human vision, two cameras at different positions observe and capture the same scene, and the depth information of the target object is recovered by the disparity value between pixels in the left and right images.

[0003] Traditional binocular stereo matching generally consists of four steps: matching cost calculation, cost aggregation, disparity calculation, and disparity optimization. It requires multi-dimensional calculations for each pixel, which consumes a large amount of cache space and has a high implementation cost, and still needs to be optimized. Summary of the Invention

[0004] This application provides an image processing method, apparatus, and readable storage medium to solve the problems of low efficiency and large storage space occupation in stereo matching processing.

[0005] Firstly, an image processing method is provided, including:

[0006] Obtain the matching cost data of the target image relative to the reference image, which includes the matching cost value of each pixel in the target image;

[0007] Based on the matching cost data, the region cost aggregation value and path cost aggregation value of the first pixel are obtained. The region cost aggregation value is determined based on the matching cost value of the pixel in the first target region of the first pixel. The path cost aggregation value is determined based on the matching cost value of the pixel on the target path of the first pixel. The first target region is a fan-shaped region between two pixel paths with the first pixel as the vertex. One pixel path is the line connecting the first pixel with the first pixel as the vertex and the pixel adjacent to the first pixel and its extension. The first target region includes only one pixel path in addition to the two pixel paths. The target path is a pixel path that does not belong to the first target region. The first pixel is any pixel in the target image.

[0008] By superimposing the aggregated cost value of the region of the first pixel and the aggregated cost value of the path, the target aggregated cost value of the first pixel is obtained.

[0009] Based on the aggregated value of the target cost, the disparity value of the first pixel is obtained.

[0010] According to the above scheme, by calculating the region cost aggregation value and path aggregation value of each pixel, the processing device can avoid calculating and storing the path aggregation values ​​of multiple paths for each pixel separately. This improves the processing efficiency of the processing device and reduces storage overhead while ensuring the stereo matching effect.

[0011] Optionally, based on the matching cost data, the region cost aggregation value of the first pixel is obtained, including:

[0012] The region cost aggregation value of the first pixel is obtained based on the region cost aggregation value of the three pixels adjacent to the first pixel in the first target region and the matching cost value of the first pixel. The region cost aggregation value of the pixels in the first target region that belong to the edge of the target image is the matching cost value.

[0013] Optionally, the aggregated cost value of the region of the first pixel is the sum of the weighted average of the aggregated cost values ​​of the regions of the three pixels and the matching cost value of the region of the first pixel.

[0014] Optionally, the three pixels are three pixels belonging to the same pixel row or the same pixel column from the following pixel points:

[0015] The pixels adjacent to the first pixel in the upper left direction, the pixels adjacent above the first pixel, the pixels adjacent to the first pixel in the upper right direction, the pixels adjacent to the first pixel in the right direction, the pixels adjacent to the first pixel in the lower right direction, the pixels adjacent below the first pixel, the pixels adjacent to the first pixel in the lower left direction, and the pixels adjacent to the first pixel in the left direction.

[0016] Optionally, the target path is the path of the pixel closest to the first target region.

[0017] Optionally, the acquisition of matching cost data of the target image relative to the reference image includes:

[0018] A first sequence of each pixel in the target image and the reference image is obtained respectively. The first sequence includes multiple grayscale comparison values ​​corresponding to a pixel. One of the grayscale comparison values ​​is the comparison result of the grayscale values ​​of two pixels symmetrical about the center of the pixel in the second target region of the pixel. The second target region is a pixel region of a preset size in the image where the pixel is located, centered on the pixel.

[0019] The matching value of the first pixel is determined based on the first sequence of the first pixel and the first sequence of the reference pixel of the first pixel in the reference image.

[0020] According to the above scheme, compared to comparing the gray values ​​of each pixel within a pixel region (i.e., the Census window) with the center pixel to obtain the Census sequence of the center pixel, comparing the gray values ​​of two pixels in a pixel pair symmetrical about the center pixel to obtain the first sequence of the center pixel can reduce cache overhead and improve processing speed. For two images containing tens of thousands of pixels, this can improve processing speed and save a significant amount of bit storage overhead.

[0021] Optionally, the matching cost data includes multiple matching cost data corresponding to multiple candidate disparity values ​​for each pixel in the target image.

[0022] The target cost aggregate value of the first pixel is obtained by superimposing the region cost aggregate value of the first pixel and the path cost aggregate value.

[0023] By superimposing the region cost aggregation value and the path cost aggregation value corresponding to the same candidate disparity value of the first pixel, the target cost aggregation value corresponding to each candidate disparity value of the first pixel is obtained.

[0024] Based on the aggregated value of the target cost, the disparity value of the first pixel is obtained, including:

[0025] Determine the minimum target cost aggregate value among the multiple target cost aggregate values ​​corresponding to the multiple candidate disparity values ​​of the first pixel;

[0026] The candidate disparity value corresponding to the minimum target cost aggregation value is determined to be the disparity value of the first pixel.

[0027] Secondly, an image processing apparatus is provided, which can be an image processing device, or the image processing apparatus can be configured in an image processing device (e.g., the image processing device is a chip, etc.), including:

[0028] The acquisition unit is used to acquire the target image and the reference image;

[0029] The processing unit is configured to acquire matching cost data of the target image relative to the reference image, the matching cost data including the matching cost value of each pixel in the target image;

[0030] The processing unit is further configured to obtain the region cost aggregation value and path cost aggregation value of the first pixel based on the matching cost data. The region cost aggregation value is determined based on the matching cost value of the pixel in the first target region of the first pixel. The path cost aggregation value is determined based on the matching cost value of the pixel on the target path of the first pixel. The first target region is a fan-shaped region between two pixel paths with the first pixel as the vertex. One pixel path is the line connecting the first pixel with the first pixel as the vertex and an adjacent pixel and its extension. The first target region includes only one pixel path in addition to the two pixel paths. The target path is a pixel path that does not belong to the first target region. The first pixel is any pixel in the target image.

[0031] The processing unit is also used to superimpose the region cost aggregation value and the path cost aggregation value of the first pixel to obtain the target cost aggregation value of the first pixel;

[0032] The processing unit is also used to aggregate the target cost value to obtain the disparity value of the first pixel.

[0033] Optionally, the processing unit is specifically used for:

[0034] The region cost aggregation value of the first pixel is obtained based on the region cost aggregation value of the three pixels adjacent to the first pixel in the first target region and the matching cost value of the first pixel. The region cost aggregation value of the pixels in the first target region that belong to the edge of the target image is the matching cost value.

[0035] Optionally, the aggregated cost value of the region of the first pixel is the sum of the weighted average of the aggregated cost values ​​of the regions of the three pixels and the matching cost value of the region of the first pixel.

[0036] Optionally, the three pixels are three pixels belonging to the same pixel row or the same pixel column from the following pixel points:

[0037] The pixels adjacent to the first pixel in the upper left direction, the pixels adjacent above the first pixel, the pixels adjacent to the first pixel in the upper right direction, the pixels adjacent to the first pixel in the right direction, the pixels adjacent to the first pixel in the lower right direction, the pixels adjacent below the first pixel, the pixels adjacent to the first pixel in the lower left direction, and the pixels adjacent to the first pixel in the left direction.

[0038] Optionally, the target path is the path of the pixel closest to the first target region.

[0039] Optionally, the processing unit is specifically used for:

[0040] A first sequence of each pixel in the target image and the reference image is obtained respectively. The first sequence includes multiple grayscale comparison values ​​corresponding to a pixel. One of the grayscale comparison values ​​is the comparison result of the grayscale values ​​of two pixels symmetrical about the center of the pixel in the second target region of the pixel. The second target region is a pixel region of a preset size in the image where the pixel is located, centered on the pixel.

[0041] The matching value of the first pixel is determined based on the first sequence of the first pixel and the first sequence of the reference pixel of the first pixel in the reference image.

[0042] Optionally, the matching cost data includes multiple matching cost data corresponding to multiple candidate disparity values ​​for each pixel in the target image, and the processing unit is specifically used for:

[0043] The target cost aggregate value of the first pixel is obtained by superimposing the region cost aggregate value of the first pixel and the path cost aggregate value.

[0044] By superimposing the region cost aggregation value and the path cost aggregation value corresponding to the same candidate disparity value of the first pixel, the target cost aggregation value corresponding to each candidate disparity value of the first pixel is obtained.

[0045] Based on the aggregated value of the target cost, the disparity value of the first pixel is obtained, including:

[0046] Determine the minimum target cost aggregate value among the multiple target cost aggregate values ​​corresponding to the multiple candidate disparity values ​​of the first pixel;

[0047] The candidate disparity value corresponding to the minimum target cost aggregation value is determined to be the disparity value of the first pixel.

[0048] Thirdly, a processing apparatus is provided, comprising: a logic circuit and a communication interface, wherein the communication interface is configured to acquire data to be processed and / or output processed data, and the logic circuit is configured to obtain processed data from the data to be processed, so that the processing apparatus performs the methods of the first aspect and any possible implementation thereof.

[0049] In one feasible design, the communication interface includes an input interface and an output interface.

[0050] Fourthly, a processing apparatus is provided, including a processor and a memory. The processor is configured to read instructions stored in the memory and to receive and transmit signals via a transceiver to execute the methods of the first aspect and any possible implementation thereof.

[0051] In one possible implementation, the processing device is a chip.

[0052] Fifthly, a computer program product is provided, comprising: a computer program (also referred to as code or instructions) that, when run, causes a computer to perform the methods described in the first aspect and any possible implementation thereof.

[0053] In a sixth aspect, a computer-readable medium is provided that stores a computer program (also referred to as code or instructions) that, when run on a computer, causes the computer to perform the methods described in the first aspect and any possible implementation thereof.

[0054] This application provides an image processing method in which the region cost aggregation value of a specified area and the path cost aggregation value of a path are calculated and stored separately for each pixel in the cost aggregation process. The region cost aggregation value and the path cost aggregation value of each pixel are then superimposed to obtain the cost aggregation value of that pixel. This method can reduce storage overhead and improve the efficiency of image processing. Attached Figure Description

[0055] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0056] Figure 1 This is a schematic diagram illustrating an application scenario applicable to the image processing method provided in this application;

[0057] Figure 2 This is a schematic diagram of the target pixel and the reference pixel provided in this application;

[0058] Figure 3 This is a schematic diagram of path cost aggregation provided in this application;

[0059] Figure 4 This is a schematic flowchart of the image processing method provided in this application;

[0060] Figure 5 This is a schematic diagram of the second target region of pixel P provided in this application;

[0061] Figure 6 This is a schematic diagram of the regional cost aggregation of the first target region provided in this application;

[0062] Figure 7 This is another schematic diagram of the regional cost aggregation of the first target area provided in this application;

[0063] Figure 8This is a schematic diagram of region cost aggregation and path cost aggregation for a single pixel provided in this application;

[0064] Figure 9 This is a schematic diagram of the image processing apparatus provided in this application;

[0065] Figure 10 This is a schematic diagram of the image processing device provided in this application;

[0066] Figure 11 This is another structural schematic diagram of the image processing device provided in this application.

[0067] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0068] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0069] Figure 1 This is a schematic diagram illustrating an application scenario applicable to the image processing method provided in this application. For example... Figure 1 As shown, this application scenario can include at least two image acquisition devices placed in different locations, such as image acquisition device 101 and image acquisition device 102. For example, the image acquisition devices can be cameras, video recorders, etc. The two image acquisition devices in different locations can respectively acquire images of the same environmental object, such as image 103 and image 104, which can be referred to as the left view and right view, respectively. The image acquisition devices can transmit the acquired environmental images to the image processing device 105, which uses stereoscopic vision technology to obtain the disparity value of the target object in the images based on images 103 and 104. It should be noted that... Figure 1 This is merely an example of an application scenario. In specific implementations, image acquisition devices 101 and 102 can be configured on two different devices or on the same device. Furthermore, image acquisition devices 101, 102, and image processing device 105 can be configured on different devices or on the same device. This application does not impose any limitations on this.

[0070] Traditional binocular stereo matching generally consists of four steps: matching cost calculation, cost aggregation, disparity calculation, and disparity optimization.

[0071] The purpose of matching cost calculation is to measure the correlation between pixels in two images. The image processing device uses one image as the target image and the other as the reference image. Based on the grayscale value of each pixel in both images, it calculates the matching cost between each pixel to be matched in the target image and each reference pixel within the disparity search range D in the reference image, resulting in a cost matrix C (also called the disparity space image, DSI) of size W×H×D (where W is the image width, H is the image height, and D is the disparity search range). For example, the image processing device can calculate the Census sequence for each pixel in both images based on the Census Transform (CT) method. In each image, the image processing device uses a predefined Census window. Each pixel is sequentially designated as the center pixel within that Census window, and its grayscale value is used as a reference grayscale value. The grayscale values ​​of each pixel in the Census window are compared to the reference grayscale value. Pixels with grayscale values ​​less than the reference grayscale value are marked as 0, and those greater than or equal to the reference grayscale value are marked as 1. The comparison results are arranged sequentially according to the pixel order, thus obtaining the Census sequence of the center pixel. The image processing device then calculates the Hamming distance between the Census sequence of each pixel in the target image and the Census sequence of each reference pixel within the disparity search range D in the reference image, based on the Census sequences of pixels in the two images. This distance is the matching cost. For example, if the disparity search range D = 3, including disparity values ​​0, 1, and 2, then... Figure 2 As shown, for a reference pixel at coordinates (h, w) in the target image, there are three reference pixels within the disparity search range in the reference image: reference pixel 1 with a disparity value of 0 and coordinates (h, w), reference pixel 2 with a disparity value of 1 and coordinates (h, w+1), and reference pixel 3 with a disparity value of 2 and coordinates (h, w+2). The image processing device calculates the Hamming distance between the Census sequence of the pixel (h, w) in the target image and the Census sequences of the three reference pixels in the reference image to obtain the matching cost value corresponding to each of the three disparity values ​​of the pixel. The matching cost value is calculated in the same way for each pixel in the target image, thus obtaining the aforementioned cost matrix C.

[0072] In stereo matching, the purpose of cost aggregation is to allow the cost value to more accurately reflect the correlation between pixels. The cost aggregation value of each pixel can be calculated based on the cost aggregation values ​​of its neighboring pixels. For example... Figure 3 As shown, pixel P contains eight adjacent pixels in eight directions: top left, top, top right, right, bottom right, bottom, bottom left, and left. A pixel path is a line connecting pixel P to one of its adjacent pixels, with pixel P as the vertex, and its extension. A pixel has a total of eight pixel paths, as shown below. Figure 3 The diagram shows the top-left path, top path, top-right path, right path, bottom-right path, bottom path, bottom-left path, and left path of pixel P. Theoretically, the preferred path aggregation scheme is to aggregate the matching cost value of pixels on each of the eight paths leading to each pixel, obtaining the aggregated path cost value for each path. Then, these aggregated path cost values ​​from all eight paths are summed to obtain the aggregated cost value for that pixel. For example... Figure 3 As shown, the cost aggregation value of pixel P is the sum of the path cost aggregation values ​​of pixel P and its eight adjacent paths. In practice, for ease of hardware implementation, the cost aggregation values ​​of the left, top-left, top, and top-right paths are typically calculated separately, and then the sum of these four path cost aggregation values ​​is used to obtain the cost aggregation value of pixel P. The image processing device can calculate the cost aggregation value corresponding to each disparity value of each pixel.

[0073] Disparity calculation involves calculating the aggregated cost value for each disparity value of each pixel after cost aggregation. The minimum aggregated cost value among the multiple disparity values ​​for a given pixel represents the optimal disparity value for that pixel. The optimal disparity values ​​of all pixels in the image form a disparity map. Further disparity optimization can be performed to improve the quality of the disparity map, such as removing erroneous disparities, appropriate smoothing, and accuracy optimization.

[0074] As can be seen from the above introduction, the stereo matching method requires multi-dimensional calculations for each pixel. For example, in the matching cost calculation, the Census sequence of each pixel in the two images needs to be calculated and stored in the cache for the next step of calculation. In the cost aggregation, the cost aggregation value of each path in multiple paths needs to be calculated and stored separately for each pixel, resulting in a large storage overhead.

[0075] This application provides an image processing method in which the region cost aggregation value of a specified area and the path cost aggregation value of a path are calculated and stored separately for each pixel in the cost aggregation process. The region cost aggregation value and the path cost aggregation value of each pixel are then superimposed to obtain the cost aggregation value of that pixel. This method can reduce storage overhead and improve the efficiency of image processing.

[0076] The image processing method provided in this application will be described below with reference to the accompanying drawings.

[0077] Figure 4This is a schematic flowchart of the image processing method provided in this application. The method can be executed by a processing device, such as... Figure 1 The image processing device 105 shown in the application scenario can be a processing device, or the processing device can be configured within a processing device, such as a chip. However, this application is not limited to this. The method includes, but is not limited to, the following steps:

[0078] S401, the processing device acquires matching cost data of the target image relative to the reference image, the matching cost data including the matching cost of each pixel in the target image.

[0079] The processing device can acquire the matching cost data of the target image relative to the reference image in the following two optional implementation methods, including but not limited to:

[0080] In Implementation Method 1, the processing device can calculate the Census sequence for each pixel in both images based on the Census transform method described above. Then, based on the Census sequences of each pixel in the two images, it obtains the matching cost value for each pixel in the target image, i.e., the matching cost data of the target image relative to the reference image. Specific implementation methods can be found in the preceding description; for brevity, they will not be repeated here.

[0081] In the second implementation method, the processing device can acquire a first sequence of each pixel in the target image and the reference image, respectively. This first sequence includes multiple grayscale comparison values ​​corresponding to a pixel. Each grayscale comparison value is a comparison result of the grayscale values ​​of two pixel pairs symmetrical about the center of the pixel in a second target region. The second target region is a pixel region of a preset size centered on the pixel in the image where the pixel is located. The processing device can determine the matching cost value of a pixel based on the first sequence of a pixel in the target image and the first sequence of reference pixels of that pixel in the reference image. This yields the matching cost value of each pixel in the target image, i.e., the matching cost data of the target image relative to the reference image.

[0082] For example Figure 5 As shown, taking the processing device acquiring the first sequence of pixel point P as an example, if the second pixel region is a 3×3 pixel region, the processing device can acquire the grayscale values ​​of the pixels within the second pixel region of pixel point P, such as... Figure 5The pixel shown is labeled with its grayscale value. The second pixel region of pixel P is a 3×3 pixel region centered on pixel P, including the eight adjacent pixels of pixel P: the pixel with a grayscale value of 35 in the upper left direction, the pixel above with a grayscale value of 31, the pixel with a grayscale value of 22 in the upper right direction, the pixel to the left with a grayscale value of 20, the pixel to the right with a grayscale value of 30, the pixel with a grayscale value of 18 in the lower left direction, the pixel below with a grayscale value of 42, and the pixel with a grayscale value of 40 in the lower right direction. The processing device compares the grayscale values ​​of every two pixels symmetrical about the center of pixel P in this second pixel region, and obtains the comparison result of the grayscale values ​​of each pair of pixels as the grayscale comparison value. For example, the comparison rule can be: if... Figure 5 If the grayscale value of a pixel within the dashed area is greater than the grayscale value of its symmetrical pixel, the grayscale comparison value is 1; if the grayscale value of a pixel within the dashed area is less than or equal to the grayscale value of its symmetrical pixel, the grayscale comparison value is 0. For example, the processing device compares the pixel with a grayscale value of 35 in the upper left direction of pixel P with its symmetrical pixel (i.e., the pixel with a grayscale value of 40 in the lower right direction of pixel P), and obtains a grayscale comparison value of 0. Comparing the pixel with a grayscale value of 31 above pixel P with its symmetrical pixel (i.e., the pixel with a grayscale value of 42 below pixel P), the grayscale comparison value is 0. Comparing the pixel with a grayscale value of 22 in the upper right direction of pixel P with its symmetrical pixel (i.e., the pixel with a grayscale value of 18 in the lower left direction of pixel P), the grayscale comparison value is 1. Furthermore, by comparing the pixel with a gray value of 20 to the left of pixel P with its symmetrical pixel (i.e., the pixel with a gray value of 30 to the lower left of pixel P), a gray-scale comparison value of 0 is obtained. Additionally, a comparison of pixel P with itself yields a gray-scale comparison value of 0. Therefore, the first sequence of pixel P is

[00100] . The processing device can use this method to obtain the first sequence of each pixel in the target image and the first sequence of each pixel in the reference image. It should be understood that the comparison rule can also be: if the gray value of a pixel in the dashed line region is greater than the gray value of its symmetrical pixel, the gray-scale comparison value is 0; if the gray value of a pixel in the dashed line region is less than or equal to the gray value of its symmetrical pixel, the gray-scale comparison value is 1. The size of the second pixel region can be determined according to specific implementations, such as a 5×5 pixel region, which is not limited in this application.

[0083] After the processing device acquires the first sequence of each pixel in the target image and the reference image, it calculates the Hamming distance between the first sequence of each pixel in the target image and the first sequence of the corresponding reference pixel in the reference image, and obtains the matching cost value of each pixel, that is, the matching cost data of the target image relative to the reference image.

[0084] Compared to Implementation 1, which compares the grayscale value of each pixel within a pixel region (i.e., the Census window) with the center pixel to obtain the Census sequence of the center pixel, Implementation 2, which compares the grayscale values ​​of two pixels in a pixel pair symmetrical about the center pixel to obtain the first sequence of the center pixel, reduces cache overhead and improves processing speed. For example, in a 3×3 pixel region, Implementation 1 performs 9 grayscale value comparisons and generates and stores a 9-bit Census sequence for each pixel. Implementation 2 performs 5 grayscale value comparisons and generates and stores a 5-bit first sequence for each pixel. For two images containing tens of thousands of pixels, this improves processing speed and saves significant bit storage overhead.

[0085] It should be noted that in the two embodiments described above, for pixels located at the edge of the image, the missing pixel positions in the pixel region can be filled by copying adjacent pixels. However, this application is not limited to this.

[0086] S402, the processing device obtains the region cost aggregation value and path cost aggregation value of the first pixel based on the matching cost data. The region cost aggregation value is determined based on the matching cost value of the pixels in the first target region of the first pixel, and the path cost aggregation value is determined based on the matching cost value of the pixels on the target path of the first pixel.

[0087] Here, the first pixel is any pixel in the target image. The first target region is a fan-shaped region between two pixel paths with the first pixel as its vertex. A pixel path is the line connecting the first pixel with the first pixel as its vertex and its adjacent pixel, plus its extension. The first target region includes only one pixel path besides these two pixel paths. The target path is a pixel path that does not belong to the first target region.

[0088] For example Figure 3 The diagram shows eight pixel paths for pixel P. The first target region of pixel P (an example of the first pixel) is a fan-shaped region between two pixel paths with pixel P as a vertex, and this first target region includes only one pixel path besides the two pixel paths shown. The target path of pixel P is a pixel path that does not belong to the first target region.

[0089] For example, the first target region can be a fan-shaped region between the upper left path and the upper right path with pixel P as the vertex. In addition to the two pixel paths, the first target region also includes the upper path. The target path of pixel P can be one of the following paths: left path, lower left path, lower path, lower right path, or right path.

[0090] For example, the first target region can be a fan-shaped region between the upper path and the right path with pixel P as the vertex, and the first target region includes the upper right path in addition to the two pixel paths. The target path of pixel P can be one of the upper left path, left path, lower left path, lower path, or lower right path.

[0091] In other words, the first target region of the first pixel includes three pixels adjacent to the first pixel. Optionally, these three pixels are three pixels belonging to the same pixel row or the same pixel column:

[0092] The pixels adjacent to the first pixel in the upper left direction, the pixels adjacent above the first pixel, the pixels adjacent to the first pixel in the upper right direction, the pixels adjacent to the first pixel in the right direction, the pixels adjacent to the first pixel in the lower right direction, the pixels adjacent below the first pixel, the pixels adjacent to the first pixel in the lower left direction, and the pixels adjacent to the first pixel in the left direction.

[0093] For example, the first target region includes the top-left adjacent pixels, the top adjacent pixels, and the top-right adjacent pixels belonging to the first pixel in the same row. The first target region is a fan-shaped area between the top-left path and the top-right path.

[0094] For example, the first target region includes the upper-right adjacent pixels, the right adjacent pixels, and the lower-right adjacent pixels belonging to the first pixel in the same column. The first target region is a fan-shaped area between the upper-right path and the lower-right path.

[0095] The processing device can obtain the region cost aggregation value of the first pixel based on the region cost aggregation value of the three pixels adjacent to the first pixel in the first target region and the matching cost value of the first pixel. The region cost aggregation value of the pixels located at the image edge in the first target region is the matching cost value.

[0096] Optionally, the processing device calculates the weighted average of the region cost aggregate values ​​of three pixels adjacent to the first pixel in the first target region, and adds the weighted average value to the matching cost value of the first pixel to obtain the region cost aggregate value of the first pixel.

[0097] In other words, the aggregated value of the region cost of the first pixel is the sum of the matching cost of the first pixel and the weighted average of the aggregated values ​​of the region costs of the three pixels adjacent to the first pixel in the first target region.

[0098] For example, the first target region is a fan-shaped region between the upper right path and the lower right path. The region cost aggregation value of the first pixel is obtained by the processing device based on the matching cost value of the first pixel and the region cost aggregation values ​​of its upper-left, upper-top, and upper-right adjacent pixels. Figure 6 As shown, each pixel is identified by its index. For pixel 44 in the fourth row and fourth column, the first target region is a fan-shaped area between the upper left and upper right paths. This first target region includes three pixels adjacent to pixel 44: upper left pixel 33, upper top pixel 34, and upper right pixel 35. The region cost aggregation value of pixel 44 is obtained by the processing device based on the matching cost value of pixel 44 and the region cost aggregation values ​​of pixels 33, 34, and 35. Similarly, the first target region of pixel 33 is a fan-shaped area between the upper left and upper right paths. The region cost aggregation value of pixel 33 is obtained by the processing device based on the matching cost value of pixel 33 and the region cost aggregation values ​​of pixels 22, 23, and 24. The region cost aggregation values ​​of pixels 34 and 35 are obtained by the processing device using the same method. For the second row of pixels, taking pixel 22 as an example, the region cost aggregation value of pixel 22 is obtained by the processing device based on the region matching cost values ​​of pixels 11, 12, and 13. Since pixels 11, 12, and 13 are pixels located at the image edge, their region matching cost values ​​are also considered. Therefore, the region cost aggregation value of pixel 44 is determined based on the matching cost values ​​of the pixels in the first target region of pixel 44.

[0099] In specific implementation, the processing device can calculate the region cost aggregation value of each pixel starting from the pixels located at the image edge in the first target region. For example, if the first target region is a fan-shaped area between the upper left and upper right paths, the processing device can calculate the region cost aggregation value of each pixel in the target image starting from the first row and first column of pixels. If the region cost aggregation value of the pixels in the first row is the matching cost value, then the region cost aggregation value of the pixels in the second row is calculated. For example, the region cost aggregation value of pixel 21 in the first column of the second row is the weighted average of the region cost aggregation values ​​of pixels 11 and 12, plus the matching cost value of pixel 21. The processing device then calculates the region cost aggregation value of pixel 22 in the second row and second column, that is, first calculates the weighted average of the region cost aggregation values ​​of pixels 11, 12, and 13, and then adds the weighted average value and the matching cost value of pixel 22 to obtain the region cost aggregation value of pixel 22. The processing device then calculates the region cost aggregation values ​​of the other pixels in the second row. After obtaining the region cost aggregation value of the pixels in the second row, the processing device calculates the region cost aggregation value of each pixel in the third row based on the region cost aggregation value of the pixels in the second row, and so on, so that the processing device can obtain the region cost aggregation value of each pixel in the target image.

[0100] In this example, for each pixel in the first column, the processing device obtains the region cost aggregate value of that pixel based on the weighted average of the region cost aggregate values ​​of the pixels above and the pixels to the upper right, and the matching cost value of that pixel. A similar method can be used for each pixel in the last column, based on the weighted average of the region cost aggregate values ​​of the pixels to the upper left and the pixels above, and the matching cost value of that pixel. However, this application is not limited to this. In another optional implementation, for pixels in the first column that do not have adjacent pixels in the upper-right direction, a virtual region cost aggregation value for an upper-right pixel can be generated according to a preset rule. For example, taking pixel 21 as an example, the matching cost value of pixel 21 can be used as the region cost aggregation value for adjacent pixels in the upper-right direction. In order to calculate the region cost aggregation value of pixel 21, the processing device can first calculate the weighted average of the virtual region cost aggregation value of the upper-right pixel (i.e., the matching cost value of pixel 21 in this example), the region cost aggregation value of pixel 11, and the region cost aggregation value of pixel 12. Then, the matching cost value of the pixel is added to the weighted average value to obtain the region cost aggregation value of pixel 21. Similarly, for each pixel in the last column, a virtual region cost aggregation value for an adjacent pixel in the upper-left direction can be generated in the same way.

[0101] The above example, using a fan-shaped region between the upper left and upper right paths as the first target region, illustrates how the processing device obtains the region cost aggregation value for each pixel in the target image. However, this application is not limited to this; the first target region can also be a fan-shaped region between other two paths, such as the upper right and lower right paths. Figure 7 As shown, the processing device can calculate the region cost aggregation value for each pixel, starting from the pixels located at the image edge in the first target region. Specifically, the processing device can calculate the region cost aggregation value for each pixel starting from the last column. The region cost aggregation value of the last column of pixels is the matching cost aggregation value. Then, based on the region cost aggregation value of the last column of pixels, the processing device calculates the region cost aggregation value of the second-to-last column of pixels. That is, the processing device calculates the weighted average of the region cost aggregation values ​​of the three pixels adjacent to a given pixel in the first target region, and then adds this weighted average to the matching cost value of that pixel to obtain the region cost aggregation value of that pixel. The processing device sequentially completes the calculation of the region cost aggregation values ​​from the last column to the first column to obtain the region cost aggregation value of each pixel in the target image.

[0102] Optionally, the target path of the first pixel is the pixel path closest to the first target region.

[0103] For example, the first target region is a fan-shaped area between the top-left path and the top-right path. The target path can be either the left path or the right path.

[0104] For example, the first target area is a fan-shaped region between the top-left path and the bottom-left path. The target path can be either the top path or the bottom path.

[0105] The processing device obtains the path cost aggregation value of the first pixel based on the matching cost value of pixels in the first target path. Specifically, the processing device obtains the path cost aggregation value of the first pixel based on the path cost aggregation values ​​of pixels adjacent to the first pixel in the first target path and the matching cost aggregation value of the first pixel. For example Figure 8 As shown, the first target region of the first pixel is the fan-shaped region between the upper left path and the upper right path, and the target path is the left path. Therefore, as follows... Figure 8The path cost aggregation value of pixel 44 shown is obtained by the processing device based on the path cost aggregation value of pixel 43 and the matching cost aggregation value of pixel 44. In a specific implementation, the processing device can calculate the path cost aggregation value of each pixel in each row, starting from the pixels in the first column. For example, for the first row, the processing device determines that the path cost aggregation value of pixel 11 is the matching cost value of pixel 11, and the path cost aggregation value of pixel 12 is the sum of the path cost aggregation value of pixel 11 and the matching cost value of pixel 12. The path cost aggregation value of pixel 13 is the sum of the path cost aggregation value of pixel 12 and the matching cost value of pixel 13. And so on, the processing device can calculate the path cost aggregation value of each pixel in the target image.

[0106] According to the above scheme, by calculating the region cost aggregation value and path aggregation value of each pixel, the processing device can avoid calculating and storing the path aggregation values ​​of multiple paths for each pixel separately. This can improve the processing efficiency of the processing device and reduce storage overhead while ensuring the stereo matching effect.

[0107] Optionally, the matching cost data of the target image relative to the reference image includes the matching cost value of each image corresponding to each disparity value of the target image relative to the reference image, that is, the matching cost data can be a matching cost matrix of W×H×D.

[0108] In one embodiment, the processing device can calculate the region cost aggregation value and the path cost aggregation value of each pixel for each disparity value. Specifically, the processing device obtains the path cost aggregation value of the first pixel based on the path cost aggregation values ​​of adjacent pixels within the first path of the first pixel that belong to the same disparity value. Furthermore, the processing device obtains the region cost aggregation value of the first pixel based on the region cost aggregation values ​​of three adjacent pixels within the first target region of the same disparity value and the matching cost value of the first pixel.

[0109] In another embodiment, when calculating the path cost aggregation value of a first pixel within a disparity value, the processing device obtains the path cost aggregation value of the first pixel based on the path cost aggregation values ​​of adjacent pixels within the target path of the first pixel that belong to the same disparity and the path cost aggregation values ​​of adjacent pixels that belong to adjacent disparities. Furthermore, when calculating the region cost aggregation value of the first pixel within a disparity value, the processing device obtains the region cost aggregation value of adjacent pixels that belong to the same disparity and the region cost aggregation value of adjacent disparities.

[0110] For example, the processing device calculates the path cost aggregation value L corresponding to the disparity value d of pixel p. rWhen (p, d), the matching cost C(p, d) is based on the disparity value d of pixel p, and the path cost aggregation value L corresponding to the disparity value d of the adjacent pixel pr in the target path. r (pr, d), and the path cost aggregation value L corresponding to the two adjacent disparity values, i.e., the disparity value d-1. r The aggregated path cost L corresponding to (pr, d-1) and disparity value d+1 is... r The minimum path cost aggregation value min within the disparity range of (pr, d+1) and its neighboring pixel pr i L r (pr, i) yields the path cost aggregation value L corresponding to the disparity value d of pixel p. r (p, d), L r (p, d) satisfy the following equation:

[0111] L r (p, d) = C(P, d) + E s，r ,

[0112]

[0113] Where P1 and P2 are penalty terms, and P1 < P2.

[0114] Pixel p has three adjacent pixels within the first target region, namely pixels p-r1, p-r2, and p-r3. The processing device calculates the region cost aggregation value L corresponding to the disparity value d of pixel p. A When (p, d), based on the matching cost C(p, d) corresponding to the disparity value d of pixel p, and the adjacent pixel pr in the first target region. k The region cost aggregation value L corresponding to the disparity value d (where k = 1, 2, 3) a (pr k d), and the two adjacent disparity values, i.e., the region cost aggregation value L corresponding to the disparity value d-1. a (pr k The aggregated region cost L corresponding to the disparity values ​​d-1 and d+1 is d-1. a (pr k The minimum region cost aggregation value min within the disparity range of d+1) and the adjacent pixel pr i L a (pr k ,i), to obtain the path cost aggregation value L corresponding to the disparity value d of pixel p. a (p, d), L a (p, d) satisfy the following equation:

[0115] L a(p, d) = C(p, d) + E s，a ,

[0116]

[0117] According to the above scheme, by calculating the region cost aggregation value and path aggregation value of each pixel, the storage cost of the region cost aggregation value of the pixel obtained by weighted averaging the region cost aggregation values ​​of three adjacent pixels and superimposing them with the matching cost value of the pixel is close to the storage cost of the path aggregation value. This avoids the processing device from calculating and storing the path aggregation values ​​of multiple paths for each pixel separately, thereby improving the processing efficiency of the processing device and reducing storage costs while ensuring the stereo matching effect.

[0118] S403, the processing device superimposes the region cost aggregation value and the path cost aggregation value of the first pixel to obtain the target cost aggregation value of the first pixel.

[0119] The processing device superimposes the region cost aggregation value and path cost aggregation value of each pixel corresponding to each disparity value calculated by S402 to obtain the target cost aggregation value corresponding to the disparity value of each pixel.

[0120] For example, the aggregated target cost of pixel p is S(p, d), which satisfies:

[0121] S(p,d)=L r (p, d) + L a (p, d)

[0122] S404, the processing device obtains the disparity value of the first pixel based on the target cost aggregation value of the first pixel.

[0123] The processing device determines the minimum target cost aggregate value among the target cost aggregate values ​​corresponding to each disparity value within the disparity range of each pixel, and the disparity value corresponding to the minimum target cost aggregate value is the optimal disparity value of the first pixel.

[0124] The image processing method provided by this application has been described above with reference to the accompanying drawings. The processing apparatus provided by this application will be described below.

[0125] Figure 9 This is a schematic diagram of the image processing apparatus 900 provided in this application. The processing apparatus can be implemented as a processing device itself or configured within a processing device through software, hardware, or a combination of both. The processing apparatus 900 includes an acquisition unit 901 and a processing unit 902. Optionally, the processing apparatus 900 further includes a storage unit 903 for storing data generated during the image processing process.

[0126] Acquisition unit 901 is used to acquire the target image and the reference image;

[0127] The processing unit 902 is used to obtain matching cost data of the target image relative to the reference image, the matching cost data including the matching cost value of each pixel in the target image;

[0128] The processing unit 902 is further configured to obtain the region cost aggregation value and the path cost aggregation value of the first pixel based on the matching cost data. The region cost aggregation value is determined based on the matching cost value of the pixel in the first target region of the first pixel. The path cost aggregation value is determined based on the matching cost value of the pixel on the target path of the first pixel. The first target region is a fan-shaped region between two pixel paths with the first pixel as the vertex. One pixel path is the line connecting the first pixel with the first pixel as the vertex and an adjacent pixel and its extension. The first target region includes only one pixel path in addition to the two pixel paths. The target path is a pixel path that does not belong to the first target region. The first pixel is any pixel in the target image.

[0129] The processing unit 902 is also used to superimpose the region cost aggregation value and the path cost aggregation value of the first pixel to obtain the target cost aggregation value of the first pixel;

[0130] The processing unit 902 is also used to obtain the disparity value of the first pixel based on the target cost aggregation value.

[0131] Optionally, the processing unit 902 is specifically used for:

[0132] The region cost aggregation value of the first pixel is obtained based on the region cost aggregation value of the three pixels adjacent to the first pixel in the first target region and the matching cost value of the first pixel. The region cost aggregation value of the pixels in the first target region that belong to the edge of the target image is the matching cost value.

[0133] Optionally, the aggregated cost value of the region of the first pixel is the sum of the weighted average of the aggregated cost values ​​of the regions of the three pixels and the matching cost value of the region of the first pixel.

[0134] Optionally, the three pixels are three pixels belonging to the same pixel row or the same pixel column from the following pixel points:

[0135] The pixels adjacent to the first pixel in the upper left direction, the pixels adjacent above the first pixel, the pixels adjacent to the first pixel in the upper right direction, the pixels adjacent to the first pixel in the right direction, the pixels adjacent to the first pixel in the lower right direction, the pixels adjacent below the first pixel, the pixels adjacent to the first pixel in the lower left direction, and the pixels adjacent to the first pixel in the left direction.

[0136] Optionally, the target path is the path of the pixel closest to the first target region.

[0137] Optionally, the processing unit 902 is specifically used for:

[0138] A first sequence of each pixel in the target image and the reference image is obtained respectively. The first sequence includes multiple grayscale comparison values ​​corresponding to a pixel. One of the grayscale comparison values ​​is the comparison result of the grayscale values ​​of two pixels symmetrical about the center of the pixel in the second target region of the pixel. The second target region is a pixel region of a preset size in the image where the pixel is located, centered on the pixel.

[0139] The matching value of the first pixel is determined based on the first sequence of the first pixel and the first sequence of the reference pixel of the first pixel in the reference image.

[0140] Optionally, the matching cost data includes multiple matching cost data corresponding to multiple candidate disparity values ​​for each pixel in the target image, and the processing unit 902 is specifically used for:

[0141] The target cost aggregate value of the first pixel is obtained by superimposing the region cost aggregate value of the first pixel and the path cost aggregate value.

[0142] By superimposing the region cost aggregation value and the path cost aggregation value corresponding to the same candidate disparity value of the first pixel, the target cost aggregation value corresponding to each candidate disparity value of the first pixel is obtained.

[0143] Based on the aggregated value of the target cost, the disparity value of the first pixel is obtained, including:

[0144] Determine the minimum target cost aggregate value among the multiple target cost aggregate values ​​corresponding to the multiple candidate disparity values ​​of the first pixel;

[0145] The candidate disparity value corresponding to the minimum target cost aggregation value is determined to be the disparity value of the first pixel.

[0146] This application also provides a processing apparatus. The processing apparatus includes: logic circuitry and a communication interface, wherein the communication interface is used to acquire data to be processed and / or output processed data, and the logic circuitry is used to process the data to be processed to obtain processed data, so that the processing apparatus executes the image processing method described in the above method embodiments. In one feasible design, the communication interface includes an input interface and an output interface.

[0147] It should be understood that the processing device provided in the embodiments of this application can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0148] Figure 10 This is a schematic diagram of the structure of the image processing device 1000 provided in an embodiment of this application. The image processing device 1000 can be applied to, for example... Figure 1 In the system shown, the processing device described in the above method embodiment performs its functions. As shown, the image processing device 1000 includes a transceiver 1001, a processor 1002, and a memory 1003. The processor 1002, transceiver 1001, and memory 1003 can communicate with each other via internal connections to transmit image processing data and / or control signals. The memory 1003 stores a computer program, and the processor 1002 executes the computer program stored in the memory to control the transceiver 1001 to acquire image data from the image acquisition device. In specific implementations, the memory can be integrated into the processor or independent of the processor.

[0149] Figure 11 This is a schematic diagram of the structure of an image processing device 1100 provided in an embodiment of this application. The image processing device 1100 performs the functions of the processing apparatus in the above-described method embodiments. As shown in the figure, the image processing device 1100 includes an image acquisition unit 1101, a processor 1102, and a memory 1103. The processor 1102, the image acquisition unit 1101, and the memory 1103 can communicate with each other through internal connection channels to transmit image processing data and / or control signals. The memory 1103 stores a computer program, and the processor 1102 executes the computer program in the memory to control the image acquisition unit 1101 to acquire image data. For example, the image acquisition unit 1101 is a camera. In specific implementations, the memory can also be integrated into the processor or independent of the processor.

[0150] This embodiment also provides a readable storage medium storing execution instructions. When at least one processor of the electronic device executes the execution instructions, the image processing method described in the above embodiment is implemented.

[0151] This embodiment also provides a computer program product including executable instructions stored in a readable storage medium. At least one processor of an electronic device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the electronic device to perform the image processing methods provided in the various embodiments described above.

[0152] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0153] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in a combination of hardware and software functional modules.

[0155] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0156] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0157] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, include: Obtain matching cost data of the target image relative to the reference image, wherein the matching cost data includes the matching cost value of each pixel in the target image; Based on the matching cost data, the region cost aggregation value and path cost aggregation value of the first pixel are obtained. The region cost aggregation value is determined based on the matching cost value of the pixel in the first target region of the first pixel. The path cost aggregation value is determined based on the matching cost value of the pixel on the target path of the first pixel. The first target region is a fan-shaped region between two pixel paths with the first pixel as the vertex. A pixel path is a line connecting the first pixel with the first pixel as the vertex and an adjacent pixel and its extension. The first target region includes only one pixel path in addition to the two pixel paths. The target path is a pixel path that does not belong to the first target region. The first pixel is any pixel in the target image. By superimposing the region cost aggregation value and the path cost aggregation value of the first pixel, the target cost aggregation value of the first pixel is obtained. Based on the target cost aggregation value, the disparity value of the first pixel is obtained.

2. The method according to claim 1, characterized in that, The step of obtaining the region cost aggregation value of the first pixel based on the matching cost data includes: The region cost aggregation value of the first pixel is obtained based on the region cost aggregation value of the three pixels adjacent to the first pixel in the first target region and the matching cost value of the first pixel. The region cost aggregation value of the pixels in the first target region that belong to the edge of the target image is the matching cost value.

3. The method according to claim 2, characterized in that, The region cost aggregation value of the first pixel is the weighted average of the region cost aggregation values ​​of the three pixels and the sum of the matching cost value of the first pixel.

4. The method according to claim 2 or 3, characterized in that, The three pixels are three pixels belonging to the same pixel row or the same pixel column from the following pixel points: The pixels adjacent to the first pixel in the upper left direction, the pixels above the first pixel, the pixels adjacent to the first pixel in the upper right direction, the pixels to the right of the first pixel, the pixels to the lower right of the first pixel, the pixels below the first pixel, the pixels adjacent to the lower left direction of the first pixel, and the pixels to the left of the first pixel.

5. The method according to any one of claims 1 to 3, characterized in that, The target path is the pixel path that is closest to the first target region.

6. The method according to any one of claims 1 to 3, characterized in that, The step of obtaining the matching cost data of the target image relative to the reference image includes: A first sequence of each pixel in the target image and the reference image is obtained respectively. The first sequence includes multiple grayscale comparison values ​​corresponding to a pixel. One of the grayscale comparison values ​​is the comparison result of the grayscale values ​​of two pixels symmetrical about the center of the pixel in the second target region of the pixel. The second target region is a pixel region of a preset size in the image where the pixel is located, centered on the pixel. The matching value of the first pixel is determined based on the first sequence of the first pixel and the first sequence of the reference pixel of the first pixel in the reference image.

7. The method according to any one of claims 1 to 3, characterized in that, The matching cost data includes multiple matching cost data corresponding to multiple candidate disparity values ​​for each pixel in the target image. The target cost aggregate value of the first pixel is obtained by superimposing the region cost aggregate value and the path cost aggregate value of the first pixel. By superimposing the region cost aggregation value and the path cost aggregation value corresponding to the same candidate disparity value of the first pixel, the target cost aggregation value corresponding to each candidate disparity value of the first pixel is obtained; The step of obtaining the disparity value of the first pixel based on the target cost aggregation value includes: Determine the minimum target cost aggregate value among the multiple target cost aggregate values ​​corresponding to the multiple candidate disparity values ​​of the first pixel; The candidate disparity value corresponding to the minimum target cost aggregation value is determined to be the disparity value of the first pixel.

8. An image processing apparatus, characterized in that, include: The acquisition unit is used to acquire the target image and the reference image; A processing unit is configured to acquire matching cost data of the target image relative to the reference image, wherein the matching cost data includes the matching cost value of each pixel in the target image; The processing unit is further configured to obtain a region cost aggregation value and a path cost aggregation value for the first pixel based on the matching cost data. The region cost aggregation value is determined based on the matching cost value of the pixel in the first target region of the first pixel. The path cost aggregation value is determined based on the matching cost value of the pixel on the target path of the first pixel. The first target region is a fan-shaped region between two pixel paths with the first pixel as the vertex. A pixel path is a line connecting the first pixel with the first pixel as the vertex and a pixel adjacent to the first pixel, and an extension line. The first target region includes only one pixel path in addition to the two pixel paths. The target path is a pixel path that does not belong to the first target region. The first pixel is any pixel in the target image. The processing unit is further configured to superimpose the region cost aggregation value and the path cost aggregation value of the first pixel to obtain the target cost aggregation value of the first pixel. The processing unit is further configured to obtain the disparity value of the first pixel based on the target cost aggregation value.

9. The apparatus according to claim 8, characterized in that, The processing unit is specifically used for: The region cost aggregation value of the first pixel is obtained based on the region cost aggregation value of the three pixels adjacent to the first pixel in the first target region and the matching cost value of the first pixel. The region cost aggregation value of the pixels in the first target region that belong to the edge of the target image is the matching cost value.

10. The apparatus according to claim 9, characterized in that, The region cost aggregation value of the first pixel is the weighted average of the region cost aggregation values ​​of the three pixels and the sum of the matching cost value of the first pixel.

11. The apparatus according to claim 9 or 10, characterized in that, The three pixels are three pixels belonging to the same pixel row or the same pixel column from the following pixel points: The pixels adjacent to the first pixel in the upper left direction, the pixels above the first pixel, the pixels adjacent to the first pixel in the upper right direction, the pixels to the right of the first pixel, the pixels to the lower right of the first pixel, the pixels below the first pixel, the pixels adjacent to the lower left direction of the first pixel, and the pixels to the left of the first pixel.

12. The apparatus according to any one of claims 8 to 10, characterized in that, The target path is the pixel path that is closest to the first target region.

13. The apparatus according to any one of claims 8 to 10, characterized in that, The processing unit is specifically used for: A first sequence of each pixel in the target image and the reference image is obtained respectively. The first sequence includes multiple grayscale comparison values ​​corresponding to a pixel. One of the grayscale comparison values ​​is the comparison result of the grayscale values ​​of two pixels symmetrical about the center of the pixel in the second target region of the pixel. The second target region is a pixel region of a preset size in the image where the pixel is located, centered on the pixel. The matching value of the first pixel is determined based on the first sequence of the first pixel and the first sequence of the reference pixel of the first pixel in the reference image.

14. The apparatus according to any one of claims 8 to 10, characterized in that, The matching cost data includes multiple matching cost data corresponding to multiple candidate disparity values ​​for each pixel in the target image. The processing unit is specifically used for: The target cost aggregate value of the first pixel is obtained by superimposing the region cost aggregate value and the path cost aggregate value of the first pixel. By superimposing the region cost aggregation value and the path cost aggregation value corresponding to the same candidate disparity value of the first pixel, the target cost aggregation value corresponding to each candidate disparity value of the first pixel is obtained; The step of obtaining the disparity value of the first pixel based on the target cost aggregation value includes: Determine the minimum target cost aggregate value among the multiple target cost aggregate values ​​corresponding to the multiple candidate disparity values ​​of the first pixel; The candidate disparity value corresponding to the minimum target cost aggregation value is determined to be the disparity value of the first pixel.

15. A processing apparatus, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the processing device to perform the method of any one of claims 1 to 7.

16. A chip, characterized in that, Includes at least one logic circuit and an input / output interface; The logic circuit is used to control the input / output interface and perform the method as described in any one of claims 1 to 7.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 7.

18. A computer program product, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images