Processing system, processing method, and program

Abstract

To provide a processing system capable of accurately performing distance measurement using a stereo image, even when a small-scale circuit is utilized.SOLUTION: A processing system comprises: a first parallax calculation part for calculating a first parallax value which is a parallax value for a first stereo image; a second parallax calculation part for calculating a second parallax value which is a parallax value for a second stereo image obtained by horizontally reducing the first stereo image; and a specification part for specifying the parallax on the basis of the first parallax value and the second parallax value.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present disclosure relates to a processing system, a processing method, and a program. [Background technology]

[0002] In the field of hazard detection, a technique for measuring the distance to an object using stereo images captured by a stereo camera is used. Patent Document 1 discloses a related technique for providing a depth map with a confidence estimate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2023-521765 Summary of the Invention [Problem to be solved by the invention]

[0004] In the field of distance measurement related to Patent Document 1, there is a demand for a technology that can perform accurate distance measurement using stereo images even when using small-scale circuits.

[0005] One of the objectives of each aspect of the present disclosure is to provide a processing system, a processing method, and a program that can solve the above-mentioned problems. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, a processing system includes a first disparity calculation unit that calculates a first disparity value that is a disparity value for a first stereo image, a second disparity calculation unit that calculates a second disparity value that is a disparity value for a second stereo image obtained by horizontally reducing the first stereo image, and an identification unit that identifies a disparity based on the first disparity value and the second disparity value.

[0007] According to another aspect of the present disclosure, a processing method includes calculating a first disparity value for a first stereo image, calculating a second disparity value for a second stereo image obtained by horizontally reducing the first stereo image, and determining a disparity based on the first disparity value and the second disparity value.

[0008] According to another aspect of the present disclosure, a program causes a computer to calculate a first disparity value that is a disparity value for a first stereo image, calculate a second disparity value that is a disparity value for a second stereo image obtained by reducing the first stereo image in a horizontal direction, and determine a disparity based on the first disparity value and the second disparity value. [Effects of the Invention]

[0009] According to each aspect of the present disclosure, even when a small-scale circuit is used, distance measurement using stereo images can be performed with high accuracy. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is an image diagram illustrating an example of processing performed by a processing system according to some embodiments of the present disclosure. [Figure 2] FIG. 1 illustrates an example of a configuration of a processing system according to some embodiments of the present disclosure. [Figure 3] FIG. 10 is a diagram illustrating an example of a configuration of a block matching processing unit according to some embodiments of the present disclosure. [Figure 4] FIG. 10 is a diagram illustrating an example of the relationship between cost values ​​(difference in gray values) and disparity (positional deviation) for images A and B according to some embodiments of the present disclosure. [Figure 5] FIG. 1 is a diagram illustrating reliability according to some embodiments of the present disclosure. [Figure 6] FIG. 10 is a diagram illustrating a process performed by a reference disparity value calculation unit according to some embodiments of the present disclosure. [Figure 7]FIG. 10 is a diagram illustrating an example of a histogram created by a reference disparity value calculation unit according to some embodiments of the present disclosure. [Figure 8] 10A and 10B are diagrams for explaining processing performed by a cost comparison search unit included in a block matching processing unit according to some embodiments of the present disclosure. [Figure 9] FIG. 10 is a diagram illustrating an example of a processing flow of a parallax synthesis unit according to some embodiments of the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating an example of disparity selection by a disparity synthesis unit according to some embodiments of the present disclosure. [Figure 11] 1A to 1C are diagrams illustrating examples of images that appear in processing performed by a processing system according to some embodiments of the present disclosure. [Figure 12] FIG. 1 illustrates an example of a configuration of a processing system according to some embodiments of the present disclosure. [Figure 13] FIG. 1 is a diagram illustrating an example of a processing flow of a processing system according to some embodiments of the present disclosure. [Figure 14] FIG. 1 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the embodiments will be described in detail with reference to the drawings. <Embodiment> A processing system 1 according to an embodiment of the present disclosure will be described with reference to the drawings. The processing system 1 is a system that can accurately perform distance measurement based on the parallax between stereo images, which are two images captured using a stereo camera, even when using a small-scale circuit.

[0012] (Overview of the processing performed by the processing system) To facilitate understanding of the description of the processing system 1 that follows, an overview of the processing performed by the processing system 1 will first be described. In order to improve the accuracy of distance measurement based on the parallax in stereo images, which are two images captured using a stereo camera, it is necessary to increase the resolution of the input images. As a result, the search range in the block matching process becomes wider, and the number of pixels that must be searched increases. Furthermore, when an object is close, the parallax between the two images becomes relatively larger than when the object is far away. Therefore, the search range in the block matching process becomes wider, and the number of pixels that must be searched increases.

[0013] FIG. 1 is a conceptual diagram illustrating an example of processing performed by a processing system 1 according to some embodiments of the present disclosure. The processing system 1 receives two images captured by a stereo camera. The processing system 1 performs block matching to determine the parallax between the two images at two different sizes: the original size and a size obtained by reducing the size by N times horizontally. For example, if the search range for the block matching process for the two images at their original size is 128 pixels and N is 4, the processing system 1 can reduce the search range for the two images reduced by N times horizontally from 128 pixels to 32 pixels (i.e., the number of pixels obtained by dividing 128 pixels by N). This reduction in the search range results in the block matching processing unit 20 (described later) that performs the block matching process preparing two search ranges each reduced by N times. This reduces the size of the logic circuitry by half N compared to processing the two images at their original size without reducing them by N times horizontally. When the search range is reduced by N times, the processing system 1 searches for (128 / N) pixels in the image. However, the processing system 1 is essentially equivalent to searching a range equivalent to that of a 128-pixel search. In other words, when the image is reduced to 1 / N in the horizontal direction, even when a small-scale circuit with a search range reduced to 1 / N is used, the processing system 1 performs a search equivalent to a 128-pixel search in an image of the original size that is not reduced to 1 / N in the horizontal direction. This is an effective process when the above-mentioned object is close, the search range in the block matching process becomes wider, and the number of pixels that must be searched increases. Note that the number of pixels in the image when the search range is reduced to 1 / N (search range for block matching process in two images of the original size / N) is the total disparity.

[0014] Furthermore, when calculating the disparity, the processing system 1 also calculates a reliability value that indicates the likelihood of the disparity value. The processing system 1 performs disparity synthesis taking this reliability into consideration. Therefore, when the image is reduced to 1 / N in the horizontal direction, the processing system 1 can identify the disparity with higher accuracy even when using a small-scale circuit with a search range reduced to 1 / N. Furthermore, as will be described later, the processing system 1 performs search using the predicted disparity as a reference value. Therefore, the processing system 1 can output a highly accurate result calculated from an image of the original size with many pixels. This allows the processing system 1 to reduce the circuit size while maintaining a wide ranging range and high ranging accuracy.

[0015] (Processing system configuration) 2 is a diagram illustrating an example of a configuration of a processing system 1 according to some embodiments of the present disclosure. As shown in FIG. 2, the processing system 1 according to one embodiment of the present disclosure includes a reduction processing unit 10, a block matching processing unit 20a (an example of a second parallax calculation unit and an example of a second reliability calculation unit), a block matching processing unit 20b (an example of a first parallax calculation unit and an example of a first reliability calculation unit), an enlargement processing unit 30, a reference parallax value calculation unit 40, and a parallax synthesis unit 50 (an example of an identification unit). The block matching processing units 20a and 20b may be collectively referred to as the block matching processing unit 20.

[0016] 3 is a diagram illustrating an example of the configuration of the block matching processing unit 20 according to some embodiments of the present disclosure. As shown in FIG. 3, the block matching processing unit 20 includes a cost calculation unit 201, a cost comparison and search unit 202, a disparity identification unit 203, and a reliability calculation unit 204.

[0017] (Processing performed by the processing system) The processing performed by each processing unit included in the processing system 1 will be described.

[0018] The reduction processing unit 10 generates an image by horizontally reducing a stereo image, which is two images captured using a stereo camera. For example, if the reduction size ratio is N, the reduction processing unit 10 thins out N to (N-1) pixels from the stereo image only in the horizontal direction to output one pixel, thereby generating an image by horizontally reducing the stereo image, which is two images captured using a stereo camera. Specifically, for example, if N is 4 (i.e., when reducing the image to one-fourth in the horizontal direction), the reduction processing unit 10 groups four pixels horizontally, retains the first pixel in each group, and removes the remaining three pixels. Then, the reduction processing unit 10 horizontally compresses the remaining pixels to generate an image by horizontally reducing the stereo image, which is two images captured using a stereo camera.

[0019] The block matching processor 20 receives two images and performs block matching to calculate the disparity between the two images. The block matching processor 20 also calculates the reliability of the disparity.

[0020] 3, the cost calculation unit 201 receives image A and image B. When the block matching processing unit 20 is block matching processing unit 20a, for example, image A is reduced image 1 and image B is reduced image 2. When the block matching processing unit 20 is block matching processing unit 20b, for example, image A is image 1 and image B is image 2.

[0021] The cost calculation unit 201 evaluates the feature amounts of the received images A and B on a pixel-by-pixel basis as cost values. For example, the cost calculation unit 201 calculates a gray value using a grayscale as a cost value for each pixel of images A and B. The gray value is a numerical value that represents the gradation in the grayscale. For example, if the gradation in the grayscale is 0 out of 0 to 255, the gray value is 8-bit 00000000. If the gradation in the grayscale is 255 out of 0 to 255, the gray value is 8-bit 11111111.

[0022] The cost comparison search unit 202 calculates a cost value for visual discrimination based on the cost value calculated by the cost calculation unit 201. Note that the cost value calculated by the cost calculation unit 201 is a gray value, and the cost value calculated by the cost comparison search unit 202 is the difference between the cost values ​​of the two images calculated by the cost calculation unit 201 (i.e., the difference in gray value).

[0023] For example, the cost comparison search unit 202 calculates, as a cost value, the difference between the gray value of a pixel to be evaluated for cost in image A and the gray value of a pixel at a position in image B that matches the position of that pixel. The cost comparison search unit 202 also calculates a cost value for the difference between the gray value of a pixel to be evaluated for cost in image A and the gray values ​​of each pixel in a range of ±(Total Disparity / 2) pixels at the position of that pixel in image B that matches the position of that pixel. Then, from these calculation results, the cost comparison search unit 202 identifies the relationship between the cost value (difference in gray values) and disparity (positional deviation) for images A and B.

[0024] FIG. 4 is a diagram showing an example of the relationship between cost values ​​(difference in gray values) and disparity (positional deviation) for images A and B according to some embodiments of the present disclosure. The vertical axis in FIG. 4 represents cost values ​​(difference in gray values). The horizontal axis in FIG. 4 represents disparity (positional deviation). Disparity is represented by the difference in the number of pixels. Cmin in FIG. 4 represents the minimum cost value. Total Disparity in FIG. 4 represents disparity resulting from a pixel count deviation of (search range for block matching processing in two images of the original size / N). Match Disparity represents the disparity resulting in the minimum cost value.

[0025] The disparity identification unit 203 identifies a Match Disparity based on the calculation result by the cost comparison search unit 202. The disparity with the smallest cost value means the disparity where the feature amounts most closely match among those calculated by the cost comparison search unit 202. Therefore, this Match Disparity identified by the disparity identification unit 203 becomes the disparity that is ultimately sought. For example, the disparity identification unit 203 identifies the smallest cost value calculated by the cost comparison search unit 202. Then, the disparity identification unit 203 identifies the Match Disparity, which is the disparity corresponding to the identified smallest cost value, as the disparity that is ultimately sought.

[0026] The reliability calculation unit 204 calculates the reliability based on the relationship between the cost value (difference in gray values) and the disparity (positional deviation) for image A and image B identified by the cost comparison search unit 202. For example, the reliability calculation unit 204 performs a product-sum calculation of the differences between each cost value and Cmin in Total Disparity, and divides the product-sum result by Total Disparity to obtain the reliability. FIG. 5 is a diagram for explaining reliability according to some embodiments of the present disclosure. The reliability calculation unit 204 calculates the reliability by finding the sum of the parts indicated by the arrows in FIG. 5 and dividing the found sum by Total Disparity. This reliability calculation can be expressed as in Equation (1).

[0027]

number

[0028] In addition, in the formula (1), D represents the disparity, and C(D) represents the cost value for each disparity.

[0029] In the block matching processing unit 20a, the parallax identified by the parallax identification unit 203 is the reduced parallax shown in Fig. 2. The parallax identification unit 203 identifies the reduced parallax for each of all pixels. Through the processing by the parallax identification unit 203, a parallax image reduced by a size ratio N is generated.

[0030] In the block matching processing unit 20a, the reliability calculated by the reliability calculation unit 204 is the reduced reliability shown in Fig. 2. The reliability calculation unit 204 calculates the reduced reliability for each of all pixels. Through the processing by this reliability calculation unit 204, a reliability image reduced by a size ratio N is generated.

[0031] The enlargement processing unit 30 restores the disparity image generated by the disparity identification unit 203 and the reliability image generated by the reliability calculation unit 204, which have been reduced by a size ratio N, to their original sizes. For example, the enlargement processing unit 30 makes (N-1) consecutive copies of each pixel in the horizontal direction so that N identical pixels are consecutively arranged in the horizontal direction in the disparity image. Then, the enlargement processing unit 30 multiplies the disparity values ​​of all pixels by N. In this way, the enlargement processing unit 30 generates a disparity image A.

[0032] Furthermore, for example, the enlargement processing unit 30 copies each pixel consecutively in the horizontal direction (N-1) times so that N identical pixels are consecutively arranged in the horizontal direction in the reliability image. Note that the reliability value remains the same. In this way, the enlargement processing unit 30 generates the reliability image A.

[0033] The reference disparity value calculation unit 40 calculates a reference disparity value for the current frame based on the disparity image A generated by the enlargement processing unit 30. FIG. 6 is a diagram illustrating processing performed by the reference disparity value calculation unit 40 according to some embodiments of the present disclosure. Part (a) of FIG. 6 illustrates an example of the disparity image A generated by the enlargement processing unit 30. Part (b) of FIG. 6 illustrates an example of a reference disparity value for the current frame. The reference disparity value calculation unit 40 first divides the entire disparity image A into cells of a specific size. The reference disparity value calculation unit 40 divides the cells into the same size as the cells divided in the processing performed by the cost comparison search unit 202 of the block matching processing unit 20 (described later). The reference disparity value calculation unit 40 calculates one reference disparity value for each cell. For example, one reference disparity value is set for each cell. This reference disparity value is calculated in advance from a calculated disparity image obtained from the previous frame. Specifically, the reference parallax value calculation unit 40 calculates the average parallax of all pixels included in the divided cells in the previous frame. The initial value of the reference parallax value is 0. That is, since there is no previous frame for the first frame, the reference parallax value calculation unit 40 sets the reference parallax value to the initial value of 0 for all cells. For the second frame and thereafter, the reference parallax value calculation unit 40 uses the reference parallax value calculated for the previous frame. As a result, for the first frame, the parallax synthesis unit 50 (described later) uses the initial value of 0 for all cells as the reference parallax value to be used for parallax synthesis. For the second frame, the parallax synthesis unit 50 (described later) uses the reference parallax value calculated for the previous frame, the first frame, as the reference parallax value to be used for parallax synthesis. For the third frame, the parallax synthesis unit 50 (described later) uses the reference parallax value calculated for the previous frame, the second frame, as the reference parallax value to be used for parallax synthesis. For the fourth frame and thereafter, the parallax synthesis unit 50 (described later) uses the reference parallax value calculated in the previous frame as the reference parallax value used for parallax synthesis.

[0034] The reference disparity value calculation unit 40 creates a histogram at a sampling interval S using all pixels in a cell as samples. FIG. 7 is a diagram showing an example of a histogram created by the reference disparity value calculation unit 40 according to some embodiments of the present disclosure. In FIG. 7, the horizontal axis represents the disparity value, and the vertical axis represents the number of pixels. The reference disparity value calculation unit 40 identifies the disparity value that is the most frequent value in the histogram as the final reference disparity value.

[0035] Similar to the block matching processing unit 20a described above, the block matching processing unit 20b receives the image 1 and the image 2, and generates a parallax image B and a reliability image B. However, unlike the cost comparison and search unit 202 provided in the block matching processing unit 20a, the cost comparison and search unit 202 provided in the block matching processing unit 20b executes processing using the reference parallax value identified by the reference parallax value calculation unit 40.

[0036] 8 is a diagram illustrating processing performed by the cost comparison search unit 202 included in the block matching processing unit 20b according to some embodiments of the present disclosure. Part (a) of FIG. 8 is a diagram illustrating example cost images (cost image A, cost image B) generated by the cost calculation unit 201 included in the block matching processing unit 20b. Part (b) of FIG. 8 is a diagram illustrating example reference disparity values ​​of a previous frame. For example, the cost comparison search unit 202 included in the block matching processing unit 20b performs processing similar to that of the cost comparison search unit 202 included in the block matching processing unit 20a, searching within a range from (the reference disparity value identified by the reference disparity value calculation unit 40−(Total Disparity / 2)) to (the reference disparity value identified by the reference disparity value calculation unit 40+(Total Disparity / 2)−1). However, when the reference disparity value specified by the reference disparity value calculation unit 40 is equal to or less than Total Disparity, the cost comparison search unit 202 included in the block matching processing unit 20b sets the search range from 0 to Total Disparity-1.

[0037] The disparity synthesis unit 50 selects and outputs a likely disparity value based on the disparity image A and the reliability image A generated by the enlargement processing unit 30, the reference disparity value generated by the reference disparity value calculation unit 40, and the disparity image B and the reliability image B generated by the block matching processing unit 20b. Fig. 9 is a diagram illustrating an example of a processing flow of the disparity synthesis unit 50 according to some embodiments of the present disclosure. Here, a process of selecting and outputting a likely disparity value by the disparity synthesis unit 50 will be described.

[0038] The parallax synthesis unit 50 receives a parallax image A, a reliability image A, a reference parallax value, a parallax image B, and a reliability image B (step S1). The parallax synthesis unit 50 determines whether the reliability value indicated by the reliability image A is equal to or greater than a constant A (step S2).

[0039] When the disparity synthesis unit 50 determines that the reliability value indicated by the reliability image A is equal to or greater than the constant A (YES in step S2), the disparity synthesis unit 50 determines whether the reliability value indicated by the reliability image A satisfies a predetermined condition (step S3). The predetermined condition here is that the reliability value indicated by the reliability image A is less than (the reference disparity value identified by the reference disparity value calculation unit 40−(Total Disparity / 2)) and greater than (the reference disparity value identified by the reference disparity value calculation unit 40+(Total Disparity / 2)−1).

[0040] If the parallax synthesis unit 50 determines that the reliability value indicated by the reliability image A satisfies a predetermined condition (YES in step S3), it selects the parallax image A (step S4).

[0041] Furthermore, if the parallax synthesis unit 50 determines that the reliability value indicated by the reliability image A is less than the constant A (NO in step S2), it determines whether the reliability value indicated by the reliability image B is equal to or greater than the constant B (step S5).

[0042] Furthermore, if the parallax synthesis unit 50 determines that the reliability value indicated by the reliability image A does not satisfy a predetermined condition (NO in step S3), the process proceeds to step S5.

[0043] If the parallax synthesis unit 50 determines that the reliability value indicated by the reliability image B is equal to or greater than the constant B (YES in step S5), it selects the parallax image B (step S6). If the parallax synthesis unit 50 determines that the reliability value indicated by the reliability image B is less than the constant B (NO in step S5), it sets the parallax to 0 (step S7).

[0044] The parallax synthesis unit 50 performs the processes of steps S2 to S7 on pixels corresponding to the positions of each pixel of the image received in the process of step S1. As a result, if the parallax is within the predicted parallax range, the parallax result of the image at full size using image 1 and image 2 with high accuracy is selected. Otherwise, the parallax result of the image obtained by reducing image 1 and image 2 to its original size is selected. FIG. 10 is a diagram illustrating an example of parallax selection by the parallax synthesis unit 50 according to some embodiments of the present disclosure. In the example shown in FIG. 10, the total disparity is set to 32. In the case of search range A shown in FIG. 10, which is M±16, parallax image A is selected. In the case of search ranges B1 and B2 shown in FIG. 10, which are outside M±16, parallax image B is selected. Note that M is a reference parallax value.

[0045] Fig. 11 is a diagram showing an example of each image appearing in the processing performed by the processing system 1 according to some embodiments of the present disclosure. Part (a) of Fig. 11 shows an example of image 1 and image 2. Part (b) of Fig. 11 shows an example of an image received by the parallax synthesis unit 50 and a parallax image of a compound word finally generated by selection by the parallax synthesis unit 50.

[0046] (advantage) The processing system 1 according to an embodiment of the present disclosure has been described above. In the processing system 1, the block matching processor 20b (an example of a first parallax calculator) calculates a parallax image B (an example of a first parallax value) that is a parallax value for image 1 and image 2 (an example of a first stereo image). The block matching processor 20a (an example of a second parallax calculator) calculates a reduced parallax (an example of a second parallax value) that is a parallax value for reduced image 1 and reduced image 2 (an example of a second stereo image) that are obtained by reducing image 1 and image 2 in the horizontal direction. The parallax synthesis unit 50 (an example of an identifier) ​​identifies the parallax based on the parallax image B and the reduced parallax.

[0047] This processing system 1 makes it possible to narrow the parallax search range at close ranges. As a result, it is possible to reduce the scale of a circuit that processes parallax in a small-scale circuit (for example, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)). Note that the narrowed parallax search range is achieved by compression (for example, by thinning out pixels), so that the range includes all of the pixels in the original parallax search range, although the number is smaller. Therefore, the effective search range of the narrowed parallax search range is the same as the original parallax search range. Therefore, the processing system 1 makes it possible to accurately perform distance measurement using stereo images even when using a small-scale circuit.

[0048] Note that the processing system 1 according to another modified example of the embodiment of the present disclosure is not limited to the above-described processing related to the cost value of the block matching processing unit 20. In the processing system 1 according to another modified example of the embodiment of the present disclosure, for example, the block matching processing unit 20 may evaluate a cost value for a feature amount on a pixel-by-pixel basis, compare and search the cost values ​​between two types of images, and perform processing to determine the disparity that minimizes the cost value.

[0049] Furthermore, in the processing system 1 according to another modified example of the embodiment of the present disclosure, the reference disparity value calculation unit 40 is not limited to calculating the reference disparity value from a calculated disparity image obtained from the previous frame. In the processing system 1 according to another modified example of the embodiment of the present disclosure, for example, the reference disparity value calculation unit 40 may calculate the reference disparity value from a disparity image obtained from the current frame.

[0050] For example, the enlargement processing unit 30 performs image processing to restore the size of a reduced image several lines before the cost calculation unit 201 of the block matching processing unit 20b performs processing to generate cost images A and B. This enables the reference parallax value calculation unit 40 to apply the results of the current frame. Note that the number of lines that the cost calculation unit 201 of the block matching processing unit 20b needs to execute in advance is the number of pixels in the vertical direction of the cell. If a line buffer capable of physically storing information for this number of lines is provided, the reference parallax value calculation unit 40 can apply the results of the current frame.

[0051] Furthermore, in the processing system 1 according to another modification of the embodiment of the present disclosure, the reference parallax value calculation unit 40 may apply a fixed reference parallax value. In the embodiment of the present disclosure, the reference parallax value calculation unit 40 calculates the reference parallax value for each cell. However, in the processing system 1 according to another modification of the embodiment of the present disclosure, if the range to focus on for distance measurement is determined in advance, the reference parallax value calculation unit 40 can set the reference parallax value to a fixed value. In this case, the reference parallax value calculation unit 40 does not need to divide the entire image into cells or calculate the reference parallax value from the previous frame, thereby enabling a smaller circuit size.

[0052] Next, a processing system 1 according to some embodiments of the present disclosure will be described. Fig. 12 is a diagram illustrating an example of the configuration of the processing system 1 according to some embodiments of the present disclosure. As shown in Fig. 12, the processing system 1 includes a first parallax calculation unit 301, a second parallax calculation unit 302, and an identification unit 303.

[0053] The first parallax calculation unit 301 calculates a first parallax value for a first stereo image. The second parallax calculation unit 302 calculates a second parallax value for a second stereo image obtained by reducing the first stereo image in the horizontal direction. The determination unit 303 determines the parallax based on the first parallax value and the second parallax value.

[0054] The first parallax calculation unit 301 can be realized, for example, by using the function of the block matching processing unit 20b illustrated in Fig. 2. The second parallax calculation unit 302 can be realized, for example, by using the function of the block matching processing unit 20a illustrated in Fig. 2. The identification unit 303 can be realized, for example, by using the function of the parallax synthesis unit 50 illustrated in Fig. 2.

[0055] Next, processing performed by the processing system 1 according to some embodiments of the present disclosure will be described. Fig. 13 is a diagram showing an example of a processing flow of the processing system 1 according to some embodiments of the present disclosure. Here, the processing of the processing system 1 will be described with reference to Fig. 13.

[0056] The first parallax calculation unit 301 calculates a first parallax value for a first stereo image (step S101). The second parallax calculation unit 302 calculates a second parallax value for a second stereo image obtained by reducing the first stereo image in the horizontal direction (step S102). The determination unit 303 determines the parallax based on the first parallax value and the second parallax value (step S103).

[0057] The processing system 1 according to some embodiments of the present disclosure has been described above. With this processing system 1, even when a small-scale circuit is used, it is possible to perform accurate distance measurement using stereo images.

[0058] The order of the processes in each embodiment of the present disclosure may be changed as long as the processes are performed appropriately.

[0059] Each embodiment of the present disclosure has been described. However, the processing system 1, the reduction processing unit 10, the block matching processing unit 20, the enlargement processing unit 30, the reference disparity value calculation unit 40, the disparity synthesis unit 50, and other control devices may have a computer system therein. The above-described processing steps are stored in the form of a program on a computer-readable recording medium, and the above processing is performed by reading and executing the program by a computer. Specific examples of computers are shown below.

[0060] 14 is a schematic block diagram showing the configuration of a computer according to at least one embodiment. As shown in FIG. 14, the computer 5 includes a CPU (Central Processing Unit) 6, a main memory 7, a storage 8, and an interface 9.

[0061] For example, the above-described processing system 1, reduction processing unit 10, block matching processing unit 20, enlargement processing unit 30, reference disparity value calculation unit 40, disparity synthesis unit 50, and other control devices are each implemented in a computer 5. The operations of the above-described processing units are stored in the storage 8 in the form of a program. The CPU 6 reads the program from the storage 8, loads it into the main memory 7, and executes the above-described processing in accordance with the program. The CPU 6 also allocates storage areas in the main memory 7 corresponding to the above-described storage units in accordance with the program.

[0062] Examples of storage 8 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), and a semiconductor memory. Storage 8 may be an internal medium directly connected to the bus of computer 5, or an external medium connected to computer 5 via interface 9 or a communication line. In addition, when this program is distributed to computer 5 via a communication line, computer 5 that receives the program may load the program into main memory 7 and execute the above-mentioned processing. In at least one embodiment, storage 8 is a non-transitory tangible storage medium.

[0063] The program may also implement some of the functions described above. Furthermore, the program may be a file that can implement the functions described above in combination with a program already recorded in the computer system, a so-called differential file (differential program).

[0064] Although several embodiments of the present disclosure have been described, these embodiments are merely examples and do not limit the scope of the disclosure. Various additions, omissions, substitutions, and modifications may be made to these embodiments without departing from the spirit of the disclosure.

[0065] Note that part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0066] (Appendix 1) a first parallax calculation unit that calculates a first parallax value that is a parallax value for the first stereo image; a second parallax calculation unit that calculates a second parallax value for a second stereo image obtained by reducing the first stereo image in a horizontal direction; an identification unit that identifies a parallax based on the first parallax value and the second parallax value; A processing system comprising:

[0067] (Appendix 2) The identification unit identifying the disparity based on a first reliability that is a reliability for the first disparity value and a second reliability that is a reliability for the second disparity value; 10. The processing system of claim 1.

[0068] (Appendix 3) a first reliability calculation unit that calculates a first reliability that is a reliability for the first disparity value; a second reliability calculation unit that calculates a second reliability that is a reliability of the second disparity value; 3. The processing system of claim 2, comprising:

[0069] (Appendix 4) a reference disparity value calculation unit that calculates a reference disparity value in a previous frame; Equipped with The first parallax calculation unit calculating the first disparity value based on the reference disparity value; 4. The processing system of any one of claims 1 to 3.

[0070] (Appendix 5) calculating a first disparity value that is a disparity value for the first stereo image; calculating a second parallax value that is a parallax value for a second stereo image obtained by reducing the first stereo image in a horizontal direction; determining a disparity based on the first disparity value and the second disparity value; A processing method comprising:

[0071] (Appendix 6) identifying the disparity based on a first reliability that is a reliability for the first disparity value and a second reliability that is a reliability for the second disparity value; Attachment 5, a processing method comprising:

[0072] (Appendix 7) Calculating a reference disparity value in a previous frame; calculating the first disparity value based on the reference disparity value; 6. The method of claim 5, further comprising:

[0073] (Appendix 8) On the computer, calculating a first disparity value that is a disparity value for the first stereo image; calculating a second parallax value that is a parallax value for a second stereo image obtained by reducing the first stereo image in a horizontal direction; determining a disparity based on the first disparity value and the second disparity value; A program that executes the following.

[0074] (Appendix 9) identifying the disparity based on a first reliability that is a reliability for the first disparity value and a second reliability that is a reliability for the second disparity value; 9. The program according to claim 8, which causes the program to execute the above steps.

[0075] (Appendix 10) Calculating a reference disparity value in a previous frame; calculating the first disparity value based on the reference disparity value; 10. The program according to claim 8 or 9, which causes the program to execute the above. [Explanation of symbols]

[0076] 1. Processing System 5. Computer 6 CPU 7. Main memory 8. Storage 9. Interface 10. Reduction processing section 20, 20a, 20b... Block matching processing section 30 Enlargement processing unit 40. Reference parallax value calculation unit 50...Parallax synthesis section 201 Cost Calculation Department 202 Cost comparison search section 203...Parallax identification part 204... Reliability calculation unit 301...first parallax calculation unit 302...Second parallax calculation unit 303...Specific section

Claims

1. a first parallax calculation unit that calculates a first parallax value that is a parallax value for the first stereo image; a second parallax calculation unit that calculates a second parallax value for a second stereo image obtained by reducing the first stereo image in a horizontal direction; an identification unit that identifies a parallax based on the first parallax value and the second parallax value; A processing system comprising:

2. The identification unit identifying the disparity based on a first reliability that is a reliability for the first disparity value and a second reliability that is a reliability for the second disparity value; The processing system of claim 1 .

3. a first reliability calculation unit that calculates a first reliability that is a reliability of the first disparity value; a second reliability calculation unit that calculates a second reliability that is a reliability of the second disparity value; The processing system of claim 2 , comprising:

4. a reference disparity value calculation unit that calculates a reference disparity value in a previous frame; Equipped with The first parallax calculation unit calculating the first parallax value based on the reference parallax value; The processing system according to any one of claims 1 to 3.

5. calculating a first disparity value that is a disparity value for the first stereo image; calculating a second parallax value for a second stereo image obtained by reducing the first stereo image in a horizontal direction; determining a disparity based on the first disparity value and the second disparity value; A processing method comprising:

6. On the computer, calculating a first disparity value that is a disparity value for the first stereo image; calculating a second parallax value for a second stereo image obtained by reducing the first stereo image in a horizontal direction; determining a disparity based on the first disparity value and the second disparity value; A program that executes the following.

Images