Robust stereo camera image processing method and system

Abstract

The invention relates to a method and a system for extracting disparity information from a stereo camera image pair, which can be used for processing camera sensor images for autonomous driving or advanced driver assistance systems in vehicles, for example. The method comprises the steps of a) receiving an uncorrected stereo camera image pair, b) performing a two-dimensional matching of the received image pair using a trained convolutional neural network, providing a two-dimensional displacement per pixel as disparity information, and c) outputting the two-dimensional displacement per pixel. An advantage of the invention is that it is robust and does not require high-precision correction. Compared to the prior art, the algorithm complexity can be reduced while preserving all information by correcting the disparity values after the matching process. The invention also relates to a vehicle comprising the above-mentioned system for extracting disparity information and a stereo camera system.

Description

Technical Field

[0001] The present invention relates to a method and system for extracting parallax information from image pairs of stereo camera devices, which can be used, for example, to process camera sensor images of autonomous driving (AD) or advanced driver assistance systems (ADAS) in vehicles. Background Technology

[0002] Automotive applications utilize stereo imaging devices; for example, EP 3176750 A1 illustrates a stereo imaging system and processing system for motor vehicles. The system is designed to perform stereo image correction and stereo matching of the corrected images, and to detect objects in the environment surrounding the motor vehicle from the matched stereo images. A data processing unit is designed to search for image elements in a corrected image from one imaging device to find the best-matching image element in a corresponding corrected image from another imaging device, wherein the search covers a two-dimensional search area. The search generates vertical displacement information, from which the vertical displacement from the image element to the best-matching image element is derived. The data processing unit is designed to calculate the pitch angle error and / or roll angle error of the imaging device or between imaging devices from the vertical displacement information.

[0003] Typical stereo camera systems require image correction before matching. This is usually for practical reasons, to reduce the complexity of matching algorithms (e.g., semi-global matching (SGM)). If the correction is performed on the horizontal line of the camera setup, then parallax matching can be reduced to a one-dimensional problem.

[0004] D. Klücker (D. ( ) et al. published a paper in the proceedings of the 2018 SIA VISION conference:

[0005] The paper, titled "Camera Based Lost Cargo Detection for Automated Driving," demonstrates a camera-based method that utilizes two different satellite stereo camera systems to extract small obstacles on the road through direct three-dimensional (3D) measurements. One of the satellite stereo camera modules is a telephoto camera.

[0006] DE 102020214622 A1 relates to a method and apparatus for evaluating image data from a stereoscopic camera device. The method for evaluating image data from a stereoscopic camera device includes the following steps:

[0007] a) Receive image data at raw resolution captured by the stereo camera device.

[0008] b) Receive or compute image data with reduced resolution.

[0009] c) Detecting objects from the two-dimensional (2D) image data of the camera module of the stereoscopic camera device.

[0010] d) Perform stereo evaluation at the original resolution in the search area of ​​the image data containing the detected object, and at a reduced resolution in the remaining areas of the image data to determine the three-dimensional (3D) information of the relevant object and its surrounding environment, and

[0011] e) Output relevant three-dimensional (3D) information.

[0012] Correcting high-resolution cameras with lens distortion involves a significant computational workload and increases the size of the resulting "linearized" image. The correction process requires interpolation between pixels, which blurs pixel information. Furthermore, some areas see increased resolution without adding more information, while others experience decreased resolution and lost information. Summary of the Invention

[0013] The purpose of this invention is to provide a robust stereoscopic image processing method.

[0014] On the one hand, it involves the automotive functions required for advanced driver assistance or autonomous driving.

[0015] Autonomous vehicles need to detect obstacles in front of them. The advantage of stereo cameras is that they can measure the depth and distance of objects.

[0016] A typical stereoscopic camera setup consists of two cameras observing in the same direction. By matching the left and right images, the parallax related to the object's depth is calculated. An alternative stereoscopic camera setup consists of a moving monocular camera, typically found in monocular cameras in driving vehicles. Structural information about the surrounding environment (“motion reconstruction structure”) can be extracted from image pairs (or image sequences) captured at consecutive time points.

[0017] The proposed method can be used for 2D / 3D object detection, 2D / 3D object tracking, 3D scene reconstruction, 3D mapping, 3D surrounding environment modeling (e.g., by fusing images from camera devices with radar / laser sensors), lost item detection, object detection, 3D road surface modeling and / or augmented reality (AR) based visualization.

[0018] The method for extracting parallax information from image pairs of a stereo camera device includes the following steps:

[0019] a) Receive uncorrected stereoscopic camera image pairs,

[0020] b) Using a trained convolutional neural network to perform two-dimensional (2D) matching on the received image pairs, providing pixel-by-pixel two-dimensional (2D) displacement as disparity information, and

[0021] c) Output pixel-level two-dimensional (2D) displacement.

[0022] The convolutional neural network may have been trained to generate a two-dimensional (2D) displacement map of each pixel from uncorrected stereo camera image pairs using supervised learning methods.

[0023] According to one embodiment, two-dimensional (2D) displacement is described as a disparity vector.

[0024] In one example, two-dimensional (2D) displacement information is used as input for subsequent perception tasks such as object detection or similar tasks.

[0025] According to one embodiment, the direction of the two-dimensional (2D) displacement is accumulated over time, providing a filtered angular direction for the image pixel location. The filtered angular direction corresponds to the epipolar direction of that particular image pixel location.

[0026] In one example, the fundamental matrix parameters of the stereo camera system are calculated from uncorrected pixel-by-pixel parallax.

[0027] According to one embodiment, two-dimensional (2D) displacement is used for parallax correction of the external parameters of the camera device. The internal parameters of the camera device can be obtained from the initial camera device calibration process.

[0028] In one example, the three-dimensional (3D) real-world coordinates in meters are calculated based on two-dimensional (2D) displacement. The camera's internal parameters can be known or estimated for use in calculating the true distance.

[0029] According to one embodiment, two-dimensional (2D) displacement is used to compute dense three-dimensional (3D) point clouds.

[0030] A system for extracting parallax information from an image pair of a stereo camera device includes an input unit, a processing unit, and an output unit.

[0031] a) The input unit is configured to receive uncorrected stereo camera image pairs.

[0032] b) The processing unit is configured to perform two-dimensional (2D) matching on the received image pairs using a trained convolutional neural network, providing pixel-by-pixel two-dimensional (2D) displacement as disparity information.

[0033] c) The output unit is configured to output two-dimensional (2D) displacement on the pixel layer.

[0034] On the other hand, the invention relates to a vehicle comprising a system for extracting parallax information and a stereo camera system. The stereo camera system is configured to generate uncorrected stereo camera image pairs and output them to the system for extracting parallax information.

[0035] The proposed system is robust and requires no high-precision correction. Compared to existing techniques, correcting the disparity values ​​after the matching process reduces algorithm complexity while preserving all information.

[0036] In other words, the proposed solution uses a stereo camera system that provides images (intensity, RGB primary colors, Bayer original (un-mosaicized)...) and disparity maps describing the displacement between two corresponding pixels in the left and right images. The proposed system requires no correction before calculating disparity. Disparity calculation typically involves pixel-by-pixel matching of the left and right images. Two-dimensional (2D) displacement is calculated using the proposed matching process. The resulting displacement is described pixel-by-pixel as a two-dimensional (2D) vector (displacement in the x and y dimensions). We apply a trainable neural network for matching, trained with shifted and rotated images. Other two-dimensional (2D) matching algorithms (e.g., block matching) can also be used. To extract true depth information, the camera's internal parameters must be considered. This necessitates post-matching correction of the disparity map. Therefore, the disparity of the two-dimensional (2D) images is used to calculate three-dimensional (3D) point measurements in the world / vehicle coordinate system.

[0037] - Accumulated corrected parallax measurements over time

[0038] After applying disparity matching to the original (distorted and uncorrected) image, the direction of the two-dimensional (2D) disparity vectors accumulates over time, resulting in a filtered angular direction that corresponds to the epipolar direction of the pixel location in that particular image. Attached Figure Description

[0039] Other embodiments and aspects of the invention may be understood from the following description and accompanying drawings.

[0040] In the attached diagram:

[0041] Figure 1 This diagram schematically illustrates the stereoscopic camera device used to detect the object.

[0042] Figure 2 This image shows two pictures taken simultaneously by a stereoscopic camera device.

[0043] Figure 3This figure is an overview of a machine learning implementation method for extracting two-dimensional (2D) displacement information from images from a stereo camera device.

[0044] Figure 4 The figure schematically illustrates a system 40 for extracting two-dimensional (2D) displacement information from a stereo camera device.

[0045] Figure 5 The left two columns of the image show the corresponding pixel region pairs acquired simultaneously from the left and right cameras, while the right column is a schematic diagram of the two-dimensional (2D) displacement of each pixel.

[0046] Figure 6 This image is an example of an uncorrected camera image showing a city road scene, with multiple pixels of two-dimensional (2D) parallax vectors overlapping.

[0047] Figure 7 The figure illustrates a comparison between a known stereo image processing method (a) in the prior art and an embodiment (b) of this processing method. Detailed Implementation

[0048] Figure 1 The illustration shows the objects 3 and 4 being detected from above by two camera modules 11 and 12 of the stereoscopic camera device.

[0049] The left stereo camera module 11 and the right stereo camera module 12 are separated by a reference width b. The optical axis of the left stereo camera module 11 is schematically represented by a straight line a1, and the optical axis of the right stereo camera module 12 is represented by a straight line a2. The detection or viewing angle range of the left stereo camera module 11 is represented by a dashed line s1, and the detection or viewing angle range of the right stereo camera module 12 is represented by a dashed line s2. The distance from the circular object 3 to the stereo camera devices 11 and 12 (perpendicular to the line representing the reference width b) is z.

[0050] Figure 2 Schematic illustration in the described situation (refer to) Figure 1 Two images captured by stereo camera modules 11 and 12.

[0051] Because the two camera modules are positioned differently in the horizontal direction, the positions of the two objects 3 and 4 are also different in the two images 21 and 22.

[0052] In image 21 of the left-hand stereo camera module 11 shown on the left, both objects 3 and 4 appear to be further to the right. The horizontal image distance dL of the circular object 3 in the left-hand image 21, measured from the left edge of the image.

[0053] In the right image 22 of the right stereo camera module, both objects 3 and 4 appear to be further to the left than in the left image 21. In the right image 22, the horizontal image distance of the circular object 3 from the left edge of the image is dR.

[0054] The displacement of circular object 3 between the left and right images 21 and 22 is the parallax d, which can be calculated in the following way:

[0055] d = |dL - dR|.

[0056] from Figure 1 and Figure 2 It is evident that the parallax d depends on the distance z and the reference width b.

[0057] For example, D. Klücker (D. As shown in Section 5.1 (further consideration: Section 5.3.1), the distance z of the object 3 to be determined from the stereoscopic imaging devices 11, 12 depends on the focal length f of the optics and the reference width b and parallax d of the stereoscopic imaging devices 11, 12.

[0058] The minimum range of distances that a stereo system can determine is determined by the maximum parallax range. The parallax d is inversely proportional to the distance z:

[0059] z = f * b / d.

[0060] Figure 3 This diagram schematically illustrates an overview of a machine learning implementation for extracting two-dimensional (2D) displacement information from images of a stereoscopic camera device. A key component is a machine learning system 30, such as a convolutional neural network (CNN), which learns during the training phase to generate a disparity vector map 33 from the stereoscopic camera device image pairs 21, 22. The disparity vector map, or disparity vector image 33, displays the disparity vector at the pixel-by-pixel level.

[0061] To train a machine learning system 30, such as a decision tree learning system, support vector machine, regression analysis-based learning system, Bayesian network, neural network, or convolutional neural network, training input data X (multiple image pairs 21, 22) and associated training target maps Y (multiple corresponding disparity vector maps 33) are provided. Using the machine learning system 30, an output map Y' (not shown) is generated from the training input data X. The goal of training is to ensure that the output map Y' is as similar as possible to the training target map Y, without overfitting. To this end, the residual deviation between the output map Y' and the training target map Y is determined, for example, using a similarity measure between the generated output map Y' and the training target map Y. These deviations are used, for example, through backpropagation to adjust the parameters of the machine learning system 30. This process is repeated until a predetermined match is achieved or signs of overfitting appear.

[0062] Then, the machine learning system 30, trained in this way, generates the output graph Y' from the newly input data X.

[0063] Figure 4 A system 40 is schematically shown that extracts two-dimensional (2D) displacement information from stereo camera devices 11 and 12. The system has an input interface 42, a data processing unit 44 with a machine learning system 30, and an output interface 46 that outputs the fused data to another unit (control unit (CU)). Two camera devices 11 and 12 can be installed on a vehicle, for example, two satellite cameras with identical optical characteristics.

[0064] Based on the image data from cameras 11 and 12, multiple Advanced Driver Assistance Systems (ADAS) functions / Autonomous Driving (AD) functions can be provided to other units (control units (CUs) by an ADAS or AD control unit for purposes such as lane detection, lane keeping assist, traffic sign detection, speed limit assist, traffic participant detection, collision warning, emergency braking assist, distance tracking control, construction site assist, highway navigation, cruise driver functions, and / or autonomous driving functions.

[0065] The machine learning system 30 may include artificial neural networks such as convolutional neural networks (CNNs).

[0066] Figure 5This illustrates three pixel region pairs 51a, 52a; 51b, 52b; 51c, 52c simultaneously acquired from the left and right camera devices 11 and 12 for each row a, b, c in columns 51 and 52. The machine learning system 30 has been trained to match 2x2 pixel blocks (511+512 up row and 513+523, 514+524 down row) or 3x2 pixel blocks 512+522 with the corresponding left and right camera device pixel region pairs (51a+52a; 51c+52c; 51b+52b). Since the camera device images are not corrected, a vertical offset component may occur. Right column 53 shows a schematic diagram of the two-dimensional (2D) displacement of each pixel (between corresponding pixel regions 51a, 52a → disparity vector grid / Figure 53a, etc.). The disparity vector 531 corresponds to the upper right pixel displacement of the matched 2x2 block 511+512 in the first row 51a, 52a. Similarly, disparity vector 534 corresponds to the lower left pixel of the right 2x2 block 514+524 in the third row 51c, 52c. This indicates that 53a; 53b; 53c are composed of the disparity vectors of each pixel. The set of disparity vectors in a row corresponds to a two-dimensional (2D) disparity field with disparity vectors that show the association between each individual left and right image pixel (displayed as small vectors) and the "redundant" output (displayed as small crosses) for any single pixel location without an explicit pixel-by-pixel correspondence, because it is not simultaneously visible in the (left / right) images of the two cameras.

[0067] Figure 6 An example is shown: an uncorrected image 61 from the left camera device 11, depicting a city road scene with overlapping multiple pixel-level two-dimensional (2D) parallax vectors 63. Distortion / distortion is clearly visible in off-center areas of the uncorrected image; for example, buildings on the left and right sides of the road are imaged as curved vertical lines. The orientation of the 2D parallax vectors 63 accumulates over time into a series of stereoscopic images, represented by (slightly) curved (almost horizontal) lines 64 that also overlap image 61. This filtered angular orientation represents the epipolar direction corresponding to the pixel location of that particular image.

[0068] Figure 7 A comparison is shown between a known stereo image processing method (a) in the prior art and an embodiment (b) of this processing method.

[0069] a. The known method requires, in the first step, to correct the original images 21 and 22 acquired from the stereo camera device, S11. In the second step, stereo matching is performed based on the corrected images, S12. Stereo matching S12 can be performed, for example, by semi-global matching or by a conventional method using a trained artificial neural network, such as a convolutional neural network (CNN), which has been trained to output disparity information when the two corrected images are input into the neural network. In a subsequent step, disparity information is calculated from the matching, S13. In the next step, a dense three-dimensional (3D) point cloud 70 can be generated from or using the disparity information.

[0070] b. The implementation of the novel processing method begins by feeding the uncorrected stereo images 21, 22 into a machine learning system, such as a convolutional neural network (CNN), which has been trained to match small pixel blocks (e.g., 1x1, 2x2, 3x2, 3x3...) between the left and right uncorrected images in two dimensions. The first step can be called CNN matching S21. Of course, the CNN uses a "minimum" pixel neighborhood for correspondence search (e.g., a 2x2 block). However, a single pixel (1x1 block) without a direct neighborhood can also obtain a correspondence value. As a result of the matching step, the disparity vectors of a large number of pixels or all pixels are extracted in the subsequent step S22. In other words, the images are fed into a processor that calculates the two-dimensional (2D) disparity field between the two camera devices.

[0071] In one embodiment, the perception function can be performed based on the acquired two-dimensional (2D) disparity field in a subsequent step S24. For this purpose, another machine learning system has been trained to output perception data 88 when a two-dimensional (2D) disparity vector map is taken as input. An example of perception data 88 is image segmentation, such as identifying roads, road signs, buildings, and objects in an image.

[0072] In a second embodiment, the disparity vector (or two-dimensional (2D) disparity field) can serve as the basis for the correction S26 of the original images 21, 22. The two-dimensional (2D) disparity field is corrected using the camera device's internal parameters to calculate the true depth information (three-dimensional (3D) point cloud). This means that a dense three-dimensional (3D) point cloud 80 can be generated in the next step.

[0073] Calculate the fundamental matrix parameters of the stereo system from uncorrected pixel-by-pixel disparity.

[0074] Without prior knowledge of real-world coordinate distances, we can use the fundamental matrices of a stereo system to calculate the vertical disparity v. r -v l The following linear approximation is used for any pixel location in the image:

[0075]

[0076] in,

[0077] ty = ty / tx, tz = tz / tx define the relative translation of the right camera relative to the left camera along the Y-axis and Z-axis.

[0078] ur-ul=du represents the horizontal parallax of depth.

[0079] vr-vl=dv is the vertical parallax.

[0080] (ul, vl) = pl is the projection of a point P onto the image plane of the left camera device in space, while (ur, vr) = pr is the projection of the same point onto the image plane of the right camera device.

[0081] f corresponds to the focal length fl (in pixels) of the left camera, and af is the difference in focal length between the two cameras: fr = (1 –

[0082] af)fl,

[0083] ax, ay, and az are the relative rotation angles of the right camera device with respect to the left camera device around the X, Y, and Z axes, respectively. The background of this equation can be found in the paper "Identification of stereo rig alignment error based on vertical disparity map" published by S. Kathragadda in the WSCG Journal (ISSN 1213-6972), Vol. 27, No. 2, 2019, pp. 151-160.

[0084] The results given by the VR-VL equation above are...

[0085] vr-vl=ty*(ur-ul)+tz / f*(ul*vr-ur*vl)+af*(vr)+ax*f*(-1)+ax / f*(-vl*vr)+ay / f*(ur*vl)+

[0086] az*(ur)

[0087] dv=ty*(-du)+tz / f*(du*vl-ul*dv)+af*(vl-dv)+ax*f*(-1)+ax / f*(vl*dv-vl^2)+

[0088] ay / f*(ul*vl-du*vl)+az*(ul-du)

[0089] dv=ty*(-du)+tz / f*(du*vu*dv)+af*(v-dv)+ax*f*(-1)+ax / f*(v*dv-v^2)+ay / f*(u*v-du*v)+az*(u-du)

[0090] Therefore, the least squares approximation can be performed on tx, ty, tz, alpha_x, alpha_y, alpha_z, and alpha_f to obtain information about the rotation parameters, translation parameters, and focus parameters between the left and right camera devices.

[0091] For a small 3x3 pixel region, we obtain the following using the pixel positions (u0, v0), (u1, v1), ..., (u8, v8) and the disparities (du0, dv0), (du1, dv1), ..., (du8, dv8)...

[0092] dv0=ty*(-du0)+tz / f*(du0*v0-u0*dv0)+af*(v0-dv0)+ax*f*(-1)+

[0093] ax / f*(v0*dv0-v0^2)+ay / f*(u0*v0-du0*v0)+az*(u0-du0)

[0094] dv1=ty*(-du1)+tz / f*(du1*v10-u*dv1)+af*(v1-dv1)+ax*f*(-1)+

[0095] ax / f*(v1*dv1-v1^2)+ay / f*(u1*v1-du1*v1)+az*(u1-du1)

[0096] dv2=ty*(-du2)+tz / f*(du2*v02-u*dv2)+af*(v2-dv2)+ax*f*(-1)+

[0097] ax / f*(v2*dv2-v2^2)+ay / f*(u2*v2-du2*v2)+az*(u2-du2)

[0098] dv3=ty*(-du3)+tz / f*(du3*v03-u*dv3)+af*(v3-dv3)+ax*f*(-1)+

[0099] ax / f*(v3*dv3-v3^2)+ay / f*(u3*v3-du3*v3)+az*(u3-du3)

[0100] dv4=ty*(-du4)+tz / f*(du4*v04-u*dv4)+af*(v4-dv4)+ax*f*(-1)+

[0101] ax / f*(v4*dv4-v4^2)+ay / f*(u4*v4-du4*v4)+az*(u4-du4)

[0102] dv5=ty*(-du5)+tz / f*(du5*v0-5u*dv5)+af*Pv5-dv5)+ax*f*(-1)+

[0103] ax / f*(v5*dv5-v5^2)+ay / f*(u5*v5-du5*v5)+az*(u5-du5)

[0104] dv6=ty*(-du6)+tz / f*(du6*v0-u6*dv6)+af*(v6-dv6)+ax*f*(-1)+

[0105] ax / f*(v6*dv6-v6^2)+ay / f*(u6*v6-du6*v6)+az*(u6-du6)

[0106] dv7=ty*(-du7)+tz / f*(du7*v0-u7*dv7)+af*(v7-dv7)+ax*f*(-1)+

[0107] ax / f*(v7*dv7-v7^2)+ay / f*(u7*v7-du7*v7)+az*(u7-du7)

[0108] dv8=ty*(-du8)+tz / f*(du8*v0-u8*dv8)+af*(v8-dv8)+ax*f*(-1)+

[0109] ax / f*(v8*dv8-v8^2)+ay / f*(u8*v8-du8*v8)+az*(u8-du8)

[0110] G = A * P

[0111] The vector P = (ty, tz / f, af, ax*f, ax / f, ay / f, az) represents the unknown camera parameters (rotation, translation, focal length, etc.) between the left and right stereo camera devices.

[0112] Parallax correction is only applied to external parameters of the camera device.

[0113] Using the method described above, all parameters of the basic matrix system can be estimated. To make the estimation of external (external) mounted parameters more robust, we can use prior knowledge (calibration) of the internal (intrinsic) parameters of the camera device being used.

[0114] For example, if the focal length f of the camera is known in advance, it is possible to avoid estimating the value using the methods described above, and instead input the focal length of the camera as a "constant".

[0115] Because the rotational parameters ax, ay, and az frequently change due to thermal expansion and vibration during the operation of the three-dimensional system, the above method can be used to estimate these parameters "only" while keeping other parameters "constant".

[0116] Calculate three-dimensional (3D) real-world coordinates in meters.

[0117] With prior knowledge of the inherent calibration of the stereo system (such as the focal length of the camera device, lens distortion, and translation between the left and right camera devices), the real-world coordinate distance in meters can be calculated in the surrounding three-dimensional (3D) environment.

[0118] The advantage of this approach is that no correction is required before matching. This saves memory and computation time while preserving image information.

[0119] During operation, corrections can be computed from matching data from a two-dimensional (2D) convolutional neural network (CNN). Unlike traditional stereo systems, when running the correction steps on the input image, it is not necessary to perform offline calibration (before the first run).

Claims

1. A method for extracting parallax information from an image pair (21, 22) of a stereoscopic camera device, comprising the following steps: a) Receive uncorrected stereoscopic camera image pairs (21, 22), b) The received image pairs are matched in two dimensions using a trained convolutional neural network (30), providing pixel-level two-dimensional displacements (33, 53) as disparity information, wherein the two-dimensional displacements (33, 53) are described as disparity vectors, and the directions of the two-dimensional displacements are accumulated over time to provide a filtered angular direction for the image pixel positions, and c) Output pixel-level two-dimensional displacement (33, 53).

2. The method according to claim 1, wherein, The two-dimensional displacement (33, 53) information is used as input for at least one subsequent sensing task (S24).

3. The method according to claim 1 or 2, wherein, The basic matrix parameters of the stereo camera system are calculated from the uncorrected pixel-by-pixel parallax.

4. The method according to claim 1 or 2, wherein, The two-dimensional displacements (33, 53) are used to correct the parallax of the external parameters of the camera device.

5. The method according to claim 1 or 2, wherein, Calculate three-dimensional real-world coordinates in meters.

6. The method according to claim 1 or 2, wherein, The two-dimensional displacements (33, 53) are used to calculate dense three-dimensional point clouds.

7. A system (40) for extracting parallax information from image pairs (21, 22) of a stereoscopic camera device, comprising an input unit (42), a processing unit (44), and an output unit (46), wherein a) The input unit (42) is configured to receive uncorrected stereoscopic camera image pairs (21, 22), b) The processing unit (44) is configured to perform two-dimensional matching on the received image pairs (21, 22) using a trained convolutional neural network (30), providing pixel-level two-dimensional displacements as disparity information, wherein, The two-dimensional displacements (33, 53) are described as disparity vectors, and the directions of these displacements are accumulated over time to provide a filtered angular direction for the image pixel positions. c) The output unit (46) is configured to output two-dimensional displacements (33, 53) at the pixel level.

8. A vehicle comprising a stereo camera system (11, 12) and a system (40) for extracting parallax information according to claim 7, wherein, The stereo camera system (11, 12) is configured to generate uncorrected stereo camera image pairs (21, 22) and output them to the system (40) for extracting parallax information.

Images