Vehicle-mounted three-dimensional looking-around system based on independent controller

Abstract

The invention provides a vehicle-mounted three-dimensional looking-around system based on an independent controller, which is characterized in that the vehicle-mounted three-dimensional looking-around system is provided with an independent three-dimensional looking-around field controller, and related calculation of three-dimensional looking-around is migrated to the three-dimensional looking-around field controller; wherein the correlation calculation comprises ISP processing, image correction and gray processing, cost calculation, cost aggregation, parallax calculation and parallax optimization. According to the vehicle-mounted three-dimensional surround view system based on the independent controller provided by the invention, the burden of huge computational power demand of three-dimensional matching on an intelligent driving domain controller can be reduced, and the computational power redundancy and safety redundancy of an intelligent driving system are increased; and the intelligent driving system with medium and low computing power can also be connected to the three-dimensional all-round looking system, so that the applicable scenes of the three-dimensional all-round looking system can be greatly increased.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to an in-vehicle stereo surround view system based on an independent controller. Background Technology

[0002] In recent years, intelligent driving systems have been rapidly gaining popularity. Surround-view cameras, as a core perception component of these systems, work collaboratively with multiple cameras to achieve 360-degree coverage of the vehicle's surroundings, playing a crucial role in environmental perception, decision support, and user experience enhancement. By utilizing four ultra-wide-angle cameras placed at the front, rear, left, and right of the vehicle, images are acquired from four directions. Combined with high-precision distortion correction algorithms and image stitching technology, a bird's-eye view (BEV) is synthesized, providing the driver or autonomous driving system with intuitive environmental information. The vehicle domain controller is becoming increasingly centralized, with various optical, acoustic, and electromagnetic sensors directly connected to the central intelligent driving controller for unified data and decision processing.

[0003] In existing technologies, most stereo surround view systems employ a single intelligent driving domain controller, such as... Figure 1 As shown, the controller of the surround-view system is integrated into the intelligent driving domain controller. In a single intelligent driving domain controller solution, the stereo matching calculation of the surround-view system is deployed on the intelligent driving domain controller. However, in this solution, the surround-view stereo system has a high demand for computing power, resulting in a heavy computing burden. Specifically: 1. Computing power requirements for a single stereo camera: Taking a stereo camera with a resolution of 1920×1080 and a maximum parallax of 80 as an example, the integer calculation required for one frame of stereo matching is approximately 1920×1080×80×87, which is approximately equal to 14.4G integer calculations.

[0004] 2. Computing power requirements of surround view cameras: If a surround view stereo system consists of four stereo cameras with a resolution of 1920×1080 and a frame rate of 20 frames per second, its theoretical computing power consumption is approximately: 14.4×20×4=1152G integer calculations.

[0005] 3. CPU computing power reference: For reference, the A55 processor, which is currently a common embedded platform, has a main frequency of 1.8GHz. Its NEON accelerator can process 16 int8 integer calculations per cycle, that is, it can process a maximum of 32.4G int8 integer calculations per second. It is necessary to configure 35.5 A55 processors to theoretically complete the above stereo surround view calculation. Summary of the Invention

[0006] This invention provides an in-vehicle stereo surround view system based on an independent controller to solve at least one of the above-mentioned technical problems.

[0007] The application provides a vehicle-mounted stereoscopic surround view system based on an independent controller, which is configured with an independent stereoscopic surround view field controller, and related calculations of stereoscopic surround view are migrated to the stereoscopic surround view field controller; wherein the related calculations include ISP processing, image correction and grayscale processing, cost calculation, cost aggregation, disparity calculation and disparity optimization.

[0008] In some embodiments, the independent controller is an ASIC, a DSP or an FPGA.

[0009] In some embodiments, the image correction and grayscale processing are used to ensure that corresponding points of left and right images are in the same row, and pixel values can be directly used for subsequent cost calculation; specifically, the image correction and grayscale processing specifically include: epipolar rectification: through camera internal and external parameters, left and right images are corrected to be parallel to epipolar lines, and the matching problem is simplified to one-dimensional search; grayscale processing: color images are converted to grayscale images.

[0010] In some embodiments, the cost calculation specifically includes: for each pixel point (i, j) in the left image, the matching cost C(i, j, d) of the pixel point (i, j) and the pixel point (i-d, j) in the right image with the same row and different disparity d is calculated.

[0011] In some embodiments, the cost function in the cost calculation includes the sum of squared differences, the sum of absolute differences and Census transformation.

[0012] In some embodiments, in the cost aggregation process, one-dimensional path cost accumulation in multiple directions is performed to approximate global energy optimization; the aggregated cost is composed of an initial cost and a regularization term.

[0013] In some embodiments, in the disparity calculation process, for each pixel, the disparity with the smallest aggregated cost is selected as the initial disparity.

[0014] In one or more specific embodiments described above, the vehicle-mounted stereoscopic surround view system based on an independent controller provided by the application can reduce the burden of the huge computing power requirement of stereoscopic matching on the intelligent driving domain controller, increase the computing power redundancy and safety redundancy of the intelligent driving system, and the medium and low computing power intelligent driving system can also access the stereoscopic surround view system, which can greatly increase the application scenarios of the stereoscopic surround view system. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0016] Figure 1 is a schematic diagram of a vehicle-mounted stereoscopic surround view system provided by the prior art; Figure 2 is a schematic diagram of a vehicle-mounted stereoscopic surround view system provided by the present application. DETAILED DESCRIPTION

[0017] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort belong to the protection scope of the present application.

[0018] In one specific embodiment, as shown in Figure 2 the present application provides a vehicle-mounted stereoscopic surround view system based on an independent controller. The vehicle-mounted stereoscopic surround view system is configured with an independent stereoscopic surround view field controller, and the related calculations of stereoscopic surround view are migrated to the stereoscopic surround view field controller. The related calculations include ISP processing, image correction and gray scale processing, cost calculation, cost aggregation, disparity calculation and disparity optimization.

[0019] In this way, the multi-step complex calculations required for stereoscopic surround view (such as ISP processing, disparity optimization, etc.) are separated from the vehicle-mounted main controller and processed by the independent domain controller, avoiding the contention for computing resources. This significantly reduces the processing delay of the system for the surrounding environment data of the vehicle, and can meet the real-time image output demand of more than 30fps in low-speed moving, congestion and other scenes, providing a smooth panoramic view for the driver.

[0020] Further, the independent controller focuses on the optimization of the stereoscopic surround view algorithm, and can more efficiently complete the core steps of ISP processing (optimizing image quality), disparity calculation and optimization (improving distance measurement accuracy), etc. It can accurately identify low obstacles (such as stone piers, flowerpots), pedestrians and road edges within the range of 0.1m-10m around the vehicle, and effectively eliminate the risk of the four blind areas in traditional driving.

[0021] Meanwhile, the domain controller architecture is used to realize the isolation of computing tasks, and the coupling degree of the surround view system and other modules such as the vehicle information entertainment system and the power control system is reduced. When the surround view algorithm needs to be upgraded or a higher resolution camera (such as upgrading from 720P to 1080P) needs to be adapted, only the firmware or hardware of the independent controller needs to be updated, without changing the main controller of the vehicle, thereby improving the flexibility and cost-effectiveness of system upgrading.

[0022] In addition, the independent controller can be specially designed for the vehicle scene (such as integrating a wide-temperature working module), and the ISP processing is used to optimize the noise reduction and enhancement of the image, so that clear images can still be stably output even in low-temperature, dusty or rainy weather, thereby avoiding the failure of the surround view system caused by the excessive load of the main controller.

[0023] The above independent controller can be an ASIC, a DSP or an FPGA. That is, a dedicated processor such as an ASIC, a DSP or an FPGA can be selected, so that higher efficiency, lower overall energy consumption of the domain control, higher frame rate and smaller processing delay can be obtained, a longer processing window can be provided for the back-end processing decision system, and the normal safety can be improved. By releasing the box single 35 A55 integer computing resource, the burden of the intelligent driving domain controller will be greatly reduced, the decision speed and accuracy will be simultaneously improved, and the vehicle safety will be benefited.

[0024] Specifically, the ASIC controller is a dedicated integrated circuit, the hardware structure of which is completely matched with the surround view computing process, and no redundant general-purpose processing module is needed, so that the power consumption is extremely low (usually 30%-50% lower than that of a general-purpose processor) and the computing efficiency is extremely high, the core task such as parallax calculation can be responded in nanoseconds, and the controller is suitable for the vehicle scene with strict limitations on power consumption and volume.

[0025] The DSP controller has strong digital signal processing capability, and is especially good at matrix operations such as image filtering and grayscale conversion in ISP processing, can quickly optimize the distorted images collected by the fisheye camera, and improve the image correction accuracy; at the same time, the software programming upgrade is supported, the camera parameters of different manufacturers can be adapted, and the compatibility is strong.

[0026] The FPGA controller adopts a programmable logic architecture, and the hardware logic can be flexibly reconfigured according to the iteration requirements of the surround view algorithm, for example, when the cost aggregation algorithm is upgraded from multi-direction one-dimensional accumulation to adaptive aggregation, the hardware does not need to be replaced to complete the function update; the parallel computing characteristic can process the image data of multiple cameras at the same time, and meet the synchronous processing requirements of more than 4 high-definition images.

[0027] In addition to the above types of controllers, ARM architecture multi-core processors, heterogeneous computing controllers (FPGA+DSP), and neural network processors (NPU) can also be used as controllers. Among them, the ARM architecture multi-core processor is based on the Cortex-A series kernel and realizes the allocation of computing tasks through multi-core parallel technology, has a certain general processing capacity and energy efficiency ratio, and the cost is lower than that of FPGA and ASIC, which is suitable for economic models with high algorithm iteration frequency requirements but moderate computing precision requirements. The heterogeneous computing controller (FPGA+DSP) adopts a heterogeneous architecture of “FPGA responsible for parallel computing + DSP responsible for serial signal processing”. The FPGA undertakes parallel tasks such as cost calculation and disparity optimization, and the DSP undertakes serial tasks such as ISP processing and data transmission. The combination of the two advantages realizes the balance of “high precision + high efficiency”, which is suitable for high-end intelligent driving assistance systems. The neural network processor (NPU) is optimized for stereo matching algorithms based on deep learning (such as end-to-end disparity estimation models), has efficient tensor operation capability, can quickly process large-scale feature extraction and matching tasks, improves the disparity calculation precision of textureless areas, and is suitable for L2+ level intelligent driving scenarios.

[0028] In some embodiments, the image correction and grayscale processing are used to ensure that the corresponding points of left and right images are in the same row, and the pixel values can be directly used for subsequent cost calculation. The image correction and grayscale processing specifically include: Radial line correction: correct left and right images to be parallel to radial lines through camera internal and external parameters, and simplify the matching problem to one-dimensional search; through the internal and external parameters obtained by camera calibration (internal parameters include focal length and principal point coordinates, and external parameters include rotation matrix and translation vector), the distorted images collected by left and right cameras are corrected to be parallel to radial lines, so that the corresponding pixel points only have disparity in the horizontal direction, the original two-dimensional matching search is simplified to one-dimensional search, which can reduce the complexity of cost calculation by more than 50%, and greatly improve the matching efficiency.

[0029] Grayscale: convert color images to grayscale images; convert RGB three-channel color images to single-channel grayscale images, which not only retains the brightness characteristics of pixels (which can meet the matching requirements), but also compresses the image data volume to 1 / 3 of the original, reduces the storage occupation and data transmission bandwidth of the independent controller, and improves the processing speed of subsequent cost calculation by 2-3 times; at the same time, it avoids the matching interference caused by color channel differences and improves the matching stability.

[0030] The two work together to ensure the “row alignment” and “data lightweight” of left and right images, provide standardized input for subsequent cost calculation, and can reduce the initial error of disparity calculation by 15%-20%, which is especially suitable for processing large distortion images collected by vehicle-mounted fisheye cameras and providing accurate basic data for panoramic view stitching.

[0031] The cost calculation specifically includes, for each pixel point (i, j) in the left image, calculating the matching cost C(i, j, d) of the pixel point (i−d, j) in the same row and different disparity d in the right image. Through cost calculation, accurate pixel matching basis can be achieved, and through pixel-by-pixel and multi-disparity cost calculation, a matching similarity quantization index of the left image pixel and the corresponding pixel in the right image is constructed, which provides original data support for subsequent cost aggregation and disparity calculation, and can effectively identify the contour boundary of the object around the vehicle (such as the boundary between the vehicle body and the road edge). Moreover, through cost calculation, the image features after polar correction can be adapted: based on the calculation logic of “searching in the same row”, the above-mentioned polar correction is technically coordinated to avoid redundant calculation of cross-row and column search, so that the cost calculation time of a single pixel is shortened to the microsecond level, ensuring that the system can process real-time image streams above 30 fps. Further, the cost calculation supports multi-scene environment adaptation, and by traversing different disparities d (the disparity range can be set according to actual needs, such as 0-64 pixels), distance measurement requirements from 0.1 m to 10 m can be covered, and various vehicle-mounted scenes such as side parking (short-distance measurement) and narrow road passing (medium-distance measurement) can be adapted.

[0032] Pixel block-based cost calculation can also be used, and an n×n pixel block (such as 3×3 or 5×5) centered on the pixel point (i, j) in the left image is taken as a unit to calculate the matching cost of the corresponding pixel block in the right image in the same row and different disparity d, which can enhance the resistance to image noise and be suitable for scenes with poor image quality such as rainy days and dust.

[0033] Alternatively, multi-scale cost calculation can be used, and the corrected left and right images are first down-sampled to generate a multi-scale image pyramid, and then cost calculation is performed at different scales, and then the multi-scale costs are fused, which can improve the matching accuracy of non-textured areas (such as walls and floors) while ensuring calculation efficiency.

[0034] Dynamic disparity range cost calculation can also be used, and the search range of the disparity d is dynamically adjusted according to the position of the pixel point (i, j) (such as setting a large disparity range for close-range areas and a small disparity range for long-range areas), which reduces the calculation amount of invalid disparity and improves the overall cost calculation efficiency by 20%-30%, which is suitable for scenes with extremely high real-time requirements.

[0035] Feature point-based cost calculation can also be used, and first, key feature points (such as corner points and edge points) in the left and right images are identified through a feature extraction algorithm (such as SIFT and ORB), and then cost calculation is performed only on the feature points, and then the cost of non-feature points is completed through feature point matching results, which can greatly reduce the calculation amount and be suitable for economic controllers with limited computing power.

[0036] The cost function in the cost calculation includes sum of squared differences, sum of absolute differences, and Census transform. The sum of squared differences (SSD) is sensitive to pixel gray difference, suitable for scenes with uniform illumination (such as noon on a sunny day), and can accurately identify object details; the sum of absolute differences (SAD) has low computational complexity and strong noise resistance, suitable for scenes with high real-time requirements; the Census transform generates binary features by comparing the gray scale of a pixel with its neighborhood, is not sensitive to changes in illumination, and is suitable for scenes with uneven illumination such as overcast days and evenings.

[0037] The computing power characteristics of the adaptive independent controller are adapted, and the SAD and SSD algorithm structures are simple and can be efficiently implemented through parallel logic of ASIC or FPGA; although the Census transform has slightly more calculation steps, it does not require multiplication operations and can be quickly processed through the logic unit of a DSP, and all three functions can adapt to the hardware architecture of the vehicle-mounted controller. By combining the use of multiple cost functions (such as weight fusion), the failure problem of a single function in a special scene (such as matching error of SSD in light mutation) can be avoided, and the robustness of cost calculation can be improved by more than 30%.

[0038] In addition to the above cost functions, gradient-based cost functions, mutual information-based cost functions, deep learning-driven cost functions, or adaptive cost function selection can also be used. Among them: The gradient-based cost function includes sum of absolute differences of gradients (SAD-G) and sum of squared differences of gradients (SSD-G), which replace the gray value with the gradient value (such as Sobel gradient) for cost calculation, which can enhance the matching sensitivity to object edges, and is suitable for identifying pedestrians, non-motor vehicles, and other clear outline targets.

[0039] The mutual information-based cost function measures the matching similarity by calculating the mutual information value of the pixel blocks of the left and right images, and has stronger adaptability to nonlinear illumination changes, and is suitable for extreme illumination scenes such as tunnel entrances (alternating strong and weak light), but has high computational complexity and requires the use of parallel computing capabilities of FPGA.

[0040] The deep learning-driven cost function extracts deep features of left and right images through a pre-trained convolutional neural network (CNN) and directly outputs pixel-level matching cost, which can achieve more accurate cost calculation in textureless areas (such as flat roads), and is suitable for high-precision perception requirements of L2+ level intelligent driving.

[0041] The adaptive cost function selection system automatically switches the cost function according to the characteristics of the real-time image (such as illumination intensity and texture complexity), for example, using SSD in high-texture areas, using Census transform in unevenly illuminated areas, and using mutual information function in low-texture areas, balancing matching accuracy and computational efficiency.

[0042] In the cost aggregation process, the global energy optimization is approximated by accumulating the cost of one-dimensional path in multiple directions; the aggregated cost is composed of the initial cost and a regularization term. By accumulating the cost of one-dimensional path in multiple directions (e.g., 8 directions: up, down, left, right, left-up, right-up, left-down, right-down), the isolated initial cost is associated with the neighborhood cost, the mismatched cost of a single pixel can be corrected, the cost distribution is more consistent with the actual contour of the object, and the noise interference in the disparity calculation is reduced. Compared with global energy optimization (e.g., belief propagation, graph cut algorithm), the computational complexity of multi-directional one-dimensional aggregation is reduced from O (N²) to O (N) (N is the number of pixels), which can be realized in real time on an independent controller; at the same time, the edge error of the disparity map is reduced by more than 25% by approximating the global optimization effect through the regularization term (restricting the smoothness of the neighborhood disparity). The smoothness constraint of the regularization term can generate continuous aggregated cost in non-texture or low-texture areas (such as walls), avoid disparity jumps caused by ambiguous initial cost, and improve the stability of distance measurement between vehicles and large-area obstacles.

[0043] In addition, adaptive scale cost aggregation, cross-scale cost aggregation, guided filter-based cost aggregation, or lightweight global aggregation algorithm can also be used. Among them: Adaptive scale cost aggregation dynamically adjusts the scale of the aggregation path according to the texture complexity of the image region (e.g., small-scale path for texture-intensive areas, large-scale path for non-texture areas), which ensures the accuracy of the edge region and enhances the smoothness of the weak texture region. The scale aggregation scheme in AANet can be referred to.

[0044] Cross-scale cost aggregation is based on a multi-scale image pyramid, which up-samples the low-scale (coarse-grained) aggregated cost and fuses it with the high-scale (fine-grained) aggregated cost. The global consistency of the coarse-scale cost is used to correct the local error of the fine-scale cost, which is suitable for scenes that require high-precision disparity.

[0045] Guided filter-based cost aggregation uses the original grayscale image as a guide image to perform guided filtering on the initial cost to achieve cost aggregation. It can achieve smooth aggregation while maintaining clear object edges, avoiding the edge fattening problem that may be caused by multi-directional path aggregation.

[0046] Lightweight global aggregation algorithm uses a simplified global energy optimization model (e.g., dynamic programming-based global aggregation) to reduce the computational complexity while ensuring that the aggregation effect is close to global optimization, which is suitable for controllers with moderate computing power such as DSP.

[0047] In some embodiments, in the disparity calculation process, for each pixel, the disparity with the minimum aggregated cost is selected as the initial disparity. The logic of "minimum cost selection" is simple and direct, and can be quickly implemented by hardware circuit. The disparity calculation time of a single pixel is only nanoseconds, ensuring that the system can output a disparity map matching the image resolution in real time (such as 1280x720 pixels). The aggregated cost has fused multi-directional neighborhood information and added a smoothing constraint. The initial disparity selected based on this can accurately reflect the true matching relationship of the pixel, and can control the disparity error in the near-range area (0.1-3m) within 1 pixel, meeting the measurement requirements of the tire-to-road distance when parking sideways. The accuracy of the initial disparity as the pre-data of the disparity optimization step directly affects the final output result. The initial disparity obtained by this method can reduce the calculation amount of the optimization step and shorten the delay of the overall processing link.

[0048] In addition, sub-pixel level disparity calculation, multi-peak cost screening disparity calculation, confidence-based disparity calculation, and dynamic threshold screening disparity calculation can also be used. Among them: After finding the minimum cost disparity d_min, the sub-pixel level disparity calculation fits the cost curve (such as parabolic fitting) near d_min (such as d_min-1, d_min, d_min+1) to calculate the sub-pixel disparity value corresponding to the minimum value of the curve. The accuracy of the disparity can be improved from the pixel level to the 0.1 pixel level, which is suitable for high-precision distance measurement scenarios (such as automatic parking of parking line positioning).

[0049] When there are multiple local minimum values in the aggregated cost, the multi-peak cost screening disparity calculation compares the difference between the local minimum values (such as the difference needs to be greater than a threshold T) to screen out a unique valid disparity. If the difference is less than the threshold, it is marked as an uncertain area, which will be supplemented by neighborhood interpolation later. This can reduce the mismatch rate in textureless areas.

[0050] In the confidence-based disparity calculation, while calculating the initial disparity, the matching confidence of each pixel is calculated through the cost ratio (such as the ratio of the minimum cost to the second minimum cost). Only the disparity with a confidence higher than a threshold is retained as the initial disparity, directly filtering suspicious matching results with low confidence and reducing the burden of subsequent optimization steps.

[0051] The dynamic threshold screening disparity calculation dynamically adjusts the threshold for disparity selection according to the texture features of the image area (such as using a low threshold in texture-dense areas and a high threshold in textureless areas), avoiding the adaptability problem of a single threshold in different areas, and improving the overall quality of the disparity map.

[0052] In the foregoing specific embodiments, the vehicle-mounted independent controller-based vehicle-mounted stereoscopic surround view system provided by the present application can reduce the burden of the huge calculation power requirement of stereoscopic matching on the intelligent driving domain controller, increase the calculation power redundancy and safety redundancy of the intelligent driving system, and the medium and low calculation power intelligent driving system can also access the stereoscopic surround view system, which can greatly increase the application scenarios of the stereoscopic surround view system.

[0053] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0054] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0055] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A vehicle-mounted stereo surround view system based on an independent controller, characterized in that, The vehicle-mounted stereo surround view system is equipped with an independent stereo surround view domain controller, which migrates the relevant calculations of stereo surround view to the stereo surround view domain controller; wherein, the relevant calculations include ISP processing, image correction and grayscale processing, cost calculation, cost aggregation, disparity calculation and disparity optimization.

2. The vehicle-mounted stereo surround view system based on an independent controller according to claim 1, characterized in that, The independent controller is an ASIC, DSP, or FPGA.

3. The vehicle-mounted stereo surround view system based on an independent controller according to claim 2, characterized in that, The image correction and grayscale processing are used to ensure that corresponding points in the left and right images are on the same line, and that the pixel values ​​can be directly used for subsequent cost calculations; specifically, the image correction and grayscale processing include: Epipolar correction: By using camera intrinsic and extrinsic parameters, the left and right images are corrected to make the epipolar lines parallel, simplifying the matching problem into a one-dimensional search; Grayscale conversion: Converting a color image to a grayscale image.

4. The vehicle-mounted stereo surround view system based on an independent controller according to claim 3, characterized in that, The cost calculation specifically includes: For each pixel (i,j) in the left image, calculate its matching cost C(i,j,d) with the pixel (i−d,j) in the same row of the right image but with different disparities d.

5. The vehicle-mounted stereo surround view system based on an independent controller according to claim 4, characterized in that, The cost function in the cost calculation includes the sum of squared differences, the sum of absolute differences, and the Census transform.

6. The vehicle-mounted stereo surround view system based on an independent controller according to claim 5, characterized in that, In the cost aggregation process, global energy optimization is approximated by accumulating the cost of one-dimensional paths in multiple directions; the aggregated cost consists of the initial cost and a regularization term.

7. The vehicle-mounted stereo surround view system based on an independent controller according to claim 6, characterized in that, In the disparity calculation process, for each pixel, the disparity with the lowest aggregation cost is selected as the initial disparity.

Images