Road three-dimensional modeling vehicle-mounted auxiliary equipment based on RGB image information

Abstract

The invention discloses a road three-dimensional modeling vehicle-mounted auxiliary device based on RGB image information, and particularly relates to the technical field of three-dimensional modeling, the vehicle-mounted auxiliary device comprises an energy supply device, a storage device, a photographing device and an image acquisition processing device, the photographing device completes road image data acquisition, and the image acquisition processing device processes the road image data. A main control chip arranged in an image acquisition and processing device is used for controlling logic and constructing an algorithm, and based on computer vision and machine vision, a mapping relation between a two-dimensional image and a three-dimensional target is established and three-dimensional geometric information is recovered; a least square algorithm in a computer vision library OpenCV is adopted to minimize the difference between a point, projected to a two-dimensional image plane through a camera model, of a three-dimensional point and a corresponding point in an actual image, a target function of the sum of re-projection errors of all observed feature points is constructed, and then an optimal solution is found through the algorithm; and initializing the main control chip and preprocessing the original image by using the algorithm built in the main control chip.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional modeling, and in particular to a vehicle-mounted auxiliary device for three-dimensional road modeling based on RGB image information. Background Art

[0002] At present, the 3D modeling technology in my country is widely used. The mainstream applications on the market are: 1. Photogrammetry CAD technology; 2. UAV oblique photogrammetry 3D modeling technology; 3. Camera and 2D laser rangefinder 3D modeling technology; 4. 3D laser radar modeling technology; 5. Real-time dynamic display technology of road 3D scenes; 6. 3D modeling vehicle-mounted mobile image acquisition system technology, etc. Among them, domestic UAV technology is in the development stage, and various related technologies and theories are becoming more mature and perfect. UAV aerial photography technology has played a huge role in the fields of aerial photography, disaster relief, earthquake disaster relief, and post-earthquake recovery and reconstruction, but there are problems such as high cost and short battery life. The 3D laser radar modeling technology does not have the problem of short battery life, and can quickly and accurately obtain the 3D shape and structure information of the target object. However, it is expensive, and there may be missing or inaccurate data for some complex surfaces or transparent objects. Therefore, further research on 3D modeling technology is crucial.

[0003] The vehicle-mounted mobile high-quality image acquisition system for 3D modeling mainly considers the following aspects: first, accurately obtain the movement trajectory of the mobile vehicle including the ground elevation, and use this as a benchmark to realize the terrain undulations of the ground 3D modeling, meeting the requirements of 3D visualization of underground pipelines and 3D digital urban management components for the realistic expression of terrain; second, obtain high-quality street view textures in ideal locations to meet the need for high-quality textures in 3D modeling of urban roads and landscapes; third, the obtained textures are associated with space to facilitate the search, integration and publishing of images; fourth, the cost factor, meeting the needs while reducing investment as much as possible. Summary of the invention

[0004] In order to solve the problems of operational complexity and environmental factors in the current existing technology, as well as high cost, limited battery life, secondary processing of exported data, low monitoring accuracy, and low deformation monitoring accuracy of ground close-range photogrammetry, a new device is provided that can operate continuously without being affected by external factors, and the exported data can be directly used for three-dimensional modeling. The device can not only reduce the photography cost and energy supply cost of the device, but also improve the accuracy of the road three-dimensional modeling device through effective parameters extracted by experiments, thereby realizing a new device with high efficiency and high benefit modeling. The present invention provides a road three-dimensional modeling vehicle-mounted auxiliary device based on RGB image information to solve the problems raised in the background technology.

[0005] In order to achieve the above-mentioned object, the present invention provides the following technical solution: a vehicle-mounted auxiliary device for road three-dimensional modeling based on RGB image information, the vehicle-mounted auxiliary device includes a power supply device, a storage device, a photographing device and an image acquisition and processing device, and the road image data acquisition is completed by the photographing device, including the following steps:

[0006] The main control chip in the image acquisition and processing device is used to control the logic and construct an algorithm. Based on computer vision and machine vision, a mapping relationship between a two-dimensional image and a three-dimensional target is established and three-dimensional geometric information is restored. The least squares algorithm in the computer vision library OpenCV is used to minimize the difference between the three-dimensional points projected onto the two-dimensional image plane through the camera model and the corresponding points in the actual image. The objective function of the sum of the reprojection errors of all observed feature points is constructed, and then the optimal solution is found through the algorithm.

[0007] Initializing the main control chip and preprocessing the original image using the algorithm built into the main control chip;

[0008] Preprocessing includes wide-angle camera distortion correction and wide-angle camera image data extraction and calibration processing, which are implemented through C++ programming language; before wide-angle camera distortion correction and wide-angle camera image data extraction and calibration processing, the wide-angle camera is calibrated first;

[0009] The steps for wide-angle camera program calibration are as follows:

[0010] S11. Import libraries and define variables:

[0011] First, import the library, introduce the core functions of the OpenCV library and the standard input and output streams. Use the std and cv namespaces to facilitate the use of standard library and OpenCV functions in the code. Use the CHECKERBOARD array to define the number of internal intersections of the checkerboard as 6 rows and 9 columns, which will be used for corner detection later.

[0012] S12. Define the data storage structure:

[0013] The 3D point container: objpoints is used to store the 3D points corresponding to each chessboard image. The 2D point container: imgpoints is used to store the 2D points corresponding to each chessboard image.

[0014] S13. Define the world coordinates of the 3D point:

[0015] Generate 3D points, using a double loop to generate the corresponding 3D coordinates (X, Y, Z) for each intersection of the chessboard. The Z coordinate is 0, indicating that the chessboard is on the same plane. All generated 3D points are stored in objp.

[0016] S14. Extract image path:

[0017] Define the image path, specify the directory where the chessboard image is located and the file format (.jpg). Read the image path, use the cv::glob function to get the paths of all image files under the specified path, and store them in the images vector.

[0018] S15. Detect corner points and store data:

[0019] First, frame is used to store the read image, gray is used to store the image converted to grayscale, and corner_pts is used to store the pixel coordinates of the detected corner points. The image is read in a loop, and each image path in the images vector is looped through and grayscale converted. Find the chessboard corner points using findChessboardCorners, and the return value is stored in the success variable. If the corner point is successfully detected, use cornerSubPix to refine the corner point position and draw the corner point on the original image. Finally, objp and corner_pts are stored in objpoints and imgpoints for subsequent camera calibration.

[0020] S16. Camera calibration:

[0021] First, declare the calibration variables, define the camera intrinsic parameter matrix cameraMatrix, distortion coefficient distCoeffs, rotation matrix R and translation vector T, and then call the calibrateCamera function, passing in the known 3D points and corresponding 2D points for camera calibration.

[0022] S17. Output results:

[0023] Output the camera's intrinsic parameter matrix, distortion coefficients, rotation vectors, and translation vectors for analysis and subsequent use;

[0024] The steps for wide-angle camera distortion correction are as follows:

[0025] S21. Import libraries and define variables:

[0026] Introduce the core functions of the OpenCV library and the standard input and output stream library, and use the cv and std namespaces to facilitate subsequent calls to OpenCV functions and C++ standard library functions;

[0027] S22. Main function starts:

[0028] Define the input and output file paths, where InputFile is the image path that needs to be distorted and OutputFile is the path where the rectified image is saved;

[0029] S23. Read image:

[0030] Use the imread function to read the input image and store it in sMatImg. If the image reading fails, that is, sMatImg is empty, an error message is output and -1 is returned to end the program.

[0031] S24. Define camera internal parameters and distortion coefficients:

[0032] Define the output image matrix, where sMatOutput is used to store the corrected image, and then define a 3x3 array cameraMat, which contains the focal length and principal point position of the camera; finally define a 1x5 array distCoef, which contains the distortion coefficient of the lens;

[0033] S25. Set image size and matrix:

[0034] Create a Size object image_size, set it to 1920x1080, convert the previously defined camera intrinsic parameters and distortion coefficients into OpenCV matrix form; then print the camera intrinsic parameter matrix and distortion coefficients;

[0035] S26. Initialize the calibration map:

[0036] Create a mapping matrix, where mapx and mapy are used to store the corrected pixel map, create a rotation matrix R and initialize it as the unit matrix; then initialize the correction map, call the initUndistortRectifyMap function, pass in the camera intrinsic parameters, distortion coefficients, rotation matrix and new camera matrix, this function will generate the mapping matrices mapx and mapy;

[0037] S27. Get the optimized camera matrix:

[0038] Call the getOptimalNewCameraMatrix function to generate an optimized camera matrix optimal_mat and output it;

[0039] S28. Process the image and save the result:

[0040] Resize the input image sMatImg to the target size and store it in sMatOutput. Then create a copy of new_image for subsequent processing. Use the remap function to correct the image distortion according to the previously generated mapping matrix. Finally, use imwrite to save the corrected image to the specified output path.

[0041] The steps for wide-angle camera image data frame extraction calibration are as follows:

[0042] S31. Import libraries and define variables:

[0043] Introduce OpenCV library for image processing, iostream for input and output operations, vector for dynamic arrays, and fstream for file operations; omit the prefixes of standard namespaces and OpenCV namespaces to simplify the code;

[0044] S32. Define the writeDataToFile function:

[0045] First, write the floating-point data to the specified file and output it in a C-style array format; Filename is the file name to be written, Data is the data vector to be written, and dataNumPerLine is the number of data written per line; then open the file and check whether it is successful, write the definition and formatting content of the array, use a loop to traverse the data, and wrap the line whenever the number of elements written reaches dataNumPerLine; do not add a comma after the last element; close the file;

[0046] S33. Define camera parameters and distortion coefficients:

[0047] Where captureOriginM and previewOriginM are the intrinsic parameter matrices of the camera, Coeff is the distortion coefficient moment, captureM and previewM are the intrinsic parameter matrices for different resolutions, and R is the rotation matrix, which is initialized to the unit matrix;

[0048] S34. Define image size:

[0049] The resolution used to store captured and preview images;

[0050] S35. Generate dedistortion and correction map:

[0051] Use the initUndistortRectifyMap function to calculate the dedistortion and correction mapping matrix; input the camera intrinsic parameters, distortion coefficient, rotation matrix, new intrinsic parameter matrix and image size, and output the dedistortion mapping matrix captureMapX, captureMapY, previewMapX, previewMapY;

[0052] S36. Flattened mapping matrix:

[0053] Create four one-dimensional floating-point vectors to store the dedistorted mapping data, and use the copyTo method to copy the mapping matrix into the flattened vectors;

[0054] S37. Write to file:

[0055] Call writeDataToFile function to write the data of preview map Y to the specified file; return 0 indicates that the program ends normally;

[0056] The preprocessed image data is passed through the motion recovery structure multi-view stereo vision algorithm embedded in the main control chip to match images from multiple perspectives, and finally the three-dimensional structure of the road scene is restored from the two-dimensional image data.

[0057] Preferably, the original image preprocessing also includes denoising, contrast enhancement, and color correction; wherein denoising and contrast enhancement are implemented by a Gaussian filtering algorithm;

[0058] Preferably, according to the epipolar geometry constraint, the camera extrinsic parameters are restored and the relationship that the corresponding points between the two images must satisfy is described. Then, the relative motion parameters of the camera are obtained by decomposing the intrinsic matrix. Then, the matching relationship between the newly added camera posture and the existing camera and the three-dimensional point cloud is optimized. Then, the coordinates of the three-dimensional points are calculated using the triangulation method through the matching points between multiple views and the known camera parameters. Finally, the camera parameters and the three-dimensional structure are refined through global nonlinear optimization, and finally the three-dimensional structure of the road scene is restored from the two-dimensional image data. The specific formula is as follows:

[0059] Epipolar geometry constraints:

[0060] Where: x 1 and x 2 are the homogeneous coordinates of the matching point pairs in the two images, E is the intrinsic matrix, which contains the relative rotation and translation information between the cameras, and T is the transposed matrix;

[0061] Decomposition of the eigenvalue matrix: E = U∑V T (2)

[0062] Where: U and V are orthogonal matrices, Σ is a diagonal matrix, R i and t i are the rotation and translation matrices of the i-th camera, P j is the projection matrix of the jth camera, X is a three-dimensional point;

[0063] Camera pose optimization: minR i ,t i ∑jeqi∑(x,X)||xP j (R i X+t i )|| 2 (3)

[0064] Where: jeqi represents the weight factor, which is used to measure the strength relationship between camera rotation, translation and projection in this formula;

[0065] Triangulate the coordinate point: X = argmin∑||x i -P i X|| 2 (4)

[0066] Where: x i is the image point in the i-th view, P i is the projection matrix of the i-th camera;

[0067] Nonlinear Optimization:

[0068] The present invention has the following advantages:

[0069] 1. The present invention uses a common digital camera as an image acquisition device, which can effectively overcome the disadvantages of low operating efficiency, high labor intensity, high cost based on three-dimensional laser scanning equipment, and limited application conditions of traditional measurement methods, and can improve the application level of GIS, and further adapt to the needs of GIS socialization services that are simple, flexible, efficient and low-cost. In addition, camera parameter estimation is the key to establishing a mapping relationship between a two-dimensional image and a three-dimensional target and restoring three-dimensional geometric information. At present, the solution scheme of the beam adjustment optimization principle for simultaneously settling camera parameters and three-dimensional point coordinates can provide a reference for the parameter processing of the present invention, realize the construction of a reasonable objective function, and obtain the purpose of high reconstruction accuracy.

[0070] 2. The present invention uses clean energy to power data collection, which can reduce investment costs, operating costs, and reduce limitations.

[0071] 3. The present invention performs frame extraction and calibration processing on image data through an image acquisition and processing device, and the exported data can be directly used for three-dimensional modeling, which effectively reduces the consumption of manpower and physical resources while improving processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A flow chart provided for the present invention;

[0073] Figure 2 An overall diagram of the vehicle-mounted auxiliary device provided by the present invention;

[0074] Figure 3 A schematic diagram of the structure of the main control chip provided by the present invention;

[0075] Figure 4 A detailed diagram of the vehicle-mounted auxiliary device provided by the present invention;

[0076] Figure 5 A detailed diagram of the PCB board of the vehicle-mounted auxiliary equipment provided by the present invention;

[0077] Figure 6 A diagram of electrical components of an embedded device camera provided by the present invention;

[0078] Figure 7 The present invention provides Figure 4 Electrical components diagram of the middle display screen;

[0079] Figure 8 This is the model output diagram provided for this embodiment. DETAILED DESCRIPTION

[0080] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] First, assemble the vehicle auxiliary equipment, including the energy supply device, storage device, and photography device (such as Figure 6 ) and an image acquisition and processing device, such as Figure 1 As shown. Through the assembled vehicle auxiliary equipment, such as Figure 2 As shown, it includes an energy supply device, a storage device, a photographic device and an image acquisition and processing device, and then completes the acquisition of road image data on this basis.

[0082] The energy supply device provides power for the entire vehicle-mounted auxiliary equipment to ensure the basic operation of the equipment. There are two ways to store energy. One is manual charging, that is, connecting the power supply through the USB interface to store electricity. However, in the case of field operations, it is often impossible to find a power source. Therefore, solar panels are added to the device as another way to store electricity. When one side’s power is exhausted, the other method can follow up in time to ensure that the remaining functions of the vehicle-mounted auxiliary equipment are uninterrupted and maintain the basic operation of the equipment.

[0083] The storage device is completed by auxiliary memory, including camera data, processed image data and final model data, which can be uploaded to the cloud server through the data storage unit, and the results can be obtained directly remotely, avoiding manual secondary processing.

[0084] The photography device is mainly composed of three wide-angle cameras and a controller. The cameras are fixed on both sides and the top of the vehicle through suction cups. The camera arm can rotate 90 to 180 degrees to obtain road image information at different heights, and the camera itself can rotate 0 to 180 degrees to capture image information of the roadbed and road surface on both sides. The rotation adjustment of the bracket arm can be achieved through buttons 1 and 2 on the controller display, and the rotation adjustment of the three cameras corresponds to buttons 3, 4, and 5 respectively. Button 6 can switch the camera for recording, and knob 7 can control the time interval for taking photos or recording according to the vehicle speed sensed by the acceleration sensor and the vehicle distance sensed by the infrared vehicle distance sensor. The image acquisition and processing device is the core device of this equipment, which mainly realizes a series of image processing and reconstruction of the final model through a built-in chip embedded with an algorithm. Detailed structure diagram of the entire device, such as Figure 4 and Figure 7 As shown, the PCB board is Figure 5 shown.

[0085] The main control chip set in the image acquisition and processing device is used to control the logic and construct an algorithm, and the relevant content in the field of computer vision and machine vision is used to restore three-dimensional geometric information from two-dimensional images. This process usually involves the estimation of camera parameters. Camera parameter estimation is the key to establishing the mapping relationship between two-dimensional images and three-dimensional targets and restoring three-dimensional geometric information. These parameters define the internal and external parameters of the camera, including focal length, principal point, distortion coefficient, and the position and orientation of the camera in the world coordinate system. In order to establish a mapping relationship between two-dimensional images and three-dimensional targets and restore three-dimensional geometric information, the least squares algorithm in the computer vision library OpenCV is used to minimize the difference between the three-dimensional points projected onto the two-dimensional image plane through the camera model and the corresponding points in the actual image. By constructing an objective function of the sum of the reprojection errors of all observed feature points, the optimal solution is found through this algorithm. The main control chip such as Figure 3 shown.

[0086] Before you start using the auxiliary device, you first need to initialize the main control chip. This includes loading the necessary drivers and configuring the chip's various parameters, such as resolution, frame rate, exposure time, etc., to ensure that the chip can work properly and meet the needs of specific applications. The auxiliary device has a built-in electrical component diagram, such as Figure 6 and 7As shown. Then, the original image is preprocessed using the built-in image processing function of the main control chip, which may include denoising, contrast enhancement, color correction and other operations to improve image quality. Among them, denoising and contrast enhancement are achieved through the Gaussian filtering algorithm. The Gaussian filtering is achieved by performing a convolution operation on the image. Its principle is based on the Gaussian function (normal distribution). The continuous Gaussian function is discretized into a Gaussian kernel, and then normalized to ensure that the sum of the Gaussian kernel elements is 1. Finally, the convolution operation is used to perform a weighted average of the Gaussian kernel and the pixel values ​​of each pixel point and its neighborhood in the image to achieve smoothing and image enhancement effects.

[0087]

[0088] Where: H i,j is the element of the Gaussian kernel, i and j are the row and column indices of the element respectively, and σ is the standard deviation, which determines the width of the Gaussian function, that is, the degree of blur.

[0089]

[0090] Where: I′(x,y) is the filtered image, I(x,y) is the original image, and H(i,j) is the Gaussian kernel.

[0091] The lens distortion correction and image data extraction and calibration are implemented in C++ programming language. The code is attached below:

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098]

[0099]

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106] The SFM-MVS (Structure from motion and multi-view stereo) algorithm embedded in the built-in chip is used to match images from multiple perspectives. It is necessary to restore the camera extrinsic parameters and describe the relationship that must be satisfied between the corresponding points of the two images based on the epipolar geometry constraints. Then, the relative motion parameters of the camera are obtained by decomposing the intrinsic matrix. The matching relationship between the newly added camera posture and the existing camera and the three-dimensional point cloud is optimized. Then, the coordinates of the three-dimensional points are calculated using triangulation method based on the matching points between multiple views and the known camera parameters. Finally, the camera parameters and three-dimensional structure are refined through global nonlinear optimization, such as removing noise, filling missing parts or smoothing surfaces to optimize model quality. Finally, the three-dimensional structure of the road scene is restored from the two-dimensional image data. The specific formulas are (4)-(8). The output model is as follows: Figure 8 shown.

[0107] Epipolar geometry constraints:

[0108] Where: x 1 and x 2 is the homogeneous coordinates of the matching point pair in the two images, E is the intrinsic matrix, which contains the relative rotation and translation information between the cameras, and T is the transposed matrix;

[0109] Decomposition of the eigenvalue matrix: E = U∑V T (5)

[0110] Where: U and V are orthogonal matrices, Σ is a diagonal matrix, R i and t i are the rotation and translation matrices of the i-th camera, P j is the projection matrix of the jth camera, X is a three-dimensional point;

[0111] Camera pose optimization: minR i ,t i ∑jeqi∑(x,X)||xP j (R i X+t i )|| 2 (6)

[0112] Where: jeqi represents the weight factor, which is used to measure the strength relationship between camera rotation, translation and projection in this formula;

[0113] Triangulate the coordinate point: X = argmin∑||x i-P i X|| 2 (7)

[0114] Where: x i is the image point in the i-th view, P i is the projection matrix of the i-th camera;

[0115] Nonlinear Optimization:

[0116] The present invention designs a new road 3D modeling device controlled by vehicle-mounted images. The device is guaranteed to operate continuously without being affected by external factors. The image data exported by the device can directly complete the 3D modeling. While reducing the photography cost and energy supply cost of the device, the accuracy of the road 3D modeling device is improved to achieve high-efficiency and high-benefit modeling.

[0117] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A vehicle-mounted auxiliary device for road three-dimensional modeling based on RGB image information, the vehicle-mounted auxiliary device includes a power supply device, a storage device, a photographing device and an image acquisition and processing device, and completes road image data acquisition through the photographing device, characterized in that: The following steps are involved: The main control chip in the image acquisition and processing device is used to control the logic and construct an algorithm. Based on computer vision and machine vision, a mapping relationship between a two-dimensional image and a three-dimensional target is established and three-dimensional geometric information is restored. The least squares algorithm in the computer vision library OpenCV is used to minimize the difference between the three-dimensional points projected onto the two-dimensional image plane through the camera model and the corresponding points in the actual image. The objective function of the sum of the reprojection errors of all observed feature points is constructed, and then the optimal solution is found through the algorithm. Initializing the main control chip and preprocessing the original image using the algorithm built into the main control chip; The preprocessing includes wide-angle camera distortion correction and wide-angle camera image data extraction and calibration processing; before the wide-angle camera distortion correction and wide-angle camera image data extraction and calibration processing, the wide-angle camera is calibrated first; The steps for wide-angle camera distortion correction are as follows: S11. Import libraries and define variables: Introduce the core functions of the OpenCV library and the standard input and output stream library, and use the cv and std namespaces to facilitate subsequent calls to OpenCV functions and C++ standard library functions; S12. Main function starts: Define the input and output file paths, where InputFile is the image path that needs to be distorted and OutputFile is the path where the rectified image is saved; S13. Read image: Use the imread function to read the input image and store it in sMatImg. If the image reading fails, that is, sMatImg is empty, an error message is output and -1 is returned to end the program. S14. Define camera internal parameters and distortion coefficients: Define the output image matrix, where sMatOutput is used to store the corrected image, and then define a 3x3 array cameraMat, which contains the focal length and principal point position of the camera; finally define a 1x5 array distCoef, which contains the distortion coefficient of the lens; S15. Set image size and matrix: Create a Size object image_size, set it to 1920x1080, convert the previously defined camera intrinsic parameters and distortion coefficients into OpenCV matrix form; then print the camera intrinsic parameter matrix and distortion coefficients; S16. Initialize the calibration map: Create a mapping matrix, where mapx and mapy are used to store the corrected pixel map, create a rotation matrix R and initialize it as the unit matrix; then initialize the correction map, call the initUndistortRectifyMap function, pass in the camera intrinsic parameters, distortion coefficients, rotation matrix and new camera matrix, this function will generate the mapping matrices mapx and mapy; S17. Get the optimized camera matrix: Call the getOptimalNewCameraMatrix function to generate an optimized camera matrix optimal_mat and output it; S18. Process the image and save the result: Resize the input image sMatImg to the target size and store it in sMatOutput. Then create a copy of new_image for subsequent processing. Use the remap function to correct the image distortion according to the previously generated mapping matrix. Finally, use imwrite to save the corrected image to the specified output path. The steps for wide-angle camera image data frame extraction calibration are as follows: S21. Import libraries and define variables: Introduce OpenCV library for image processing, iostream for input and output operations, vector for dynamic arrays, and fstream for file operations; omit the prefixes of standard namespaces and OpenCV namespaces to simplify the code; S22. Define the writeDataToFile function: First, write the floating-point data to the specified file and output it in a C-style array format; Filename is the file name to be written, Data is the data vector to be written, and dataNumPerLine is the number of data written per line; then open the file and check whether it is successful, write the definition and formatting content of the array, use a loop to traverse the data, and wrap the line whenever the number of elements written reaches dataNumPerLine; do not add a comma after the last element; close the file; S23. Define camera parameters and distortion coefficients: Where captureOriginM and previewOriginM are the intrinsic parameter matrices of the camera, Coeff is the distortion coefficient moment, captureM and previewM are the intrinsic parameter matrices for different resolutions, and R is the rotation matrix, which is initialized to the unit matrix; S24. Define image size: The resolution used to store captured and preview images; S25. Generate dedistortion and correction maps: Use the initUndistortRectifyMap function to calculate the dedistortion and correction mapping matrix; input the camera intrinsic parameters, distortion coefficient, rotation matrix, new intrinsic parameter matrix and image size, and output the dedistortion mapping matrix captureMapX, captureMapY, previewMapX, previewMapY; S26. Flattened mapping matrix: Create four one-dimensional floating-point vectors to store the dedistorted mapping data, and use the copyTo method to copy the mapping matrix into the flattened vectors; S27. Write to file: Call writeDataToFile function to write the data of preview map Y to the specified file; return 0 indicates that the program ends normally; The preprocessed image data is passed through the motion recovery structure multi-view stereo vision algorithm embedded in the main control chip to match images from multiple perspectives, and finally the three-dimensional structure of the road scene is restored from the two-dimensional image data.

2. The vehicle-mounted auxiliary device for road 3D modeling based on RGB image information according to claim 1, characterized in that: According to the epipolar geometry constraints, the camera extrinsic parameters are restored and the relationship that the corresponding points between the two images must satisfy is described. Then, the relative motion parameters of the camera are obtained by decomposing the intrinsic matrix. Then, the matching relationship between the newly added camera posture and the existing camera and the 3D point cloud is optimized. Then, the coordinates of the 3D points are calculated using the triangulation method through the matching points between multiple views and the known camera parameters. Finally, the camera parameters and 3D structure are refined through global nonlinear optimization, and the 3D structure of the road scene is finally restored from the 2D image data. The specific formula is as follows: Epipolar geometry constraints: Where: x1 and x2 are the homogeneous coordinates of the matching point pairs in the two images, E is the intrinsic matrix, which contains the relative rotation and translation information between the cameras, and T is the transposed matrix; Decomposition of the eigenvalue matrix: E = U∑V T (2) Where: U and V are orthogonal matrices, Σ is a diagonal matrix, R i and t i are the rotation and translation matrices of the i-th camera, P j is the projection matrix of the jth camera, X is a three-dimensional point; Camera pose optimization: min R i ,t i ∑jeqi∑(x,X)||xP j (R i X+t i )|| 2 (3) Where: jeqi represents the weight factor, which is used to measure the strength relationship between camera rotation, translation and projection in this formula; Triangulate the coordinate point: X = argmin∑||x i -P i X|| 2 (4) Where: x i is the image point in the i-th view, P i is the projection matrix of the i-th camera; Nonlinear Optimization:

Images