A processing method and system for an intelligent driving system, a medium, an intelligent driving system, and an intelligent driving vehicle

Abstract

This invention discloses a processing method, system, medium, in-vehicle device, and intelligent driving vehicle for intelligent driving systems, comprising: fixing a real camera at a predetermined installation position on the vehicle and constructing a real calibration scene in conjunction with a real calibration board; replicating a simulated calibration scene with reference to the real calibration scene; using the real camera to capture images of the real calibration board to obtain real images, and extracting the two-dimensional pixel coordinates of the corner points of the real calibration board from the real images; obtaining the three-dimensional spatial coordinates of the corner points of the virtual calibration board in the simulated calibration scene in the world coordinate system; inputting the two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera into a camera perspective projection model to solve for the camera extrinsic parameters; configuring the camera extrinsic parameters to a virtual camera in the simulated calibration scene, and using the virtual camera to capture images of the simulated test scene to obtain simulated test images; and using the simulated test images to process the intelligent driving system.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving system simulation technology, and in particular to a processing method, system, medium, intelligent driving system, and intelligent driving vehicle for intelligent driving systems. Background Technology

[0002] In the development of intelligent driving systems, simulation-based testing has become a core method for verifying the performance of perception algorithms, decision-making, planning, and control systems. Its key advantage lies in its ability to complete technical verification in a low-cost, high-efficiency, and high-safety manner early in the development process. As a crucial perception sensor in intelligent driving systems, the quality of the simulation results of vehicle-mounted cameras directly determines the effectiveness of the system testing. Ensuring a high degree of consistency between the images generated by the simulated camera and the real camera under the same physical position and orientation is a core technical challenge in intelligent driving system testing. This high consistency is specifically manifested in the consistency between the pixel coordinates of the calibration board corner points captured by the real camera and the corresponding corner point coordinates in the simulated image, as well as the uniformity of their shooting perspectives and the absence of tilt distortion. This is a crucial prerequisite for the practical application of intelligent driving systems.

[0003] In existing technologies, the position and attitude of the simulation camera are mainly set and fine-tuned by human experience. There is a lack of standardized debugging foundation, which makes it impossible to achieve high-precision parameter measurement. This can easily cause the external parameters of the simulation camera to be out of sync with the actual parameters of the real camera. As a result, there are significant differences between the simulation test images and the images taken by the real camera in the same physical position and attitude, which ultimately reduces the credibility of the test results of the intelligent driving system. Summary of the Invention

[0004] To address the technical problem of low reliability in test results of intelligent driving systems in existing technologies, this invention provides a processing method, system, medium, in-vehicle equipment, and intelligent driving vehicle for intelligent driving systems. It replicates the simulated calibration scene by referencing the real calibration scene, maintaining consistency between the two scenes. Based on the three-dimensional spatial coordinates of corner points in the simulated calibration scene and the two-dimensional pixel coordinates of corner points in the real calibration scene, combined with a camera perspective projection model, the camera extrinsic parameters are calculated. This improves the consistency between simulated test images and images captured by the real camera, fundamentally avoiding significant differences between the simulated test images and images captured by the real camera. It eliminates the problem of test result distortion caused by image deviation, effectively improving the reliability of intelligent driving test results.

[0005] To address the aforementioned technical problems, a first aspect of the present invention discloses a processing method for an intelligent driving system, the method comprising: A real camera is fixed at a predetermined installation position on the vehicle, and a real calibration scenario is constructed by combining it with a real calibration board. The simulation calibration scenario is replicated with reference to the real calibration scenario to make the simulation calibration scenario consistent with the real calibration scenario; The real camera is used to capture a real image of the real calibration board, and the two-dimensional pixel coordinates of the corner points of the real calibration board are extracted from the real image. Obtain the three-dimensional spatial coordinates of the corner points of the virtual calibration board in the world coordinate system in the simulation calibration scenario; wherein, the corner points of the virtual calibration board correspond one-to-one with the corner points of the real calibration board; The two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera are input into the camera perspective projection model to solve for the camera extrinsic parameters. Configure the camera extrinsic parameters to the virtual camera in the simulation calibration scene, and use the virtual camera to take pictures of the simulation test scene to obtain simulation test images; The intelligent driving system is tested using the simulated test images; after the test is completed, the intelligent driving system is deployed in the vehicle for application.

[0006] Optionally, the step of replicating the simulation calibration scenario based on the real calibration scenario to make the simulation calibration scenario consistent with the real calibration scenario specifically includes: The simulation calibration scene is replicated at a 1:1 scale with reference to the real calibration scene to make the simulation calibration scene consistent with the real calibration scene.

[0007] Optionally, the reference real calibration scene is replicated at a 1:1 scale to make the simulation calibration scene consistent with the real calibration scene, specifically including: Determine the calibration parameters in the reference real calibration scenario; wherein, the calibration parameters include: world coordinate system, calibration plate position and size parameters, camera intrinsic parameters and lens distortion parameters; The simulation calibration scene is replicated at a 1:1 scale according to the calibration parameters, so that the world coordinate system, the calibration plate position and size parameters, the camera intrinsic parameters and the lens distortion parameters of the simulation calibration scene and the real calibration scene are consistent.

[0008] Optionally, the step of inputting the two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera into the camera perspective projection model to solve for the camera extrinsic parameters specifically includes: The two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera are input into the camera perspective projection model: s×[u; v; 1] = K×[R|t]×[X; Y; Z;1], resulting in multiple sets of extrinsic parameter solution equations. The multiple sets of extrinsic parameter solution equations are solved jointly to obtain the camera extrinsic parameters; where s represents the scale factor, (u, v) represents the two-dimensional pixel coordinates of the corner points of the real calibration board, (X, Y, Z) represents the three-dimensional spatial coordinates of the corner points of the virtual calibration board, K represents the camera intrinsic parameters of the real camera, [R|t] represents the camera extrinsic parameters, R is the rotation matrix, and t is the translation vector.

[0009] Optionally, the camera extrinsic parameters are obtained by jointly solving the multiple sets of extrinsic parameter equations, specifically including: By jointly solving the multiple sets of extrinsic parameter equations, the target candidate solutions for the camera extrinsic parameters are obtained; The target candidate solution is configured into the virtual camera, and the virtual camera is used to capture the virtual calibration board to obtain a virtual calibration image; Extract the two-dimensional pixel coordinates of the corner points of the virtual calibration board from the virtual calibration image; The reprojection error is calculated using the two-dimensional pixel coordinates of the corner points of the virtual calibration board and the two-dimensional pixel coordinates of the corner points of the real calibration board. The target candidate solution is iteratively optimized with the goal of minimizing the reprojection error until the target candidate solution with the minimum reprojection error is obtained. The target candidate solution with the minimum reprojection error is then used as the camera extrinsic parameter.

[0010] Optionally, the step of calculating the reprojection error using the two-dimensional pixel coordinates of the corner points of the virtual calibration board and the two-dimensional pixel coordinates of the corner points of the real calibration board specifically includes: Calculate the Euclidean distance between the two-dimensional pixel coordinates of the corner point of the virtual calibration board and the two-dimensional pixel coordinates of the corner point of the real calibration board; The average error or root mean square error is calculated using the Euclidean distances between all corner points. The average error or the root mean square error is used as the reprojection error.

[0011] A second aspect of the present invention discloses a processing system for an intelligent driving system, the system comprising: The module is used to fix the real camera to the predetermined installation position on the vehicle and build a real calibration scene in combination with the real calibration board. The replication module is used to replicate the simulation calibration scenario with reference to the real calibration scenario, so that the simulation calibration scenario and the real calibration scenario are consistent; The extraction module is used to capture a real image of the real calibration board using the real camera, and to extract the two-dimensional pixel coordinates of the corner points of the real calibration board from the real image. The acquisition module is used to acquire the three-dimensional spatial coordinates of the corner points of the virtual calibration board in the simulation calibration scene in the world coordinate system; wherein, the corner points of the virtual calibration board and the corner points of the real calibration board correspond one-to-one; The solution module is used to input the two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera into the camera perspective projection model, and solve for the camera extrinsic parameters. The configuration module is used to configure the camera extrinsic parameters to the virtual camera of the simulation calibration scene, and use the virtual camera to take pictures of the simulation test scene to obtain the simulation test image; The testing module is used to test the intelligent driving system using the simulated test images; the intelligent driving system is then deployed in a vehicle after the test is completed.

[0012] A third aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the above-described method.

[0013] A fourth aspect of the present invention discloses an intelligent driving system, which is obtained by testing using the method described in the first aspect.

[0014] A fifth aspect of the present invention discloses an intelligent driving vehicle, including the intelligent driving system as described in the third aspect.

[0015] Through one or more technical solutions of the present invention, the present invention has the following beneficial effects or advantages: The processing flow for the intelligent driving system in this technical solution first involves fixing a real camera at a predetermined installation position on the vehicle and constructing a real calibration scene. Simultaneously, a simulated calibration scene identical to the real calibration scene is replicated, laying the scene foundation for accurate matching of the extrinsic parameters of the virtual and real cameras and ensuring the geometric consistency of the virtual and real calibration scenes. Next, the two-dimensional pixel coordinates of the corner points of the real calibration board are extracted by capturing images with the real camera. Simultaneously, the three-dimensional spatial coordinates of the virtual calibration board corner points in the simulated calibration scene, corresponding one-to-one with the real corner points, are obtained. Combined with the key imaging parameter of the real camera's intrinsic parameters, these three types of parameters are input into the camera. Perspective projection models are used to solve extrinsic parameters, providing a reliable basis for accurate solutions to camera extrinsic parameters from the perspectives of data input and model calculation. The accurate camera extrinsic parameters obtained are then configured into the virtual camera in the simulation calibration scene, ensuring that the pose of the virtual camera is completely consistent with that of the real camera fixed at the predetermined installation position on the vehicle. This ensures that the shooting perspectives of the two cameras are highly consistent under the same physical position and posture, fundamentally avoiding significant differences between simulated images and images captured by real cameras. This eliminates the problem of test result distortion caused by image deviation and effectively improves the credibility of test results for intelligent driving systems.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart of a processing method for an intelligent driving system according to an embodiment of the present invention is shown; Figure 2 A schematic diagram of a processing system for an intelligent driving system according to an embodiment of the present invention is shown. Detailed Implementation

[0018] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0019] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0020] Firstly, such as Figure 1 As shown, the processing method for intelligent driving systems provided in this embodiment of the invention includes at least the following steps: The S101 uses a real camera fixed at a predetermined installation position on the vehicle, combined with a real calibration board to construct a real calibration scenario.

[0021] In the real world, a real camera to be calibrated is fixed at a predetermined installation position on the vehicle, and a real calibration board with a known pattern (such as a checkerboard or dot matrix) is placed in front of the vehicle to construct a real calibration scene.

[0022] The calibration parameters involved in a real calibration scenario include: world coordinate system, calibration board position and size parameters, camera intrinsic parameters, and lens distortion parameters. The spatial positions of the calibration board and camera are defined and described in the world coordinate system. Calibration board position and size parameters include the calibration board's position and geometric dimensions. The real calibration board uses a checkerboard grid or a dot matrix, with corner points corresponding to the precise intersection of adjacent black and white squares in the checkerboard or the center of a circular target in the dot matrix. The real camera's intrinsic parameters include focal length pixel values, principal point pixel coordinates, and other parameters. Lens distortion parameters include radial distortion coefficients and tangential distortion coefficients.

[0023] S102, the simulation calibration scene is replicated with reference to the real calibration scene so that the simulation calibration scene is consistent with the real calibration scene.

[0024] Specifically, the simulation calibration scene is replicated at a 1:1 scale with reference to the real calibration scene, so that the simulation calibration scene is consistent with the real calibration scene.

[0025] First, determine the calibration parameters in the reference real calibration scenario, including: world coordinate system, calibration plate position and size parameters, camera intrinsic parameters, and lens distortion parameters.

[0026] Secondly, the simulation calibration scene is replicated at a 1:1 scale according to the calibration parameters, so that the world coordinate system, the calibration plate position and size parameters, the camera intrinsic parameters and the lens distortion parameters of the simulation calibration scene and the real calibration scene are consistent.

[0027] Specifically, firstly, a world coordinate system for the simulation environment is constructed based on the real-world coordinate system of the actual calibration scene, ensuring that the origin, coordinate axis directions, and units of measurement are completely consistent between the two, thus establishing a unified spatial reference with a 1:1 replication. Secondly, a virtual calibration board model is created at a 1:1 scale according to the physical size and pattern type of the real calibration board, and it is precisely placed in the simulation world coordinate system at the exact same three-dimensional spatial coordinate position as the real calibration board in the real-world coordinate system, while maintaining the same placement posture, ensuring that the position and size parameters of the virtual calibration board are without deviation from those of the real calibration board. Finally, a virtual camera is created, and the pre-calibrated camera intrinsic parameters and lens distortion parameters, which are completely consistent with those of the real camera, are configured into the virtual camera to ensure that the hardware imaging parameters of the virtual camera are consistent with those of the real camera.

[0028] This ensures complete equivalence between the simulated calibration scenario and the real calibration scenario in terms of spatial reference, calibration board state, and camera imaging characteristics, laying a core foundation for subsequent solution of camera extrinsic parameters based on the corner coordinates of the virtual and real calibration boards and ensuring consistency between simulated test images and real vehicle-mounted camera imaging.

[0029] S103, use the real camera to take a picture of the real calibration board to obtain a real image, and extract the two-dimensional pixel coordinates of the corner points of the real calibration board from the real image.

[0030] In this process, a real camera captures an image of the real calibration board, obtaining a true image containing all corner points of the board. A corner detection algorithm is then used to detect corner points in this image, yielding the two-dimensional pixel coordinates of the corner points. Examples of corner detection algorithms include the Shi-Tomasi and Harris algorithms, but these are not considered limitations.

[0031] S104, Obtain the three-dimensional spatial coordinates of the corner points of the virtual calibration board in the simulation calibration scene in the world coordinate system.

[0032] The virtual calibration board is constructed based on the geometric dimensions and spatial position parameters of the real calibration board. The spatial position of each corner point has been defined in the modeling stage and belongs to the known parameters in the simulation environment. Therefore, the three-dimensional spatial coordinates (X,Y,Z) of all corner points of the virtual calibration board in the world coordinate system can be obtained directly from the simulation system.

[0033] S105, input the two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera into the camera perspective projection model, and solve for the camera extrinsic parameters.

[0034] Specifically, the two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera are input into the camera perspective projection model: s×[u;v;1]=K×[R|t]×[X;Y;Z;1], to obtain multiple sets of extrinsic parameter solution equations. The multiple sets of extrinsic parameter solution equations are solved jointly to obtain the camera extrinsic parameters.

[0035] Where s represents the scale factor, (u, v) represents the two-dimensional pixel coordinates of the corner point of the real calibration board, (X, Y, Z) represents the three-dimensional spatial coordinates of the corner point of the virtual calibration board, K represents the camera intrinsic parameters of the real camera in matrix form, [R|t] represents the camera extrinsic parameters, R is the rotation matrix, and t is the translation vector.

[0036] The coordinate data of a single corner point can correspond to a set of extrinsic parameter solution equations. By solving the multiple sets of extrinsic parameter solution equations formed by all corner points, the camera extrinsic parameters of XXX can be obtained.

[0037] The specific solution process is as follows: S201, solve the multiple sets of extrinsic parameter equations together to obtain the target candidate solution of the camera extrinsic parameters.

[0038] The solution algorithm can employ PnP (Perspective-n-Point) related algorithms, such as EPnP (Efficient Perspective-n-Point), UPnP (Uncalibrated Perspective-n-Point), and AP3P (Accurate Perspective-3-Point). Specifically, the EPnP algorithm is suitable for scenarios with a sufficient number of calibration board corner points, the UPnP algorithm is suitable for scenarios with imaging noise interference, and the AP3P algorithm is suitable for scenarios requiring rapid preliminary calculations.

[0039] The extrinsic parameter equations corresponding to all corner points are solved by combining them using a preset algorithm, and the target candidate solution of the camera extrinsic parameters is output. The target candidate solution includes the rotation matrix R and the translation vector t.

[0040] S202, the target candidate solution is configured into the virtual camera, and the virtual camera is used to photograph the virtual calibration board to obtain a virtual calibration image.

[0041] The configuration operation includes synchronizing the rotation matrix R and translation vector t in the target candidate solution to the extrinsic parameter configuration module of the virtual camera, so that the spatial pose of the virtual camera is consistent with the initial pose of the real camera.

[0042] During the shooting process, the imaging parameters of the virtual camera (including resolution, focal length, distortion coefficient, and exposure parameters) are completely matched with the imaging parameters of the real camera. Furthermore, the size, corner distribution density, and spatial placement of the virtual calibration board correspond one-to-one with the real calibration board, ensuring that the imaging effect of the virtual calibration image is comparable to that of the real calibration image.

[0043] The virtual calibration image is generated by rendering using a virtual simulation module. The image format is consistent with that of the real calibration image, including but not limited to JPG, PNG, and RAW formats.

[0044] S203, extract the two-dimensional pixel coordinates of the corner points of the virtual calibration board from the virtual calibration image.

[0045] Specifically, the corner detection algorithm, which is consistent with the corner extraction of the real calibration board, is used to extract the two-dimensional pixel coordinates of the corners of the virtual calibration board.

[0046] S204, calculate the reprojection error using the two-dimensional pixel coordinates of the corner points of the virtual calibration board and the two-dimensional pixel coordinates of the corner points of the real calibration board.

[0047] In calculating the reprojection error, the Euclidean distance between the two-dimensional pixel coordinates of the virtual calibration board corner points and the two-dimensional pixel coordinates of the real calibration board corner points is calculated. Specifically, the virtual and real calibration board corner points are assigned corresponding numbers, and for each pair of virtual-real corner points with the same number, the Euclidean distance between them is calculated using their two-dimensional pixel coordinates. The average error or root mean square error is calculated using the Euclidean distances of all corner points; the average error or the root mean square error is then used as the reprojection error.

[0048] S205, iteratively optimize the target candidate solution with the goal of minimizing the reprojection error until the target candidate solution with the minimum reprojection error is obtained, and use the target candidate solution with the minimum reprojection error as the camera extrinsic parameter.

[0049] Specifically, it is determined whether the reprojection error meets the preset convergence conditions. The preset convergence conditions are: the reprojection error is less than a preset threshold or the change in reprojection error between two adjacent iterations is less than the convergence threshold. If either condition is met, the target candidate solution is determined to be optimized to its optimum, and no further iteration is needed. If either or both of the above conditions are not met, the pitch angle, yaw angle, and roll angle of the rotation matrix R corresponding to the target candidate solution, as well as the x, y, and z axis coordinates of the translation vector t, are adjusted. The adjustment step size is dynamically adjusted according to the error magnitude; the larger the error, the larger the adjustment step size, and the smaller the error, the smaller the adjustment step size. The step size range is 0.001 to 0.01, resulting in a new target candidate solution. Steps S202 to S204 are repeated for the new target candidate solution until the reprojection error meets the preset convergence conditions.

[0050] S106, Configure the camera extrinsic parameters to the virtual camera of the simulation calibration scene, and use the virtual camera to take pictures of the simulation test scene to obtain the simulation test image.

[0051] During configuration, the final determined camera extrinsic parameters (including rotation matrix R and translation vector t) are synchronized to the virtual camera extrinsic parameter control module of the simulation calibration scene.

[0052] The simulation test scenarios are typical road conditions that simulate the actual application of intelligent vehicle-mounted equipment, including but not limited to urban road scenarios, highway scenarios, rural road scenarios, low light night scenarios, and adverse weather scenarios such as rain, snow, and fog.

[0053] During the shooting process, the imaging parameters of the virtual camera are completely consistent with those of the real camera. The simulation rendering engine generates simulated test images that conform to the real imaging effect. The image format is consistent with the output format of the real vehicle camera, including but not limited to JPG, PNG, and RAW formats.

[0054] S107, The intelligent driving system is tested using the simulated test image.

[0055] The intelligent driving system, once tested, will be deployed in vehicles for application.

[0056] The simulated test images are input into each functional module of the intelligent driving system according to the image input format requirements of the real vehicle intelligent driving system, and the tests of each functional module of the intelligent driving system and the full-process joint test are carried out in sequence.

[0057] Taking the environmental perception module of the intelligent driving system as an example, the pixel coordinates and contour ranges of environmental targets such as lane lines, traffic signs, pedestrians, and vehicles are extracted from the simulated test image. These are then compared with the preset ground truth data of the targets in the simulated test image to calculate the extraction deviation and verify the perception accuracy. If the accuracy is insufficient, the cause of the perception deviation is first identified, such as an unreasonable lane line feature extraction threshold or insufficient adaptability of the target contour recognition algorithm to lighting and occlusion scenes in the simulated test image. Then, the algorithm parameters of the environmental perception module are adjusted accordingly, such as the feature extraction threshold and contour fitting coefficient, or the target detection and feature extraction logic is optimized. After adjustment, the simulated test image is re-input into the module for perception accuracy retesting. This iterative optimization is repeated until the perception accuracy reaches the accuracy standard of the intelligent driving system.

[0058] The intelligent driving system, once tested, will be deployed in vehicles for application.

[0059] The above describes the processing flow for the intelligent driving system in this technical solution. First, a real camera is fixed at the predetermined installation position on the vehicle to construct a real calibration scene. Simultaneously, a simulated calibration scene that is completely identical to the real calibration scene is replicated, laying the scene foundation for accurate matching of the extrinsic parameters of the virtual and real cameras and ensuring the geometric consistency of the virtual and real calibration scenes. Then, the two-dimensional pixel coordinates of the corner points of the real calibration board are extracted by capturing images with the real camera. At the same time, the three-dimensional spatial coordinates of the virtual calibration board corner points in the simulated calibration scene, which correspond one-to-one with the real corner points, are obtained. Combined with the key imaging parameter of the real camera's intrinsic parameters, the above three types of parameters are input together. The camera perspective projection model is used to solve for extrinsic parameters, providing a reliable basis for accurate solutions to camera extrinsic parameters from the perspectives of data input and model calculation. The accurate camera extrinsic parameters obtained are then configured into the virtual camera in the simulation calibration scene, ensuring that the pose of the virtual camera is completely consistent with that of the real camera fixed at the predetermined installation position on the vehicle. This ensures that the shooting angles of the two cameras are highly consistent under the same physical position and posture, fundamentally avoiding significant differences between the simulated images and the images captured by the real camera. This eliminates the problem of test result distortion caused by image deviation and effectively improves the credibility of test results for intelligent driving systems.

[0060] Furthermore, the entire process only requires referencing real cameras and calibration boards to replicate the simulated calibration scenario, resulting in a high degree of automation. It eliminates the need for expensive precision measuring equipment and engineers to repeatedly manually adjust parameters, significantly shortening calibration time and reducing costs. Moreover, this solution is not limited by specific vehicle models or fixed camera installation locations, allowing for rapid application to calibration tasks across different vehicle models and camera positions, demonstrating excellent versatility and scalability.

[0061] Secondly, based on the same inventive concept as the processing method for intelligent driving systems provided in the first aspect of the embodiments described above, the present invention also provides a processing system for intelligent driving systems, see below. Figure 2 The system includes: Module 201 is used to fix the real camera to the predetermined installation position on the vehicle and construct a real calibration scene in conjunction with the real calibration board; The replication module 202 is used to replicate the simulation calibration scene with reference to the real calibration scene, so that the simulation calibration scene and the real calibration scene are consistent; Extraction module 203 is used to capture a real image of the real calibration board using the real camera, and extract the two-dimensional pixel coordinates of the corner points of the real calibration board from the real image; The acquisition module 204 is used to acquire the three-dimensional spatial coordinates of the corner points of the virtual calibration board in the simulation calibration scene in the world coordinate system; wherein, the corner points of the virtual calibration board and the corner points of the real calibration board correspond one-to-one. The solving module 205 is used to input the two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera into the camera perspective projection model, and solve the camera extrinsic parameters from the camera perspective projection model. The configuration module 206 is used to configure the camera extrinsic parameters to the virtual camera of the simulation calibration scene, and use the virtual camera to take pictures of the simulation test scene to obtain the simulation test image; The test module 207 is used to test the intelligent driving system using the simulated test images; wherein, after the test is completed, the intelligent driving system is deployed in the vehicle for application.

[0062] It should be noted that the specific methods by which each module performs operations in the processing system for the intelligent driving system provided in the embodiments of the present invention have been described in detail in the method embodiments provided in the first aspect above. The specific implementation process can be referred to the method embodiments provided in the first aspect above, and will not be described in detail here.

[0063] Thirdly, based on the same inventive concept as the processing method for intelligent driving systems provided in the first aspect of the embodiments described above, the embodiments of the present invention also disclose a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0064] Fourthly, based on the same inventive concept as the processing method for the intelligent driving system provided in the first aspect of the embodiments described above, the present invention also discloses an intelligent driving system, which is obtained by testing the method described in the first aspect.

[0065] Fifthly, based on the same inventive concept as the processing method for the intelligent driving system provided in the first aspect of the embodiments described above, the present invention also discloses an intelligent driving vehicle, including the intelligent driving system as described in the third aspect.

[0066] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0067] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A processing method for an intelligent driving system, characterized in that, The method includes: A real camera is fixed at a predetermined installation position on the vehicle, and a real calibration scenario is constructed by combining it with a real calibration board. The simulation calibration scenario is replicated with reference to the real calibration scenario to make the simulation calibration scenario consistent with the real calibration scenario; The real camera is used to capture a real image of the real calibration board, and the two-dimensional pixel coordinates of the corner points of the real calibration board are extracted from the real image. Obtain the three-dimensional spatial coordinates of the corner points of the virtual calibration board in the world coordinate system in the simulation calibration scenario; wherein, the corner points of the virtual calibration board correspond one-to-one with the corner points of the real calibration board; The two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera are input into the camera perspective projection model to solve for the camera extrinsic parameters. Configure the camera extrinsic parameters to the virtual camera in the simulation calibration scene, and use the virtual camera to take pictures of the simulation test scene to obtain simulation test images; The intelligent driving system is tested using the simulated test images; after the test is completed, the intelligent driving system is deployed in the vehicle for application.

2. The method as described in claim 1, characterized in that, The step of replicating the simulation calibration scene by referring to the real calibration scene, so that the simulation calibration scene is consistent with the real calibration scene, specifically includes: The simulation calibration scene is replicated at a 1:1 scale with reference to the real calibration scene to make the simulation calibration scene consistent with the real calibration scene.

3. The method as described in claim 2, characterized in that, The simulation calibration scene is replicated at a 1:1 scale with the real calibration scene to ensure consistency between the simulation calibration scene and the real calibration scene. Specifically, this includes: Determine the calibration parameters in the reference real calibration scenario; wherein, the calibration parameters include: world coordinate system, calibration plate position and size parameters, camera intrinsic parameters and lens distortion parameters; The simulation calibration scene is replicated at a 1:1 scale according to the calibration parameters, so that the world coordinate system, the calibration plate position and size parameters, the camera intrinsic parameters and the lens distortion parameters of the simulation calibration scene and the real calibration scene are consistent.

4. The method as described in claim 1, characterized in that, The step of inputting the two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera into the camera perspective projection model to solve for the camera extrinsic parameters specifically includes: The two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera are input into the camera perspective projection model: s×[u; v; 1] = K×[R|t] ×[X; Y; Z;1], resulting in multiple sets of extrinsic parameter solution equations. The multiple sets of extrinsic parameter solution equations are solved jointly to obtain the camera extrinsic parameters; where s represents the scale factor, (u, v) represents the two-dimensional pixel coordinates of the corner points of the real calibration board, (X, Y, Z) represents the three-dimensional spatial coordinates of the corner points of the virtual calibration board, K represents the camera intrinsic parameters of the real camera, [R|t] represents the camera extrinsic parameters, R is the rotation matrix, and t is the translation vector.

5. The method as described in claim 4, characterized in that, By jointly solving the multiple sets of extrinsic parameter equations, the camera extrinsic parameters are obtained, specifically including: By jointly solving the multiple sets of extrinsic parameter equations, the target candidate solutions for the camera extrinsic parameters are obtained; The target candidate solution is configured into the virtual camera, and the virtual camera is used to capture the virtual calibration board to obtain a virtual calibration image; Extract the two-dimensional pixel coordinates of the corner points of the virtual calibration board from the virtual calibration image; The reprojection error is calculated using the two-dimensional pixel coordinates of the corner points of the virtual calibration board and the two-dimensional pixel coordinates of the corner points of the real calibration board. The target candidate solution is iteratively optimized with the goal of minimizing the reprojection error until the target candidate solution with the minimum reprojection error is obtained. The target candidate solution with the minimum reprojection error is then used as the camera extrinsic parameter.

6. The method as described in claim 5, characterized in that, The calculation of the reprojection error using the two-dimensional pixel coordinates of the corner points of the virtual calibration board and the two-dimensional pixel coordinates of the corner points of the real calibration board specifically includes: Calculate the Euclidean distance between the two-dimensional pixel coordinates of the corner point of the virtual calibration board and the two-dimensional pixel coordinates of the corner point of the real calibration board; The average error or root mean square error is calculated using the Euclidean distances between all corner points. The average error or the root mean square error is used as the reprojection error.

7. A processing system for an intelligent driving system, characterized in that, The system includes: The module is used to fix the real camera to the predetermined installation position on the vehicle and build a real calibration scene in combination with the real calibration board. The replication module is used to replicate the simulation calibration scenario with reference to the real calibration scenario, so that the simulation calibration scenario and the real calibration scenario are consistent; The extraction module is used to capture a real image of the real calibration board using the real camera, and to extract the two-dimensional pixel coordinates of the corner points of the real calibration board from the real image. The acquisition module is used to acquire the three-dimensional spatial coordinates of the corner points of the virtual calibration board in the simulation calibration scene in the world coordinate system; wherein, the corner points of the virtual calibration board and the corner points of the real calibration board correspond one-to-one; The solution module is used to input the two-dimensional pixel coordinates of the corner points of the real calibration board, the three-dimensional spatial coordinates of the corner points of the virtual calibration board, and the camera intrinsic parameters of the real camera into the camera perspective projection model, and solve for the camera extrinsic parameters. The configuration module is used to configure the camera extrinsic parameters to the virtual camera of the simulation calibration scene, and use the virtual camera to take pictures of the simulation test scene to obtain the simulation test image; The testing module is used to test the intelligent driving system using the simulated test images; the intelligent driving system is then deployed in a vehicle after the test is completed.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-6.

9. An intelligent driving system, characterized in that, The intelligent driving system is obtained by testing using the method described in any one of claims 1-6.

10. An intelligent driving vehicle, comprising the processing system as described in claim 9.

Images