Parameter calibration method and device of virtual camera, electronic equipment and storage medium

Abstract

This application provides a method, apparatus, electronic device, and storage medium for calibrating the parameters of a virtual camera. The method includes: acquiring real image information of a preset real calibration location using a real camera; acquiring virtual image information of a preset virtual calibration location using a virtual camera, wherein the preset real calibration location and the preset virtual calibration location correspond one-to-one; parsing the real image information and the virtual image information respectively to obtain the real calibration pixel coordinates and the virtual calibration pixel coordinates; calculating the target parameters of the virtual camera based on the real calibration pixel coordinates and the virtual calibration pixel coordinates; and optimizing the virtual camera parameters based on the target parameters. Therefore, the calibrated virtual camera can directly output high-fidelity data, reducing the workload of repeated real-vehicle data acquisition and manual calibration, shortening the ADAS algorithm iteration cycle, reducing R&D costs, and enhancing the universality and reusability of the same set of simulation data across multiple platforms and vehicle models.

Description

Technical Field

[0001] This application relates to the field of intelligent driving function development simulation technology, and in particular to a method, apparatus, electronic device and storage medium for parameter calibration of a virtual camera. Background Technology

[0002] In the development of ADAS (Advanced Driver-Assistance Systems), virtual simulation has become a core means to shorten algorithm iteration cycles and reduce road testing costs. As a digital sensor in the simulation environment, the fidelity of the output image from a virtual camera directly determines whether the perception algorithm can be effectively trained and validated on the simulation side. Therefore, improving the calibration accuracy of virtual cameras has key engineering value for accelerating the implementation of advanced intelligent driving functions. The industry generally adopts a geometric alignment method, first using a calibration board to determine the intrinsic, extrinsic, and distortion coefficients of the real camera in a real-world scenario, and then directly writing these parameters into the virtual camera.

[0003] However, due to the inherent differences between the rendering pipeline and lighting model of the simulation engine and the real physical world, simply copying parameters will still result in corner offsets and distortion residuals at the pixel level between virtual and real images. To eliminate this deviation, developers can only repeatedly collect data and manually fine-tune it during the actual vehicle stage, resulting in long calibration cycles, poor cross-platform portability, and high costs of repeated development. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for calibrating the parameters of a virtual camera. The aim is to use the pixel coordinates of a real image as the true value to reverse-optimize the intrinsic parameters and distortion coefficients of the virtual camera, so that the simulation output is aligned with the actual shooting result at the pixel level. This improves the problems of insufficient simulation fidelity and high cost of repeated corrections on the actual vehicle caused by lens distortion error and rendering pipeline error remaining in traditional geometric calibration.

[0005] To achieve the above objectives, the first aspect of this application proposes a method for parameter calibration of a virtual camera, comprising: acquiring real image information of a preset real calibration location using a real camera; acquiring virtual image information of a preset virtual calibration location using a virtual camera, wherein the preset real calibration location and the preset virtual calibration location correspond one-to-one; parsing the real image information and the virtual image information respectively to obtain the real calibration pixel coordinates and the virtual calibration pixel coordinates; calculating the target parameters of the virtual camera based on the real calibration pixel coordinates and the virtual calibration pixel coordinates, and optimizing the virtual camera parameters based on the target parameters.

[0006] The virtual camera parameter calibration method described in this application enables the calibrated virtual camera to directly output high-fidelity data, reducing the workload of repeated data collection and manual calibration on real vehicles, shortening the iteration cycle of ADAS algorithms, reducing R&D costs, and enhancing the universality and reusability of the same set of simulation data across multiple platforms and vehicle models.

[0007] A second aspect of this application provides a parameter calibration device for a virtual camera, comprising: a first acquisition module for acquiring real image information of a preset real calibration location using a real camera; a second acquisition module for acquiring virtual image information of a preset virtual calibration location using a virtual camera, wherein the preset real calibration location and the preset virtual calibration location correspond one-to-one; a parsing module for parsing the real image information and the virtual image information respectively to obtain the real calibration pixel coordinates and the virtual calibration pixel coordinates; and a calibration module for calculating the target parameters of the virtual camera based on the real calibration pixel coordinates and the virtual calibration pixel coordinates, and optimizing the virtual camera parameters based on the target parameters.

[0008] According to the parameter calibration device of the virtual camera in the embodiments of this application, the calibrated virtual camera can directly output high-fidelity data, reduce the workload of repeated data collection and manual calibration of real vehicles, shorten the iteration cycle of ADAS algorithm, reduce R&D costs, and enhance the universality and reusability of the same set of simulation data across multiple platforms and multiple vehicle models.

[0009] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned parameter calibration method for a virtual camera.

[0010] According to the electronic device of the present application, when the processor executes the program stored in the memory, it implements the aforementioned virtual camera parameter calibration method. The calibrated virtual camera can directly output high-fidelity data, reducing the workload of repeated data collection and manual calibration on real vehicles, shortening the iteration cycle of ADAS algorithms, reducing R&D costs, and enhancing the universality and reusability of the same set of simulation data across multiple platforms and vehicle models.

[0011] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor for use in the aforementioned virtual camera parameter calibration method.

[0012] According to the computer-readable storage medium of the embodiments of this application, the computer program stored thereon is executed to implement the aforementioned virtual camera parameter calibration method. The calibrated virtual camera can directly output high-fidelity data, reducing the workload of repeated data collection and manual calibration on real vehicles, shortening the iteration cycle of ADAS algorithms, reducing R&D costs, and enhancing the universality and reusability of the same set of simulation data across multiple platforms and vehicle models. Attached Figure Description

[0013] Figure 1 This is a flowchart of a virtual camera parameter calibration method provided in some embodiments of this application; Figure 2 This is a flowchart of a virtual camera parameter calibration method provided in a specific embodiment of this application; Figure 3 This is a block diagram of a parameter calibration device for a virtual camera provided in some embodiments of this application; Figure 4 This is a block diagram of an electronic device provided in some embodiments of this application. Detailed Implementation

[0014] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0015] It should be noted that the virtual camera parameter calibration method of this application is applied to a specific parameter calibration system, which includes a real calibration location and a virtual calibration location. For example, the real calibration location includes a vehicle containing a real camera and a real-world calibration location.

[0016] The real camera serves as the imaging unit for the ADAS system to perceive the external environment. The real-world calibration site consists of several calibration plates arranged in specific positions and is used to calibrate the real camera.

[0017] It should be noted that the ADAS in this application embodiment is an on-board electronic control system that integrates radar, camera, ultrasonic waves and domain controller. It can perceive the environment in real time while driving and automatically intervene in the lateral / longitudinal movement of the vehicle to realize functions such as lane keeping, automatic emergency braking, adaptive cruise, traffic sign recognition, blind spot monitoring, and parking assistance, thereby reducing the driver's burden, reducing the risk of collision and improving driving comfort and fuel economy. With the upgrading of regulations and the popularization of intelligent driving, ADAS has expanded from high-end models to mainstream passenger cars and commercial vehicles, becoming the core cornerstone for realizing L2 and above autonomous driving.

[0018] In some embodiments, the real calibration site can be further configured with various sensors and target devices according to actual testing needs. For example, the real calibration site may further include corner radars distributed at the four corners of the site for millimeter-wave radar detection and calibration; simultaneously, various types of visual targets can be set up to meet different visual perception needs, including camera targets for front-view or rear-view camera calibration, telephoto camera targets for high-precision calibration of telephoto lenses, and ground-based camera targets laid on the ground for surround-view camera calibration. Furthermore, the calibration site can be equipped with a power distribution box to provide power support for the entire calibration system. Thus, this parameter calibration system can support comprehensive calibration needs from single visual sensors to multi-sensor fusion, providing a more comprehensive data foundation for improving the recognition fidelity of various virtual sensors in the simulation environment.

[0019] It is understood that in the embodiments of this application, for real cameras and real-world calibration locations, virtual cameras and virtual calibration locations are set up in the simulation environment to completely correspond to them.

[0020] A virtual camera is a digital model that reproduces the hardware parameters, optical characteristics, and imaging mechanism of a real camera in a simulation environment to generate images or video data equivalent to the real scene. A virtual calibration site is reconstructed in the simulation environment based on the size and design elements of the real calibration site and is used exclusively for virtual camera calibration.

[0021] Reference Figure 1 This application provides a method for calibrating the parameters of a virtual camera, comprising: S1 acquires real image information of a preset real calibration location using a real camera.

[0022] The preset real calibration site refers to a space comprised of several calibration boards arranged according to the designed dimensions and layout in a real environment. The preset real calibration site can be set by technicians based on actual conditions, and there are no specific restrictions. Real image information refers to image data, including the pixel coordinates of the corner points of the calibration boards, captured by a real camera within this site from multiple known poses.

[0023] Specifically, in a pre-defined real calibration location, a real camera is precisely controlled to capture images of the physical calibration board from multiple known poses, resulting in a set of real image information. It should be noted that the poses described in this embodiment can be the shooting angles of the real camera.

[0024] S2, acquire virtual image information of the preset virtual calibration location through a virtual camera, wherein the preset real calibration location and the preset virtual calibration location correspond one-to-one.

[0025] The preset virtual calibration site refers to a 3D model scene reconstructed in the simulation engine at a 1:1 scale according to the board size, grid spacing, surface reflectivity, and spatial layout of the real calibration site. Furthermore, there is a one-to-one correspondence between the preset real calibration site and the preset virtual calibration site; that is, the position, orientation, and number of each real calibration board are unique in the virtual scene, and the virtual calibration board has completely identical geometric parameters. The preset virtual calibration site can be set by technicians according to the actual situation, and there are no specific restrictions.

[0026] Virtual image information refers to image data generated by rendering a virtual camera in a simulation environment, where the camera is placed in the exact same pose as the real camera, and the data includes the pixel coordinates of the corner points of the virtual calibration board.

[0027] Specifically, a three-dimensional preset virtual calibration site with geometric dimensions, surface characteristics, and spatial arrangement that are completely consistent with the real calibration site is imported into the simulation engine. Then, the initial values ​​of the virtual camera's intrinsic and extrinsic parameters are set to be the same as those of the real camera. The virtual camera is then positioned sequentially according to the pose sequence of the real camera. The rendering pipeline is called for each pose to generate the corresponding virtual image. All the resulting virtual images constitute the virtual image information, which is used for subsequent pixel-level difference comparison with the real image.

[0028] S3 parses the real image information and the virtual image information respectively to obtain the real and virtual marker pixel coordinates.

[0029] In this context, the true calibration pixel coordinates refer to the pixel-level two-dimensional coordinates of each corner point or marker center on the true calibration board, extracted from real image information, on the image plane. The virtual calibration pixel coordinates refer to the pixel-level two-dimensional coordinates of the corresponding corner point or marker center on the virtual calibration board, extracted from virtual image information, on the image plane.

[0030] It should be noted that calibration points refer to the locations of feature points on the calibration board used for camera calibration. Calibration points typically use dedicated markers whose 3D world coordinates are known and pixel-level detectable in the image, used to establish the correspondence between 3D space and the 2D image. Multiple calibration points are often set up in the calibration area. Specifically, the same feature extraction process is performed on both the real and virtual image information to automatically identify and accurately locate all feature points on the calibration board, obtaining a one-to-one set of pixel coordinates between the real and virtual sides, i.e., the real calibration point pixel coordinates and the virtual calibration point pixel coordinates.

[0031] S4. Calculate the target parameters of the virtual camera based on the real and virtual target pixel coordinates, and optimize the virtual camera parameters based on the target parameters.

[0032] The target parameters of the virtual camera refer to the parameters of the virtual camera that need to be updated to make the feature points of the virtual image consistent with the feature points of the real image in terms of pixel position.

[0033] Specifically, based on the real-world pixel coordinates, the relevant parameters of the virtual camera are iteratively adjusted until the pixel deviation between the virtual image feature points and the real image feature points is minimized globally. The resulting optimal parameters are the target parameters of the virtual camera and are directly written into the simulation model to complete the parameter optimization.

[0034] According to the virtual camera parameter calibration method of this application, firstly, a calibration board space is set up in a real environment according to the design, and real image information containing corner coordinates is obtained by taking pictures from multiple poses using a real camera. Then, a virtual calibration board space of the same size and layout is reconstructed at a 1:1 scale in the simulation engine, and virtual image information is obtained by rendering the virtual camera in the same pose. Subsequently, the same feature extraction is performed on the two types of images to obtain the corresponding real calibration pixel coordinates and virtual calibration pixel coordinates. Finally, based on the real calibration pixel coordinates, the relevant parameters of the virtual camera are adjusted through optimization and iteration to minimize the pixel deviation between the virtual calibration pixel coordinates and the real corner points, and the optimal parameters are written into the virtual camera to complete the calibration.

[0035] The virtual camera parameter calibration method described in this application uses real imaging as a benchmark and corrects the intrinsic parameters and distortion parameters of the virtual camera through pixel-level differences. This ensures that the simulated image achieves pixel-level consistency with the actual shooting result in terms of corner position, geometric shape, and lens distortion characteristics, thereby improving the visual fidelity and measurement reliability of the simulated scene. The calibrated virtual camera can directly output high-fidelity data, reducing the workload of repeated data acquisition and manual calibration on real vehicles, shortening the iteration cycle of ADAS algorithms, reducing R&D costs, and enhancing the universality and reusability of the same set of simulation data across multiple platforms and vehicle models.

[0036] In some embodiments of this application, before acquiring virtual image information of a preset virtual calibration location through a virtual camera, the parameter calibration method of the virtual camera further includes: determining the initial parameters of the real camera based on the real image information, wherein the initial parameters include an intrinsic parameter matrix, distortion coefficients, a rotation matrix, and a translation vector; and initializing the virtual camera parameters based on the initial parameters.

[0037] The initial parameters of the real camera refer to the parameters used to describe the imaging geometry of the real camera, which are obtained by performing a conventional camera calibration algorithm on real image information. These initial parameters are directly used as the startup values ​​of the virtual camera.

[0038] Furthermore, the initial parameters of the real camera include the intrinsic parameter matrix, distortion coefficients, and rotation matrix and translation vector corresponding to each shooting pose.

[0039] Specifically, the intrinsic parameter matrix can refer to a 3×3 matrix composed of focal length and principal point coordinates in pixels, used to describe the projection ratio and center offset of the camera from the three-dimensional camera coordinates to the two-dimensional image plane.

[0040] For example, the intrinsic parameter matrix (denoted as K) can be represented as: K=

[0041] Among them, f x f y This represents the focal length in pixels. f x =F / P X Where F represents the physical focal length (in millimeters), P X This represents the physical size of one pixel on the sensor (in millimeters per pixel). It also represents the magnification factor of the camera in the x and y directions. C u C V These represent the principal point coordinates, which are the pixel coordinates of the intersection of the camera's optical axis and the imaging plane. Ideally, the principal point coordinates are at the exact center of the image, but in reality, there will be slight deviations due to sensor mounting.

[0042] The distortion coefficient can refer to the vector composed of radial distortion parameters and tangential distortion parameters, which is used to quantify the image point shift caused by lens shape and installation errors.

[0043] For example, the distortion coefficient (denoted as D) can be expressed as: D=[k1,k2,p1,p2,k3] Where k1, k2, and k3 represent radial distortion coefficients, used to simulate the degree of light bending caused by the lens shape, resulting in barrel distortion (image edges bulging outward) or pincushion distortion (image edges contracting inward). p1 and p2 represent tangential distortion coefficients, used to simulate distortion caused by lens manufacturing and installation errors, resulting in the lens not being parallel to the imaging plane.

[0044] A rotation matrix is ​​a matrix used to rotate a point in the world coordinate system to the camera coordinate system. A rotation matrix can be represented by a 3×3 orthogonal matrix. The rotation matrix (denoted as R) is calculated from the angles of rotation about the X, Y, and Z axes, respectively. It satisfies R^{-1} = R^T and det(R) = 1.

[0045] A translation vector (denoted as T) can be defined as a vector representing the displacement from the origin of the world coordinate system to the origin of the camera coordinate system. A translation vector can be represented by a 3×1 vector. P_cam = R * P_world + T (P_cam represents the coordinates of the point in the camera coordinate system, and P_world represents the coordinates of the point in the world coordinate system).

[0046] The rotation matrix and translation vector together constitute the extrinsic parameters of the camera.

[0047] Specifically, after the real image is acquired, the intrinsic parameter matrix, distortion coefficient, and initial parameters such as rotation matrix and translation vector corresponding to all shooting poses of the real camera are calculated based on the real image information. Then, this set of complete parameters is seamlessly written into the virtual camera model through a script or interface, so that the virtual camera is completely consistent with the real camera in terms of imaging geometry, distortion characteristics and spatial pose.

[0048] This application embodiment achieves an accurate initial model by directly injecting the real calibration results into the virtual camera, so that subsequent pixel-level optimization only requires fine-tuning to converge, thereby shortening the calibration time and avoiding local optima caused by initial value errors, thus improving the one-time calibration success rate and simulation fidelity of the virtual camera.

[0049] In some embodiments of this application, the real image information includes multiple real images. Acquiring real image information of a preset real calibration location through a real camera includes: acquiring multiple preset shooting angles; controlling the real camera to take pictures of the preset real calibration location based on the multiple preset shooting angles to obtain multiple real images.

[0050] Among them, multiple preset shooting angles can refer to a set of discrete poses formed by different combinations of pitch angles, yaw angles and altitudes in the real calibration space, which are used to cover the typical working range of camera field of view, depth of field and distortion distribution.

[0051] Specifically, a set of shooting poses covering changes in field of view, angle, and distance are planned within a preset real calibration location. The real camera is then driven to position itself in each pose and take pictures of the preset real calibration location to obtain a multi-angle real image sequence.

[0052] This application embodiment captures the imaging features of the entire field of view, focal length, and distortion range of the lens at once by setting multiple preset shooting angles, providing a complete data foundation for subsequent virtual camera parameter optimization, thereby improving the robustness and generalization accuracy of the calibration results.

[0053] In some embodiments of this application, the virtual image information includes multiple virtual images. Acquiring virtual image information of a preset virtual calibration location through a virtual camera includes controlling the virtual camera to take pictures of the preset virtual calibration location from multiple preset shooting angles to obtain multiple virtual images, wherein the multiple virtual images correspond one-to-one with real images.

[0054] Specifically, a preset shooting angle sequence that corresponds exactly to the real acquisition is imported into the simulation engine. The virtual camera is then driven by a script to be positioned to each pose in sequence. After locking the initial intrinsic and extrinsic parameters consistent with those of the real camera, the rendering pipeline is called in batches to generate multiple virtual images that correspond one-to-one with the real images in terms of viewpoint, number, and total number. This provides a strictly matched data source for subsequent pixel-level difference calculations.

[0055] This application embodiment generates virtual images that correspond strictly one-to-one with real images in batches from multiple preset shooting angles, forming paired datasets with zero viewpoint deviation and zero missing data, providing sufficient and paired samples for pixel-level reprojection error calculation, thereby improving the convergence speed and calibration accuracy of subsequent parameter optimization.

[0056] In some embodiments of this application, real image information and virtual image information are parsed to obtain real and virtual marker pixel coordinates, including: parsing multiple real images to obtain the real marker pixel coordinates corresponding to each real image; and parsing multiple virtual images to obtain the virtual marker pixel coordinates corresponding to each virtual image.

[0057] It is understandable that for images captured from different preset shooting angles, each image may contain one or more marker pixel coordinates; that is, the number of visible markers in each image varies with viewpoint, occlusion, and depth of field. Specifically, the same pixel corner detection algorithm is run synchronously on a group of real and virtual images, batch-outputting the corresponding real marker pixel coordinates in the real image information and the corresponding virtual marker pixel coordinates in the virtual image information. This achieves a one-to-one pairing of real and virtual marker pixel coordinates at the viewpoint and feature point levels.

[0058] This application embodiment extracts pixel corner coordinates from real and virtual images in batches, establishes a paired dataset with a viewpoint-corner dual-layer index, reduces systematic errors introduced by detection differences, and improves the accuracy and convergence speed of subsequent parameter optimization.

[0059] In some embodiments of this application, the target parameters of the virtual camera are calculated based on the real and virtual marker pixel coordinates, including: determining the real and virtual images corresponding to each preset shooting angle from multiple real and virtual images; calculating the pixel coordinate difference between the real and virtual marker pixel coordinates of the real image and the virtual marker pixel coordinates of the virtual image corresponding to each preset shooting angle; and calculating the target parameters of the virtual camera based on the preset shooting angle, the pixel coordinate difference, the real and virtual marker pixel coordinates.

[0060] The pixel coordinate difference refers to the distance vector on the image plane between the real and virtual coordinates of a corner point in the real image and the virtual coordinates of the corresponding corner point in the virtual image, under the same shooting angle. It is used to quantify the reprojection deviation of the corner point caused by the error of the virtual camera parameters.

[0061] Specifically, the real image and the virtual image at the same preset shooting angle are extracted sequentially, and the pixel coordinate difference between the real and virtual marker pixel coordinates is calculated point by point. Furthermore, based on the preset shooting angle, pixel coordinate difference, real marker pixel coordinates, and virtual marker pixel coordinates, the target parameters of the virtual camera are calculated.

[0062] This application embodiment directly locates the source of virtual camera parameter error by quantifying the difference between real and virtual pixel coordinates point by point and summarizing it into a global reprojection error. Furthermore, based on the preset shooting angle, pixel coordinate difference, real-calibrated pixel coordinates, and virtual-calibrated pixel coordinates, the target parameters of the virtual camera are calculated, thereby reducing the simulation-real-shot gap and improving the efficiency and fidelity of virtual camera calibration.

[0063] In some embodiments of this application, the target parameters of the virtual camera are calculated based on a preset shooting angle, pixel coordinate difference, real-target pixel coordinates, and virtual-target pixel coordinates. This includes: obtaining a preset projection function of the virtual camera; calculating multiple sets of camera parameters based on the preset projection function, preset shooting angle, pixel coordinate difference, real-target pixel coordinates, and virtual-target pixel coordinates; and determining the target parameters from the multiple sets of camera parameters based on a preset optimization algorithm and pixel coordinate difference. The target parameters include an intrinsic parameter matrix and distortion coefficients.

[0064] The preset projection function refers to the complete imaging model function that maps the corner points of the calibration board in the three-dimensional world coordinate system to the two-dimensional image plane. Its inputs are the corner point world coordinates, rotation matrix, translation vector, intrinsic parameter matrix and distortion coefficient, and the output is the predicted pixel coordinates.

[0065] The preset optimization algorithm is an algorithm used to minimize pixel coordinate differences. For example, this preset optimization algorithm can use a non-linear optimization algorithm to adjust the relevant parameters of the virtual camera with the real-world pixel coordinates as the target.

[0066] It should be noted that the above-mentioned preset projection function and preset optimization algorithm can be set by technical personnel according to the actual situation, and there are no specific restrictions.

[0067] As a specific embodiment of this example, the preset projection function can be expressed as: Project(K_virtual,D_virtual,Pose_i,P_j), Where K_virtual represents the intrinsic parameter matrix of the virtual camera, including the focal length and principal point coordinates. D_virtual represents the distortion coefficient vector of the virtual camera, describing the radial and tangential distortion of the lens. Pose_i represents the extrinsic parameters corresponding to the i-th preset shooting angle, including the rotation matrix and translation vector, defining the position and orientation of the camera coordinate system relative to the world coordinate system. P_j represents the virtual calibration pixel coordinates of the j-th corner point on the calibration board.

[0068] Furthermore, the preset projection function takes the corner point P_j in the real-world coordinate system, the corresponding pose Pose_i, the current virtual camera intrinsic matrix K_virtual, and the distortion coefficients D_virtual as input, and outputs the predicted pixel coordinates. Then, using a predefined loss function, the sum of the reprojection errors (i.e., pixel coordinate differences) of all corner pixel coordinates (denoted as L) is calculated. The calculation method of the sum of pixel coordinate differences can be expressed as: L=Σ||Project(K_virtual,D_virtual,Pose_i,P_j)-u_real_ij||².

[0069] Where u_real_ij is the real-world pixel coordinate of the j-th corner point detected on the i-th real image.

[0070] Using the real-valued pixel coordinates as the ground truth, and combining the pixel coordinate differences to construct the reprojection error, error samples under different (K, D) combinations are generated in batches to form multiple sets of candidate camera parameters. Finally, a nonlinear least squares algorithm is used to minimize the sum of squared differences of all pixel coordinates. The optimal intrinsic parameter matrix and distortion coefficient are obtained by iterative convergence from multiple sets of candidate parameters, which are used as the target parameters of the virtual camera.

[0071] This application embodiment maps the corner point in world coordinates to predicted pixel coordinates through a preset projection function, and constructs a reprojection error loss function with the real detection coordinates as the true value. The intrinsic parameter matrix and distortion coefficient of the virtual camera are optimized in one go through a nonlinear optimization algorithm, so that the virtual image and the real image can be aligned at the pixel level under all shooting angles, thereby reducing the simulation fidelity error, shortening the ADAS algorithm development cycle and reducing R&D costs.

[0072] As a specific embodiment of this application, refer to Figure 2 The parameter calibration method for the virtual camera in this application may include: S201 acquires real image information of a preset real calibration location using a real camera.

[0073] S202, acquire virtual image information of a preset virtual calibration location through a virtual camera.

[0074] S203, analyzes the real image information and the virtual image information respectively.

[0075] S204 determines the real image information and virtual image information corresponding to each preset shooting angle.

[0076] S205, calculate the pixel coordinate difference.

[0077] S206 generates multiple sets of camera parameters based on a preset projection function, a preset shooting angle, pixel coordinate difference, real-pointing pixel coordinates, and virtual-pointing pixel coordinates.

[0078] S207 determines target parameters from multiple sets of camera parameters based on a preset optimization algorithm and pixel coordinate differences.

[0079] S208 optimizes the virtual camera parameters based on the target parameters.

[0080] This application also provides a parameter calibration device for a virtual camera, referring to... Figure 3 The device 300 includes: a first acquisition module 310, a second acquisition module 320, a parsing module 330, and a calibration module 340.

[0081] The first acquisition module 310 is used to acquire real image information of a preset real calibration location using a real camera; the second acquisition module 320 is used to acquire virtual image information of a preset virtual calibration location using a virtual camera, wherein the preset real calibration location and the preset virtual calibration location correspond one-to-one; the parsing module 330 is used to parse the real image information and the virtual image information respectively to obtain the real calibration pixel coordinates and the virtual calibration pixel coordinates; the calibration module 340 is used to calculate the target parameters of the virtual camera based on the real calibration pixel coordinates and the virtual calibration pixel coordinates, and optimize the virtual camera parameters based on the target parameters.

[0082] In some embodiments of this application, before the first acquisition module 310 acquires virtual image information of a preset virtual calibration location through a virtual camera, it is further configured to: determine the initial parameters of the real camera based on the real image information, wherein the initial parameters include an intrinsic parameter matrix, distortion coefficients, a rotation matrix, and a translation vector; and initialize the virtual camera parameters based on the initial parameters.

[0083] In some embodiments of this application, the first acquisition module 310 determines that the real image information includes multiple real images. Specifically, the acquisition of real image information of a preset real calibration location by a real camera is used to: acquire multiple preset shooting angles; and control the real camera to take pictures of the preset real calibration location based on the multiple preset shooting angles to obtain multiple real images.

[0084] In some embodiments of this application, the second acquisition module 320 determines that the virtual image information includes multiple virtual images. Specifically, the virtual image information of the preset virtual calibration location is acquired by the virtual camera. This is specifically used to: control the virtual camera to take pictures of the preset virtual calibration location based on multiple preset shooting angles to obtain multiple virtual images, wherein the multiple virtual images correspond one-to-one with the real images.

[0085] In some embodiments of this application, the parsing module 330 parses the real image information and the virtual image information respectively to obtain the real marker pixel coordinates and the virtual marker pixel coordinates. Specifically, it is used to: parse multiple real images respectively to obtain the real marker pixel coordinates corresponding to each real image; and parse multiple virtual images respectively to obtain the virtual marker pixel coordinates corresponding to each virtual image.

[0086] In some embodiments of this application, the calibration module 340 calculates the target parameters of the virtual camera based on the real calibration pixel coordinates and the virtual calibration pixel coordinates. Specifically, it is used to: determine the real image and virtual image corresponding to each preset shooting angle from multiple real images and multiple virtual images; calculate the pixel coordinate difference between the real calibration pixel coordinates of the real image and the virtual calibration pixel coordinates of the virtual image corresponding to each preset shooting angle; and calculate the target parameters of the virtual camera based on the preset shooting angle, the pixel coordinate difference, the real calibration pixel coordinates, and the virtual calibration pixel coordinates.

[0087] In some embodiments of this application, the calibration module 340 calculates the target parameters of the virtual camera based on a preset shooting angle, pixel coordinate difference, real-pointed pixel coordinates, and virtual-pointed pixel coordinates. Specifically, it is used to: obtain a preset projection function of the virtual camera; calculate multiple sets of camera parameters based on the preset projection function, preset shooting angle, pixel coordinate difference, real-pointed pixel coordinates, and virtual-pointed pixel coordinates; and determine the target parameters from the multiple sets of camera parameters based on a preset optimization algorithm and pixel coordinate difference. The target parameters include an intrinsic parameter matrix and distortion coefficients.

[0088] This application also provides an electronic device, which is described in reference to... Figure 4 The electronic device 400 includes a memory 410, a processor 420, and a computer program stored in the memory 410 and executable on the processor 420. The processor 420 executes the program to implement the aforementioned parameter calibration method for the virtual camera.

[0089] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned virtual camera parameter calibration method.

[0090] In this application, "multiple" refers to two or more.

[0091] In this application, unless otherwise expressly defined, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0092] The terms “first,” “second,” “third,” “fourth,” etc., in this application (if present) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0093] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0094] Unless otherwise specified, all steps in this application may be performed sequentially or randomly. For example, if the method includes steps A and B, it means that the method may include steps A and B performed sequentially, or it may include steps B and A performed sequentially. For example, if the method may also include step C, it means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or it may include steps A, C, and B, or it may include steps C, A, and B, etc.

[0095] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for calibrating the parameters of a virtual camera, characterized in that, include: Acquire real image information of a pre-defined, real-calibration location using a real camera; Virtual image information of a preset virtual calibration location is acquired through a virtual camera, wherein the preset real calibration location corresponds one-to-one with the preset virtual calibration location; The real image information and the virtual image information are parsed respectively to obtain the real marker pixel coordinates and the virtual marker pixel coordinates; The target parameters of the virtual camera are calculated based on the real and virtual target pixel coordinates, and the virtual camera parameters are optimized based on the target parameters.

2. The parameter calibration method for a virtual camera according to claim 1, characterized in that, Before acquiring virtual image information of a preset virtual calibration location via a virtual camera, the method further includes: The initial parameters of the real camera are determined based on the real image information, wherein the initial parameters include an intrinsic parameter matrix, distortion coefficients, a rotation matrix, and a translation vector; The virtual camera is initialized according to the initial parameters.

3. The parameter calibration method for a virtual camera according to claim 1, characterized in that, The real image information includes multiple real images, wherein acquiring the real image information of the preset real calibration location through a real camera includes: Acquire multiple preset shooting angles; The real camera is controlled to take pictures of the preset real location from the multiple preset shooting angles to obtain the multiple real images.

4. The parameter calibration method for a virtual camera according to claim 3, characterized in that, The virtual image information includes multiple virtual images, wherein the virtual image information of a preset virtual calibration location acquired through a virtual camera includes: The virtual camera is controlled to take pictures of the preset virtual calibration location based on the multiple preset shooting angles to obtain the multiple virtual images, wherein the multiple virtual images correspond one-to-one with the real images.

5. The parameter calibration method for a virtual camera according to claim 4, characterized in that, The step of parsing the real image information and the virtual image information respectively to obtain the real marker pixel coordinates and the virtual marker pixel coordinates includes: The multiple real images are analyzed to obtain the real location pixel coordinates corresponding to each real image; The multiple virtual images are analyzed to obtain the virtual marker pixel coordinates corresponding to each virtual image.

6. The parameter calibration method for a virtual camera according to claim 5, characterized in that, The step of calculating the target parameters of the virtual camera based on the real-valued pixel coordinates and the virtual-valued pixel coordinates includes: From the plurality of real images and the plurality of virtual images, determine the real image and virtual image corresponding to each preset shooting angle; Calculate the pixel coordinate difference between the real and virtual marker pixel coordinates of the real image and the virtual image corresponding to each preset shooting angle. The target parameters of the virtual camera are calculated based on the preset shooting angle, the pixel coordinate difference, the real target pixel coordinates, and the virtual target pixel coordinates.

7. The parameter calibration method for a virtual camera according to claim 6, characterized in that, The step of calculating the target parameters of the virtual camera based on the preset shooting angle, the pixel coordinate difference, the real-target pixel coordinates, and the virtual-target pixel coordinates includes: Obtain the preset projection function of the virtual camera; Based on the preset projection function, the preset shooting angle, the pixel coordinate difference, the real locator pixel coordinates, and the virtual locator pixel coordinates, multiple sets of camera parameters are calculated. Based on a preset optimization algorithm and the pixel coordinate difference, the target parameters are determined from the multiple sets of camera parameters, wherein the target parameters include an intrinsic parameter matrix and distortion coefficients.

8. A parameter calibration device for a virtual camera, characterized in that, include: The first acquisition module is used to acquire real image information of a preset real calibration location through a real camera; The second acquisition module is used to acquire virtual image information of a preset virtual calibration location through a virtual camera, wherein the preset real calibration location corresponds one-to-one with the preset virtual calibration location; The parsing module is used to parse the real image information and the virtual image information respectively to obtain the real marker pixel coordinates and the virtual marker pixel coordinates; The calibration module is used to calculate the target parameters of the virtual camera based on the real calibration pixel coordinates and the virtual calibration pixel coordinates, and to optimize the parameters of the virtual camera based on the target parameters.

9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the parameter calibration method for a virtual camera as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the parameter calibration method for the virtual camera as described in any one of claims 1-7.

Images