Camera calibration simulation method and system, storage medium and equipment

Abstract

The invention discloses a camera calibration simulation method. The method comprises the following steps: obtaining a VTD model of a target for camera calibration established based on three-dimensional modeling software; obtaining a VTD simulation scene for camera calibration, and importing the simulation vehicle model provided with the camera and the VTD model of the target to corresponding positions of the VTD simulation scene; generating a corresponding configuration file according to the internal parameters and the external parameters calibrated by the camera; and performing VTD camera simulation processing according to the imported VTD simulation scene and configuration, evaluating a simulation result, adjusting the size and position of the target according to the evaluation result, and obtaining and outputting an optimized target arrangement scheme. The invention further discloses a corresponding system, a storage medium and a device. By implementing the method, the target arrangement scheme of the calibration laboratory can be efficiently, accurately and conveniently obtained, so that the camera calibration laboratory can be built.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a camera calibration simulation method, system, storage medium and device based on complex traffic scene visual simulation (Virtual Test Drive, VTD). Background Art

[0002] Camera calibration is a very important task in driver assistance systems. Its main purpose is to determine the spatial relationship between the camera and the vehicle, so as to help the algorithm accurately calculate the distance, position and direction between the vehicle and the surrounding environment, and provide necessary data support for driver assistance and autonomous driving. The accuracy of camera calibration will seriously affect the accuracy of visual technology judgment and recognition.

[0003] Camera calibration is usually completed in a calibration laboratory. Generally, camera calibration is completed in a calibration laboratory by photographing calibration plates at different positions and angles. Among them, the calibration plate is usually a specially designed plane or three-dimensional plate covered with some special marking points, called corner points. By placing the calibration plate at different angles and positions, taking multiple calibration plate images on the camera, finding the pixel position of the corner point through image processing and calculation, and solving the camera's intrinsic and extrinsic parameters after optimization. Reasonable target style and accurate target position can greatly improve the speed of accurate camera calibration.

[0004] At present, the design of calibration laboratories mainly adopts the method of on-site debugging, and adjusts the style and position of the target through continuous experiments to achieve the optimal design. Due to the large number of on-site calibration debugging times, the design cycle of the calibration board is relatively long and costly; the calibration room needs to be redesigned for unused camera external parameters, and its maintainability is poor. At the same time, in the design of the calibration room, the design of traditional camera calibration laboratories usually uses manual image coordinate collection or corner detection algorithms to obtain the coordinates of the feature points of the calibration object image, which will have original errors and its accuracy is difficult to guarantee.

[0005] In the prior art, to build a camera calibration laboratory, a large indoor or outdoor calibration room is needed. At the same time, the target size and style need to be repeatedly designed, and the physical object needs to be made. The position of the target needs to be adjusted continuously for debugging. The existing method of building a calibration laboratory has the disadvantages of long design cycle, high cost, and high manpower consumption. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a camera calibration simulation method, system, storage medium and device, which can efficiently, accurately and conveniently obtain the target arrangement scheme of the calibration laboratory.

[0007] In order to solve the above technical problems, as one aspect of the present invention, a camera calibration simulation method is provided, which at least includes the following steps:

[0008] Obtain a VTD model of a target for camera calibration based on 3D modeling software;

[0009] Obtain a VTD simulation scene for camera calibration, and import a simulated vehicle model with a camera installed and a VTD model of the target into corresponding positions of the VTD simulation scene;

[0010] Generate corresponding configuration files based on the intrinsic and extrinsic parameters of camera calibration;

[0011] The VTD camera simulation is performed according to the imported VTD simulation scene and configuration file, the simulation results are evaluated, and the size and position of the target are adjusted according to the evaluation results to obtain the optimized target layout plan.

[0012] Wherein, the step of obtaining a VTD model of a target for camera calibration established based on three-dimensional modeling software further comprises:

[0013] A 3D modeling software is used to build a 3D low-precision model of the target used for camera calibration, the 3D low-precision model is UV unfolded, and corresponding color textures are affixed to form a 3D high-precision model, and the 3D high-precision model is converted into a VTD model format.

[0014] Wherein, obtaining a VTD simulation scene for camera calibration, importing a simulated vehicle model with a camera installed and a VTD model of the target into corresponding positions of the VTD simulation scene, further comprising:

[0015] Build a static scene for VTD camera calibration in the VTD network modeler and import the VTD model of the target to the predetermined position;

[0016] Build a dynamic scene for simulation calibration in the scene editor and import a simulated vehicle model equipped with a camera.

[0017] The step of generating a corresponding configuration file according to the intrinsic parameters and extrinsic parameters of the camera calibration further includes:

[0018] In the VTD simulation scene configuration file, complete the external parameter settings for camera calibration based on the location information of the camera installed on the vehicle;

[0019] Use the lens distortion tool to configure the camera's resolution, focal length, FOV angle, pixel size, and distortion table parameters, generate the corresponding configuration file, and complete the setting of the camera calibration parameters.

[0020] The step of performing VTD camera simulation processing according to the imported VTD simulation scene and configuration file and evaluating the simulation result further includes:

[0021] According to the expected number of targets to be placed, the feasible area for placing each target is confirmed through the expected projection effect, and then the target position is dynamically adjusted according to the target size and placement distance requirements;

[0022] According to the position of the target in the vehicle coordinate system, the true value of the coordinates of the corner points in the target in the pixel coordinate system is solved, and the rationality of the target layout is evaluated;

[0023] When all of the following conditions are met, the current target layout plan is considered to be a valid plan; otherwise, dynamic adjustment is required:

[0024] Checkerboard target area S chess Satisfying conditions: 0.8% ≤ S chess ≤5%;

[0025] The total number of identified target corner points is equal to the total number of expected target corner points;

[0026] And the target center pixel coordinates (u, v) meet the following conditions:

[0027] Where S is the total area of ​​the image; w and v are the image height and width.

[0028] The step of adjusting the size and position of the target according to the evaluation result to obtain an optimized target arrangement scheme further includes:

[0029] The pre-set simulation evaluation program is used to make judgments, and the position of the target is continuously adjusted according to the evaluation results, and the longitudinal, lateral and angular optimal ranges of the target corresponding to the vehicle model are output;

[0030] Repeat the above operation by changing the vehicle model to obtain the optimal longitudinal, lateral and angular intervals of the target corresponding to each vehicle model interval, and use the common available position interval of each interval as the optimized target placement position to obtain the optimized target layout plan.

[0031] The step of continuously adjusting the target position according to the evaluation results and outputting the optimal longitudinal, lateral and angular intervals of the target corresponding to the vehicle model further includes:

[0032] Determine whether the longitudinal evaluation criteria are met, and if not, adjust the longitudinal distance between the target and the camera by a predetermined step length until the condition is met;

[0033] Determine whether the longitudinal evaluation criteria are met. If not, adjust the lateral position of the outermost target each time to ensure that the outermost target falls within the camera field of view and does not overlap until all target conditions are met;

[0034] Determine whether the angle evaluation criteria are met. If not, further adjust the horizontal and vertical distances between targets or the target azimuth. If a suitable result is still not obtained, adjust the vertical distance between the target and the camera and repeat the above judgment.

[0035] If the above three evaluation criteria are met, the optimal longitudinal, lateral and angular ranges of the target corresponding to the vehicle model are obtained.

[0036] Among them, it further includes: outputting the optimized target arrangement plan to build a camera calibration laboratory according to the optimized target arrangement plan.

[0037] Accordingly, another aspect of the present invention further provides a system for designing a camera calibration laboratory based on VTD simulation, which at least includes:

[0038] A target model acquisition unit, used to obtain a VTD model of a target for camera calibration established based on three-dimensional modeling software;

[0039] A simulation scene building unit, used to obtain a VTD simulation scene for camera calibration, and import a simulation vehicle model with a camera installed and a VTD model of the target into corresponding positions of the VTD simulation scene;

[0040] A camera internal and external parameter setting unit, used to generate a corresponding configuration file according to the camera calibrated internal and external parameters;

[0041] The simulation debugging processing unit is used to perform VTD camera simulation processing according to the imported VTD simulation scene and configuration file, evaluate the simulation results, and adjust the size and position of the target according to the evaluation results to obtain an optimized target layout plan.

[0042] Wherein, the simulation debugging processing unit further includes:

[0043] The initial area determination unit is used to confirm the feasible area for placing each target according to the expected number of targets to be placed and the expected projection effect, and then dynamically adjust the target position according to the size of the target and the placement distance requirements;

[0044] The rationality evaluation unit is used to solve the true value of the coordinates of the corner points in the target in the pixel coordinate system according to the position of the target in the vehicle coordinate system, and evaluate the rationality of the target layout; when all the following conditions are met, the current target layout plan is determined to be a valid plan; otherwise, dynamic adjustment is required:

[0045] Checkerboard target area S chess Satisfying conditions: 0.8% ≤ S chess ≤5%;

[0046] The total number of identified target corner points is equal to the total number of expected target corner points;

[0047] And the target center pixel coordinates (u, v) meet the following conditions:

[0048] Where S is the total area of ​​the image; w and v are the image height and width of the image;

[0049] The optimal interval determination unit is used to make judgments according to a preset simulation evaluation program, continuously adjust the position of the target according to the evaluation results, and output the longitudinal, lateral and angular optimal intervals of the target corresponding to the vehicle type;

[0050] The optimized scheme obtaining unit is used to repeat the above operation by changing the vehicle model to obtain the longitudinal, lateral and angular optimal intervals of the targets corresponding to each vehicle model interval, and use the common available position intervals of each interval as the optimized target placement position to obtain the optimized target layout plan.

[0051] Wherein, the optimal interval determination unit further includes:

[0052] A judgment processing unit is used to judge whether the longitudinal evaluation standard is met. If not, the longitudinal distance between the target and the camera is adjusted by a predetermined step length until the condition is met; to judge whether the longitudinal evaluation standard is met. If not, the lateral position of the outermost target is adjusted each time to ensure that the outermost target falls within the camera field of view and does not overlap until all target conditions are met;

[0053] Determine whether the angle evaluation criteria are met. If not, further adjust the horizontal and vertical distances between targets or the target azimuth. If a suitable result is still not obtained, adjust the vertical distance between the target and the camera and repeat the above judgment. If the above three evaluation criteria are all met, obtain the optimal longitudinal, lateral and angular ranges of the target corresponding to this vehicle model.

[0054] Among them, further include:

[0055] The target arrangement scheme output unit is used to output the optimized target arrangement scheme so as to build a camera calibration laboratory according to the optimized target arrangement scheme.

[0056] Accordingly, as another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned method are implemented.

[0057] Correspondingly, as another aspect of the present invention, a computing device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned method when executing the computer program.

[0058] The implementation of the embodiments of the present invention has the following beneficial effects:

[0059] The present invention provides a camera calibration simulation method, system, storage medium and device. By using VTD to build a camera calibration simulation environment, a camera calibration laboratory is designed, thereby providing a solution for realizing an automated camera calibration room;

[0060] In the embodiment of the present invention, the pixel value of the target corner point is calculated by automatically receiving the simulation image; and the range of the area where each target can be placed is determined according to the distance of the target placement, the expected imaging position, and the target size. At the same time, an evaluation method for target imaging and a process for dynamically adjusting the target position are provided, which can greatly reduce the number of on-site debugging in the calibration room and shorten the design cycle of the calibration board.

[0061] At the same time, the method provided by the present invention can be adapted to vehicle models of different sizes, and to different camera types such as fisheye cameras and pinhole cameras; it supports the switching of different target graphics, and can conveniently and accurately complete the calibration of all directions, thereby improving the design efficiency of the calibration laboratory;

[0062] In summary, the method for building a camera calibration laboratory online provided by the present invention can reduce the number of on-site vehicle debugging in the calibration room, saving costs and design resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, without paying creative labor, other drawings obtained based on these drawings still belong to the scope of the present invention.

[0064] Figure 1 A schematic diagram of the main process of an embodiment of a camera calibration simulation method provided by the present invention;

[0065] Figure 2 The calibration plate model designed by three-dimensional modeling software involved in the present invention;

[0066] Figure 3 The size, style and position layout of the VTD target involved in the present invention;

[0067] Figure 4The simulation result of the VTD camera involved in the present invention;

[0068] Figure 5 The actual recognition result obtained according to the arrangement scheme of the VTD output calibration plate involved in the present invention;

[0069] Figure 6 A comparison between the actual recognition result and the simulation result obtained according to the arrangement scheme of the VTD output calibration plate involved in the present invention;

[0070] Figure 7 This is a schematic diagram of a calibration room layout scheme output by the present invention;

[0071] Figure 8 A schematic structural diagram of an embodiment of a camera calibration simulation system provided by the present invention;

[0072] Fig. 9 for Figure 8 Schematic diagram of the structure of the simulation debugging processing unit. DETAILED DESCRIPTION

[0073] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0074] like Figure 1 FIG. 1 is a schematic diagram showing the main process of an embodiment of a camera calibration simulation method provided by the present invention. Figures 2 to 7 As shown, in this embodiment, the method at least includes the following steps:

[0075] Step S10, obtaining a VTD model of a target for camera calibration established based on three-dimensional modeling software;

[0076] Specifically, in one example, step S10 further includes:

[0077] Use 3D modeling software to build a 3D low-precision model of the target (such as a calibration plate) used for camera calibration, UV (horizontal and vertical coordinates) unfold the 3D low-precision model, paste the corresponding color texture to form a 3D high-precision model, and convert the 3D high-precision model into VTD model format.

[0078] More specifically, in a specific example, a low-precision model of the calibration plate (such as Figure 2 As shown), UV unfold the low-precision model; UV unfolding is the process of unfolding the surface of a three-dimensional model into a two-dimensional plane, so that it can be more conveniently edited and adjusted when designing materials and textures for the model.

[0079] Then import the UV map into the corresponding texture material design software (such as Photoshop), make the model map in Photoshop according to the requirements of the target texture material and export it to Blender software in RGB format for UV mapping. The UV mapping operation can give the corresponding material texture in the form of a picture, thereby adding more details, edges and contours to achieve a more realistic effect, and finally complete the design of the high-precision model of the calibration plate.

[0080] Step S11, obtaining a VTD simulation scene for camera calibration, and importing a simulated vehicle model with a camera installed and a VTD model of the target into corresponding positions of the VTD simulation scene;

[0081] Specifically, in one example, the step S11 further includes:

[0082] Build a static scene for VTD camera calibration in the VTD road network modeler, and import the VTD model of the target to the predetermined position. The main content of the static scene construction is the placement of the calibration plate and the modeling of its surrounding environment, including the ground, surrounding roads, buildings, etc.

[0083] In a specific example, the import process can be implemented as follows: export the three-dimensional model generated by Blender software in OBJ format, where the OBJ file usually contains the vertex information, material information and texture coordinate (UV) information of the model. In the same directory as the exported OBJ file, a .mtl file will be generated, which contains material information related to the model, such as the path of the texture file, the lighting model, etc. Put the OBJ file generated by Blender, the related .mtl file and the texture map file (such as PNG format) into the corresponding VTD model library; then configure the size of the model bounding box (Bounding Box), the map path, the model center position and other parameters in the configuration file (.dat file), and the VTD model of the target used for external camera calibration can be successfully imported into the VTD simulation scene.

[0084] Build a dynamic scene for simulation calibration in the Scenario editor and import a simulated vehicle model equipped with a camera.

[0085] The VTD simulation scene for camera calibration can be found in Figure 3 shown.

[0086] Step S12, generating a corresponding configuration file according to the intrinsic parameters and extrinsic parameters of the camera calibration;

[0087] It is understandable that when using a vehicle for camera calibration, the vehicle needs to run in a static scene along a specific path and capture images of the calibration plate along the way. In order to simulate the driving and shooting conditions of the vehicle, the vehicle parameters, driving trajectory, internal and external parameters of the camera, etc. need to be defined in this step, and the camera image is dynamically generated as the vehicle moves for subsequent camera calibration and parameter estimation analysis.

[0088] Specifically, in one example, the step S12 further includes:

[0089] In the VTD simulation scene configuration file, the external parameter settings of the camera calibration are completed according to the position information of the camera installed on the vehicle; specifically, by defining the position and direction of the camera installation, the external parameters of the camera, namely the position, orientation and rotation of the camera, can be determined. These parameters are very important in subsequent geometric calculations and visual measurements, and can be used to calculate the geometric relationship between the camera and the target, and realize visual applications such as 3D reconstruction.

[0090] The lens distortion tool is used to configure the camera's resolution, focal length, field of view (FOV) angle, pixel size, distortion table parameters, etc., generate the corresponding configuration file, and complete the setting of the camera calibration parameters. Among them, the resolution defines the size and clarity of the image; the focal length is one of the optical characteristics of the camera, which determines the viewing angle of the image; the FOV angle refers to the field of view of the camera; the pixel size refers to the size of each pixel on the screen; the distortion table refers to the distortion phenomenon in the camera imaging process, which needs to be corrected to improve the measurement accuracy.

[0091] It is understandable that in step S12, the camera parameters are calibrated to achieve more accurate visual measurement and computer vision algorithms. By setting external and internal camera parameters, accurate camera positioning, distortion correction, projection transformation and other processing can be achieved, thereby improving the accuracy and reliability of visual measurement and algorithms.

[0092] Step S13, performing VTD camera simulation processing according to the imported VTD simulation scene and configuration file, evaluating the simulation results, and performing simulation debugging according to the evaluation results, dynamically adjusting the size and position of the target, and obtaining an optimized target layout plan;

[0093] Specifically, in one example, step S13 further includes:

[0094] Step S130, the boundary range where targets corresponding to different depths of field can be placed, according to the number of targets expected to be placed, through the expected projection effect (or predetermined placement rules), confirm the feasible area for placement of each target, and then dynamically adjust the target position according to the size of the target and the placement distance requirements;

[0095] In the present invention, an automated image receiving, processing, and evaluation program is designed. Image receiving can be obtained by TCP, shared memory, or video injection. This embodiment can use a shared memory method to obtain simulation images in real time, and can realize multi-IG synchronous simulation and simultaneous acquisition of multi-IG simulation results;

[0096] Step S131, performing VTD camera simulation processing according to the VTD simulation scene and configuration (such as Figure 4 That is, a simulation result), according to the position of the target in the vehicle coordinate system, the true value of the coordinates of the corner points in the target in the pixel coordinate system is solved, and the rationality of the target layout is evaluated;

[0097] When all of the following conditions are met, the current target layout plan is considered to be a valid plan; otherwise, dynamic adjustment is required:

[0098] Checkerboard target area S chess Satisfying conditions: 0.8% ≤ S chess ≤5%; this condition aims to reduce the target area S chess Restricted to a certain range;

[0099] The total number of identified target corner points is equal to the total number of expected target corner points. The purpose of this condition is to calibrate that all the required target corner points are distributed within the camera's field of view.

[0100] And the target center pixel coordinates (u, v) meet the following conditions:

[0101] Where s is the total area of ​​the image; w and v are the image height and width of the image. The purpose of this condition is to limit the target center pixel coordinate value (u, v) to a certain area of ​​the image.

[0102] It can be understood that without considering distortion, the process of projecting an object from the world coordinate system to the pixel coordinate system is as follows:

[0103]

[0104] Among them, M is the internal parameter matrix and N is the external parameter matrix.

[0105] Step S132, making a judgment according to a preset simulation evaluation program, continuously adjusting the position of the target according to the evaluation result, and outputting the longitudinal, lateral and angular optimal intervals of the target corresponding to the vehicle type;

[0106] Further including:

[0107] Determine whether the longitudinal evaluation criteria are met, and if not, adjust the longitudinal distance between the target and the camera by a predetermined step length until the condition is met;

[0108] Determine whether the longitudinal evaluation criteria are met. If not, adjust the lateral position of the outermost target each time to ensure that the outermost target falls within the camera field of view and does not overlap until all target conditions are met;

[0109] Determine whether the angle evaluation criteria are met. If not, further adjust the horizontal and vertical distances between targets or the target azimuth. If a suitable result is still not obtained, adjust the vertical distance between the target and the camera and repeat the above judgment.

[0110] If the above three evaluation criteria are met, the optimal longitudinal, lateral and angular ranges of the target corresponding to the vehicle model are obtained.

[0111] like Figure 5 and Figure 6 As shown, an example diagram of the actual recognition result obtained according to the arrangement scheme of the VTD output calibration board involved in the present invention during the adjustment process, as well as an example diagram comparing the actual recognition result and the simulation result obtained according to the arrangement scheme of the VTD output calibration board.

[0112] Step S1303, repeat the above operation by changing the vehicle type, obtain the longitudinal, lateral and angular optimal intervals of the target corresponding to each vehicle type interval, and use the common available position interval of each interval as the optimized target placement position to obtain the optimized target layout plan, such as Figure 7 A schematic diagram of an optimized target arrangement scheme is shown in FIG.

[0113] It is understandable that when making the above judgments, the settings and weights of the evaluation criteria can be confirmed in advance, which can avoid unnecessary calculations and judgments and improve the efficiency of the program; when adjusting the target position, it is necessary to consider the influence of different directions, such as longitudinal, lateral, rotation, etc., and at the same time, it is necessary to ensure that all targets are in the camera's field of view and are not blocked; determine the appropriate step size based on the specific situation. Too large or too small will lead to poor performance.

[0114] Step S14, outputting the optimized target arrangement scheme, so as to build a camera calibration laboratory according to the optimized target arrangement scheme.

[0115] Specifically, according to the optimized target arrangement scheme outputted from step S14, a camera calibration laboratory is arranged, and the simulation results are compared with the measured results to confirm the correctness of the simulation results, thereby completing the closed-loop design of the simulation arrangement of the camera calibration laboratory.

[0116] It can be understood that in the method provided by the present invention, a simulation environment for camera calibration is built with the help of VTD, and a camera calibration laboratory is designed, thereby providing a solution for an automated calibration room. In the present invention, the simulated image can be automatically received, the pixel value of the target corner point can be calculated, and the range of the area where each target can be placed can be determined according to the distance of the target placement, the expected imaging position, and the target size. An evaluation method for target imaging and a process for dynamically adjusting the target position are provided, which can greatly reduce the number of on-site debugging in the calibration room and shorten the design cycle of the calibration board; at the same time, the simulation technology provided by the present invention can be adapted to vehicle models of different sizes, to different camera types such as fisheye cameras and pinhole cameras, and supports the switching of different target graphics, and can conveniently and accurately complete calibration in all directions, thereby improving the design efficiency of the calibration laboratory.

[0117] like Figure 7 FIG. 1 is a schematic diagram showing a structure of a system for designing a camera calibration laboratory based on VTD simulation according to an embodiment of the present invention. Figure 8 As shown, in this embodiment, the system 1 for designing a camera calibration laboratory based on VTD simulation includes at least:

[0118] The target model acquisition unit 10 is used to obtain a VTD model of a target for camera calibration established based on three-dimensional modeling software;

[0119] A simulation scene building unit 11 is used to obtain a VTD simulation scene for camera calibration, and import a simulation vehicle model with a camera installed and a VTD model of the target into corresponding positions of the VTD simulation scene;

[0120] The camera internal and external parameter setting unit 12 is used to generate a corresponding configuration file according to the internal and external parameters of the camera calibration;

[0121] The simulation debugging processing unit 13 is used to perform VTD camera simulation processing according to the imported VTD simulation scene and configuration, evaluate the simulation results, and perform simulation debugging according to the evaluation results, dynamically adjust the size and position of the target, and obtain an optimized target layout plan;

[0122] The target arrangement scheme output unit 14 is used to output the optimized target arrangement scheme so as to build a camera calibration laboratory according to the optimized target arrangement scheme.

[0123] More specifically, the simulation debugging processing unit 13 further includes:

[0124] The initial area determination unit 130 is used to determine the feasible area for placement of each target according to the number of targets to be placed and the expected projection effect, and then dynamically adjust the target position according to the size of the target and the placement distance requirements;

[0125] The rationality evaluation unit 131 is used to perform VTD camera simulation processing, and solve the coordinate true value of the corner point of the target in the pixel coordinate system according to the position of the target in the vehicle coordinate system, and evaluate the rationality of the target arrangement; when all the following conditions are met, the current target arrangement scheme is determined to be a valid scheme; otherwise, dynamic adjustment is required:

[0126] Checkerboard target area S chess Satisfying conditions: 0.8% ≤ S chess ≤5%;

[0127] The total number of identified target corner points is equal to the total number of expected target corner points;

[0128] And the target center pixel coordinates (u, v) meet the following conditions:

[0129] Where S is the total area of ​​the image; w and v are the image height and width of the image;

[0130] The optimal interval determination unit 132 is used to make a judgment according to a preset simulation evaluation program, continuously adjust the position of the target according to the evaluation result, and output the longitudinal, lateral and angular optimal intervals of the target corresponding to the vehicle type;

[0131] The optimized scheme obtaining unit 133 is used to repeat the above operation by changing the vehicle type to obtain the longitudinal, lateral and angular optimal intervals of the targets corresponding to each vehicle type interval, and use the common available position intervals of each interval as the optimized target placement position to obtain the optimized target layout scheme.

[0132] In a specific example, the optimal interval determining unit 132 further includes:

[0133] The judgment processing unit is used to judge whether the longitudinal evaluation standard is met. If not, the longitudinal distance between the target and the camera is adjusted with a predetermined step length until the condition is met; it is used to judge whether the longitudinal evaluation standard is met. If not, the lateral position of the outermost target is adjusted each time to ensure that the outermost target falls within the camera field of view and does not overlap until all target conditions are met; it is used to judge whether the angle evaluation standard is met. If not, the lateral and longitudinal distances between the targets or the target azimuth are further adjusted. If a suitable result is still not obtained, the longitudinal distance between the target and the camera needs to be adjusted, and the above judgment is repeated; if all the above three evaluation standards are met, the longitudinal, lateral and angular optimal intervals of the target corresponding to this vehicle model are obtained.

[0134] For more details, please refer to and combine the above Figures 1 to 7 The description is not repeated here.

[0135] In another aspect of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the above-mentioned Figures 1 to 7 For more details, please refer to and combine the above Figures 1 to 7 The description is not repeated here.

[0136] In another aspect of the present invention, a computing device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the aforementioned Figures 1 to 7 For more details, please refer to and combine the above Figures 1 to 7 The description is not repeated here.

[0137] The implementation of the embodiments of the present invention has the following beneficial effects:

[0138] The present invention provides a camera calibration simulation method, system, storage medium and device. By using VTD to build a camera calibration simulation environment, a camera calibration laboratory is designed, thereby providing a solution for realizing an automated camera calibration room;

[0139] In the embodiment of the present invention, the pixel value of the target corner point is calculated by automatically receiving the simulation image; and the range of the area where each target can be placed is determined according to the distance of the target placement, the expected imaging position, and the target size. At the same time, an evaluation method for target imaging and a process for dynamically adjusting the target position are provided, which can greatly reduce the number of on-site debugging in the calibration room and shorten the design cycle of the calibration board.

[0140] At the same time, the method provided by the present invention can be adapted to vehicle models of different sizes, and to different camera types such as fisheye cameras and pinhole cameras; it supports the switching of different target graphics, and can conveniently and accurately complete the calibration of all directions, thereby improving the design efficiency of the calibration laboratory;

[0141] In summary, the method for building a camera calibration laboratory online provided by the present invention can reduce the number of on-site vehicle debugging in the calibration room, saving costs and design resources.

[0142] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] The above disclosure is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A camera calibration simulation method, characterized in that: At least the following steps are included: Obtain a VTD model of a target for camera calibration based on 3D modeling software; Obtain a VTD simulation scene for camera calibration, and import a simulated vehicle model with a camera installed and a VTD model of the target into corresponding positions of the VTD simulation scene; Generate corresponding configuration files based on the intrinsic and extrinsic parameters of camera calibration; The VTD camera simulation is performed according to the imported VTD simulation scene and configuration file, the simulation results are evaluated, and the size and position of the target are adjusted according to the evaluation results to obtain the optimized target layout plan.

2. The method according to claim 1, characterized in that The step of obtaining a VTD model of a target for camera calibration established based on three-dimensional modeling software further includes: A 3D modeling software is used to build a 3D low-precision model of the target used for camera calibration, the 3D low-precision model is UV unfolded, and corresponding color textures are affixed to form a 3D high-precision model, and the 3D high-precision model is converted into a VTD model format.

3. The method according to claim 2, characterized in that Obtaining a VTD simulation scene for camera calibration, importing a simulated vehicle model with a camera installed and a VTD model of the target into corresponding positions of the VTD simulation scene, further comprising: Build a static scene for VTD camera calibration in the VTD network modeler and import the VTD model of the target to the predetermined position; Build a dynamic scene for simulation calibration in the scene editor and import a simulated vehicle model equipped with a camera.

4. The method according to claim 3, characterized in that The generating of a corresponding configuration file according to the intrinsic parameters and extrinsic parameters of the camera calibration further comprises: In the VTD simulation scene configuration file, complete the external parameter settings for camera calibration based on the location information of the camera installed on the vehicle; Use the lens distortion tool to configure the camera's resolution, focal length, FOV angle, pixel size, and distortion table parameters, generate the corresponding configuration file, and complete the setting of the camera calibration parameters.

5. The method according to any one of claims 1 to 4, characterized in that: The performing VTD camera simulation processing according to the imported VTD simulation scene and configuration file and evaluating the simulation result further includes: According to the expected number of targets to be placed, the feasible area for placing each target is confirmed through the expected projection effect, and then the target position is dynamically adjusted according to the target size and placement distance requirements; Perform VTD camera simulation processing, solve the true value of the target's corner point in the pixel coordinate system according to the target's position in the vehicle coordinate system, and evaluate the rationality of the target layout; When all of the following conditions are met, the current target layout plan is considered to be a valid plan; otherwise, dynamic adjustment is required: Checkerboard target area S chess Satisfying conditions: 0.8% ≤ S chess ≤5%; The total number of identified target corner points is equal to the total number of expected target corner points; And the target center pixel coordinates (u, v) meet the following conditions: Where S is the total area of ​​the image; w and v are the image height and width.

6. The method according to claim 5, characterized in that The step of adjusting the size and position of the target according to the evaluation result to obtain an optimized target arrangement scheme further includes: The pre-set simulation evaluation program is used to make judgments, and the position of the target is continuously adjusted according to the evaluation results, and the longitudinal, lateral and angular optimal ranges of the target corresponding to the vehicle model are output; Repeat the above operation by changing the vehicle model to obtain the optimal longitudinal, lateral and angular intervals of the target corresponding to each vehicle model interval, and use the common available position interval of each interval as the optimized target placement position to obtain the optimized target layout plan.

7. The method according to claim 6, characterized in that The method of continuously adjusting the target position according to the evaluation result and outputting the longitudinal, lateral and angular optimal intervals of the target corresponding to the vehicle model further includes: Determine whether the longitudinal evaluation criteria are met, and if not, adjust the longitudinal distance between the target and the camera by a predetermined step length until the condition is met; Determine whether the longitudinal evaluation criteria are met. If not, adjust the lateral position of the outermost target each time to ensure that the outermost target falls within the camera field of view and does not overlap until all target conditions are met; Determine whether the angle evaluation standard is met. If not, further adjust the horizontal and vertical distances between targets or the target azimuth. If a suitable result is still not obtained, adjust the vertical distance between the target and the camera and repeat the above determination. If the above three evaluation criteria are met, the optimal longitudinal, lateral and angular ranges of the target corresponding to the vehicle model are obtained.

8. The method according to claim 7, characterized in that Further including: The optimized target arrangement scheme is outputted, so as to build a camera calibration laboratory according to the optimized target arrangement scheme.

9. A camera calibration simulation system, characterized in that: At least: A target model acquisition unit, used to obtain a VTD model of a target for camera calibration established based on three-dimensional modeling software; A simulation scene building unit, used to obtain a VTD simulation scene for camera calibration, and import a simulation vehicle model with a camera installed and a VTD model of the target into corresponding positions of the VTD simulation scene; A camera internal and external parameter setting unit, used to generate a corresponding configuration file according to the camera calibrated internal and external parameters; The simulation debugging processing unit is used to perform VTD camera simulation processing according to the imported VTD simulation scene and configuration file, evaluate the simulation results, and adjust the size and position of the target according to the evaluation results to obtain an optimized target layout plan.

10. The system according to claim 9, characterized in that The simulation debugging processing unit further comprises: The initial area determination unit is used to confirm the feasible area for placing each target according to the expected number of targets to be placed and the expected projection effect, and then dynamically adjust the target position according to the size of the target and the placement distance requirements; The rationality evaluation unit is used to perform VTD camera simulation processing and solve the true value of the coordinates of the corner points in the target in the pixel coordinate system according to the position of the target in the vehicle coordinate system, and evaluate the rationality of the target layout; when all the following conditions are met, the current target layout plan is determined to be a valid plan; otherwise, dynamic adjustment is required: Checkerboard target area S chess Satisfying conditions: 0.8% ≤ S chess ≤5%; The total number of identified target corner points is equal to the total number of expected target corner points; And the target center pixel coordinates (u, v) meet the following conditions: Where S is the total area of ​​the image; w and v are the image height and width of the image; The optimal interval determination unit is used to make judgments according to a preset simulation evaluation program, continuously adjust the position of the target according to the evaluation results, and output the longitudinal, lateral and angular optimal intervals of the target corresponding to the vehicle type; The optimized scheme obtaining unit is used to repeat the above operation by changing the vehicle model to obtain the longitudinal, lateral and angular optimal intervals of the targets corresponding to each vehicle model interval, and use the common available position intervals of each interval as the optimized target placement position to obtain the optimized target layout plan.

11. The system according to claim 10, characterized in that The optimal interval determination unit further comprises: The judgment processing unit is used to judge whether the longitudinal evaluation standard is met. If not, the longitudinal distance between the target and the camera is adjusted with a predetermined step length until the condition is met; it is used to judge whether the longitudinal evaluation standard is met. If not, the lateral position of the outermost target is adjusted each time to ensure that the outermost target falls within the camera field of view and does not overlap until all target conditions are met; it is used to judge whether the angle evaluation standard is met. If not, the lateral and longitudinal distances between the targets or the target azimuth are further adjusted. If a suitable result is still not obtained, the longitudinal distance between the target and the camera is adjusted, and the above judgment is repeated; if all the above three evaluation standards are met, the longitudinal, lateral and angular optimal intervals of the target corresponding to the vehicle model are obtained.

12. The system according to any one of claims 9 to 11, characterized in that: Further including: The target arrangement scheme output unit is used to output the optimized target arrangement scheme so as to build a camera calibration laboratory according to the optimized target arrangement scheme.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

14. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images