A visual evaluation method based on multi-camera calibration and related equipment

Abstract

The application discloses a kind of based on multi-camera calibration visual evaluation method and related equipment.Method includes: every interval preset frame number obtains a target image on target road, until target quantity target image is obtained;Respectively, target quantity target 3D point cloud under target quantity groups world coordinate system is converted to target image;Target quantity groups target 3D point cloud is merged, and target point cloud map corresponding to the target road is obtained;According to the target point cloud map, the omnidirectional multi-camera of the target vehicle is evaluated.The visual evaluation method based on multi-camera calibration provided in the application can automatically evaluate the online calibration accuracy of the omnidirectional camera of vehicle, and brings guarantee for user driving safety.

Description

Technical Field

[0001] This invention relates to the field of vehicle extrinsic parameters, and in particular to a visualization evaluation method and related equipment based on multi-camera calibration. Background Technology

[0002] In fields such as autonomous driving, mapping and localization, as well as perception and detection, all require raw data from multiple sensors, including but not limited to multiple cameras, LiDAR, RTK GPS, and IMU. Each sensor has its own advantages and disadvantages, making multi-sensor fusion particularly important. This allows for leveraging the strengths of multiple sensors and mitigating their weaknesses, achieving the goals of various modules and algorithms in autonomous driving. The essence of fusion lies in understanding the extrinsic parameter relationships between each sensor; otherwise, each sensor operates independently, failing to achieve the purpose of fusion.

[0003] Among these, the extrinsic parameters between the vehicle's panoramic cameras and between the cameras and the vehicle are among the most important parameters. There are many algorithms and methods for extrinsic parameter calibration between panoramic cameras, such as: 1) factory calibration, which refers to a specific calibration workshop where the vehicle is calibrated by setting up a checkerboard or other types of calibration boards around it or on the ground; 2) online calibration, which means that during vehicle operation, the sensors collect external environmental data in real time and perform calibration in real time.

[0004] Factory calibration generally offers high accuracy, but it's a one-time process. When autonomous vehicles are on the road, users inevitably encounter various bumps and vibrations, causing sensor positional deviations. This leads to inaccurate extrinsic parameters between sensors, significantly impacting the output accuracy of the autonomous driving algorithm. In such cases, online calibration is crucial. Firstly, it allows for real-time monitoring of extrinsic parameter accuracy and to detect significant changes. Secondly, it enables updating of extrinsic parameters if they are found to be faulty.

[0005] However, online calibration relies on real-time acquisition of external environmental data. If the constraints provided by the external environmental data are insufficient, it can lead to large deviations in the accuracy of the calibration results. Therefore, there is an urgent need for a visual evaluation scheme for periscope multi-camera extrinsic parameter calibration to verify the accuracy of the calibration results and the robustness of the calibration algorithm.

[0006] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0007] To address the aforementioned shortcomings of existing technologies, a visualization evaluation method and related equipment based on multi-camera calibration are provided. This aims to solve the problem that existing technologies lack visualization evaluation methods for extrinsic parameter calibration of panoramic multi-camera systems, and that large deviations in calibration results pose safety hazards to users.

[0008] A first aspect of the present invention provides a visualization evaluation method based on multi-camera calibration, comprising:

[0009] One target image is acquired at preset frame intervals on the target road until the target number of target images are acquired. The target images are AVM images captured by the panoramic multi-camera of the target vehicle.

[0010] Each of the target images described in the target number of images is converted into a target 3D point cloud in the world coordinate system of the target number of groups;

[0011] The target 3D point clouds of the target quantity group are merged to obtain the target point cloud map corresponding to the target road;

[0012] The panoramic multi-camera system of the target vehicle is evaluated based on the target point cloud map.

[0013] The visualization evaluation method based on multi-camera calibration, wherein acquiring a target image at preset frame intervals on the target road includes:

[0014] At each preset number of frames along the target road, a first image, a second image, a third image, a fourth image, a fifth image, and a sixth image captured by a panoramic multi-camera on the target vehicle are acquired, wherein the first image is captured by a first panoramic camera, the second image is captured by a second panoramic camera, the third image is captured by a third panoramic camera, the fourth image is captured by a fourth panoramic camera, the fifth image is captured by a fifth panoramic camera, and the sixth image is captured by a sixth panoramic camera.

[0015] The first image, the second image, the third image, the fourth image, the fifth image, and the sixth image are subjected to distortion correction processing to obtain the first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image. The first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image are BEV images.

[0016] The first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image are merged to obtain the target image.

[0017] The visualization evaluation method based on multi-camera calibration, wherein obtaining the preset number of frames includes:

[0018] Obtain the visible distance of the target vehicle from the panoramic multi-camera and the real-time speed of the target vehicle;

[0019] The preset number of frames is obtained based on the visible distance and the real-time speed.

[0020] The visualization evaluation method based on multi-camera calibration, wherein converting the target number of target images into target 3D point clouds in the target number set world coordinate system includes:

[0021] Obtain the intrinsic parameters and installation height corresponding to the panoramic multi-camera of the target vehicle, and transform each pixel in the target image to the coordinate system of the corresponding virtual camera of the target vehicle to obtain the AVM point cloud;

[0022] The real-time pose of the target vehicle is obtained, and the AVM point cloud in the coordinate system of the virtual camera is transformed to the world coordinate system based on the real-time pose to obtain the target 3D point cloud.

[0023] The visualization evaluation method based on multi-camera calibration, wherein the evaluation of the panoramic multi-camera system of the target vehicle based on the target point cloud map includes:

[0024] The target point cloud image is downsampled to obtain a lightweight point cloud image;

[0025] The panoramic multi-camera system of the target vehicle is evaluated based on the lightweight point cloud map.

[0026] The visualization evaluation method based on multi-camera calibration, wherein the evaluation of the panoramic multi-camera system of the target vehicle based on the lightweight point cloud map includes:

[0027] Extract semantic information from the lightweight point cloud to obtain the target semantic map;

[0028] The panoramic multi-camera system of the target vehicle is evaluated based on the target semantic map.

[0029] The visualization evaluation method based on multi-camera calibration, wherein the evaluation of the panoramic multi-camera view of the target vehicle based on the target semantic map includes:

[0030] Extract lane lines, zebra crossings, and lane boundary lines from the target semantic map;

[0031] The accuracy of the panoramic multi-camera system of the target vehicle is determined based on the smoothness and consistency of lane lines, zebra crossings, and lane boundary lines in the target semantic map.

[0032] A second aspect of the present invention provides a visualization evaluation device based on multi-camera calibration, comprising:

[0033] The image acquisition module is used to acquire one target image at a preset frame interval on the target road until a target number of target images are acquired. The target images are AVM images captured by the panoramic multi-camera of the target vehicle.

[0034] The conversion module is used to convert the target number of target images into target 3D point clouds in the world coordinate system of the target number of groups;

[0035] The merging module is used to merge the target 3D point clouds of the target quantity group to obtain the target point cloud map corresponding to the target road;

[0036] An evaluation module is used to evaluate the panoramic multi-camera system of the target vehicle based on the target point cloud map.

[0037] A third aspect of the present invention provides a terminal comprising: a processor and a storage medium communicatively connected to the processor, the storage medium being adapted to store a plurality of instructions, and the processor being adapted to invoke the instructions in the storage medium to execute the steps of implementing the visualization evaluation method based on multi-camera calibration as described in any of the preceding claims.

[0038] In a fourth aspect, the present invention provides a storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the visualization evaluation method based on multi-camera calibration as described in any of the preceding claims.

[0039] Beneficial Effects: Compared with existing technologies, this invention provides a visualization evaluation method and related equipment based on multi-camera calibration. In this method, a target image is acquired at preset frame intervals along the target road until a target number of images are acquired. These target images are AVM images captured by the panoramic multi-camera system of the target vehicle. Then, the target number of target images are converted into target 3D point clouds in a target number of world coordinate systems, and these target number of 3D point clouds are merged to obtain a target point cloud map corresponding to the target road. Finally, the panoramic multi-camera system of the target vehicle is evaluated based on the target point cloud map. This visualization evaluation method based on multi-camera calibration allows for coordinate transformation of pixels in images captured by the panoramic multi-camera system of the target vehicle to obtain 3D point clouds. These points are then stitched together to obtain a 3D image of the target vehicle driving on the target road. The performance of the panoramic multi-camera system of the target vehicle is evaluated based on the 3D image, thus obtaining the accuracy of online calibration of the target vehicle and improving driving safety for users. Attached Figure Description

[0040] Figure 1A flowchart illustrating an embodiment of the visualization evaluation method based on multi-camera calibration provided by the present invention;

[0041] Figure 2 A panoramic camera installation diagram of an embodiment of the visualization evaluation method based on multi-camera calibration provided by the present invention;

[0042] Figure 3 The evaluation flowchart is shown in an embodiment of the visualization evaluation method based on multi-camera calibration provided by the present invention.

[0043] Figure 4 Example 1 of a 3D point cloud image in an embodiment of the visualization evaluation method based on multi-camera calibration provided by the present invention;

[0044] Figure 5 Example 2 of 3D point cloud map in an embodiment of the visualization evaluation method based on multi-camera calibration provided by the present invention;

[0045] Figure 6 Example 3 of the 3D point cloud map in the embodiment of the visualization evaluation method based on multi-camera calibration provided by the present invention;

[0046] Figure 7 Example 4 of the 3D point cloud map in the embodiments of the visualization evaluation method based on multi-camera calibration provided by the present invention;

[0047] Figure 8 Example 5 of the 3D point cloud map in the embodiments of the visualization evaluation method based on multi-camera calibration provided by the present invention;

[0048] Figure 9 This is a schematic diagram of an embodiment of the visualization evaluation device based on multi-camera calibration provided by the present invention;

[0049] Figure 10 A schematic diagram of the structure of an embodiment of the terminal provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0051] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0052] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0053] Example 1

[0054] The visualization evaluation method based on multi-camera calibration provided in this embodiment can be executed by a terminal, which can be, but is not limited to, a smart car, a computer, etc. The following explanation uses a smart car as an example. Figure 1 As shown, the visualization evaluation method based on multi-camera calibration provided in this embodiment includes the following steps:

[0055] This embodiment provides a visualization evaluation method based on multi-camera calibration. For example... Figure 1 As shown, the visualization evaluation method based on multi-camera calibration provided by this invention includes the following steps:

[0056] S100. Acquire one target image at preset frame intervals on the target road until the target number of target images are acquired. The target images are AVM images captured by the panoramic multi-camera of the target vehicle.

[0057] One target image is acquired at preset frame intervals along the target road, including:

[0058] At each preset number of frames along the target road, a first image, a second image, a third image, a fourth image, a fifth image, and a sixth image captured by a panoramic multi-camera on the target vehicle are acquired, wherein the first image is captured by a first panoramic camera, the second image is captured by a second panoramic camera, the third image is captured by a third panoramic camera, the fourth image is captured by a fourth panoramic camera, the fifth image is captured by a fifth panoramic camera, and the sixth image is captured by a sixth panoramic camera.

[0059] The first image, the second image, the third image, the fourth image, the fifth image, and the sixth image are subjected to distortion correction processing to obtain the first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image. The first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image are BEV images.

[0060] The first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image are merged to obtain the target image.

[0061] Specifically, refer to Figure 2 , Figure 2 This is an installation diagram of the panoramic cameras of the target vehicle described in this embodiment. In this embodiment, the target vehicle includes six panoramic cameras: a first panoramic camera c1, a second panoramic camera c2, a third panoramic camera c3, a fourth panoramic camera c4, a fifth panoramic camera c5, and a sixth panoramic camera c6. C1 and C4 are the front-view and rear-view cameras, respectively, while C2, C3, C5, and C6 are all side-view cameras.

[0062] In this embodiment, the image captured by c1 is the first image, the image captured by c2 is the second image, the image captured by c3 is the third image, the image captured by c4 is the fourth image, the image captured by c5 is the fifth image, and the image captured by c6 is the sixth image.

[0063] Reference Figure 3 After acquiring the first image, the second image, the third image, the fourth image, the fifth image, and the sixth image captured by the six panoramic cameras, distortion correction processing is required to obtain six BEV images: the first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image.

[0064] Then, the first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image are merged to obtain the target image. The target image is an AVM image captured by the panoramic multi-camera system of the target vehicle.

[0065] Specifically, a target image needs to be acquired at preset frame intervals until the target number of target images are acquired on the same target road.

[0066] In this embodiment, obtaining the preset number of frames includes:

[0067] Obtain the visible distance of the target vehicle from the panoramic multi-camera and the real-time speed of the target vehicle;

[0068] The preset number of frames is obtained based on the visible distance and the real-time speed.

[0069] To facilitate the stitching of the target images, it is necessary to select images of the preceding and following frames with appropriate intervals.

[0070] Specifically, because image frame rates are generally high, such as 10–30Hz, the overlapping area between two adjacent frames is too large. If they were stitched together, they would overlap and become unwatchable. Therefore, it is necessary to select two frames at an appropriate distance for stitching.

[0071] Specifically, the visible distance of the target vehicle from the panoramic multi-camera and the real-time speed of the target vehicle are obtained. The visible distance is divided by the real-time speed to obtain the time it takes for the vehicle to travel the road surface corresponding to the current frame. Then, the appropriate preset number of frames can be obtained based on the frame rate corresponding to the panoramic camera.

[0072] The target number is a pre-set parameter; in this embodiment, the preset number is 5-10 images. In other embodiments, if a more accurate evaluation result is required, more target images can be used to evaluate the multi-camera online calibration results of the target vehicle.

[0073] Refer again Figure 1 As can be seen, the visualization evaluation method based on multi-camera calibration further includes the following steps:

[0074] S200. Convert the target number of images into target 3D point clouds in the world coordinate system of the target number group.

[0075] The step of converting the target number of images into a target number of 3D point clouds in the world coordinate system includes:

[0076] S210. Obtain the intrinsic parameters and installation height corresponding to the panoramic multi-camera of the target vehicle, and transform each pixel in the target image to the coordinate system of the corresponding virtual camera of the target vehicle to obtain the AVM point cloud.

[0077] S220. Obtain the real-time pose of the target vehicle, and transform the AVM point cloud in the coordinate system of the virtual camera to the world coordinate system based on the real-time pose to obtain the target 3D point cloud.

[0078] Specifically, due to AVM Figure 1 Generally, it is established in the vehicle's virtual camera coordinate system, where the virtual camera is positioned in a bird's-eye view from the roof of the target vehicle. The intrinsic and extrinsic parameters of the virtual camera are predefined.

[0079] Therefore, since the intrinsic parameters of the virtual camera are known, and the height of the virtual camera on the roof of the vehicle is known, each pixel in the 2D AVM image can be transformed into the coordinate system of the virtual camera to generate the 3D AVM point cloud. In this way, each pixel becomes a 3D point, and the entire AVM image becomes a 3D point cloud.

[0080] Then, by obtaining the real-time pose of the target vehicle corresponding to the current frame, the AVM point cloud is transformed into the world coordinate system, thus obtaining the target 3D point cloud.

[0081] This yields the target 3D point cloud as described in the target number group.

[0082] S300. Merge the target 3D point clouds of the target quantity group to obtain the target point cloud map corresponding to the target road.

[0083] Since the pixels in the target image are all converted into 3D points in the world coordinate system, the target number group of the target 3D point cloud can be easily merged and saved to obtain continuous road surface conditions, which is convenient for subsequent visualization.

[0084] S400. Evaluate the panoramic multi-camera system of the target vehicle based on the target point cloud map.

[0085] In this embodiment, the display software is CloudCompare, which displays the target 3D point cloud of the target number group to observe the smoothness, fit, etc. of the connection points, and compares the parameter results of online calibration and workshop calibration.

[0086] Reference Figure 4 , Figure 4The left and right images show the stitched 3DAVM road point cloud maps generated by the target vehicle using workshop calibration parameters and online calibration parameters, respectively. Based on road smoothness and fit, it can be seen that online calibration is more effective than workshop calibration, with higher extrinsic parameter accuracy.

[0087] Reference Figure 5 , Figure 5 In the two boxes on the left, the lane lines have obvious ghosting and misalignment, while the lane lines on the right do not have ghosting, indicating that the calibration effect on the right is better and the external parameter accuracy is higher.

[0088] Reference Figure 6 , Figure 6 The narrowing of the lower lane in the left image indicates an issue with the pitch angle of the extrinsic parameters. In contrast, the consistent lane width in the right image suggests better calibration on the right.

[0089] Reference Figure 7 , Figure 7 In the left image, the lane widths at the two boxes are inconsistent, indicating a problem with the extrinsic parameter pitch angle. The right image, however, shows a natural connection and is a complete road, indicating better calibration and higher extrinsic parameter accuracy.

[0090] Reference Figure 8 , Figure 8 The misaligned lane lines in the left image indicate a problem with the extrinsic parameters. In contrast, the lane lines in the right image are consistently aligned, suggesting a better calibration and higher extrinsic parameter accuracy.

[0091] In addition to evaluating the target vehicle through manual assessment of the visualized images generated by the panoramic multi-camera system, the evaluation can also be performed automatically through further image processing.

[0092] Further, the evaluation of the panoramic multi-camera system of the target vehicle based on the target point cloud map includes:

[0093] S410. Perform downsampling processing on the target point cloud map to obtain a lightweight point cloud map;

[0094] S420. Evaluate the panoramic multi-camera system of the target vehicle based on the lightweight point cloud map.

[0095] The evaluation of the panoramic multi-camera system of the target vehicle based on the lightweight point cloud map includes:

[0096] S421. Extract semantic information from the lightweight point cloud map to obtain the target semantic map;

[0097] S422. Evaluate the panoramic multi-camera system of the target vehicle based on the target semantic map.

[0098] The evaluation of the panoramic multi-camera system of the target vehicle based on the target semantic map includes:

[0099] Extract lane lines, zebra crossings, and lane boundary lines from the target semantic map;

[0100] The accuracy of the panoramic multi-camera system of the target vehicle is determined based on the smoothness and consistency of lane lines, zebra crossings, and lane boundary lines in the target semantic map.

[0101] Specifically, the target point cloud map is a colored road map. The target point cloud map is further downsampled to become a lightweight colored road point cloud map, resulting in the lightweight point cloud map. Then, semantic information is extracted from the lightweight point cloud map to obtain the target semantic map. By pre-setting semantic thresholds, the panoramic multi-camera system of the target vehicle can be evaluated based on the target semantic map.

[0102] Furthermore, online mapping can be achieved based on the target point cloud map. For example, the number of targets can be increased to a sufficient number to create a regional map.

[0103] Specifically, when the extrinsic parameters are very accurate, the stitched 3D point cloud map can serve as an initial version of the point cloud map, and it also contains color information. By downsampling and generating a more lightweight point cloud color map, it can also be used as an input map to assist in localization and road detection.

[0104] In summary, this embodiment provides a visualization evaluation method based on multi-camera calibration. It acquires a target image on the target road at preset frame intervals until a target number of such images are obtained. These target images are AVM images captured by the panoramic multi-camera system of the target vehicle. Then, the target number of these target images are converted into a target number of target 3D point clouds in a target number of world coordinate systems. These target number of target 3D point clouds are then merged to obtain a target point cloud map corresponding to the target road. Finally, the panoramic multi-camera system of the target vehicle is evaluated based on the target point cloud map. This visualization evaluation method based on multi-camera calibration allows for coordinate transformation of pixels in images captured by the panoramic multi-camera system of the target vehicle to obtain 3D point clouds. These points are then stitched together to obtain a 3D image of the target vehicle driving on the target road. The performance of the panoramic multi-camera system of the target vehicle is evaluated based on the 3D image, thus obtaining the accuracy of online calibration of the target vehicle and improving driving safety for users.

[0105] It should be understood that although the steps in the flowcharts shown in the accompanying drawings are displayed sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of the steps in this invention, and these steps can be executed in other orders. Moreover, at least a portion of the steps in this invention may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0106] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0107] Example 2

[0108] Based on the above embodiments, the present invention also provides a visualization evaluation device based on multi-camera calibration, the functional module diagram of which is shown below. Figure 9 As shown, the visualization evaluation device based on multi-camera calibration includes:

[0109] The image acquisition module is used to acquire one target image at a preset frame interval on the target road until a target number of target images are acquired. The target images are AVM images captured by the panoramic multi-camera of the target vehicle, as specifically described in Embodiment 1.

[0110] The conversion module is used to convert the target number of target images into target 3D point clouds in the world coordinate system of the target number of groups, as described in Embodiment 1.

[0111] The merging module is used to merge the target 3D point clouds of the target quantity group to obtain the target point cloud map corresponding to the target road, as specifically described in Embodiment 1;

[0112] An evaluation module is used to evaluate the panoramic multi-camera system of the target vehicle based on the target point cloud map, as specifically described in Embodiment 1.

[0113] Example 3

[0114] Based on the visualization evaluation method based on multi-camera calibration described in Embodiment 1 above, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 10 As shown. The terminal includes a memory 10 and a processor 20. The memory 10 stores a visualization evaluation program based on multi-camera calibration. When the processor 20 executes the computer program, it can perform at least the following steps:

[0115] One target image is acquired at preset frame intervals on the target road until the target number of target images are acquired. The target images are AVM images captured by the panoramic multi-camera of the target vehicle.

[0116] Each of the target images described in the target number of images is converted into a target 3D point cloud in the world coordinate system of the target number of groups;

[0117] The target 3D point clouds of the target quantity group are merged to obtain the target point cloud map corresponding to the target road;

[0118] The panoramic multi-camera system of the target vehicle is evaluated based on the target point cloud map.

[0119] The step of acquiring a target image at preset frame intervals on the target road includes:

[0120] At each preset number of frames along the target road, a first image, a second image, a third image, a fourth image, a fifth image, and a sixth image captured by a panoramic multi-camera on the target vehicle are acquired, wherein the first image is captured by a first panoramic camera, the second image is captured by a second panoramic camera, the third image is captured by a third panoramic camera, the fourth image is captured by a fourth panoramic camera, the fifth image is captured by a fifth panoramic camera, and the sixth image is captured by a sixth panoramic camera.

[0121] The first image, the second image, the third image, the fourth image, the fifth image, and the sixth image are subjected to distortion correction processing to obtain the first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image. The first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image are BEV images.

[0122] The first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image are merged to obtain the target image.

[0123] Obtaining the preset frame number includes:

[0124] Obtain the visible distance of the target vehicle from the panoramic multi-camera and the real-time speed of the target vehicle;

[0125] The preset number of frames is obtained based on the visible distance and the real-time speed.

[0126] The step of converting the target number of target images into a target number of target 3D point clouds in the world coordinate system includes:

[0127] Obtain the intrinsic parameters and installation height corresponding to the panoramic multi-camera of the target vehicle, and transform each pixel in the target image to the coordinate system of the corresponding virtual camera of the target vehicle to obtain the AVM point cloud;

[0128] The real-time pose of the target vehicle is obtained, and the AVM point cloud in the coordinate system of the virtual camera is transformed to the world coordinate system based on the real-time pose to obtain the target 3D point cloud.

[0129] The step of evaluating the panoramic multi-camera system of the target vehicle based on the target point cloud map includes:

[0130] The target point cloud image is downsampled to obtain a lightweight point cloud image;

[0131] The panoramic multi-camera system of the target vehicle is evaluated based on the lightweight point cloud map.

[0132] The step of evaluating the panoramic multi-camera system of the target vehicle based on the lightweight point cloud map includes:

[0133] Extract semantic information from the lightweight point cloud to obtain the target semantic map;

[0134] The panoramic multi-camera system of the target vehicle is evaluated based on the target semantic map.

[0135] The step of evaluating the panoramic multi-camera system of the target vehicle based on the target semantic map includes:

[0136] Extract lane lines, zebra crossings, and lane boundary lines from the target semantic map;

[0137] The accuracy of the panoramic multi-camera system of the target vehicle is determined based on the smoothness and consistency of lane lines, zebra crossings, and lane boundary lines in the target semantic map.

[0138] Example 4

[0139] The present invention also provides a storage medium storing one or more programs that can be executed by one or more processors to implement the steps of the visualization evaluation method based on multi-camera calibration described in the above embodiments.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A visualization evaluation method based on multi-camera calibration, characterized in that, include: One target image is acquired at a preset number of frames on the target road until the target number of target images are acquired. The target images are AVM images captured by the panoramic multi-camera of the target vehicle. Each of the target images described in the target number of images is converted into a target 3D point cloud in the world coordinate system of the target number of groups; The target 3D point clouds of the target quantity group are merged to obtain the target point cloud map corresponding to the target road; The panoramic multi-camera system of the target vehicle is evaluated based on the target point cloud map. The evaluation of the panoramic multi-camera system of the target vehicle based on the target point cloud map includes: The target point cloud map is downsampled to obtain a lightweight point cloud map, which is a colored road map. The panoramic multi-camera system of the target vehicle is evaluated based on the lightweight point cloud map. The evaluation of the panoramic multi-camera system of the target vehicle based on the lightweight point cloud map includes: Semantic information is extracted from the lightweight point cloud map to obtain a target semantic map, and the panoramic multi-camera of the target vehicle is evaluated based on the target semantic map using a preset semantic threshold. The panoramic multi-camera system of the target vehicle is evaluated based on the target semantic map; The evaluation of the panoramic multi-camera system of the target vehicle based on the target semantic map includes: Extract lane lines, zebra crossings, and lane boundary lines from the target semantic map; The accuracy of the target vehicle's panoramic multi-camera system is determined based on the smoothness and consistency of lane lines, zebra crossings, and lane boundary lines in the target semantic map. The evaluation of the target vehicle's panoramic multi-camera system based on the target point cloud map includes: The target 3D point cloud is displayed using the display software to observe the smoothness and fit of the connection points, and to compare the parameter results using online calibration and workshop calibration.

2. The visualization evaluation method based on multi-camera calibration according to claim 1, characterized in that, The step of acquiring a target image at preset frame intervals on the target road includes: At each preset number of frames along the target road, a first image, a second image, a third image, a fourth image, a fifth image, and a sixth image captured by a panoramic multi-camera on the target vehicle are acquired, wherein the first image is captured by a first panoramic camera, the second image is captured by a second panoramic camera, the third image is captured by a third panoramic camera, the fourth image is captured by a fourth panoramic camera, the fifth image is captured by a fifth panoramic camera, and the sixth image is captured by a sixth panoramic camera. The first image, the second image, the third image, the fourth image, the fifth image, and the sixth image are subjected to distortion correction processing to obtain the first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image. The first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image are BEV images. The first target image, the second target image, the third target image, the fourth target image, the fifth target image, and the sixth target image are merged to obtain the target image.

3. The visualization evaluation method based on multi-camera calibration according to claim 1, characterized in that, Obtaining the preset frame number includes: Obtain the visible distance of the target vehicle from the panoramic multi-camera and the real-time speed of the target vehicle; The preset number of frames is obtained based on the visible distance and the real-time speed.

4. The visualization evaluation method based on multi-camera calibration according to claim 1, characterized in that, The step of converting the target number of images into a target number of 3D point clouds in the world coordinate system includes: Obtain the intrinsic parameters and installation height corresponding to the panoramic multi-camera of the target vehicle, and transform each pixel in the target image to the coordinate system of the corresponding virtual camera of the target vehicle to obtain the AVM point cloud; The real-time pose of the target vehicle is obtained, and the AVM point cloud in the coordinate system of the virtual camera is transformed to the world coordinate system based on the real-time pose to obtain the target 3D point cloud.

5. A visualization evaluation device based on multi-camera calibration, characterized in that, The device includes: The image acquisition module is used to acquire one target image at a preset frame interval on the target road until a target number of target images are acquired. The target images are AVM images captured by the panoramic multi-camera of the target vehicle. The conversion module is used to convert the target number of target images into target 3D point clouds in the world coordinate system of the target number of groups; The merging module is used to merge the target 3D point clouds of the target quantity group to obtain the target point cloud map corresponding to the target road; An evaluation module is used to evaluate the panoramic multi-camera system of the target vehicle based on the target point cloud map. The evaluation of the panoramic multi-camera system of the target vehicle based on the target point cloud map includes: The target point cloud map is downsampled to obtain a lightweight point cloud map, which is a colored road map. The panoramic multi-camera system of the target vehicle is evaluated based on the lightweight point cloud map. The evaluation of the panoramic multi-camera system of the target vehicle based on the lightweight point cloud map includes: Semantic information is extracted from the lightweight point cloud map to obtain a target semantic map, and the panoramic multi-camera of the target vehicle is evaluated based on the target semantic map using a preset semantic threshold. The panoramic multi-camera system of the target vehicle is evaluated based on the target semantic map; The evaluation of the panoramic multi-camera system of the target vehicle based on the target semantic map includes: Extract lane lines, zebra crossings, and lane boundary lines from the target semantic map; The accuracy of the target vehicle's panoramic multi-camera system is determined based on the smoothness and consistency of lane lines, zebra crossings, and lane boundary lines in the target semantic map. The evaluation of the target vehicle's panoramic multi-camera system based on the target point cloud map includes: The target 3D point cloud is displayed using the display software to observe the smoothness and fit of the connection points, and to compare the parameter results using online calibration and workshop calibration.

6. A terminal, characterized in that, The terminal includes: a processor and a storage medium communicatively connected to the processor. The storage medium is adapted to store multiple instructions, and the processor is adapted to call the instructions in the storage medium to execute the steps of implementing the visualization evaluation method based on multi-camera calibration as described in any one of claims 1-4.

7. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the visualization evaluation method based on multi-camera calibration as described in any one of claims 1-4.

Images