A calibration precision evaluation method and device, electronic equipment, and storage medium

Abstract

The application discloses a calibration precision evaluation method and device, an electronic device, and a storage medium. The method comprises the following steps: acquiring a road image of a road traffic sign captured by a camera and a local high-precision map point cloud corresponding to the camera; acquiring a virtual point cloud corresponding to the road traffic sign in the road image according to a calibration parameter of the camera; performing point particle matching on the local high-precision map point cloud and the virtual point cloud to obtain point particles that are mutually matched in the local high-precision map point cloud and the virtual point cloud; and evaluating the calibration precision of the camera according to a position deviation between the mutually matched point particles to obtain an evaluation result. The technical solution of the application can avoid the limitations of the evaluation method based on a calibration reference, and can improve the reliability of the evaluation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a calibration accuracy evaluation method and device, an electronic device, and a storage medium. BACKGROUND

[0002] In an intelligent transportation system, through wireless communication and Internet technology, real-time information exchange between vehicles and between vehicles and roads is achieved, that is, vehicle-road cooperation is achieved. A roadside perception system provides super-vision perception information for the realization of vehicle-road cooperation. The calibration accuracy of a camera, which is one of the most important sensors in the roadside perception system, determines the perception accuracy of the camera for obstacles on the road. Therefore, it is necessary to evaluate the calibration accuracy of the camera.

[0003] The evaluation method in the prior art usually sets a calibration reference object manually, selects a small number of marker points on the calibration reference object, performs orthographic projection onto an image, then calculates the pixel coordinate deviation between the projection points and the original pixel points of the image, and evaluates the calibration parameters of the camera according to the total error or average error of the projection of these marker points.

[0004] The above evaluation method needs to set a calibration reference object manually, and the limitations of the setting of the calibration reference object will result in limitations in the calibration accuracy evaluation process, and the number of marker points that can be selected on the calibration reference object is small, so that the evaluation result is unreliable. SUMMARY

[0005] Based on the above problems existing in the prior art, the embodiments of the present application provide a calibration accuracy evaluation method and device, an electronic device, and a storage medium to overcome the limitations of manually setting a calibration reference object and improve the reliability of the calibration accuracy evaluation result.

[0006] The embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, the embodiments of the present application provide a calibration accuracy evaluation method, which comprises:

[0008] obtaining a road image containing a road traffic sign and a local high-definition map point cloud corresponding to the road image, which are captured by a camera;

[0009] obtaining a virtual point cloud corresponding to the road traffic sign in the road image according to the calibration parameters of the camera;

[0010] performing point-grain matching on the local high-definition map point cloud and the virtual point cloud to obtain point grains in the local high-definition map point cloud that are mutually matched with the virtual point cloud;

[0011] evaluating the calibration accuracy of the camera according to the positional deviation between the mutually matched point grains to obtain an evaluation result.

[0012] Optionally, the step of performing particle matching between the local high-precision map point cloud and the virtual point cloud to obtain mutually matching particles in the local high-precision map point cloud and the virtual point cloud includes:

[0013] The least Euclidean distance matching algorithm is used to determine the ground truth particles that match each particle in the virtual point cloud from the local high-precision map point cloud.

[0014] Optionally, the step of performing particle matching between the local high-precision map point cloud and the virtual point cloud to obtain mutually matching particles in the local high-precision map point cloud and the virtual point cloud includes:

[0015] The virtual point cloud and the local high-precision map point cloud are visualized using visualization software;

[0016] Based on the visualization results of the virtual point cloud and the local high-precision map point cloud, the matching points in the local high-precision map point cloud and the virtual point cloud are determined by the particle selection command.

[0017] Optionally, the evaluation results include visual evaluation results, and the evaluation of the camera's calibration accuracy based on the positional deviation between the mutually matched dots includes:

[0018] The visualization software's point cloud visualization display interface is cropped, and the point cloud visualization display interface overlays the virtual point cloud and the local high-precision map point cloud on the same screen, and the virtual point cloud and the local high-precision map point cloud have different visualization effects.

[0019] The captured image is used as the visualization evaluation result.

[0020] Optionally, the evaluation result includes a calibration error evaluation result, wherein evaluating the calibration accuracy of the camera based on the positional deviation between the matched dots includes:

[0021] Obtain the positional deviation value between each matching particle;

[0022] Using the distance information between the virtual point cloud particles and the camera as the horizontal axis and the positional deviation corresponding to each particle as the vertical axis, a calibration error fitting curve is constructed.

[0023] The calibration error fitting curve is determined as the calibration error evaluation result.

[0024] Optionally, obtaining the virtual point cloud corresponding to the road traffic signs in the road image based on the camera's calibration parameters includes:

[0025] Target detection is performed on the road image to obtain the area information of road traffic signs;

[0026] Based on the area information of the road traffic sign, obtain the image pixel coordinates of the foreground pixel of the road traffic sign;

[0027] Based on the camera's calibration parameters and the image pixel coordinates of the foreground pixels of the road traffic sign, a virtual point cloud corresponding to the foreground pixels of the road traffic sign is obtained.

[0028] Optionally, the step of performing target detection on the road image to obtain the area information of road traffic signs includes:

[0029] The road image is converted into a mask image based on an adaptive threshold algorithm. The intersection of the mask image and the road image is then processed to obtain the mask sub-image of the road traffic sign.

[0030] The step of obtaining the image pixel coordinates of the foreground pixels of the road traffic sign based on the area information of the road traffic sign includes:

[0031] Based on the pixel values ​​of the pixels in the Mask sub-image of the road traffic sign, the image pixel coordinates of the foreground pixels of the road traffic sign are obtained.

[0032] Secondly, embodiments of this application also provide a calibration accuracy evaluation device, the device comprising:

[0033] The acquisition unit is used to acquire road images containing road traffic signs captured by the camera and the corresponding local high-precision map point cloud of the road images;

[0034] The computing unit is used to obtain the virtual point cloud corresponding to the road traffic signs in the road image based on the calibration parameters of the camera.

[0035] A matching unit is used to perform particle matching between the local high-precision map point cloud and the virtual point cloud to obtain mutually matching particles in the local high-precision map point cloud and the virtual point cloud.

[0036] The evaluation unit is used to evaluate the calibration accuracy of the camera based on the positional deviation between the mutually matched dots and obtain the evaluation result.

[0037] Thirdly, embodiments of this application also provide an electronic device, including:

[0038] Processor; and

[0039] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform a calibration accuracy evaluation method.

[0040] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform a calibration accuracy evaluation method.

[0041] The above-mentioned technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: The embodiments of this application first acquire road images captured by the camera and corresponding local high-precision map point clouds, then obtain a large number of pixels from the road surface traffic signs in the road images as reference points, determine the virtual point cloud corresponding to these pixels based on the camera's calibration parameters, then determine the matching particles in the virtual point cloud and the local high-precision map point cloud, and finally evaluate the camera's calibration accuracy based on the positional deviation between these matching particles.

[0042] The evaluation process in this application embodiment does not require manually setting calibration reference objects on the road surface, thus avoiding the limitations of evaluation methods based on calibration reference objects; and the road traffic signs in the road image generally occupy the global area or most of the road image area, so the evaluation method can use a sufficient number of reference points, and the number of reference points can cover the entire image area or most of the image area, which can improve the reliability of the evaluation results. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0044] Figure 1 This is a flowchart illustrating a calibration accuracy evaluation method in an embodiment of this application;

[0045] Figure 2 This is a schematic diagram of a mask image of a road image shown in an embodiment of this application;

[0046] Figure 3 This is a schematic diagram of the Mask sub-image of a road traffic sign shown in an embodiment of this application;

[0047] Figure 4 This is a schematic diagram illustrating the visualization effect of local high-precision map point cloud and virtual point cloud in the embodiments of this application;

[0048] Figure 5 This is a schematic diagram of the structure of a calibration accuracy evaluation device shown in the embodiments of this application;

[0049] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0052] The execution entity of the calibration accuracy evaluation method provided in this application embodiment can be a roadside device (e.g., a roadside camera or roadside computing device), a server, or a cloud control platform; it can also be a perception system, a fused perception system, or a planning / control system integrating the above systems, such as an assisted driving system or an autonomous driving system. The execution entity of the calibration accuracy evaluation method in this application embodiment can be software or hardware.

[0053] Please refer to Figure 1 , Figure 1 Taking roadside equipment as an example, this application provides a method for evaluating calibration accuracy. Figure 1 As shown, the calibration accuracy evaluation method provided in this application embodiment may include the following steps S110 to S140:

[0054] Step S110: Obtain a road image captured by the camera, including road traffic signs, and a local high-precision map point cloud corresponding to the road image.

[0055] This application embodiment can evaluate the accuracy of calibration parameters for roadside cameras or vehicle-mounted cameras. A roadside camera refers to a camera mounted on a roadside pole with a fixed position and state, while a vehicle-mounted camera refers to a monocular camera mounted around a vehicle body. When evaluating the accuracy of calibration parameters for a roadside camera, a road image containing road traffic signs is captured by the roadside camera, and the corresponding local high-precision map point cloud is determined based on the position of the roadside pole where the roadside camera is located. When evaluating the accuracy of calibration parameters for a vehicle-mounted camera, a road image containing road traffic signs can be captured by the vehicle-mounted camera while the vehicle is traveling on the road, and the vehicle's position information at the time the road image was captured can be obtained. The corresponding local high-precision map point cloud is then obtained based on the vehicle's position information.

[0056] The road traffic signs in this application embodiment are mainly used to control and guide traffic, and are usually composed of various lines marked on the road surface (stop lines, pedestrian crossings, lane dividing lines, etc., dotted or solid lines), arrow lines (turn left, turn right, turn left and go straight, turn right and go straight, etc.), text, delineators, etc.

[0057] The high-definition map (HD Map) in this embodiment can be derived from a high-definition map designed for vehicle driving, especially a high-definition map designed for autonomous vehicles. The HD Map can contain elements for vehicle navigation, such as road information, intersection information, traffic signal information, and lane rule information.

[0058] Step S120: Based on the calibration parameters of the camera, obtain the virtual point cloud corresponding to the road traffic sign in the road image.

[0059] Camera calibration parameters indicate the transformation relationship between image pixel coordinates and world coordinates. Generally, camera calibration parameters include intrinsic parameters and extrinsic parameters. Intrinsic parameters, abbreviated as K, include focal length, principal point coordinates, and distortion parameters; extrinsic parameters, abbreviated as Tcw, include rotation matrix and translation vector. The transformation relationship between image pixel coordinates and camera coordinates can be obtained based on the intrinsic parameters K, while the transformation relationship between camera coordinates and world coordinates can be obtained based on the extrinsic parameters Tcw.

[0060] The embodiments of this application can identify the pixels of road traffic signs in road images, and project the pixels of road traffic signs in road images based on calibration parameters to obtain a virtual point cloud.

[0061] In this application, image pixel coordinates refer to the coordinates of the pixel at the location of the reference point in the image; these coordinates are two-dimensional. World coordinates are the three-dimensional coordinates of every real point in a geographic region. It can be understood that in the physical world, the same point can have different coordinate values ​​in different coordinate systems. In this application, the world coordinates of the reference point can be coordinates in any coordinate system. For example, the world coordinates of the reference point can be three-dimensional coordinates composed of the longitude, latitude, and altitude corresponding to the reference point; it can also be three-dimensional coordinates composed of the X, Y, and Z coordinates in the natural coordinate system corresponding to the reference point; or it can be other forms of coordinates, as long as these coordinates can uniquely determine the location of the reference point in the geographic region. This application does not limit the specific form of coordinates used. Optionally, the world coordinates used in this application are world coordinates provided by high-precision map point cloud data, such as WGS84 coordinates.

[0062] Step S130: Perform particle matching on the local high-precision map point cloud and the virtual point cloud to obtain the matching particles in the local high-precision map point cloud and the virtual point cloud.

[0063] It is understood that in this application's embodiments, "particles" refers to each point in the point cloud. The number of particles constituting the virtual point cloud is far less than the number of particles in a local high-definition map point cloud. Theoretically, there are more particles in a local high-definition map point cloud.

[0064] The pixels that match each pixel in the virtual point cloud. Therefore, embodiments of this application can use manual or automatic matching methods to match pixels in two point clouds, and determine the camera calibration error based on the positional deviation between the matched pixels.

[0065] Step S140: Evaluate the calibration accuracy of the camera based on the positional deviation between the mutually matched dots to obtain the evaluation result.

[0066] like Figure 1 As shown in the calibration accuracy evaluation method, this embodiment first acquires road images captured by the camera and corresponding local high-precision map point clouds. Then, it obtains a large number of pixels from road surface traffic signs in the road images as reference points, and determines the virtual point cloud corresponding to these pixels based on the camera's calibration parameters.

[0067] The system identifies matching pixels in the virtual point cloud and the local high-precision map point cloud, and evaluates the camera calibration accuracy based on the positional deviations between these five matching pixels. The evaluation process in this embodiment is automated.

[0068] To set up calibration references on the calibration road surface, the limitations of evaluation methods based on calibration references can be avoided; and since road traffic signs in road images generally occupy the global area or most of the road image area, the evaluation method can use a sufficient number of reference points, and the number of reference points can cover the entire image area or most of the image area, which can improve the reliability of the evaluation results.

[0069] The camera in this application can be a bullet camera, a fisheye camera, or a dome camera. A road image captured by a bullet camera is a bullet-view image, which can be equivalent to an image acquired by a bullet camera along a fixed angle, such as... Figure 2 or Figure 3 As shown, images from a gun's perspective can exhibit the characteristic of near objects appearing larger than distant ones, but without significant distortion. Fisheye or dome cameras have a wider field of view and are capable of capturing images...

[0070] The image covers a considerable area before and after multiple lanes within a certain angle, but exhibits significant barrel and pincushion distortion. 5. Based on the distortion characteristics of camera imaging, in some embodiments of this application, when the camera is a fisheye camera or a spherical camera, acquiring the road image of road traffic signs captured by the camera includes:

[0071] Distortion correction is performed on road images captured by fisheye cameras or spherical cameras to obtain distorted road images. Distortion correction methods such as checkerboard correction, lateral unfolding, and latitude and longitude correction can be used. Of course, in practical applications, those skilled in the art can also choose other distortion correction methods.

[0072] Accordingly, acquiring the virtual point cloud corresponding to the road traffic signs in the road image based on the camera's calibration parameters includes:

[0073] Based on the camera's calibration parameters, a virtual point cloud corresponding to the road traffic signs in the distortion-free road image is obtained.

[0074] In some embodiments of this application, the virtual point cloud corresponding to the road traffic sign can be obtained through the following steps:

[0075] Target detection is performed on the road image to obtain the area information of road traffic signs;

[0076] Based on the area information of the road traffic sign, obtain the image pixel coordinates of the foreground pixel of the road traffic sign;

[0077] Based on the camera's calibration parameters and the image pixel coordinates of the foreground pixels of the road traffic sign, a virtual point cloud corresponding to the foreground pixels of the road traffic sign is obtained.

[0078] In some possible implementations of this embodiment, target detection is performed on the road image to obtain the area information of road traffic signs, including:

[0079] The road image is converted into a mask image based on an adaptive threshold algorithm; the intersection of the mask image and the road image is then processed to obtain the mask sub-image of the road traffic sign.

[0080] A mask image is also called a masked image or a binarized image. Figure 2 Taking the lane line mask image shown as an example, the pixel value of the foreground object of interest in the lane line mask image is 255, and the pixel value of the background area of ​​no interest is 0. After obtaining... Figure 2 The mask image shown can be used to extract lane line mask sub-images within a specified area through image intersection operations. The lane line mask sub-images within the specified area are shown below. Figure 3 As shown.

[0081] This embodiment uses an adaptive thresholding algorithm and image intersection operation to determine the region information of road traffic signs, so as to obtain the foreground pixels of the road traffic signs from the mask image. In practical applications, other object detection algorithms can also be used to detect the region information of road traffic signs, such as object detection of road images based on neural network models.

[0082] Accordingly, based on the area information of the road traffic sign, the image pixel coordinates of the foreground pixels of the road traffic sign are obtained, including:

[0083] Based on the pixel values ​​of the pixels in the Mask sub-image of the road traffic sign, the image pixel coordinates of the foreground pixels of the road traffic sign are obtained. That is, the pixels with a pixel value of 255 in the Mask sub-image of the road traffic sign are taken as foreground pixels, and the pixels with a pixel value of 0 are taken as background pixels. According to the camera calibration parameters, the virtual dot particles corresponding to each foreground pixel are obtained, and all virtual dot particles constitute the virtual point cloud corresponding to the road traffic sign.

[0084] This application embodiment can use either a manual matching method or an automatic matching method to perform granular matching between local high-precision map point clouds and virtual point clouds. The automatic matching method includes:

[0085] The least Euclidean distance matching algorithm is used to determine the ground truth particles that match each particle in the virtual point cloud from the local high-precision map point cloud.

[0086] This embodiment constructs a first table file based on the latitude and longitude information of each pixel in the local high-precision map point cloud, and a second table file based on the latitude and longitude information of each pixel in the virtual point cloud. The row numbers in the first and second table files represent the pixel sequence numbers, the first column number represents the longitude information, and the second column number represents the latitude information. Since the number of pixels in the virtual point cloud is much smaller than the number of pixels in the local high-precision map point cloud, the second table file can be traversed for each pixel in the virtual point cloud to calculate the ground truth pixel with the minimum Euclidean distance to the pixel in the virtual point cloud. Thus, pixel matching between the local high-precision map point cloud and the virtual point cloud is completed.

[0087] Manual matching methods include:

[0088] The virtual point cloud and the local high-precision map point cloud are visualized using visualization software; based on the visualization results of the virtual point cloud and the local high-precision map point cloud, the matching point particles in the local high-precision map point cloud and the virtual point cloud are determined by the particle selection command.

[0089] For example, importing a local high-precision map point cloud into QGIS visualization software can yield results such as... Figure 3The high-precision map shown is then used to visualize the virtual point cloud, which is then imported into the QGIS visualization software to obtain the following result: Figure 3 The point cloud visualization interface shown is for... Figure 3 The point cloud image shown is magnified to a certain extent. Based on the user's touch or click operation, a point selection instruction is generated, enabling the user to independently select matching points, improving the flexibility of point selection. Moreover, allowing the user to select points can reduce the amount of computer computation and improve evaluation efficiency.

[0090] After obtaining the matched points in the local high-precision map point cloud and the virtual point cloud, the calibration accuracy of the camera can be evaluated based on the positional deviation between the matched points. This application embodiment can evaluate the calibration accuracy from two aspects, which will be described in detail below.

[0091] Firstly, the calibration accuracy is visually evaluated.

[0092] The point cloud visualization interface of the visualization software is cropped, and the cropped image is used as the visualization evaluation result. The calibration accuracy of the camera can be viewed intuitively through the visualization evaluation result.

[0093] like Figure 4 As shown, the point cloud visualization display interface overlays virtual point clouds and local high-precision map point clouds on the same screen, and the point particles of virtual point clouds and local high-precision map point clouds have different visualization effects. Figure 4 The example shows that the pixels of the local high-definition map point cloud are black, while the pixels of the virtual point cloud are gray. In other implementations, the shape and / or color of the pixels of the two point clouds are different. For example, the pixels of the local high-definition map point cloud are black circular pixels, while the pixels of the virtual point cloud are red circular pixels or red triangular pixels.

[0094] Secondly, the calibration accuracy is quantitatively evaluated, including the calibration error evaluation results.

[0095] Obtain the positional deviation value between each matching point particle. Using the distance information between the point particles in the virtual point cloud and the camera as the horizontal axis and the positional deviation corresponding to each point particle as the vertical axis, construct a calibration error fitting curve. Determine the calibration error fitting curve as the calibration error evaluation result.

[0096] like Figure 4 As shown, from Figure 4 From the left region to the right region, the positional offset between matching particles increases progressively, i.e. Figure 4 In the left region, the matching dots almost completely overlap, and there is no positional offset between the virtual and real dots. Therefore, the calibration error in this region is relatively small. Figure 4Significant positional shifts exist between the matching dots in the right region, indicating a large calibration error in this area.

[0097] It is evident that the calibration error exhibits a regular relationship with the distance between the particles and the camera; that is, the closer the particles are to the camera, the smaller the calibration error, and the farther the particles are from the camera, the larger the calibration error. Therefore, this embodiment constructs a calibration error fitting curve using the distance information between the particles and the camera as the horizontal axis and the corresponding positional deviation of the particles as the vertical axis. This calibration error fitting curve is used to achieve a quantitative evaluation of the calibration accuracy.

[0098] This application embodiment also provides a calibration accuracy evaluation device 500, such as... Figure 5 The diagram shows a structural schematic of a calibration accuracy evaluation device according to an embodiment of this application. The device 500 is applied to roadside equipment and includes: an acquisition unit 510, a calculation unit 520, a matching unit 530, and an evaluation unit 540, wherein:

[0099] The acquisition unit 510 is used to acquire a road image containing road traffic signs captured by a camera and a local high-precision map point cloud corresponding to the road image;

[0100] The computing unit 520 is used to obtain a virtual point cloud corresponding to the road traffic signs in the road image based on the calibration parameters of the camera.

[0101] The matching unit 530 is used to perform particle matching between the local high-precision map point cloud and the virtual point cloud to obtain the mutually matching particles in the local high-precision map point cloud and the virtual point cloud.

[0102] Evaluation unit 540 is used to evaluate the calibration accuracy of the camera based on the positional deviation between the mutually matched dots and obtain an evaluation result.

[0103] In one embodiment of this application, the matching unit 530 is used to determine, from the local high-precision map point cloud, the ground truth particle that matches each particle in the virtual point cloud using a least Euclidean distance matching algorithm.

[0104] In one embodiment of this application, the matching unit 530 is used to visualize the virtual point cloud and the local high-precision map point cloud through visualization software; based on the visualization results of the virtual point cloud and the local high-precision map point cloud, the matching point particles in the local high-precision map point cloud and the virtual point cloud are determined by a particle selection instruction.

[0105] In one embodiment of this application, the evaluation result includes a visualization evaluation result. The evaluation unit 540 is used to capture an image of the point cloud visualization display interface of the visualization software. The point cloud visualization display interface displays the virtual point cloud and the local high-precision map point cloud on the same screen, and the points of the virtual point cloud and the local high-precision map point cloud have different visualization effects. The captured image is used as the visualization evaluation result.

[0106] In one embodiment of this application, the evaluation result includes a calibration error evaluation result. The evaluation unit 540 is used to obtain the positional deviation value between each matching point particle; construct a calibration error fitting curve with the distance information between the point particles in the virtual point cloud and the camera as the horizontal axis and the positional deviation corresponding to each point particle as the vertical axis; and determine the calibration error fitting curve as the calibration error evaluation result.

[0107] In one embodiment of this application, the computing unit 520 is used to perform target detection on the road image to obtain the region information of the road traffic sign; based on the region information of the road traffic sign, to obtain the image pixel coordinates of the foreground pixels of the road traffic sign; and based on the calibration parameters of the camera and the image pixel coordinates of the foreground pixels of the road traffic sign, to obtain the virtual point cloud corresponding to the foreground pixels of the road traffic sign.

[0108] In one embodiment of this application, the calculation unit 520 is further configured to convert the road image into a Mask image based on an adaptive threshold algorithm, perform intersection processing on the Mask image and the road image to obtain a Mask sub-image of the road traffic sign, and obtain the image pixel coordinates of the foreground pixels of the road traffic sign based on the pixel values ​​of the pixels in the Mask sub-image of the road traffic sign.

[0109] It is understood that the above-mentioned calibration accuracy evaluation device can realize each step of the calibration accuracy evaluation method provided in the foregoing embodiments. The relevant explanations of the calibration accuracy evaluation method are applicable to the calibration accuracy evaluation device, and will not be repeated here.

[0110] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 6 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0111] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0112] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0113] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a calibration accuracy evaluation device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0114] Acquire road images captured by a camera, including road traffic signs, and corresponding local high-precision map point clouds of the road images;

[0115] Based on the camera's calibration parameters, obtain the virtual point cloud corresponding to the road traffic signs in the road image;

[0116] Perform particle matching between the local high-precision map point cloud and the virtual point cloud to obtain the matching particles in the local high-precision map point cloud and the virtual point cloud;

[0117] The calibration accuracy of the camera is evaluated based on the positional deviation between the matched dots, and an evaluation result is obtained.

[0118] The above is as stated in this application. Figure 1The calibration accuracy evaluation device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads the information from the memory and, in conjunction with its hardware, completes the steps of the aforementioned calibration accuracy evaluation method.

[0119] The electronic device can also perform Figure 1 The method executed by the calibration accuracy evaluation device is described above, and the function of the calibration accuracy evaluation device in the embodiment shown in 1 is realized. The embodiments of this application will not be described again here.

[0120] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method performed by the calibration accuracy evaluation device in the illustrated embodiment will not be described again in this application.

[0121] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0126] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0127] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0128] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0129] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0130] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for evaluating calibration accuracy, characterized in that, The method includes: Acquire road images captured by a camera, including road traffic signs, and corresponding local high-precision map point clouds of the road images; Based on the camera's calibration parameters, obtain the virtual point cloud corresponding to the road traffic signs in the road image; Perform particle matching between the local high-precision map point cloud and the virtual point cloud to obtain the matching particles in the local high-precision map point cloud and the virtual point cloud; The calibration accuracy of the camera is evaluated based on the positional deviation between the mutually matched dots, and the evaluation result is obtained. The step of performing granular matching between the local high-precision map point cloud and the virtual point cloud includes: The local high-precision map point cloud and the virtual point cloud are matched using manual or automatic matching methods; The evaluation results include calibration error evaluation results. The evaluation of the camera's calibration accuracy based on the positional deviations between the matched dots includes: Obtain the positional deviation value between each matching particle; Using the distance information between the virtual point cloud particles and the camera as the horizontal axis and the positional deviation corresponding to each particle as the vertical axis, a calibration error fitting curve is constructed. The calibration error fitting curve is determined as the calibration error evaluation result.

2. The calibration accuracy evaluation method as described in claim 1, characterized in that, The step of performing particle matching between the local high-precision map point cloud and the virtual point cloud to obtain mutually matching particles in the local high-precision map point cloud and the virtual point cloud includes: The least Euclidean distance matching algorithm is used to determine the ground truth particles that match each particle in the virtual point cloud from the local high-precision map point cloud.

3. The calibration accuracy evaluation method as described in claim 1, characterized in that, The step of performing particle matching between the local high-precision map point cloud and the virtual point cloud to obtain mutually matching particles in the local high-precision map point cloud and the virtual point cloud includes: The virtual point cloud and the local high-precision map point cloud are visualized using visualization software; Based on the visualization results of the virtual point cloud and the local high-precision map point cloud, the matching points in the local high-precision map point cloud and the virtual point cloud are determined by the particle selection command.

4. The calibration accuracy evaluation method as described in claim 3, characterized in that, The evaluation results include visual evaluation results. The evaluation of the camera's calibration accuracy based on the positional deviations between the matched dots includes: The visualization software's point cloud visualization display interface is cropped, and the point cloud visualization display interface displays the virtual point cloud and the local high-precision map point cloud on the same screen, and the virtual point cloud and the local high-precision map point cloud have different visualization effects. The captured image is used as the visualization evaluation result.

5. The calibration accuracy evaluation method as described in claim 1, characterized in that, The step of obtaining the virtual point cloud corresponding to the road traffic signs in the road image based on the camera's calibration parameters includes: Target detection is performed on the road image to obtain the area information of road traffic signs; Based on the area information of the road traffic sign, obtain the image pixel coordinates of the foreground pixel of the road traffic sign; Based on the camera's calibration parameters and the image pixel coordinates of the foreground pixels of the road traffic sign, a virtual point cloud corresponding to the foreground pixels of the road traffic sign is obtained.

6. The calibration accuracy evaluation method as described in claim 5, characterized in that, The step of performing target detection on the road image to obtain the area information of road traffic signs includes: The road image is converted into a mask image based on an adaptive threshold algorithm. The intersection of the mask image and the road image is then processed to obtain the mask sub-image of the road traffic sign. The step of obtaining the image pixel coordinates of the foreground pixels of the road traffic sign based on the area information of the road traffic sign includes: Based on the pixel values ​​of the pixels in the Mask sub-image of the road traffic sign, the image pixel coordinates of the foreground pixels of the road traffic sign are obtained.

7. A calibration accuracy evaluation device, characterized in that, The device includes: The acquisition unit is used to acquire road images containing road traffic signs captured by the camera and the corresponding local high-precision map point cloud of the road images; The computing unit is used to obtain the virtual point cloud corresponding to the road traffic signs in the road image based on the calibration parameters of the camera. A matching unit is used to perform particle matching between the local high-precision map point cloud and the virtual point cloud to obtain mutually matching particles in the local high-precision map point cloud and the virtual point cloud. An evaluation unit is used to evaluate the calibration accuracy of the camera based on the positional deviation between the mutually matched dots and obtain an evaluation result. The matching unit is specifically used for: The local high-precision map point cloud and the virtual point cloud are matched using manual or automatic matching methods; The evaluation results include calibration error evaluation results, and the evaluation unit is specifically used for: Obtain the positional deviation value between each matching particle; Using the distance information between the virtual point cloud particles and the camera as the horizontal axis and the positional deviation corresponding to each particle as the vertical axis, a calibration error fitting curve is constructed. The calibration error fitting curve is determined as the calibration error evaluation result.

8. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the calibration accuracy evaluation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of applications, cause the electronic device to perform the calibration accuracy evaluation method according to any one of claims 1 to 6.

Images