External parameter calibration matrix evaluation device, method and electronic equipment for camera and radar

Abstract

The present invention discloses an external parameter calibration matrix evaluation device, method and electronic device for cameras and radars. The device includes: a calibration plate, including a first calibration plate and a second calibration plate arranged front and back; a support component, the first end of the support component is connected to the calibration plate; a motion platform, the motion platform is connected to the second end of the support component; a controller is connected to the motion platform, controls the radar to emit laser to the first calibration plate of the calibration plate, so that part of the laser passes through the circular hole on the first calibration plate, and projects a circular point cloud on the second calibration plate, and controls the camera to shoot the first calibration plate to obtain a target image, and evaluates the external parameter calibration matrix of the camera and radar based on the circular point cloud and the target image to obtain an evaluation result. The present invention solves the technical problem of low evaluation accuracy of the external parameter calibration matrix of cameras and radars in the related art.

Description

Technical Field

[0001] The present invention relates to the field of intelligent robots, and in particular to an external parameter calibration matrix evaluation device, method and electronic equipment for cameras and radars. Background Art

[0002] The movement of intelligent robots relies on intelligent control, and the perception of scenes through different sensors is the basis of intelligent control. Among them, the most widely used in the field of intelligent robots is the combination of monocular cameras and laser radars. The accuracy of the external parameter calibration between the two directly determines the accuracy of the intelligent robot's scene perception. Therefore, it is very important to evaluate the external parameter calibration results of monocular cameras and 3D laser radars.

[0003] The calibration accuracy evaluation method of the external parameter table of the monocular camera and the lidar in the related art usually manually sets the calibration reference object, selects the feature points on the calibration reference object, calculates the pixel error between the point cloud projection of the feature points in the 3D lidar on the image and the feature points on the image, and uses the sum or average of the projection errors of the feature points as the result for evaluation. The above evaluation process directly uses the feature points of the calibration reference object collected by the lidar and camera as the evaluation data. However, due to the errors in the collected data, the evaluation accuracy of the external parameter calibration matrix of the camera and radar in the related art is low.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] Embodiments of the present invention provide an apparatus, method and electronic device for evaluating an extrinsic parameter calibration matrix of a camera and a radar, so as to at least solve the technical problem of low evaluation accuracy of the extrinsic parameter calibration matrix of a camera and a radar in the related art.

[0006] According to one aspect of an embodiment of the present invention, an extrinsic parameter calibration matrix evaluation device for a camera and a radar is provided, comprising: a calibration plate, comprising a first calibration plate and a second calibration plate arranged front and rear, the first calibration plate being a calibration plate facing the radar and the camera; a supporting component, a first end of the supporting component being connected to the calibration plate; a motion platform, the motion platform being connected to the second end of the supporting component, and being used to move the calibration plate to a first target distance from the radar and the camera when receiving an extrinsic parameter calibration matrix evaluation instruction for the radar and the camera; a controller being connected to the motion platform, and being used to control the radar to emit a laser to the first calibration plate of the calibration plate when the calibration plate is at the first target distance from the radar and the camera, so that part of the laser passes through the circular hole on the first calibration plate and projects a circular point cloud on the second calibration plate, and controls the camera to shoot the first calibration plate to obtain a target image, and evaluates the extrinsic parameter calibration matrix of the camera and the radar based on the circular point cloud and the target image to obtain an evaluation result, wherein the evaluation result is used to describe the calibration accuracy of the extrinsic parameter calibration matrix.

[0007] Furthermore, a circular hole is provided on the first calibration plate, and a plurality of first line segments and a plurality of second line segments are drawn on the first calibration plate, wherein the plurality of first line segments are arranged at equal intervals in a horizontal direction on the first calibration plate, and adjacent first line segments are spaced a first distance apart; the plurality of second line segments are arranged at equal intervals in a vertical direction on the first calibration plate, and adjacent second line segments are spaced a second distance apart; a distance between a first endpoint of a first target line segment among the plurality of first line segments and a second endpoint of a second target line segment among the plurality of second line segments in a camera coordinate system of the camera is a first preset distance, the first target line segment is the first line segment arranged in a horizontal direction among the plurality of first line segments, and the second target line segment is the first line segment arranged in a vertical direction among the plurality of second line segments; a distance between a center of the circular hole and a first endpoint of the first target line segment in a radar coordinate system of the radar is a second preset distance.

[0008] According to another aspect of an embodiment of the present invention, a method for evaluating an extrinsic calibration matrix of a camera and a radar is provided, comprising: in response to receiving an extrinsic calibration matrix evaluation instruction of a radar and a camera, moving the calibration plate to a first target distance from the radar and the camera by controlling a motion platform in an extrinsic calibration matrix evaluation device of the camera and the radar; in response to the calibration plate being the first target distance from the radar and the camera, controlling the radar to emit a laser to a first calibration plate of the calibration plate so that part of the laser passes through a circular hole on the first calibration plate, projects a circular point cloud on the second calibration plate, and controls the camera to shoot the first calibration plate to obtain a target image; evaluating the extrinsic calibration matrix of the camera and the radar based on the circular point cloud and the target image to obtain an evaluation result, wherein the evaluation result is used to describe the calibration accuracy of the extrinsic calibration matrix.

[0009] Furthermore, the extrinsic calibration matrix of the camera and the radar is evaluated based on the circular point cloud and the target image to obtain an evaluation result, including: determining the target position coordinates and the target pixel coordinates corresponding to the first endpoint of the first target line segment based on the circular point cloud and the target image; projecting the target position coordinates onto the target image based on the extrinsic calibration matrix to obtain the projected pixel coordinates of the first endpoint; and evaluating the extrinsic calibration matrix based on the pixel distance error between the target pixel coordinates and the projected pixel coordinates to obtain an evaluation result.

[0010] Furthermore, based on the circular point cloud and the target image, the target position coordinates and target pixel coordinates corresponding to the first endpoint of the first target line segment are determined, including: based on the circular point cloud, the first target distance and the second preset distance, the target position coordinates of the first endpoint of the first target line segment are determined; based on the target image, the number of multiple first line segments, the number of multiple second line segments and the first preset distance, the target pixel coordinates of the first endpoint of the first target line segment are determined.

[0011] Furthermore, based on the circular point cloud, the first target distance and the second preset distance, the target position coordinates of the first endpoint of the first target line segment are determined, including: determining the first center coordinates of the first center of the circular point cloud in the radar coordinate system; based on the first center coordinates and the first target distance, determining the second center coordinates of the second center of the circular hole in the camera coordinate system; based on the second center coordinates and the second preset distance, determining the target position coordinates of the first endpoint of the first target line segment.

[0012] Further, based on the target image, the number of the plurality of first line segments, the number of the plurality of second line segments and the first preset distance, the target pixel coordinates of the first endpoint of the first target line segment are determined, including: determining a first target pixel value of the first endpoint of the first target line segment based on the first pixel coordinates of the target image in the camera coordinate system, the plurality of second pixel coordinates and the number of the plurality of first line segments, wherein the first pixel coordinate is the pixel coordinate of the first endpoint of the first target line segment, and the plurality of second pixel coordinates are the pixel coordinates of the first endpoints of the plurality of line segments other than the first target line segment in the plurality of first line segments; determining a second target pixel value of the second endpoint of the second target line segment based on the third pixel coordinates of the target image in the camera coordinate system, the plurality of fourth pixel coordinates and the number of the plurality of second line segments, wherein the third pixel coordinate is the pixel coordinate of the second endpoint of the second target line segment, and the plurality of fourth pixel coordinates are the pixel coordinates of the second endpoints of the plurality of line segments other than the second target line segment in the plurality of second line segments; determining the target pixel coordinates of the first endpoint of the first target line segment based on the first target pixel value, the second target pixel value and the first preset distance.

[0013] Furthermore, based on the first pixel coordinate of the target image in the camera coordinate system, multiple second pixel coordinates and the number of multiple first line segments, a first target pixel value of the first endpoint of the first target line segment is determined, including: determining a first target pixel coordinate among the multiple second pixel coordinates whose horizontal coordinate differs from the first pixel coordinate by a first preset value; and determining the first target pixel value based on the line segment corresponding to the first target pixel coordinate, the first pixel coordinate and the number of multiple first line segments.

[0014] Furthermore, based on the third pixel coordinate of the target image in the camera coordinate system, multiple fourth pixel coordinates and the number of multiple second line segments, the second target pixel value of the second endpoint of the second target line segment is determined, including: determining a second target pixel coordinate among the multiple fourth pixel coordinates whose vertical coordinate differs from the third pixel coordinate by a second preset value; determining the second target pixel value based on the line segment corresponding to the second target pixel coordinate, the third pixel coordinate and the number of multiple second line segments.

[0015] According to another aspect of an embodiment of the present invention, an external parameter calibration matrix evaluation device for a camera and a radar is also provided, including: a first control module, for responding to receiving an external parameter calibration matrix evaluation instruction for the radar and the camera, by controlling a motion platform in the external parameter calibration matrix evaluation device for the camera and the radar, to move the calibration plate to a first target distance from the radar and the camera; a second control module, for responding to the calibration plate being the first target distance from the radar and the camera, to control the radar to emit a laser to a first calibration plate of the calibration plate, so that part of the laser passes through the circular hole on the first calibration plate, projects a circular point cloud on the second calibration plate, and controls the camera to shoot the first calibration plate to obtain a target image; an evaluation module, for evaluating the external parameter calibration matrix of the camera and the radar based on the circular point cloud and the target image to obtain an evaluation result, wherein the evaluation result is used to describe the calibration accuracy of the external parameter calibration matrix.

[0016] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising: a memory for storing an executable program; and a processor for running the program, wherein the above-mentioned camera and radar extrinsic calibration matrix evaluation method is executed when the program is running.

[0017] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the storage medium is located is controlled to execute the above-mentioned camera and radar extrinsic parameter calibration matrix evaluation method.

[0018] According to another aspect of an embodiment of the present invention, a computer program product is also provided. When the computer program is executed by a processor, the computer program implements the above-mentioned extrinsic calibration matrix evaluation method for cameras and radars.

[0019] In an embodiment of the present invention, an external parameter calibration matrix evaluation device for a camera and a radar is provided, comprising: a calibration plate, comprising a first calibration plate and a second calibration plate arranged front and back, the first calibration plate being a calibration plate facing the radar and the camera; a support component, the first end of the support component being connected to the calibration plate; a motion platform, the motion platform being connected to the second end of the support component; and a controller being connected to the motion platform. It is easy to notice that the support component in the external parameter calibration matrix evaluation device for a camera and a radar is the middle connecting part of the entire device, the first end of the support component is connected to the calibration plate, the second end is connected to the motion platform, and the motion platform is then connected to the controller to form a complete evaluation device. The above-mentioned extrinsic calibration matrix evaluation device controls the radar through a controller to emit laser to the first calibration plate when the camera and the calibration plate are at a first target distance, so that the laser passing through the first calibration plate projects a circular point cloud on the second calibration plate, and uses the pinhole imaging principle to filter the point cloud emitted by the laser, and only retains points with smaller errors. At the same time, the camera is controlled to shoot the first calibration plate to obtain the target image, and then the extrinsic calibration matrix of the camera and radar is evaluated based on the circular point cloud and the target image to obtain an evaluation result. The above process extracts more accurate point clouds and pixel points of feature points, thereby improving the evaluation accuracy of the extrinsic calibration matrix, and further solves the technical problem of low evaluation accuracy of the extrinsic calibration matrix of cameras and radars in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0021] Figure 1 An extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of a camera coordinate system in an extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of a radar coordinate system in an extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of a first calibration board of an extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention;

[0025] Figure 5 is a flow chart of a method for evaluating an extrinsic calibration matrix of a camera and a radar according to an embodiment of the present invention;

[0026] Figure 6Schematic diagram of an extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] According to an embodiment of the present invention, a device for evaluating an extrinsic calibration matrix of a camera and a radar is provided. Figure 1 is an extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention, such as Figure 1 As shown, the device includes: a calibration plate 102 , a supporting component 104 , a motion platform 106 and a controller 108 .

[0030] The calibration plate 102 includes a first calibration plate and a second calibration plate arranged front and back, wherein the first calibration plate is a calibration plate facing the radar and the camera;

[0031] A support member 104, a first end of the support member is connected to the calibration plate;

[0032] A motion platform 106, connected to the second end of the supporting member, for moving the calibration plate to a first target distance from the radar and a first target distance from the camera when receiving an external parameter calibration matrix evaluation instruction of the radar and the camera;

[0033] The controller 108 is connected to the motion platform, and is used to control the radar to emit laser to the first calibration plate of the calibration plate when the calibration plate is at a first target distance from the radar and at a first target distance from the camera, so that part of the laser light passes through the circular hole on the first calibration plate and projects a circular point cloud on the second calibration plate, and controls the camera to shoot the first calibration plate to obtain a target image, and evaluates the extrinsic calibration matrix of the camera and the radar based on the circular point cloud and the target image to obtain an evaluation result, wherein the evaluation result is used to describe the calibration accuracy of the extrinsic calibration matrix.

[0034] The above-mentioned calibration plate may refer to a plane with specific patterns or features. The patterns on the calibration plate may include but are not limited to equally spaced lines, black and white grids, and dots. The patterns on the calibration plate help cameras and radars identify and process feature points in images. The functions of the calibration plate include but are not limited to helping determine the internal and external parameters of the camera, calibrating camera lens distortion, etc.

[0035] The above-mentioned first calibration plate may refer to a calibration plate facing the radar and the camera. The first calibration plate may include but is not limited to circular holes, a plurality of lines arranged equidistantly in the horizontal direction, and a plurality of lines arranged equidistantly in the vertical direction. The arrangement of the circular holes, the plurality of lines arranged equidistantly in the horizontal direction, and the plurality of lines arranged equidistantly in the vertical direction on the above-mentioned first calibration plate may be determined according to actual needs and is not limited here. The position of the circular holes on the above-mentioned first calibration plate may be any position on the calibration plate and may be determined according to actual needs and is not limited here. The number and distribution position of the plurality of lines arranged equidistantly in the horizontal direction and the plurality of lines arranged equidistantly in the vertical direction may be determined according to actual needs and are not limited here. The functions of the first calibration plate may include but are not limited to receiving the laser of the radar, filtering the laser with large error, etc.

[0036] The second calibration plate may refer to a calibration plate located behind the first calibration plate, and is used to receive a circular point cloud projected by a laser passing through a circular hole on the first calibration plate.

[0037] In an optional embodiment, the calibration plate in the external parameter calibration matrix evaluation device of the camera and radar includes a first calibration plate and a second calibration plate. The first calibration plate is used to receive the laser of the radar, and the camera simultaneously records the laser passing through the surface of the first calibration plate. The second calibration plate is located behind the first calibration plate and is used to receive the circular point cloud projected by the laser passing through the circular hole on the first calibration plate. More accurate point clouds and pixel points of feature points are recorded by the above-mentioned first calibration plate and second calibration plate, and are used in the subsequent evaluation of the external parameter calibration matrix, thereby improving the evaluation accuracy of the external parameter calibration matrix.

[0038] The supporting component mentioned above may refer to a structural component used to connect the calibration plate and the motion platform to ensure that the calibration plate remains stable during movement.

[0039] The above-mentioned external parameter calibration matrix may refer to a matrix used to describe the relative position and posture relationship between the camera and the radar. The calibration accuracy may be determined by evaluating the external parameter calibration matrix.

[0040] The above-mentioned evaluation instructions may refer to instructions used to trigger and guide the controller to perform external parameter calibration matrix evaluation tasks. The types of evaluation instructions include but are not limited to starting evaluation, adjusting the position of the calibration plate, emitting laser and other instruction operations. The role of the evaluation instructions is mainly to guide the controller to perform the evaluation task of the external parameter calibration matrix according to predetermined procedures and parameters to ensure the accuracy and reliability of the evaluation process.

[0041] The setting of evaluation instructions may include but is not limited to the following situations:

[0042] In the first case, the evaluation instructions are manually determined according to the requirements of the evaluation task and manually input into the device.

[0043] In the second case, the system automatically generates evaluation instructions according to the evaluation task requirements and automatically applies them to the device.

[0044] The above evaluation instructions are set for example only. The specific evaluation instructions can be determined according to the actual situation of the device and the requirements of the evaluation task, and are not limited here.

[0045] The above-mentioned first target distance may refer to a preset distance between a calibration plate and the radar and camera. The setting of the first target distance helps to maintain the relative position relationship between the calibration plate, the radar and the camera during the calibration process, thereby improving the calibration accuracy. The specific setting of the first target distance may be determined according to actual conditions and is not limited here.

[0046] In an optional embodiment, the external parameter calibration matrix evaluation device of the camera and radar includes a calibration plate, a supporting component and a moving platform, wherein the supporting component is a connecting device between the calibration plate and the moving platform, ensuring that the calibration plate remains relatively stable during the movement. After receiving the evaluation instruction, the moving platform moves the calibration plate to a position away from the camera and the radar at a first target distance, thereby facilitating the radar to emit a laser to the calibration plate at the first target distance for subsequent external parameter calibration matrix evaluation.

[0047] The controller may refer to a key component for realizing the evaluation of the camera and radar extrinsic calibration matrix, which is responsible for receiving evaluation instructions and controlling the coordinated work of the motion platform, radar and camera. The types of controllers include but are not limited to the use of hardware devices such as computers or microcontrollers for control, and may also be controlled by software programs. The specific type of controller may be determined according to the device requirements and is not limited here.

[0048] The control of each component in the device by the above controller includes but is not limited to the following situations:

[0049] In the first case, the motion platform is controlled to move the calibration plate to a position at a first target distance from the radar and the camera.

[0050] In a second case, when the calibration plate is located at a first target distance from the radar and the camera, the radar is controlled to emit laser to a first calibration plate of the calibration plate.

[0051] In a third case, when the calibration plate is located at a first target distance from the radar and the camera, the camera is controlled to shoot the first calibration plate.

[0052] The above controllers are only examples of the control of various components in the device. The specific controller control situation should be determined according to the needs of the device and is not limited here.

[0053] The circular point cloud may refer to a point cloud pattern formed on the second calibration plate after the laser passes through the circular hole of the first calibration plate. By determining the spatial position information of the circular point cloud, the extrinsic calibration matrix may be evaluated.

[0054] The above-mentioned target image can be an image of the first calibration plate taken during the calibration process. The role of the target image includes but is not limited to providing feature point information from the camera's perspective, matching with the laser point cloud, etc. The relative position and posture between the camera and the radar can be calculated by determining the target image.

[0055] The above evaluation results may refer to the results obtained after evaluating the external parameter calibration matrices of the camera and the radar based on the circular point cloud and the target image.

[0056] In an optional embodiment, in an external parameter calibration matrix evaluation device for a camera and a radar, a controller is connected to a motion platform, and controls the motion platform to move the calibration plate to a position at a first target distance from the radar and the camera, and controls the radar to emit a laser to the first calibration plate, so that the laser passing through the circular hole on the first calibration plate projects a circular point cloud on the second calibration plate, and at the same time controls the camera to photograph the first calibration plate to obtain a target image. In the above process, the controller obtains a point cloud and pixel points of more accurate feature points by controlling each component in the device, and can thereby effectively evaluate and improve the evaluation accuracy of the external parameter calibration matrix of the camera and the radar.

[0057] In an embodiment of the present invention, an external parameter calibration matrix evaluation device for a camera and a radar is provided, comprising: a calibration plate, comprising a first calibration plate and a second calibration plate arranged front and back, the first calibration plate being a calibration plate facing the radar and the camera; a support component, the first end of the support component being connected to the calibration plate; a motion platform, the motion platform being connected to the second end of the support component; and a controller being connected to the motion platform. It is easy to notice that the support component in the external parameter calibration matrix evaluation device for a camera and a radar is the middle connecting part of the entire device, the first end of the support component is connected to the calibration plate, the second end is connected to the motion platform, and the motion platform is then connected to the controller to form a complete evaluation device. The above-mentioned extrinsic calibration matrix evaluation device controls the radar through a controller to emit laser to the first calibration plate when the camera and the calibration plate are at a first target distance, so that the laser passing through the first calibration plate projects a circular point cloud on the second calibration plate, and uses the pinhole imaging principle to filter the point cloud emitted by the laser, and only retains points with smaller errors. At the same time, the camera is controlled to shoot the first calibration plate to obtain the target image, and then the extrinsic calibration matrix of the camera and radar is evaluated based on the circular point cloud and the target image to obtain an evaluation result. The above process extracts more accurate point clouds and pixel points of feature points, thereby improving the evaluation accuracy of the extrinsic calibration matrix, and further solves the technical problem of low evaluation accuracy of the extrinsic calibration matrix of cameras and radars in related technologies.

[0058] Optionally, a circular hole is provided on the first calibration plate, and multiple first line segments and multiple second line segments are drawn on the first calibration plate, wherein the multiple first line segments are arranged at equal intervals in the horizontal direction on the first calibration plate, and adjacent first line segments are spaced by a first distance; the multiple second line segments are arranged at equal intervals in the vertical direction on the first calibration plate, and adjacent second line segments are spaced by a second distance; the distance between the first endpoint of a first target line segment among the multiple first line segments and the second endpoint of a second target line segment among the multiple second line segments in the camera coordinate system of the camera is a first preset distance, the first target line segment is the first line segment arranged in the horizontal direction among the multiple first line segments, and the second target line segment is the first line segment arranged in the vertical direction among the multiple second line segments; the distance between the center of the circular hole and the first endpoint of the first target line segment in the radar coordinate system of the radar is a second preset distance.

[0059] The first line segments may refer to line segments arranged at equal intervals along the horizontal direction on the first calibration plate, and are used to calibrate the horizontal direction parameters of the camera.

[0060] The second line segments may refer to line segments arranged at equal intervals along the vertical direction on the first calibration plate, and are used to calibrate the vertical direction parameters of the camera.

[0061] The first distance mentioned above may refer to the interval distance between a plurality of first line segments.

[0062] The second distance mentioned above may refer to the interval distance between the plurality of second line segments.

[0063] The first target line segment may refer to a first line segment arranged along a horizontal direction among a plurality of first line segments, and may be used to determine a reference point in a camera coordinate system.

[0064] The first endpoint may refer to a starting point of a first line segment arranged along a horizontal direction among a plurality of first line segments.

[0065] The second target line segment may refer to the first line segment arranged along the vertical direction among the plurality of second line segments, and may be used to determine a reference point in the camera coordinate system.

[0066] The second endpoint may refer to a starting point of a first line segment arranged along a vertical direction among the plurality of second line segments.

[0067] The first preset distance may refer to the distance between the first endpoint of the first target line segment and the second endpoint of the second target line segment in the camera coordinate system.

[0068] The second preset distance may refer to the distance between the center of the circular hole and the first endpoint of the first target line segment in the radar coordinate system.

[0069] Figure 2 is a schematic diagram of a camera coordinate system in an extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention, such as Figure 2 As shown in the figure, the camera coordinate system is composed of the x-axis and the y-axis.

[0070] Figure 3 is a schematic diagram of a radar coordinate system in an extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention, such as Figure 3 As shown, the left side is the external parameter calibration matrix evaluation device for the camera and radar, wherein 302 is the first calibration plate, 304 is the second calibration plate, 306 is the support component, 308 is the motion platform, 310 is the circular hole on the first calibration plate, and the right side is the three-dimensional coordinate system of x-axis, y-axis and z-axis established with the radar as the center, namely the radar coordinate system, wherein 312 is the radar. The external parameter calibration matrix evaluation device for the camera and radar of the embodiment of the present invention is located on the left side of the radar, and the calibration plate plane faces the laser radar to receive the laser emitted by the radar. The external parameter calibration matrix evaluation device includes a first calibration plate, a second calibration plate, a support component and a motion platform, and the first calibration plate includes a circular hole for filtering the laser emitted by the radar. Figure 3 It reflects the relative position of the radar coordinate system and the external parameter calibration matrix evaluation device, which facilitates the subsequent determination of the positions of multiple feature points in the evaluation device relative to the radar coordinate system based on the radar coordinate system.

[0071] In an optional embodiment, according to the camera imaging principle, the actual size of the object , Camera focal length , the distance between the camera and the object , object pixel size There is a relationship between:

[0072] ;

[0073] In the formula, is the actual size of the object, is the camera focal length, is the distance between the camera and the object, is the object size in pixels.

[0074] Then the distance between the camera and the object is When , the actual distance corresponding to each pixel of the camera is:

[0075] ;

[0076] In the formula, is the real distance corresponding to each pixel of the camera, is the actual size of the object, is the pixel size of the object, is the distance between the camera and the object, is the camera focal length.

[0077] Figure 4 is a schematic diagram of a first calibration board of an extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention, such as Figure 4 As shown, the first calibration plate includes a circular hole, n first line segments arranged in the horizontal direction, and n second line segments arranged in the vertical direction. The first line segments arranged in the horizontal direction are , the second line segments arranged in the vertical direction are , the horizontal spacing between the left endpoints of the n first line segments is , the vertical spacing between the upper endpoints of the n second line segments is , wherein the circular hole on the first calibration plate can be set at any position on the first calibration plate, and the position of the circular hole can be adjusted according to actual needs, which is not limited here. The distance between the first endpoint of the first target line segment and the second endpoint of the second target line segment in the first calibration plate in the camera coordinate system, that is, the first preset distance, can be expressed as , the distance between the center of the circular hole and the first endpoint of the first target line segment in the radar coordinate system, that is, the second preset distance can be expressed as .

[0078] According to another aspect of an embodiment of the present invention, a method for evaluating an extrinsic calibration matrix of a camera and a radar is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0079] Figure 5 is a flow chart of a method for evaluating an extrinsic calibration matrix of a camera and a radar according to an embodiment of the present invention. Figure 5 As shown, the method comprises the following steps:

[0080] Step S502, in response to receiving an extrinsic calibration matrix evaluation instruction for the radar and the camera, the calibration plate is moved to a first target distance from the radar and a first target distance from the camera by controlling a motion platform in an extrinsic calibration matrix evaluation device for the camera and the radar.

[0081] The motion platform in the above-mentioned external parameter calibration matrix evaluation device for controlling the camera and radar includes the following methods:

[0082] The first method is to control the movement of the motion platform through sensors and computer software programming.

[0083] The second method is to design a controller to control the movement of the motion platform according to the characteristics of the motion platform.

[0084] The above-mentioned method of controlling the movement of the motion platform is only an example. The specific control method should be determined according to the actual situation and is not limited here.

[0085] In an optional embodiment, after receiving the external parameter calibration matrix evaluation instructions of the radar and camera, the motion platform is controlled to move the calibration plate to a position at a first target distance from the camera and the radar. The above process sets the first target distance in advance, and then the motion platform moves the calibration plate to the specified position. This can accurately control the position of the calibration plate, thereby ensuring that the relative position relationship between the radar and the camera during the calibration process is consistent, thereby improving the accuracy and reliability of the calibration.

[0086] Step S504, in response to the calibration plate being at a first target distance from the radar and the camera, the radar is controlled to emit laser to the first calibration plate of the calibration plate, so that part of the laser light passes through the circular hole on the first calibration plate, and a circular point cloud is projected on the second calibration plate, and the camera is controlled to shoot the first calibration plate to obtain a target image.

[0087] The circular point cloud may refer to a set of points with the same radius obtained during the laser radar emission process. These points usually represent a circular area and can be used to assist in the detection and positioning of feature points.

[0088] The above-mentioned control radar emits laser to the first calibration plate of the calibration plate in the following ways:

[0089] The first method is to control the radar to emit laser through hardware control, which may include the physical connection and configuration between the laser transmitter, radar system and calibration board.

[0090] The second method is to control the radar to emit laser through software control, which can specifically include the programming and setting operations of the laser transmitter and radar system.

[0091] The third method is to control the radar to emit laser through timing control, which can specifically include the timing of laser emission and radar reception, and control the radar to emit laser within the set time.

[0092] The above method of controlling the radar to emit laser to the first calibration plate of the calibration plate is only an example, and the specific method should be determined according to system requirements and actual conditions, and is not limited here.

[0093] The above-mentioned control camera to shoot the first calibration plate includes the following methods:

[0094] The first method is to control the camera to shoot the first calibration plate by automatic triggering. Specifically, it can include using an external trigger such as a sensor to automatically trigger the camera to shoot. When the calibration plate enters a preset area, the trigger sends a signal to the camera, and the camera automatically shoots after receiving the signal.

[0095] The second method is to control the camera to shoot the first calibration plate through software programming, which may specifically include implementing software programming control by using a software development kit or a general image acquisition library provided by the camera manufacturer.

[0096] The above-mentioned method of controlling the camera to shoot the first calibration plate is only an example. The specific method should be determined according to system requirements and actual conditions and is not limited here.

[0097] In an optional embodiment, when the calibration plate is located at a first target distance from both the radar and the camera, the radar is controlled to emit laser light to the calibration plate, and the camera is photographed by the camera, thereby achieving calibration of the radar and the camera. Specifically, first determine the first target distance between the calibration plate and the radar and the camera, and then control the radar to emit laser light to the calibration plate so that the laser passes through the circular hole on the calibration plate and projects a circular point cloud on the second calibration plate. At the same time, control the camera to photograph the first calibration plate to obtain a target image. In this way, the system can achieve accurate calibration of the radar and the camera, improve detection accuracy and reliability, and simplify the calibration process and improve calibration efficiency.

[0098] Step S506, evaluating the extrinsic calibration matrices of the camera and the radar based on the circular point cloud and the target image to obtain an evaluation result, wherein the evaluation result is used to describe the calibration accuracy of the extrinsic calibration matrix.

[0099] The above evaluation results may refer to the evaluation results of the accuracy of the external parameter calibration matrix obtained by quantifying the deviation in the calibration process through some quantitative indicators. The quantitative indicators may include but are not limited to the difference, average error, etc. The role of the evaluation results is not limited to reflecting the accuracy of the external parameter calibration matrix, facilitating error analysis and reliability evaluation, etc.

[0100] In an optional embodiment, the camera and radar extrinsic calibration matrix evaluation method evaluates the accuracy of the extrinsic calibration matrix by comparing the differences between the circular point clouds observed by the camera and the radar. If the evaluation result shows that the accuracy of the extrinsic calibration matrix is ​​high, it can be considered that the data obtained by the camera and the radar can be well fused and aligned. The above process extracts more accurate point clouds and pixel points of feature points, so that the calibration accuracy between the camera and the radar can be effectively evaluated, thereby improving the evaluation accuracy of the extrinsic calibration matrix.

[0101] Optionally, the extrinsic calibration matrix of the camera and the radar is evaluated based on the circular point cloud and the target image to obtain an evaluation result, including: determining the target position coordinates and the target pixel coordinates corresponding to the first endpoint of the first target line segment based on the circular point cloud and the target image; projecting the target position coordinates onto the target image based on the extrinsic calibration matrix to obtain the projected pixel coordinates of the first endpoint; and evaluating the extrinsic calibration matrix based on the pixel distance error between the target pixel coordinates and the projected pixel coordinates to obtain an evaluation result.

[0102] The above-mentioned target position coordinates may refer to the position of the first endpoint of the first target line segment in the radar coordinate system.

[0103] The target pixel coordinates may refer to the position of the first endpoint of the first target line segment in the camera coordinate system.

[0104] The projected pixel coordinates may refer to the position of the first endpoint of the first target line segment in the radar coordinate system projected to the camera coordinate system.

[0105] The above pixel distance error may refer to the difference between the target pixel coordinates and the projected pixel coordinates, usually in pixels. The role of the pixel distance error includes but is not limited to evaluating the accuracy of the extrinsic calibration matrix.

[0106] In an optional embodiment, firstly, based on the circular point cloud and the target image, determine the position of the first endpoint of the first target line segment in the radar coordinate system, i.e., the target position coordinates, and the position of the first endpoint of the first target line segment in the camera coordinate system, i.e., the target pixel coordinates; then, based on the extrinsic calibration matrix, project the target position coordinates onto the target image to obtain the projected pixel coordinates of the first endpoint of the first target line segment; finally, calculate the pixel distance error between the target pixel coordinates and the projected pixel coordinates, and evaluate the accuracy of the extrinsic calibration matrix. If the evaluation result shows that the error is small, it means that the accuracy of the extrinsic calibration matrix is ​​high. If the evaluation result shows that the error is large, it means that the accuracy of the extrinsic calibration matrix is ​​low, and recalibration can be performed as needed. Through the above method, the accuracy of the extrinsic calibration matrix of the camera and radar can be effectively evaluated, and the reliability of the calibration results can be improved.

[0107] Optionally, based on the circular point cloud and the target image, the target position coordinates and target pixel coordinates corresponding to the first endpoint of the first target line segment are determined, including: determining the target position coordinates of the first endpoint of the first target line segment based on the circular point cloud, the first target distance and the second preset distance; determining the target pixel coordinates of the first endpoint of the first target line segment based on the target image, the number of multiple first line segments, the number of multiple second line segments and the first preset distance.

[0108] In an optional embodiment, a method for determining target position coordinates and target pixel coordinates based on a circular point cloud and a target image is as follows: first, based on the circular point cloud, the first target distance and the second preset distance, the target position coordinates of the first endpoint of the first target line segment in the radar coordinate system are determined, and then based on the target image, the number of multiple first line segments, the number of multiple second line segments and the first preset distance, the target pixel coordinates of the first endpoint of the first target line segment in the camera coordinate system are determined. Through the above process, the position of the target feature point in the radar coordinate system and the position in the camera coordinate system can be accurately determined, providing important position information for the subsequent determination of the actual position of the target feature point.

[0109] Optionally, based on the circular point cloud, the first target distance and the second preset distance, the target position coordinates of the first endpoint of the first target line segment are determined, including: determining the first center coordinates of the first center of the circular point cloud in the radar coordinate system; based on the first center coordinates and the first target distance, determining the second center coordinates of the second center of the circular hole in the camera coordinate system; based on the second center coordinates and the second preset distance, determining the target position coordinates of the first endpoint of the first target line segment.

[0110] The first circle center mentioned above may refer to the circle center of the circular point cloud.

[0111] The first center coordinates may refer to the coordinates of the center of the circular point cloud in the radar coordinate system. By determining the first center coordinates, the position and shape of the circular point cloud can be described more accurately.

[0112] The second center mentioned above may refer to the center of the circular hole.

[0113] The second center coordinates may refer to the coordinates of the center of the circular hole in the camera coordinate system. By determining the second center coordinates, the information in the radar coordinate system may be converted to the camera coordinate system, so as to facilitate the subsequent calculation of the error magnitude through the coordinates of the feature points in different coordinate systems.

[0114] In an optional embodiment, the target position coordinates of the first endpoint on the first target segment are determined by the circular point cloud, the first target distance and the second preset distance. This process involves the conversion of distance from the radar coordinate system to the camera coordinate system, as well as the identification and positioning of the target feature points. It can achieve accurate positioning and tracking of the target feature points and obtain the true position of the target feature points.

[0115] In an optional embodiment, when the controller controls the motion platform to move the calibration plate to a position at a first target distance from the radar and the camera, the radar is controlled to emit laser light to the calibration plate, wherein the first target distance is recorded as At this time, the coordinates of the center of the circular point cloud in the radar coordinate system, that is, the coordinates of the first center of the circle, are calculated as , combined with the first target distance , the coordinates of the center of the circular hole, that is, the coordinates of the second center, are calculated as , and finally according to the coordinates of the second circle center The second preset distance , calculate the target position coordinates of the first endpoint of the first target line segment, and the target position coordinate formula is as follows:

[0116] ;

[0117] In the formula, represents the target position coordinates, Indicates the position of the target feature point on the x-axis in the radar coordinate system. Indicates the position of the target feature point on the y-axis in the radar coordinate system. Indicates the position of the target feature point on the z-axis in the radar coordinate system.

[0118] Optionally, determining the target pixel coordinates of the first endpoint of the first target line segment based on the target image, the number of the plurality of first line segments, the number of the plurality of second line segments and the first preset distance includes: determining a first target pixel value of the first endpoint of the first target line segment based on the first pixel coordinates of the target image in the camera coordinate system, the plurality of second pixel coordinates and the number of the plurality of first line segments, wherein the first pixel coordinates are the pixel coordinates of the first endpoint of the first target line segment, and the plurality of second pixel coordinates are the pixel coordinates of the first endpoints of the plurality of line segments other than the first target line segment in the plurality of first line segments; determining a second target pixel value of the second endpoint of the second target line segment based on the third pixel coordinates of the target image in the camera coordinate system, the plurality of fourth pixel coordinates and the number of the plurality of second line segments, wherein the third pixel coordinates are the pixel coordinates of the second endpoint of the second target line segment, and the plurality of fourth pixel coordinates are the pixel coordinates of the second endpoints of the plurality of line segments other than the second target line segment in the plurality of second line segments; determining the target pixel coordinates of the first endpoint of the first target line segment based on the first target pixel value, the second target pixel value and the first preset distance.

[0119] The first pixel coordinates may refer to the coordinates of a specific point in the target image, and here are the pixel coordinates of the first endpoint of the first target line segment.

[0120] The plurality of second pixel coordinates may refer to the pixel coordinates of the first endpoints of the plurality of first line segments except the first target line segment, and may be used to assist in calculation or comparison to determine the coordinates of the target line segment endpoint.

[0121] The first target pixel value may refer to an accurate pixel value of the first endpoint of the first target line segment obtained by calculation.

[0122] The third pixel coordinates may refer to the coordinates of a specific point in the target image, and here are the pixel coordinates of the second endpoint of the second target line segment.

[0123] The above-mentioned multiple fourth pixel coordinates may refer to the pixel coordinates of the second endpoints of the multiple second line segments except the second target line segment, and may be used to assist calculation or comparison to determine the coordinates of the target line segment endpoint.

[0124] The second target pixel value may refer to an accurate pixel value of the second endpoint of the second target line segment obtained by calculation.

[0125] In an optional embodiment, the pixel value of the first endpoint of the first target line segment of the target image in the camera coordinate system, i.e., the first pixel coordinate, and the pixel coordinates of the first endpoints of the other line segments except the first target line segment among the multiple first line segments, i.e., the multiple second pixel coordinates, can be jointly determined as the first target pixel value; then, the pixel value of the second endpoint of the second target line segment of the target image in the camera coordinate system, i.e., the third pixel coordinate, and the pixel coordinates of the second endpoints of the other line segments except the second target line segment among the multiple second line segments, i.e., the multiple fourth pixel coordinates, can be jointly determined as the second target pixel value; finally, the target pixel coordinates of the first endpoint of the first target line segment are calculated by combining the first target pixel value, the second target pixel value, and a preset distance parameter. The above process associates the image coordinates with the camera coordinate system, and combines the information of the multiple line segments, so as to more accurately determine the position of the target line segment endpoint.

[0126] Optionally, based on the first pixel coordinate of the target image in the camera coordinate system, multiple second pixel coordinates and the number of multiple first line segments, the first target pixel value of the first endpoint of the first target line segment is determined, including: determining a first target pixel coordinate among the multiple second pixel coordinates whose horizontal coordinate differs from the first pixel coordinate by a first preset value; determining the first target pixel value based on the line segment corresponding to the first target pixel coordinate, the first pixel coordinate and the number of multiple first line segments.

[0127] The above-mentioned first preset value may refer to a pre-set numerical value, which is used to determine the pixel points among multiple second pixel coordinates whose horizontal coordinates differ from the first pixel coordinates by a specific value. The size of the first preset value may include but is not limited to 1 pixel, 2 pixels, and 3 pixels. The specific size of the first preset value should be determined according to actual needs and is not limited here.

[0128] In an optional embodiment, in the camera coordinate system, the method for determining the first target pixel value is as follows: First, obtain the first target line segment on the first calibration plate The pixel coordinates of the first endpoint of , obtaining the pixel coordinates of the first endpoints of the plurality of first line segments on the first calibration plate except the first target line segment, that is, the plurality of second pixel coordinate markers ,in Indicates The first line segment.

[0129] Then, the plurality of second pixel coordinates of the plurality of first line segments are detected, and the first target pixel coordinate whose x-axis coordinate value differs from the first pixel coordinate by a first preset value is found among the plurality of second pixel coordinates corresponding to the plurality of first line segments, and the line segment corresponding to the first target pixel coordinate is recorded as , then we can get the first target segment The first target pixel value of the first endpoint is as follows:

[0130] ;

[0131] In the formula, a first target pixel value representing a first endpoint of a first target line segment, The x-axis first pixel coordinate value of the first endpoint of the first target line segment, represents the number of multiple first line segments, Indicates a line segment corresponding to a first target pixel coordinate whose x-axis coordinate value differs from the first pixel coordinate by a first preset value.

[0132] Optionally, based on the third pixel coordinate of the target image in the camera coordinate system, multiple fourth pixel coordinates and the number of multiple second line segments, the second target pixel value of the second endpoint of the second target line segment is determined, including: determining a second target pixel coordinate among the multiple fourth pixel coordinates whose longitudinal coordinate differs from the third pixel coordinate by a second preset value; determining the second target pixel value based on the line segment corresponding to the second target pixel coordinate, the third pixel coordinate and the number of multiple second line segments.

[0133] The above-mentioned second preset value may refer to a pre-set numerical value, which is used to determine the pixel point whose horizontal coordinate differs from the third pixel coordinate by a specific value among multiple fourth pixel coordinates. The size of the second preset value may include but is not limited to 1 pixel, 2 pixels and 3 pixels, etc. The specific size of the second preset value should be determined according to actual needs and is not limited here.

[0134] In an optional embodiment, in the camera coordinate system, the method for determining the second target pixel value is as follows: first, obtain the second target line segment among the plurality of second line segments on the first calibration plate. The pixel coordinates of the second endpoint, that is, the third pixel coordinates are marked as , and obtain the pixel coordinates of the second endpoints of the plurality of line segments other than the second target line segment in the plurality of second line segments, that is, the plurality of fourth pixel coordinates are marked as ,in Indicates The second line segment.

[0135] Then, the plurality of fourth pixel coordinates of the plurality of second line segments are detected, and the second target pixel coordinate whose y-axis coordinate value differs from the third pixel coordinate by a second preset value is found among the plurality of fourth pixel coordinates corresponding to the plurality of second line segments, and the second line segment corresponding to the second target pixel coordinate is recorded as , then we can get the second target segment The second target pixel value of the second endpoint is as follows:

[0136] ;

[0137] In the formula, a second target pixel value representing the second endpoint of the second target line segment, The third pixel coordinate value of the second endpoint of the second target line segment on the y-axis, represents the number of multiple second line segments, Indicates a line segment corresponding to a second target pixel coordinate whose y-axis coordinate value differs from the third pixel coordinate by a second preset value.

[0138] In an optional embodiment, the steps for calculating the calibration error of the camera and radar extrinsic parameters are as follows:

[0139] Step 1: When the controller controls the motion platform to move the calibration plate to a position at a first target distance from the radar and the camera, the radar is controlled to emit laser light to the calibration plate, where the first target distance is recorded as At this time, the coordinates of the center of the circular point cloud in the radar coordinate system, that is, the coordinates of the first center of the circle, are calculated as , combined with the first target distance , the coordinates of the center of the circular hole, that is, the coordinates of the second center, are calculated as , and finally according to the coordinates of the second circle center The second preset distance , calculate the target position coordinates of the first endpoint of the first target line segment, and the target position coordinate formula is as follows:

[0140] ;

[0141] In the formula, represents the target position coordinates, Indicates the position of the target feature point on the x-axis in the radar coordinate system. Indicates the position of the target feature point on the y-axis in the radar coordinate system. Indicates the position of the target feature point on the z-axis in the radar coordinate system.

[0142] Step 2: Control the camera to shoot the first calibration plate to obtain the target image. Based on the captured target image, first obtain the first target line segment on the first calibration plate in the camera coordinate system. The pixel coordinates of the first endpoint of , obtaining the pixel coordinates of the first endpoints of the plurality of first line segments on the first calibration plate except the first target line segment, that is, the plurality of second pixel coordinate markers ,in Indicates first line segments; then detecting multiple second pixel coordinates of the multiple first line segments, finding the first target pixel coordinate that differs from the first pixel coordinate by 1 pixel in the x-axis coordinate value among the multiple second pixel coordinates corresponding to the multiple first line segments, and recording the line segment corresponding to the first target pixel coordinate as , then we can get the first target segment The first target pixel value of the first endpoint is as follows:

[0143] ;

[0144] In the formula, a first target pixel value representing a first endpoint of a first target line segment, The x-axis first pixel coordinate value of the first endpoint of the first target line segment, represents the number of multiple first line segments, Indicates the line segment corresponding to the first target pixel coordinate whose x-axis coordinate value differs from the first pixel coordinate by 1 pixel.

[0145] Then, in the camera coordinate system, obtain the second target line segment among the multiple second line segments on the first calibration plate The pixel coordinates of the second endpoint, that is, the third pixel coordinates are marked as , and obtain the pixel coordinates of the second endpoints of the plurality of line segments other than the second target line segment in the plurality of second line segments, that is, the plurality of fourth pixel coordinates are marked as ,in Indicates Then, the plurality of fourth pixel coordinates of the plurality of second line segments are detected, and the second target pixel coordinate whose y-axis coordinate value differs from the third pixel coordinate by 1 pixel is found among the plurality of fourth pixel coordinates corresponding to the plurality of second line segments, and the line segment corresponding to the second target pixel coordinate is recorded as , then we can get the second target segment The second target pixel value of the second endpoint is as follows:

[0146] ;

[0147] In the formula, a second target pixel value representing the second endpoint of the second target line segment, The third pixel coordinate value of the second endpoint of the second target line segment on the y-axis, represents the number of multiple second line segments, Indicates the line segment corresponding to the second target pixel coordinate whose y-axis coordinate value differs from the third pixel coordinate by 1 pixel.

[0148] Step 3: Based on the distance between the first endpoint of the first target line segment and the second endpoint of the second target line segment, that is, the first preset distance , get the target pixel coordinates of the first endpoint of the first target line segment. The target pixel coordinate formula is as follows:

[0149] ;

[0150] In the formula, represents the target pixel coordinates, represents the first target pixel value, represents the x-axis pixel value in the first preset distance, represents the second target pixel value, Indicates the y-axis pixel value in the first preset distance.

[0151] Step 4: By projecting the target position coordinates onto the target image, the projection pixel coordinates of the first endpoint are obtained as .

[0152] Step 5: Calculate the projection pixel coordinates With the target pixel coordinates The pixel distance error is used as the calibration error of the camera and radar extrinsic parameters.

[0153] According to another aspect of an embodiment of the present invention, a device for evaluating an extrinsic calibration matrix of a camera and a radar is provided. The device can execute the extrinsic calibration matrix evaluation method of the camera and the radar of the above embodiment. The specific implementation method and preferred application scenario are the same as those of the above embodiment and will not be repeated here.

[0154] Figure 6 is a schematic diagram of an extrinsic calibration matrix evaluation device for a camera and a radar according to an embodiment of the present invention. Figure 6 As shown, the device includes the following: a first control module 602 , a second control module 604 , and an evaluation module 606 .

[0155] A first control module is configured to, in response to receiving an external parameter calibration matrix evaluation instruction of the radar and the camera, move the calibration plate to a first target distance from the radar and the first target distance from the camera by controlling a motion platform in an external parameter calibration matrix evaluation device of the camera and the radar;

[0156] a second control module, for controlling the radar to emit laser light to the first calibration plate of the calibration plate in response to the calibration plate being at a first target distance from the radar and the camera, so that part of the laser light passes through the circular hole on the first calibration plate, and a circular point cloud is projected on the second calibration plate, and controlling the camera to shoot the first calibration plate to obtain a target image;

[0157] The evaluation module is used to evaluate the extrinsic calibration matrix of the camera and the radar based on the circular point cloud and the target image to obtain an evaluation result, wherein the evaluation result is used to describe the calibration accuracy of the extrinsic calibration matrix.

[0158] Optionally, the evaluation module includes: a first determination unit, which determines the target position coordinates and target pixel coordinates corresponding to the first endpoint of the first target line segment based on the circular point cloud and the target image; a projection unit, which projects the target position coordinates onto the target image based on the external parameter calibration matrix to obtain the projected pixel coordinates of the first endpoint; and an evaluation unit, which evaluates the external parameter calibration matrix based on the pixel distance error between the target pixel coordinates and the projected pixel coordinates to obtain an evaluation result.

[0159] Optionally, the first determination unit includes: a first determination subunit, which determines the target position coordinates of the first endpoint of the first target line segment based on the circular point cloud, the first target distance and the second preset distance; and a second determination subunit, which determines the target pixel coordinates of the first endpoint of the first target line segment based on the target image, the number of multiple first line segments, the number of multiple second line segments and the first preset distance.

[0160] Optionally, the first determination subunit includes: determining the first center coordinates of the first center of the circular point cloud in the radar coordinate system; determining the second center coordinates of the second center of the circular hole in the camera coordinate system based on the first center coordinates and the first target distance; determining the target position coordinates of the first endpoint of the first target line segment based on the second center coordinates and the second preset distance.

[0161] Optionally, the second determination subunit includes: determining a first target pixel value of a first endpoint of a first target line segment based on a first pixel coordinate of a target image in a camera coordinate system, a plurality of second pixel coordinates, and the number of a plurality of first line segments, wherein the first pixel coordinate is the pixel coordinate of the first endpoint of the first target line segment, and the plurality of second pixel coordinates are the pixel coordinates of the first endpoints of a plurality of line segments other than the first target line segment among the plurality of first line segments; determining a second target pixel value of a second endpoint of a second target line segment based on a third pixel coordinate of a target image in a camera coordinate system, a plurality of fourth pixel coordinates, and the number of a plurality of second line segments, wherein the third pixel coordinate is the pixel coordinate of the second endpoint of the second target line segment, and the plurality of fourth pixel coordinates are the pixel coordinates of the second endpoints of a plurality of line segments other than the second target line segment among the plurality of second line segments; determining the target pixel coordinate of the first endpoint of the first target line segment based on the first target pixel value, the second target pixel value, and a first preset distance.

[0162] Optionally, based on the first pixel coordinate of the target image in the camera coordinate system, multiple second pixel coordinates and the number of multiple first line segments, the first target pixel value of the first endpoint of the first target line segment is determined, including: determining a first target pixel coordinate among the multiple second pixel coordinates whose horizontal coordinate differs from the first pixel coordinate by a first preset value; determining the first target pixel value based on the line segment corresponding to the first target pixel coordinate, the first pixel coordinate and the number of multiple first line segments.

[0163] Optionally, based on the third pixel coordinate of the target image in the camera coordinate system, multiple fourth pixel coordinates and the number of multiple second line segments, the second target pixel value of the second endpoint of the second target line segment is determined, including: determining a second target pixel coordinate among the multiple fourth pixel coordinates whose longitudinal coordinate differs from the third pixel coordinate by a second preset value; determining the second target pixel value based on the line segment corresponding to the second target pixel coordinate, the third pixel coordinate and the number of multiple second line segments.

[0164] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising: a memory for storing an executable program; and a processor for running the program, wherein the above-mentioned camera and radar extrinsic calibration matrix evaluation method is executed when the program is running.

[0165] The memory in the above steps can be one of the memories such as random access memory RAM, read-only memory ROM and non-volatile memory, which is used to store programs and data in electronic devices. It is a temporary or permanent storage space that enables the processor to access and execute the information stored therein; the processor is the core component of the electronic device that executes program instructions, and is responsible for executing programs and data stored in the memory and processing various computing tasks. The types of memory and processor are not limited here.

[0166] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the storage medium is located is controlled to execute the above-mentioned camera and radar extrinsic parameter calibration matrix evaluation method.

[0167] The computer-readable storage medium in the above steps can be a physical device for storing data, including semiconductors, magnetic cores, magnetic drums, magnetic tapes, laser disks, etc. Different media have different storage densities, access speeds and cost-effectiveness. Computer-readable storage media can allow computer systems to read and access information stored therein, which can be temporary or permanent, depending on the method and purpose of storing data.

[0168] According to another aspect of an embodiment of the present invention, a computer program product is also provided. When the computer program is executed by a processor, the computer program implements the above-mentioned extrinsic calibration matrix evaluation method for cameras and radars.

[0169] The computer program product in the above steps may refer to a software product consisting of a series of computer programs, codes and related documents. These programs and codes are written to achieve specific functions and requirements, and usually include application programs, operating systems, database management systems, etc. The core of the computer program product is the program, that is, a collection of codes that perform tasks according to specific rules and logic.

[0170] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0171] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0172] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0173] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0174] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0175] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An external parameter calibration matrix evaluation device for cameras and radars, characterized in that: include: The calibration plate includes a first calibration plate and a second calibration plate arranged front and back, wherein the first calibration plate is a calibration plate facing the radar and the camera; A supporting component, a first end of which is connected to the calibration plate; a motion platform connected to the second end of the supporting member, and configured to move the calibration plate to a first target distance from the radar and the camera and to the first target distance from the camera upon receiving an external parameter calibration matrix evaluation instruction of the radar and the camera; a controller connected to the motion platform, and configured to control the radar to emit laser light to a first calibration plate of the calibration plate when the calibration plate is at the first target distance from the radar and at the first target distance from the camera, so that part of the laser light passes through the circular hole on the first calibration plate and projects a circular point cloud on the second calibration plate; and control the camera to shoot the first calibration plate to obtain a target image, and evaluate the extrinsic calibration matrix of the camera and the radar based on the circular point cloud and the target image to obtain an evaluation result, wherein the evaluation result is used to describe the calibration accuracy of the extrinsic calibration matrix; The first calibration plate is provided with a circular hole, and a plurality of first line segments and a plurality of second line segments are drawn on the first calibration plate, wherein: The plurality of first line segments are arranged at equal intervals along the horizontal direction on the first calibration plate, and adjacent first line segments are spaced a first distance apart; The plurality of second line segments are arranged at equal intervals along the vertical direction on the first calibration plate, and adjacent second line segments are spaced apart by a second distance; A distance between a first endpoint of a first target line segment among the plurality of first line segments and a second endpoint of a second target line segment among the plurality of second line segments in a camera coordinate system of the camera is a first preset distance, the first target line segment is a first line segment among the plurality of first line segments arranged along a horizontal direction, and the second target line segment is a first line segment among the plurality of second line segments arranged along a vertical direction; The distance between the center of the circular hole and the first endpoint of the first target line segment in the radar coordinate system of the radar is a second preset distance.

2. A method for evaluating an extrinsic calibration matrix for a camera and a radar, characterized in that: The extrinsic calibration matrix evaluation device for cameras and radars according to claim 1, the method comprising: In response to receiving an extrinsic calibration matrix evaluation instruction for a radar and a camera, the calibration plate is moved to a first target distance from the radar and to the first target distance from the camera by controlling a motion platform in an extrinsic calibration matrix evaluation device for the camera and the radar; In response to the calibration plate being at the first target distance from the radar and the camera, the radar is controlled to emit laser light to the first calibration plate of the calibration plate, so that part of the laser light passes through the circular hole on the first calibration plate, and a circular point cloud is projected on the second calibration plate, and the camera is controlled to shoot the first calibration plate to obtain a target image; Based on the circular point cloud and the target image, the extrinsic calibration matrices of the camera and the radar are evaluated to obtain an evaluation result, wherein the evaluation result is used to describe the calibration accuracy of the extrinsic calibration matrix.

3. The method for evaluating the extrinsic calibration matrix of a camera and a radar according to claim 2, characterized in that: The external parameter calibration matrices of the camera and the radar are evaluated based on the circular point cloud and the target image to obtain evaluation results, including: Based on the circular point cloud and the target image, determining the target position coordinates and the target pixel coordinates corresponding to the first endpoint of the first target line segment; Projecting the target position coordinates onto the target image based on the extrinsic calibration matrix to obtain the projected pixel coordinates of the first endpoint; The extrinsic calibration matrix is ​​evaluated based on a pixel distance error between the target pixel coordinates and the projection pixel coordinates to obtain an evaluation result.

4. The method for evaluating the extrinsic calibration matrix of a camera and a radar according to claim 3, characterized in that: Determining target position coordinates and target pixel coordinates corresponding to a first endpoint of the first target line segment based on the circular point cloud and the target image includes: Determining target position coordinates of a first endpoint of the first target line segment based on the circular point cloud, the first target distance, and the second preset distance; Based on the target image, the number of the plurality of first line segments, the number of the plurality of second line segments, and the first preset distance, a target pixel coordinate of a first endpoint of the first target line segment is determined.

5. The method for evaluating the extrinsic calibration matrix of a camera and a radar according to claim 4, characterized in that: Determining the target position coordinates of the first endpoint of the first target line segment based on the circular point cloud, the first target distance and the second preset distance includes: Determine the coordinates of a first center of the circular point cloud in a radar coordinate system; Determine the second center coordinates of the second center of the circular hole in the camera coordinate system based on the first center coordinates and the first target distance; The target position coordinates of the first endpoint of the first target line segment are determined based on the second circle center coordinates and the second preset distance.

6. The method for evaluating the extrinsic calibration matrix of a camera and a radar according to claim 4, characterized in that: Determining a target pixel coordinate of a first endpoint of the first target line segment based on the target image, the number of the plurality of first line segments, the number of the plurality of second line segments, and the first preset distance includes: Determine a first target pixel value of a first endpoint of the first target line segment based on a first pixel coordinate of the target image in a camera coordinate system, a plurality of second pixel coordinates, and the number of the plurality of first line segments, wherein the first pixel coordinate is a pixel coordinate of the first endpoint of the first target line segment, and the plurality of second pixel coordinates are pixel coordinates of first endpoints of a plurality of line segments other than the first target line segment in the plurality of first line segments; Determine a second target pixel value of a second endpoint of the second target line segment based on a third pixel coordinate of the target image in the camera coordinate system, a plurality of fourth pixel coordinates, and the number of the plurality of second line segments, wherein the third pixel coordinate is the pixel coordinate of the second endpoint of the second target line segment, and the plurality of fourth pixel coordinates are the pixel coordinates of the second endpoints of the plurality of line segments other than the second target line segment in the plurality of second line segments; The target pixel coordinates of the first endpoint of the first target line segment are determined based on the first target pixel value, the second target pixel value, and the first preset distance.

7. The method for evaluating the extrinsic calibration matrix of a camera and a radar according to claim 6, characterized in that: Determining a first target pixel value of a first endpoint of the first target line segment based on a first pixel coordinate of the target image in the camera coordinate system, a plurality of second pixel coordinates, and the number of the plurality of first line segments, comprises: Determine a first target pixel coordinate among the plurality of second pixel coordinates, the abscissa of which differs from the first pixel coordinate by a first preset value; The first target pixel value is determined based on a line segment corresponding to the first target pixel coordinate, the first pixel coordinate, and the number of the plurality of first line segments.

8. The method for evaluating the extrinsic calibration matrix of a camera and a radar according to claim 6, characterized in that: Determining a second target pixel value of a second endpoint of the second target line segment based on a third pixel coordinate of the target image in the camera coordinate system, a plurality of fourth pixel coordinates, and the number of the plurality of second line segments, comprises: Determine a second target pixel coordinate among the plurality of fourth pixel coordinates, the vertical coordinate of which differs from the third pixel coordinate by a second preset value; The second target pixel value is determined based on the line segment corresponding to the second target pixel coordinate, the third pixel coordinate, and the number of the plurality of second line segments.

9. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 2 to 8 when running.

Images