A processing method and device for calibrating external parameters of a vehicle-mounted camera

Abstract

Embodiments of the present application relate to a kind of processing method and device for calibrating the external parameter of vehicle camera, the method comprises: confirming first camera, first radar and first calibration board;First camera is called to obtain first image by image shooting;First radar is called to obtain first point cloud by scanning;The center point of four circular regions of calibration board in first image is identified;The center point of four circular regions of calibration board in first point cloud is identified;Four matching point pairs are generated;Camera external parameter is estimated according to four matching point pairs;First image is converted to generate first calibration binary image by binary image conversion;First point cloud in four circular regions of calibration board is regarded as first calibration point cloud;According to first calibration point cloud and first calibration binary image, first external parameter matrix is fine-tuned;And fine-tuning result is regarded as camera external parameter calibration output.The present application can complete calibration without using any complex calibration equipment, and vehicle does not need to be sent to fixed place.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a processing method and device for calibrating external parameters of a vehicle-mounted camera. BACKGROUND

[0002] An autonomous vehicle needs to periodically calibrate the external parameters of a vehicle-mounted camera. The commonly used camera external parameter calibration scheme is a technical scheme based on a hand-eye calibration method. This calibration method requires special calibration equipment to cooperate. These calibration equipment are not portable devices, so the vehicle needs to be sent to a designated location to complete the calibration each time the camera external parameter calibration is performed. SUMMARY

[0003] The present application aims to overcome the defects of the prior art, and provides a processing method and device for calibrating external parameters of a vehicle-mounted camera, an electronic device and a computer readable storage medium. Four pairs of matching points in a pixel coordinate system and a laser radar coordinate system are obtained by taking a simple calibration board as a reference. The camera external parameters are estimated based on the four pairs of matching points. The estimated camera external parameters are fine-tuned based on a calibration point cloud. Through the method of the present application, the external parameter calibration can be completed without using any complex calibration equipment. The calibration board can be placed on the vehicle, and the staff can perform autonomous calibration at any location without the need to send the vehicle to a fixed calibration site.

[0004] When fusing the data of the camera and the laser radar, the problem of mispositioning of image data and point cloud data can be avoided.

[0005] To achieve the above-mentioned purpose, the first aspect of the embodiment of the present application provides a processing method for calibrating external parameters of a vehicle-mounted camera, which comprises:

[0006] A vehicle-mounted camera of a vehicle is selected as a corresponding first camera, and a vehicle-mounted laser radar of the vehicle is selected as a corresponding first radar. A calibration board arranged in a vertical ground manner in an overlapping area of a shooting range of the first camera and a scanning range of the first radar is selected as a corresponding first calibration board. The shape of the first calibration board is rectangular. The first calibration board includes four circular image areas, i.e., a first, a second, a third and a fourth circular area, which are recorded as a left upper, a right upper, a right lower and a left lower circular area. The area of the first calibration board other than the four circular image areas is recorded as a first bottom plate area. The four circular image areas are uniformly covered by a light-colored high-reflective material film. The first bottom plate area is not covered by the high-reflective material film, and the color of the first bottom plate area is black.

[0007] The first camera is called to perform image shooting, and the image obtained by shooting is taken as a corresponding first image. The first radar is called to perform scanning, and the laser radar point cloud obtained by scanning is taken as a corresponding first point cloud.

[0008] identifying pixel coordinates of four center points of the circular regions of the calibration board in the first image to generate corresponding first, second, third and fourth center point pixel coordinates; and identifying lidar coordinates of the four center points of the circular regions of the calibration board in the first point cloud to generate corresponding first, second, third and fourth center point radar coordinates;

[0009] composing the obtained first center point pixel coordinates and the first center point radar coordinates into a corresponding first matching point pair; and composing the obtained second center point pixel coordinates and the second center point radar coordinates into a corresponding second matching point pair; and composing the obtained third center point pixel coordinates and the third center point radar coordinates into a corresponding third matching point pair; and composing the obtained fourth center point pixel coordinates and the fourth center point radar coordinates into a corresponding fourth matching point pair;

[0010] performing camera extrinsic parameter pre-estimation according to the first, second, third and fourth matching point pairs to generate a corresponding first extrinsic parameter matrix; performing binary image conversion on the first image to generate a corresponding first calibration binary image; extracting point clouds in the first, second, third and fourth circular regions in the first point cloud as a corresponding first calibration point cloud; performing extrinsic parameter matrix fine-tuning on the first extrinsic parameter matrix according to the first calibration point cloud and the first calibration binary image to generate a corresponding second extrinsic parameter matrix; and outputting the second extrinsic parameter matrix as a calibration result of the current vehicle-mounted camera extrinsic parameter calibration.

[0011] Preferably, the identifying pixel coordinates of four center points of the circular regions of the calibration board in the first image to generate corresponding first, second, third and fourth center point pixel coordinates specifically comprises:

[0012] performing binary image conversion on the first image to obtain a corresponding first binary image; the size of the first binary image remains consistent with the first image; the color of the first base plate region on the first binary image is white, and the color of the remaining regions outside the first base plate region is black;

[0013] sequentially connecting four vertices of the first base plate region on the first binary image to obtain a corresponding quadrilateral region, which is referred to as a first quadrilateral region; the first quadrilateral region includes four black circular regions corresponding to the first, second, third and fourth circular regions;

[0014] extracting edge points of the four black circular regions in the first quadrilateral region to generate corresponding first, second, third and fourth edge point sets;

[0015] The first, second, third and fourth edge point sets are respectively subjected to Hough transform circle detection processing to obtain corresponding four circle center points, denoted as first, second, third and fourth circle center points; and pixel coordinates of the first, second, third and fourth circle center points are taken as corresponding first, second, third and fourth center point pixel coordinates.

[0016] Preferably, each point in the first point cloud corresponds to a first point cloud data; the first point cloud data includes a first laser radar coordinate and a first radar reflection intensity; the first laser radar coordinate includes a first depth coordinate component.

[0017] Preferably, the laser radar coordinates of the four circle region center points of the calibration board in the first point cloud are identified to generate corresponding first, second, third and fourth center point radar coordinates, specifically including:

[0018] The points in the first point cloud with the first radar reflection intensity higher than a preset high reflection intensity threshold are recorded as corresponding first high return points; and all the first high return points are extracted from the first point cloud to form a corresponding second point cloud;

[0019] The second point cloud is subjected to point cloud clustering to obtain four clustering sub-point clouds; and the four clustering sub-point clouds are respectively marked as corresponding first, second, third and fourth sub-point clouds according to the corresponding relationship with the upper left, upper right, lower right and lower left four circle image regions on the first calibration board; and the first, second, third and fourth sub-point clouds are respectively subjected to edge point cloud identification to obtain corresponding first, second, third and fourth edge point clouds.

[0020] Four spherical center points are obtained by spherical fitting on the first, second, third and fourth edge point clouds, denoted as first, second, third and fourth spherical center points; and laser radar coordinates of the first, second, third and fourth spherical center points are taken as corresponding first, second, third and fourth center point radar coordinates.

[0021] Preferably, the first, second, third and fourth matching point pairs are used to pre-estimate a camera extrinsic parameter matrix, specifically including:

[0022] The pixel-laser radar coordinate conversion equation is determined as

[0023]

[0024] wherein (x, y, z) is a three-dimensional coordinate of a laser radar coordinate coordinate system, (u, v) is a two-dimensional coordinate of a pixel coordinate system, A is a known camera intrinsic parameter matrix, is a camera extrinsic parameter matrix to be calibrated, r 11 , r12 13 21 22 23 31 32 33 t1, t2 and t3 are matrix parameters of the camera extrinsic matrix;

[0025] the first center point pixel coordinates and the first center point radar coordinates of the first matched point pair are substituted into the image-lidar coordinate conversion equation to obtain a corresponding first equation; the second center point pixel coordinates and the second center point radar coordinates of the second matched point pair are substituted into the image-lidar coordinate conversion equation to obtain a corresponding second equation; the third center point pixel coordinates and the third center point radar coordinates of the third matched point pair are substituted into the image-lidar coordinate conversion equation to obtain a corresponding third equation; and the fourth center point pixel coordinates and the fourth center point radar coordinates of the fourth matched point pair are substituted into the image-lidar coordinate conversion equation to obtain a corresponding fourth equation;

[0026] the first, second, third and fourth equations are solved based on the PnP algorithm to obtain twelve matrix parameters r 11 12 13 21 22 23 31 32 33 the parameter values of the twelve matrix parameters are substituted into the camera extrinsic matrix to obtain a corresponding first extrinsic matrix.

[0027] Preferably, the size of the first calibration binary image is consistent with the first image; the color of the first bottom plate region on the first calibration binary image is white, and the color of the remaining regions outside the first bottom plate region is black.

[0028] Preferably, the points in the first point cloud in the first, second, third and fourth circular regions are extracted as corresponding first calibration point clouds, specifically including:

[0029] the first, second, third and fourth sub-point clouds of the first point cloud form the corresponding first calibration point cloud.

[0030] Preferably, the first extrinsic matrix is fine-tuned according to the first calibration point cloud and the first calibration binary image to generate a corresponding second extrinsic matrix, specifically including:​​​​​​​​​​​​​​​

[0031] Step 81, determining the first relationship and the second relationship of three-axis Euler angles θ x 、θ y 、θ z in the camera coordinate system and the rotation matrix in the camera extrinsic matrix;

[0032] The first relationship is:

[0033] θ x =atan2(r 32 ,r 33 )

[0034]

[0035] θ z =atan2(r 21 ,r 11 )

[0036] The second relationship is:

[0037]

[0038] Step 82, bringing the matrix parameters r 11 , r 21 , r 31 , r 32 , r 33 of the first extrinsic matrix into the first relationship for calculation, and taking the values of the three Euler angles θ x , θ y , θ z obtained as the corresponding first Euler angle second Euler angle and third Euler angle

[0039] Step 83, setting the corresponding first, second and third parameter fine-tuning ranges according to the preset first, second and third fine-tuning thresholds a1, a2, a3 and the first, second and third Euler angles ; and setting the corresponding fourth, fifth and sixth parameter fine-tuning ranges according to the preset fourth, fifth and sixth fine-tuning thresholds a4, a5, a6 and the matrix parameters t1, t2, t3 of the first extrinsic matrix; and setting the corresponding first, second, third, fourth, fifth and sixth sliding step lengths △a1, △a2, △a3, △a4, △a5, △a6 according to the preset sliding number N and the first, second, third, fourth, fifth and sixth fine-tuning thresholds a1, a2, a3, a4, a5, a6;

[0040] wherein,

[0041] the first parameter fine tuning range is

[0042] the second parameter fine tuning range is

[0043] the third parameter fine tuning range is

[0044] the fourth parameter fine tuning range is [t1-a4, t1+a4],

[0045] the fifth parameter fine tuning range is [t2-a5, t2+a5],

[0046] the sixth parameter fine tuning range is [t3-a6, t3+a6],

[0047] Δa1=2*a1 / N, Δa2=2*a2 / N, Δa3=2*a3 / N,

[0048] Δa4=2*a4 / N, Δa5=2*a5 / N, Δa6=2*a6 / N;

[0049] Step 84, the first index i, the first variable θ x,i , the second variable θ y,i , the third variable θ z,i , the fourth variable t 1,i , the fifth variable t 2,i and the sixth variable t 3,i are initialized:

[0050] i=0,

[0051]

[0052]

[0053]

[0054] t 1,i =t1-a4,

[0055] t 2,i =t2-a5,

[0056] t 3,i =t3-a6;

[0057] Step 85, the first, second, third variable θ x,i , θ y,i , θ z,i are substituted into the second relationship to obtain a set of the latest matrix parameters r 11,i , r 12,i , r 13,i, r 21,i , r 22,i , r 23,i , r 31,i , r 32,i , r 33,I ; and using the latest matrix parameter r 11,i , r 12,i , r 13,i , r 21,i , r 22,i , r 23,i , r 31,i , r 32,i , r 33,I and the fourth, fifth, sixth variables t 1,i , t 2,i , t 3,i , the matrix parameters of the first extrinsic matrix are replaced to obtain the corresponding first fine-tuned extrinsic matrix

[0058] Step 86, based on the first fine-tuned extrinsic matrix, the laser radar coordinates of each point in the first calibration point cloud are converted from the laser radar coordinate system to the pixel coordinate system to generate the corresponding first pixel coordinates; and the pixel points on the first calibration binary image that match each of the first pixel coordinates are recorded as the corresponding first pixel points; and the pixel values of all the first pixel points are summed to generate the corresponding first pixel sum S i ;

[0059] Step 87, add 1 to the first index i; and according to the first, second, third, fourth, fifth and sixth sliding steps △a1, △a2, △a3, △a4, △a5, △a6 and the new first index i, the first variable θ x,i , the second variable θ y,i , the third variable θ z,i , the fourth variable t 1,i , the fifth variable t 2,i and the sixth variable t 3,i are reset.

[0060]

[0061]

[0062]

[0063] t 1,i = (t1-a4) + △a4*i,

[0064] t 2,i = (t2-a5) + △a5*i,

[0065] t 3,i = (t3-a6)+△a6*i;

[0066] Step 88: Determine whether the first index i is greater than the number of slides N; if yes, proceed to step 89; otherwise, proceed to step 85.

[0067] Step 89, from the obtained N+1 sums of the first pixels S i The minimum value is selected as the corresponding minimum pixel sum S. min ; and the sum of the minimum pixels S min The first fine-tuning extrinsic matrix is ​​used as the second extrinsic matrix.

[0068] A second aspect of the present invention provides an apparatus for implementing the calibrating extrinsic parameters of a vehicle-mounted camera as described in the first aspect above. The apparatus includes: a preparation module, a matching point pair processing module, and an extrinsic parameter matrix processing module.

[0069] The preparation module is used to select one vehicle-mounted camera as the corresponding first camera and one vehicle-mounted LiDAR as the corresponding first radar; and to set a calibration plate perpendicular to the ground in the overlapping area of ​​the first camera's shooting range and the first radar's scanning range as the corresponding first calibration plate; the first calibration plate is rectangular in shape, and includes four circular image areas on the first calibration plate, labeled as the first, second, third, and fourth circular areas (upper left, upper right, lower right, and lower left), and the area on the first calibration plate other than the four circular image areas is labeled as the first base plate area; the four circular image areas are uniformly covered by a light-colored highly reflective material film, and the first base plate area is not covered by a highly reflective material film and is black in color;

[0070] The preparation module is also used to call the first camera to capture an image and use the captured image as the corresponding first image; and to call the first radar to scan and use the scanned lidar point cloud as the corresponding first point cloud;

[0071] The matching point pair processing module is configured to identify pixel coordinates of four center points of circular regions of a calibration board in the first image to generate corresponding first, second, third and fourth center point pixel coordinates, identify lidar coordinates of the four center points of the circular regions of the calibration board in the first point cloud to generate corresponding first, second, third and fourth center point radar coordinates, form a corresponding first matching point pair by combining the first center point pixel coordinates and the first center point radar coordinates, form a corresponding second matching point pair by combining the second center point pixel coordinates and the second center point radar coordinates, form a corresponding third matching point pair by combining the third center point pixel coordinates and the third center point radar coordinates, and form a corresponding fourth matching point pair by combining the fourth center point pixel coordinates and the fourth center point radar coordinates.

[0072] The extrinsic parameter matrix processing module is configured to perform camera extrinsic parameter pre-estimation based on the first, second, third and fourth matching point pairs to generate a corresponding first extrinsic parameter matrix, perform binary graph conversion on the first image to generate a corresponding first calibration binary graph, extract point clouds in the first, second, third and fourth circular regions in the first point cloud as a corresponding first calibration point cloud, perform extrinsic parameter matrix fine-tuning on the first extrinsic parameter matrix based on the first calibration point cloud and the first calibration binary graph to generate a corresponding second extrinsic parameter matrix, and output the second extrinsic parameter matrix as a calibration result of vehicle-mounted camera calibration.

[0073] The third aspect of the embodiment of the present application provides an electronic device, comprising a memory, a processor and a transceiver.

[0074] The processor is configured to be coupled with the memory, read and execute instructions in the memory to realize the method steps of the first aspect.

[0075] The transceiver is coupled with the processor, and the transceiver is controlled by the processor to perform message transceiving.

[0076] The fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores computer instructions, and when the computer instructions are executed by a computer, the computer instructions make the computer execute the instructions of the method of the first aspect.

[0077] The embodiment of the present application provides a kind of calibration vehicle camera external parameter processing method, device, electronic equipment and computer readable storage medium;With a simple calibration board as reference, obtain four pairs of matching points in pixel coordinate system and laser radar coordinate system;Camera external parameter is estimated based on four pairs of matching points;Again, the estimated camera external parameter is fine-tuned based on a calibration point cloud.Through the method of the present application, external parameter calibration can be completed without using any complex calibration equipment;Calibration board is placed on the car, and staff can independently calibrate at any position, without sending the vehicle to fixed calibration site. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1 A kind of calibration vehicle camera external parameter processing method provided for the embodiment of the present application schematic diagram;

[0079] Figure 2a The schematic diagram of the first calibration board provided for the embodiment of the present application;

[0080] Figure 2b The first binary diagram provided for the embodiment of the present application schematic diagram;

[0081] Figure 3 The module structure diagram of a kind of calibration vehicle camera external parameter processing device provided for the embodiment of the present application;

[0082] Figure 4 The structural schematic diagram of a kind of electronic equipment provided for the embodiment of the present application. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical scheme and advantage of the present application more clear, the present application will be described in further detail below with reference to the drawings, obviously, the described embodiment is only a part of the embodiment of the present application, not all the embodiments. Based on the embodiment in the present application, all other embodiments obtained by those skilled in the art without doing creative work belong to the scope of protection of the present application.

[0084] The embodiment of the present application provides a kind of calibration vehicle camera external parameter processing method, as Figure 1 As shown in the schematic diagram of a kind of calibration vehicle camera external parameter processing method provided for the embodiment of the present application, the method mainly includes the following steps:

[0085] Step 1, select a vehicle camera as a corresponding first camera, and select a vehicle laser radar as a corresponding first radar;And the calibration board arranged in the overlapping area of the shooting range of the first camera and the scanning range of the first radar in the vertical ground way is used as the corresponding first calibration board;

[0086] The first calibration board is rectangular in shape, and the first calibration board includes four circular image area regions, namely a first, second, third, and fourth circular region, at the upper left, upper right, lower right, and lower left, respectively. The area of the first calibration board other than the four circular image area regions is referred to as a first bottom plate region.

[0087] Here, before camera extrinsic calibration is performed through subsequent steps, a pair of cameras and a laser radar are selected as a corresponding first camera and first radar, and a calibration board, namely a first calibration board, is placed. The shooting and scanning ranges of the first camera and the first radar have an overlapping region, and the first calibration board is placed in the overlapping region. When the first calibration board is placed, it is preferably perpendicular to the ground. The distance of the first calibration board from the vehicle body can be set according to the image proportion of the first calibration board in the image captured by the first camera, such as a distance range reference of 1 / 4 to 3 / 4 of the imaging proportion in the case of good imaging clarity. The first calibration board is as shown in the schematic diagram of the first calibration board provided by the first embodiment of the present application. Figure 2a The first calibration board itself does not have specific requirements on the material, and can be made of hard paperboard, hard plastic board, or ceramic board, etc. The light-colored high-reflectivity material film on the first calibration board can be a pasted material film, such as a common aluminum foil or aluminum film, or a film formed by using paint to paint or spray.

[0088] Step 2: call the first camera to capture an image and take the captured image as a corresponding first image; and call the first radar to scan and take the scanned laser radar point cloud as a corresponding first point cloud.

[0089] Each point in the first point cloud corresponds to a first point cloud data; the first point cloud data includes a first laser radar coordinate and a first radar reflection intensity; the first laser radar coordinate includes a first depth coordinate component.

[0090] Here, according to the known point cloud features, the feature data of each point in the first point cloud, namely the first point cloud data, includes a three-dimensional point cloud coordinate and a reflection intensity, namely the first radar reflection intensity. The coordinate system of the point cloud coordinate is the laser radar coordinate system by default, so the present embodiment takes it as the first laser radar coordinate. If the laser radar coordinate system is a right-handed coordinate system with the x-axis pointing forward, the y-axis pointing left, and the z-axis pointing up, then the x-axis component of the first laser radar coordinate (x, y, z) is the first depth coordinate component. Similarly, each pixel in the first image also corresponds to a pixel coordinate (u, v) of a two-dimensional pixel coordinate system and a single-channel or multi-channel pixel feature.

[0091] Step 3, identifying the pixel coordinates of the four center points of the circular regions of the calibration board in the first image to generate corresponding first, second, third and fourth center point pixel coordinates; and identifying the lidar coordinates of the four center points of the circular regions of the calibration board in the first point cloud to generate corresponding first, second, third and fourth center point radar coordinates;

[0092] Specifically comprising: step 31, identifying the pixel coordinates of the four center points of the circular regions of the calibration board in the first image to generate corresponding first, second, third and fourth center point pixel coordinates;

[0093] Specifically comprising: step 311, performing binary graph conversion on the first image to obtain a corresponding first binary graph;

[0094] The size of the first binary graph is consistent with that of the first image; the color of the first base plate region on the first binary graph is white, and the color of the remaining regions outside the first base plate region is black;

[0095] Here, in the binary graph conversion of the embodiment of the application, the first image is first converted into a grayscale graph, then the binary graph conversion is performed based on the grayscale graph, and then the converted binary graph is subjected to noise elimination based on the known geometric shape of the calibration board, and the obtained first binary graph is as shown in the first binary graph schematic diagram provided by the embodiment of the application. Figure 2b The purpose of the binary graph conversion on the first image is to set as many pixel points outside the first calibration board on the first image as background pixel points (the color of the background pixel points is pre-set as black), and set the pixel points on the first base plate region of the first calibration board as foreground pixel points (the color of the foreground pixel points is pre-set as white), because the four circular image regions on the first calibration board are covered with light-colored film, so the color of the pixel points on the four circular image regions should also be the same as that of the background pixel points, i.e., black;

[0096] Step 312, sequentially connecting the four vertices of the first base plate region on the first binary graph to obtain a corresponding rectangular region, which is recorded as a first quadrilateral region;

[0097] The first quadrilateral region includes four black circular regions corresponding to the first, second, third and fourth circular regions;

[0098] Here, the embodiment of the application first confirms the four vertices of the first base plate region on the first binary graph, and then sequentially connects the four vertices in a clockwise or counterclockwise direction to obtain the first quadrilateral region;

[0099] It should be noted that the embodiment of the present application supports multiple implementation manners when determining the four vertices of the first bottom plate region, one of which is: traversing each white pixel point on the first binary image; when traversing, taking the white pixel point currently traversed as a current pixel point, and judging whether black and white pixel points exist simultaneously in the four or eight neighborhoods of the current pixel point, and if so, marking the current pixel point as an edge pixel point; when the traversal ends, sorting all the edge pixel points in a clockwise or counterclockwise direction to generate a corresponding edge pixel point sequence, the last edge pixel point of the edge pixel point sequence coincides with the first edge pixel point; and calculating the vector angle cosine of each edge pixel point in the edge pixel point sequence to the front and rear two adjacent edge pixel points to obtain a corresponding first cosine value; and taking the four edge pixel points with the smallest absolute value of the first cosine value as the four vertices of the first quadrilateral region.

[0100] Step 313, extracting the edge points of the four black circular regions in the first quadrilateral region to generate corresponding first, second, third and fourth edge point sets.

[0101] Here, after obtaining the first quadrilateral region, the center of the first quadrilateral region can be used to equally divide the first quadrilateral region into four sub-regions, and the four sub-regions are correspondingly recorded as first, second, third and fourth sub-regions according to the corresponding relationship with the upper left, upper right, lower right and lower left four circular image regions of the first calibration plate. The color of the background pixel points in each sub-region is white, and the color of the foreground pixel points is black. In the first, second, third or fourth sub-region, each black pixel point is traversed, and when traversing, the white pixel point currently traversed is taken as a current pixel point, and whether black and white pixel points exist simultaneously in the four or eight neighborhoods of the current pixel point is judged, and if so, the current pixel point is marked as an edge point. When the traversal ends, all the edge points obtained in the current sub-region are combined to form a corresponding first, second, third or fourth edge point set.

[0102] Step 314, performing Hough transform circular detection processing on the first, second, third and fourth edge point sets respectively to obtain four circular center points corresponding to the first, second, third and fourth circular center points; and taking the pixel coordinates of the first, second, third and fourth circular center points as the corresponding first, second, third and fourth center point pixel coordinates.

[0103] Here, the circular detection principle of the Hough transform can be known from the disclosed technical documents, which will not be described one by one here; simply speaking, before the first, second, third and fourth edge point sets are subjected to the Hough transform circular detection processing, the conversion relationship of the pixel coordinate system to the Hough circle space (a, b, r) and the conversion relationship of the Hough circle space (a, b, r) to the pixel coordinate system are known, when the Hough transform circular detection processing is performed, all the edge points in the first, second, third or fourth edge point set are projected onto the Hough circle space to obtain a plurality of three-dimensional cone surfaces according to the conversion relationship of the pixel coordinate system to the Hough circle space, and the intersection coordinates (a0, b0, r0) of all the three-dimensional cone surfaces in the r height are extracted as the circular feature data corresponding to the current edge point set, and the current circular feature data (a0, b0, r0) is subjected to the coordinate system conversion according to the conversion relationship of the Hough circle space to the pixel coordinate system to obtain the center point of the circle corresponding to the current edge point set in the pixel coordinate system, i.e. the first, second, third or fourth circular center point, and the point coordinates of the first, second, third and fourth circular center points in the pixel coordinate system are the first, second, third and fourth center point pixel coordinates;

[0104] Step 32, the laser radar coordinates of the four circular region center points of the calibration board in the first point cloud are identified to generate corresponding first, second, third and fourth center point radar coordinates;

[0105] Specifically includes: step 321, the points in the first point cloud whose first radar reflection intensity is higher than a preset high reflection intensity threshold are recorded as corresponding first high return points; and all the first high return points are extracted from the first point cloud to form a corresponding second point cloud;

[0106] Here, as known from the foregoing, the four circular regions on the first calibration board are uniformly covered by a thin film of light-colored high-reflection material, so the first radar reflection intensity of each point in the first point cloud corresponding to the four circular regions will be much higher than that of each point outside the four circular regions. The present embodiment filters each point in the first point cloud based on a pre-set high-reflection empirical threshold, i.e. the high-reflection intensity threshold, and the obtained second point cloud is mainly composed of sub-point clouds of the four circular regions;

[0107] Step 322, the second point cloud is subjected to point cloud clustering to obtain four clustered sub-point clouds; and the four clustered sub-point clouds are respectively marked as corresponding first, second, third and fourth sub-point clouds according to the corresponding relationship with the left upper, right upper, right lower and left lower four circular image regions on the first calibration board; and the first, second, third and fourth sub-point clouds are respectively subjected to edge point cloud identification to obtain corresponding first, second, third and fourth edge point clouds;

[0108] Step 323, by spherical surface fitting to the first, second, third and fourth edge point clouds respectively, four corresponding spherical center points are obtained, which are recorded as the first, second, third and fourth spherical center points; and the laser radar coordinates of the first, second, third and fourth spherical center points are taken as the first, second, third and fourth center point radar coordinates.

[0109] Here, based on the least square method, the first left upper circular edge point cloud set, the first right upper circular edge point cloud set, the first right lower circular edge point cloud set and the first left lower circular edge point cloud set are respectively subjected to point cloud spherical surface fitting, so as to obtain the spherical center points of the four fitted spherical surfaces, that is, the first, second, third and fourth spherical center points, and the point coordinates of the first, second, third and fourth spherical center points in the laser radar coordinate system are the first, second, third and fourth center point radar coordinates.

[0110] Step 4, the obtained first center point pixel coordinates and the first center point radar coordinates form a corresponding first matching point pair; and the obtained second center point pixel coordinates and the second center point radar coordinates form a corresponding second matching point pair; and the obtained third center point pixel coordinates and the third center point radar coordinates form a corresponding third matching point pair; and the obtained fourth center point pixel coordinates and the fourth center point radar coordinates form a corresponding fourth matching point pair.

[0111] Here, the four circular regions on the first calibration board correspond to a group of matching point pairs respectively; the first circular region corresponds to the first matching point pair, the second circular region corresponds to the second matching point pair, the third circular region corresponds to the third matching point pair, and the fourth circular region corresponds to the fourth matching point pair.

[0112] Step 5, according to the first, second, third and fourth matching point pairs, a first extrinsic parameter matrix is generated by camera extrinsic parameter pre-estimation; a first calibration binary image is generated by performing binary image conversion on the first image; the point clouds in the first point cloud that are in the first, second, third and fourth circular regions are extracted as a corresponding first calibration point cloud; and a second extrinsic parameter matrix is generated by extrinsic parameter matrix fine-tuning according to the first calibration point cloud and the first calibration binary image; and the second extrinsic parameter matrix is taken as the calibration result of the present vehicle-mounted camera extrinsic parameter calibration.

[0113] Specifically, it includes: step 51, generating a corresponding first extrinsic parameter matrix by camera extrinsic parameter pre-estimation according to the first, second, third and fourth matching point pairs.

[0114] Here, the embodiment of the application first estimates the camera extrinsic parameter based on four pairs of matching point pairs;

[0115] Specifically, it includes: step 511, determining that the pixel-laser radar coordinate conversion equation is:

[0116]

[0117] wherein (x, y, z) is a three-dimensional coordinate of a laser radar coordinate coordinate system, (u, v) is a two-dimensional coordinate of a pixel coordinate system, A is a known camera intrinsic matrix, is a camera extrinsic matrix to be calibrated, r 11 , r 12 , r 13 , r 21 , r 22 , r 23 , r 31 , r 32 , r 33 , t1, t2 and t3 are matrix parameters of the camera extrinsic matrix;

[0118] Here, the pixel-laser radar coordinate conversion equation is a known coordinate conversion equation, and the derivation process can be obtained from the disclosed technical literature, which will not be described here.

[0119] Step 512, the first center point pixel coordinates and the first center point radar coordinates of the first matching point pair are substituted into the image-laser radar coordinate conversion equation to obtain the corresponding first equation; and the second center point pixel coordinates and the second center point radar coordinates of the second matching point pair are substituted into the image-laser radar coordinate conversion equation to obtain the corresponding second equation; and the third center point pixel coordinates and the third center point radar coordinates of the third matching point pair are substituted into the image-laser radar coordinate conversion equation to obtain the corresponding third equation; and the fourth center point pixel coordinates and the fourth center point radar coordinates of the fourth matching point pair are substituted into the image-laser radar coordinate conversion equation to obtain the corresponding fourth equation.

[0120] Step 513, based on the PnP algorithm, the first, second, third and fourth equations are solved to obtain the parameter values of the twelve matrix parameters r 11 , r 12 , r 13 , r 21 , r 22 , r 23 , r 31 , r 32 , r 33 , t1, t2 and t3, and the parameter values of the twelve matrix parameters are substituted into the camera extrinsic matrix to obtain the corresponding first extrinsic matrix.

[0121] Here, the PnP (Perspective-n-Point) algorithm is an algorithm for solving a three-dimensional to two-dimensional projection conversion matrix, and the PnP algorithm can estimate a corresponding projection conversion matrix based on a known n pairs of three-dimensional-two-dimensional matching points, where n≥3; that is, in the case that four pairs of first center point pixel coordinates (u, v) + first center point radar coordinates (x, y, z) and a camera intrinsic matrix A are known, the camera extrinsic matrix can be estimated by the PnP algorithm to obtain a corresponding first extrinsic matrix;

[0122] Step 52, performing a binary graph conversion on the first image to generate a corresponding first calibration binary graph;

[0123] The size of the first calibration binary graph is consistent with that of the first image; the color of the first base plate region on the first calibration binary graph is white, and the color of the remaining regions outside the first base plate region is black.

[0124] Here, the current step is similar to the foregoing step 311, and thus will not be repeated here.

[0125] Step 53, extracting the point clouds in the first, second, third, and fourth circular regions from the first point cloud as a corresponding first calibration point cloud;

[0126] Specifically, the first calibration point cloud is composed of the first, second, third, and fourth sub-point clouds of the first point cloud.

[0127] Here, the first calibration point cloud is actually a point cloud composed of the first, second, third, and fourth sub-point clouds obtained in the foregoing step 322.

[0128] Step 54, performing an extrinsic matrix fine-tuning on the first extrinsic matrix according to the first calibration point cloud and the first calibration binary graph to generate a corresponding second extrinsic matrix;

[0129] Specifically, step 541, determining a first relationship and a second relationship between three-axis Euler angles θ x , θ y , θ z in the camera coordinate system and a rotation matrix in the camera extrinsic matrix.

[0130] The first relationship is:

[0131] θ x =atan2(r 32 , r 33 )

[0132]

[0133] θ z=atan2(r 21 ,r 11 )

[0134] The second relationship is:

[0135]

[0136] Here, the first, second and third Euler angles θ x , θ y , θ z in the camera coordinate system and the first and second relationships of the rotation matrix in the camera extrinsic matrix are known conversion relationships, and the derivation process can be referred to relevant public technical documents, which will not be described here.

[0137] Step 542, the matrix parameters r 11 , r 21 , r 31 , r 32 , r 33 of the first extrinsic matrix are brought into the first relationship for calculation, and the values of the three Euler angles θ x , θ y , θ z obtained are taken as the corresponding first Euler angle second Euler angle and third Euler angle

[0138] Step 543, according to the preset first, second and third fine-tuning thresholds a1, a2, a3 and the first, second and third Euler angles , the corresponding first, second and third parameter fine-tuning ranges are set; and according to the preset fourth, fifth and sixth fine-tuning thresholds a4, a5, a6 and the matrix parameters t1, t2, t3 of the first extrinsic matrix, the corresponding fourth, fifth and sixth parameter fine-tuning ranges are set; and according to the preset sliding number N and the first, second, third, fourth, fifth and sixth fine-tuning thresholds a1, a2, a3, a4, a5, a6, the corresponding first, second, third, fourth, fifth and sixth sliding steps △a1, △a2, △a3, △a4, △a5, △a6 are set.

[0139] Wherein,

[0140] The first parameter fine-tuning range is

[0141] The second parameter fine-tuning range is

[0142] The third parameter fine-tuning range is

[0143] The fourth parameter fine-tuning range is [t1-a4, t1+a4],

[0144] The fifth parameter fine tuning range is [t2-a5, t2+a5],

[0145] The sixth parameter fine tuning range is [t3-a6, t3+a6],

[0146] Δa1=2*a1 / N, Δa2=2*a2 / N, Δa3=2*a3 / N,

[0147] Δa4=2*a4 / N, Δa5=2*a5 / N, Δa6=2*a6 / N;

[0148] Step 544, the first index i, the first variable θ x,i , the second variable θ y,i , the third variable θ z,i , the fourth variable t 1,i , the fifth variable t 2,i and the sixth variable t 3,i are initialized:

[0149] i=0,

[0150]

[0151]

[0152]

[0153] t 1,i =t1-a4,

[0154] t 2,i =t2-a5,

[0155] t 3,i =t3-a6;

[0156] Step 545, the first, second, third variable θ x,i , θ y,i , θ z,i are substituted into the second relationship to obtain a set of the latest matrix parameters r 11,i , r 12,i , r 13,i , r 21,i , r 22,i , r 23,i , r 31,i , r 32,i , r 33,I ; and the latest matrix parameters r 11,i , r 12,i , r 13,i , r 21,i , r 22,i , r 23,i , r 31,i, r 32,i , r 33,I and the fourth, fifth, sixth variables t 1,i , t 2,i , t 3,i Substitute the matrix parameters of the first extrinsic matrix to obtain the corresponding first fine-tuned extrinsic matrix

[0157] Step 546, based on the first fine-tuned extrinsic matrix, the laser radar coordinates of each point in the first calibration point cloud are converted from the laser radar coordinate system to the pixel coordinate system to generate the corresponding first pixel coordinates; and the pixel points on the first calibration binary image that match each first pixel coordinate are recorded as the corresponding first pixel points; and the pixel values of all first pixel points are summed to generate the corresponding first pixel sum S i ;

[0158] Step 547, add 1 to the first index i; and according to the first, second, third, fourth, fifth and sixth sliding steps △a1, △a2, △a3, △a4, △a5, △a6 and the new first index i, reset the first variable θ x,i , the second variable θ y,i , the third variable θ z,i , the fourth variable t 1,i , the fifth variable t 2,i and the sixth variable t 3,i ;

[0159]

[0160]

[0161]

[0162] t 1,i = (t1-a4) + △a4*i,

[0163] t 2,i = (t2-a5) + △a5*i,

[0164] t 3,i = (t3-a6) + △a6*i;

[0165] Step 548, determine whether the first index i is greater than the sliding number N; if yes, go to step 549; if not, go to step 545;

[0166] Step 549, select the minimum value from the obtained N+1 first pixel sums S i as the corresponding minimum pixel sum S min ; and the minimum pixel sum S minThe corresponding first fine-tuned extrinsic parameter matrix is taken as the corresponding second extrinsic parameter matrix;

[0167] In step 55, the second extrinsic parameter matrix is taken as the calibration result of the calibration of the extrinsic parameter of the current vehicle-mounted camera.

[0168] Here, the current step 5 is actually to take the first parameter fine-tuning range The second parameter fine-tuning range The third parameter fine-tuning range The fourth parameter fine-tuning range [t1-a4, t1+a4], the fifth parameter fine-tuning range [t2-a5, t2+a5], and the sixth parameter fine-tuning range [t3-a6, t3+a6] are parameter sliding ranges, and the first, second, third, fourth, fifth, and sixth sliding steps △a1, △a2, △a3, △a4, △a5, and △a6 are single-step sliding steps. From the starting position to the ending position of the first, second, third, fourth, fifth, and sixth parameter fine-tuning range, N+1 times of sliding are performed to obtain N+1 sets of fine-tuning variables (the first variable θ x,i , the second variable θ y,i , the third variable θ z,i , the fourth variable t 1,i , the fifth variable t 2,i , and the sixth variable t 3,i ); and each time a set of fine-tuning variables is obtained, the matrix parameters of the first extrinsic parameter matrix are fine-tuned based on the fine-tuning variables of the current time to obtain the corresponding first fine-tuned extrinsic parameter matrix, and the pixel coordinates of each point of the first calibration point cloud are converted based on the first fine-tuned extrinsic parameter matrix to obtain the corresponding first pixel coordinates, and the sum of the pixel values corresponding to all the first pixel coordinates on the first calibration binary image is obtained to obtain the corresponding first pixel sum S i ; finally, the minimum value is selected from N+1 first pixel sums S i corresponding to N+1 sets of fine-tuning variables as the minimum pixel sum S min , and the first fine-tuned extrinsic parameter matrix corresponding to the set of fine-tuning variables corresponding to the minimum pixel sum S min is taken as the output result of the camera extrinsic parameter calibration.

[0169] Figure 3 A module structure diagram of a processing device for calibrating the extrinsic parameter of a vehicle-mounted camera is provided in Embodiment Two of the application. The device is a terminal device or a server for implementing the aforementioned method embodiments, or a device capable of enabling the terminal device or the server to implement the aforementioned method embodiments, such as a device or a chip system of the terminal device or the server. As shown in the figure, the device comprises a preparation module 201, a matching point pair processing module 202, and an extrinsic parameter matrix processing module 203. Figure 3

[0170] ​The preparation module 201 is configured to select one vehicle-mounted camera of the vehicle as a corresponding first camera, and select one vehicle-mounted laser radar of the vehicle as a corresponding first laser radar; and set a calibration board arranged in a vertical ground manner in an overlapping area of a shooting range of the first camera and a scanning range of the first laser radar as a corresponding first calibration board; the first calibration board is in a rectangular shape, and the first calibration board includes four circular image area regions of upper left, upper right, lower right and lower left, which are recorded as a corresponding first, second, third and fourth circular region, and an area of the first calibration board except the four circular image area regions is recorded as a first bottom plate region; the four circular image area regions are uniformly covered by a thin film of light-colored high-reflective material, and the first bottom plate region is not covered by the thin film of high-reflective material and the color of the first bottom plate region is black.

[0171] The preparation module 201 is further configured to call the first camera to perform image shooting and take the shot image as a corresponding first image; and call the first laser radar to perform scanning and take the scanned laser radar point cloud as a corresponding first point cloud.

[0172] The matching point pair processing module 202 is configured to identify pixel coordinates of center points of the four circular regions of the calibration board in the first image to generate corresponding first, second, third and fourth center point pixel coordinates; identify laser radar coordinates of the center points of the four circular regions of the calibration board in the first point cloud to generate corresponding first, second, third and fourth center point radar coordinates; generate a corresponding first matching point pair by combining the obtained first center point pixel coordinates and the first center point radar coordinates; generate a corresponding second matching point pair by combining the obtained second center point pixel coordinates and the second center point radar coordinates; generate a corresponding third matching point pair by combining the obtained third center point pixel coordinates and the third center point radar coordinates; and generate a corresponding fourth matching point pair by combining the obtained fourth center point pixel coordinates and the fourth center point radar coordinates.

[0173] The extrinsic matrix processing module 203 is configured to perform camera extrinsic parameter pre-estimation according to the first, second, third and fourth matching point pairs to generate a corresponding first extrinsic matrix; perform binary graph conversion on the first image to generate a corresponding first calibration binary graph; extract point clouds in the first, second, third and fourth circular regions from the first point cloud as a corresponding first calibration point cloud; perform extrinsic matrix fine-tuning on the first extrinsic matrix according to the first calibration point cloud and the first calibration binary graph to generate a corresponding second extrinsic matrix; and output the second extrinsic matrix as a calibration result of the vehicle-mounted camera extrinsic parameter calibration.

[0174] The processing device for calibrating vehicle-mounted camera extrinsic parameters provided in the embodiment of the application can execute the method steps in the method embodiments, and has similar implementation principles and technical effects, which will not be described here in detail.

[0175] It should be noted that the division of the various modules of the above apparatus is only a logical functional division, and in actual implementation, all or part of them can be integrated into one physical entity, or can be physically separated. These modules can all be implemented in the form of software invoked by a processing element; all can be implemented in the form of hardware; or some modules can be implemented in the form of software invoked by a processing element, and some modules can be implemented in the form of hardware. For example, the preparation module can be a separately established processing element, or can be integrated in a certain chip of the above apparatus, in addition, it can also be stored in the form of program code in the memory of the above apparatus, and the function of the above determination module is invoked and executed by a certain processing element of the above apparatus. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or can be independently implemented. The processing element described herein can be an integrated circuit with signal processing capability. In the implementation process, each step of the above method or each module can be completed by the integrated logic circuit of the hardware in the processor element or the instructions in the form of software.

[0176] For example, the above modules can be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), etc. For another example, when a certain module above is implemented in the form of program code invoked by a processing element, the processing element can be a general purpose processor, such as a central processing unit (CPU) or other processor that can invoke program code. For another example, these modules can be integrated together to implement in the form of system on a chip (SOC).

[0177] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0178] Figure 4 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be the aforementioned terminal device or server, or it can be a terminal device or server connected to the aforementioned terminal device or server that implements the method of the embodiments of the present invention. Figure 4 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing method embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0179] exist Figure 4The system bus 305 mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or only one type of bus. The communication interface is used to realize the communication between the database access device and other devices (such as the client, the read-write library and the read-only library). The memory can include a Random Access Memory (RAM), and can also include a Non-Volatile Memory, such as at least one disk memory.

[0180] The processor mentioned above can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), a Graphics Processing Unit (GPU), etc.; can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0181] It should be noted that the embodiment of the present application also provides a computer readable storage medium, which stores instructions, when the instructions are run on a computer, the computer executes the method and process provided in the above embodiment.

[0182] The embodiment of the present application also provides a chip for running instructions, which is used to execute the processing steps described in the foregoing method embodiment.

[0183] The embodiment of the present application provides a processing method and device for calibrating external parameters of a vehicle-mounted camera, an electronic device and a computer readable storage medium; four pairs of matching points in a pixel coordinate system and a laser radar coordinate system are obtained by taking a simple calibration board as a reference; the camera external parameters are estimated based on the four pairs of matching points; and the estimated camera external parameters are fine-tuned based on a calibration point cloud. Through the method of the present application, the external parameter calibration can be completed without using any complex calibration equipment; the calibration board is placed on the vehicle, and the staff can perform autonomous calibration at any position without the need to send the vehicle to a fixed calibration site.

[0184] Those skilled in the art should further appreciate that the elements and algorithms described in connection with the examples disclosed herein can be embodied in electronic hardware, computer software, or in combinations of both. To clearly illustrate this interchangeability of hardware and software, various examples have been described herein in terms of their general functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0185] The steps of a method or algorithm described in connection with the examples disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0186] The specific implementation described above is further to the purposes, technical solutions, and beneficial effects of the present application. It should be understood that the above description is merely a specific implementation of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A processing method for calibrating extrinsic parameters of a vehicle-mounted camera, characterized in that, The method comprises: selecting a vehicle-mounted camera of a vehicle as a corresponding first camera, and selecting a vehicle-mounted laser radar of the vehicle as a corresponding first radar; and setting a calibration board arranged in a vertical ground manner in an overlapping area of a shooting range of the first camera and a scanning range of the first radar as a corresponding first calibration board; the first calibration board is in a rectangular shape, and the first calibration board comprises four circular image area regions of upper left, upper right, lower right and lower left, which are recorded as corresponding first, second, third and fourth circular regions; and an area of the first calibration board other than the four circular image area regions is recorded as a first bottom plate region; the four circular image area regions are uniformly covered by a thin film of light-colored high-reflection material, the first bottom plate region is not covered by the thin film of high-reflection material, and the color of the first bottom plate region is black; calling the first camera to shoot an image and taking the shot image as a corresponding first image; and calling the first radar to scan and taking a laser radar point cloud obtained by scanning as a corresponding first point cloud; identifying pixel coordinates of center points of the four circular regions of the calibration board in the first image to generate corresponding first, second, third and fourth center point pixel coordinates; and identifying laser radar coordinates of the center points of the four circular regions of the calibration board in the first point cloud to generate corresponding first, second, third and fourth center point radar coordinates; composing the obtained first center point pixel coordinates and first center point radar coordinates into a corresponding first matching point pair; composing the obtained second center point pixel coordinates and second center point radar coordinates into a corresponding second matching point pair; composing the obtained third center point pixel coordinates and third center point radar coordinates into a corresponding third matching point pair; and composing the obtained fourth center point pixel coordinates and fourth center point radar coordinates into a corresponding fourth matching point pair; generating a corresponding first extrinsic parameter matrix according to the first, second, third and fourth matching point pairs; performing binary image conversion on the first image to generate a corresponding first calibration binary image; extracting point clouds in the first, second, third and fourth circular regions from the first point cloud as a corresponding first calibration point cloud; performing extrinsic parameter matrix fine tuning on the first extrinsic parameter matrix according to the first calibration point cloud and the first calibration binary image to generate a corresponding second extrinsic parameter matrix; and outputting the second extrinsic parameter matrix as a calibration result of vehicle-mounted camera extrinsic parameter calibration.

2. The method of claim 1, wherein, The identification of the pixel coordinates of the center points of the four circular regions of the calibration board in the first image to generate the corresponding first, second, third and fourth center point pixel coordinates specifically comprises: performing binary image conversion on the first image to obtain a corresponding first binary image; the size of the first binary image remains consistent with that of the first image; the color of the first bottom plate region on the first binary image is white, and the color of the remaining regions other than the first bottom plate region is black; The four vertices of the first floor area are sequentially connected on the first binary image to obtain a corresponding quadrilateral area, which is referred to as a first quadrilateral area; the first, second, third, and fourth circular areas on the first quadrilateral area correspond to four black circular areas; Edge points of the four black circular areas in the first quadrilateral area are extracted to generate corresponding first, second, third, and fourth edge point sets; Hough transform circular detection processing is performed on the first, second, third, and fourth edge point sets to obtain four circular center points, which are referred to as first, second, third, and fourth circular center points; and pixel coordinates of the first, second, third, and fourth circular center points are taken as corresponding first, second, third, and fourth center point pixel coordinates.

3. The processing method for calibrating an extrinsic parameter of a vehicle-mounted camera according to claim 1, characterized in that, Each point in the first point cloud corresponds to a first point cloud data; the first point cloud data includes a first laser radar coordinate and a first radar reflection intensity; the first laser radar coordinate includes a first depth coordinate component.

4. The method of claim 3, wherein, The laser radar coordinates of the four circular area center points of the calibration board in the first point cloud are identified to generate corresponding first, second, third, and fourth center point radar coordinates, specifically including: Points with the first radar reflection intensity higher than a preset high reflection intensity threshold in the first point cloud are recorded as corresponding first high reflection points; and all the first high reflection points are extracted from the first point cloud to form a corresponding second point cloud; Point cloud clustering is performed on the second point cloud to obtain four clustering sub-point clouds; the four clustering sub-point clouds are respectively marked as corresponding first, second, third, and fourth sub-point clouds according to the corresponding relationship with the upper left, upper right, lower right, and lower left four circular image areas on the first calibration board; and edge point cloud identification is performed on the first, second, third, and fourth sub-point clouds to obtain corresponding first, second, third, and fourth edge point clouds; Spherical surface fitting is performed on the first, second, third, and fourth edge point clouds to obtain four spherical surface center points, which are referred to as first, second, third, and fourth spherical surface center points; and laser radar coordinates of the first, second, third, and fourth spherical surface center points are taken as corresponding first, second, third, and fourth center point radar coordinates.

5. The method of claim 1, wherein, The first, second, third, and fourth matching point pairs are used to generate a corresponding first extrinsic parameter matrix, specifically including: The pixel-laser radar coordinate conversion equation is determined as Wherein (x, y, z) is the three-dimensional coordinates of the laser radar coordinate coordinate system, (u, v) is the two-dimensional coordinates of the pixel coordinate system, A is the known camera intrinsic matrix, is the camera extrinsic matrix to be calibrated, r 11 , r 12 , r 13 , r 21 , r 22 , r 23 , r 31 , r 32 , r 33 , t1, t2 and t3 are the matrix parameters of the camera extrinsic matrix; the first center point pixel coordinate and the first center point radar coordinate of the first matched point pair are substituted into an image-lidar coordinate conversion equation to obtain a corresponding first equation; the second center point pixel coordinate and the second center point radar coordinate of the second matched point pair are substituted into the image-lidar coordinate conversion equation to obtain a corresponding second equation; the third center point pixel coordinate and the third center point radar coordinate of the third matched point pair are substituted into the image-lidar coordinate conversion equation to obtain a corresponding third equation; and the fourth center point pixel coordinate and the fourth center point radar coordinate of the fourth matched point pair are substituted into the image-lidar coordinate conversion equation to obtain a corresponding fourth equation; solving the first, second, third and fourth equations based on the PnP algorithm to obtain twelve matrix parameters r 11 12 13 21 22 23 31 32 33 , t1, t2 and t3, and substituting the parameter values of the twelve matrix parameters into the camera extrinsic parameter matrix to obtain the corresponding first extrinsic parameter matrix.​​​​​​​​ 6. The processing method of claim 1, wherein, a size of the first calibration binary image is consistent with the first image; a color of the first floor area on the first calibration binary image is white, and a color of a remaining area outside the first floor area is black.

7. The method of claim 4, wherein, The first, second, third, and fourth sub-point clouds in the first point cloud are extracted as corresponding first calibration point clouds. The first, second, third, and fourth sub-point clouds of the first point cloud form the corresponding first calibration point clouds.

8. The method of claim 6, wherein, The first calibration point clouds and the first calibration binary image are used to fine-tune the first extrinsic parameter matrix to generate a corresponding second extrinsic parameter matrix, and the fine-tuning of the first extrinsic parameter matrix includes: Step 81, determine the first relationship and the second relationship of three-axis Euler angles θ x , θ y , θ z and the rotation matrix in the camera extrinsic matrix ​ The first relationship is: The second relationship is: Step 82, the matrix parameters r 11 , r 21 , r 31 , r 32 , r 33 of the first extrinsic matrix are substituted into the first relationship formula for calculation, and the values of the three Euler angles θ x , θ y , θ z obtained are taken as the corresponding first Euler angle second Euler angle and third Euler angle Step 83, according to the preset first, second and third fine-tuning thresholds a1, a2, a3 and the first, second and third Euler angles corresponding first, second and third parameter fine-tuning ranges are set; and according to the preset fourth, fifth and sixth fine-tuning thresholds a4, a5, a6 and the matrix parameters t1, t2, t3 of the first extrinsic parameter matrix, corresponding fourth, fifth and sixth parameter fine-tuning ranges are set; and according to the preset sliding number N and the first, second, third, fourth, fifth and sixth fine-tuning thresholds a1, a2, a3, a4, a5, a6, corresponding first, second, third, fourth, fifth and sixth sliding step lengths △a1, △a2, △a3, △a4, △a5, △a6 are set; wherein, The first parameter fine tuning range is The second parameter fine tuning range is The third parameter fine tuning range is The fourth parameter fine-tuning range is [t1-a4, t1+a4], The fifth parameter fine-tuning range is [t2-a5, t2+a5], The sixth parameter fine-tuning range is [t3-a6, t3+a6], △a1=2*a1 / N,△a2=2*a2 / N,△a3=2*a3 / N, △a4=2*a4 / N,△a5=2*a5 / N,△a6=2*a6 / N; Step 84, the first index i, the first variable θ x,i , the second variable θ y,i , the third variable θ z,i , the fourth variable t 1,i , the fifth variable t 2,i and the sixth variable t 3,i are initialized: i＝0， t 1,i = t1-a4, t 2,i = t2-a5, t 3,i = t3-a6; Step 85, substituting the first, second, third variables θ x,i , θ y,i , θ z,i into the second relationship formula to obtain a set of latest matrix parameters r 11,i , r 12,i , r 13,i , r 21,i , r 22,i , r 23,i , r 31,i , r 32,i , r 33,I ; and using the latest matrix parameters r 11,i , r 12,i , r 13,i , r 21,i , r 22,i , r 23,i , r 31,i , r 32,i , r 33,I and the fourth, fifth, sixth variables t 1,i , t 2,i , t 3,i replace the matrix parameters of the first external parameter matrix to obtain a corresponding first fine-tuning external parameter matrix Step 86, based on the first fine-tuning external parameter matrix, the laser radar coordinates of each point in the first calibration point cloud are converted from the laser radar coordinate system to the pixel coordinate system to generate corresponding first pixel coordinates; and the pixel points on the first calibration binary image that match each of the first pixel coordinates are recorded as corresponding first pixel points; and the pixel values of all the first pixel points are summed to generate a corresponding first pixel sum S i ; Step 87, add 1 to the first index i; and reset the first variable θ x,i , the second variable θ y,i , the third variable θ z,i , the fourth variable t 1,i , the fifth variable t 2,i , and the sixth variable t 3,i according to the first, second, third, fourth, fifth, and sixth sliding steps Δa1, Δa2, Δa3, Δa4, Δa5, Δa6 and the new first index i. t 1,i = (t1-a4) + Δa4*i, t 2,i = (t2-a5) + a5*i, t 3,i = (t3-a6) + Da6*i; Step 88, determining whether the first index i is greater than the sliding number N; if yes, going to step 89; if not, going to step 85; Step 89, selecting the minimum value from the N+1 sums S of the first pixels as the corresponding minimum pixel sum S i Step 90, selecting the first fine-tuned extrinsic parameter matrix corresponding to the minimum pixel sum S as the corresponding second extrinsic parameter matrix. min Step 91, selecting the minimum value from the N+1 sums S of the second pixels as the corresponding minimum pixel sum S min Step 92, selecting the first fine-tuned extrinsic parameter matrix corresponding to the minimum pixel sum S as the corresponding second extrinsic parameter matrix.

9. An apparatus for performing the processing method of calibrating the extrinsic parameters of a vehicle camera according to any one of claims 1-8, characterized in that, The device includes a preparation module, a matched point pair processing module, and an extrinsic parameter matrix processing module. The preparation module is configured to select a vehicle-mounted camera of a vehicle as a corresponding first camera, select a vehicle-mounted lidar of the vehicle as a corresponding first lidar, and set a calibration board arranged in a vertical ground manner in an overlapping area of a shooting range of the first camera and a scanning range of the first lidar as a corresponding first calibration board. The first calibration board has a rectangular shape, and includes four circular image areas, i.e., a first, a second, a third, and a fourth circular image area, on the first calibration board. The four circular image areas are uniformly covered by a light-colored high-reflective material film. The first calibration board has no high-reflective material film covering the first floor area, and the first floor area has a black color. The preparation module is further configured to call the first camera to take an image and take the image as a corresponding first image; and call the first radar to scan and take a laser radar point cloud obtained by scanning as a corresponding first point cloud. The matching point pair processing module is configured to identify pixel coordinates of four center points of the circular regions of the calibration board in the first image to generate corresponding first, second, third, and fourth center point pixel coordinates; identify laser radar coordinates of the four center points of the circular regions of the calibration board in the first point cloud to generate corresponding first, second, third, and fourth center point radar coordinates; generate a first matching point pair by combining the first center point pixel coordinates and the first center point radar coordinates; generate a second matching point pair by combining the second center point pixel coordinates and the second center point radar coordinates; generate a third matching point pair by combining the third center point pixel coordinates and the third center point radar coordinates; and generate a fourth matching point pair by combining the fourth center point pixel coordinates and the fourth center point radar coordinates. The extrinsic parameter matrix processing module is configured to generate a first extrinsic parameter matrix by performing camera extrinsic parameter pre-estimation based on the first, second, third, and fourth matching point pairs; perform binary graph conversion on the first image to generate a first calibration binary graph; extract point clouds in the first, second, third, and fourth circular regions from the first point cloud as a first calibration point cloud; perform extrinsic parameter matrix fine-tuning on the first extrinsic parameter matrix based on the first calibration point cloud and the first calibration binary graph to generate a second extrinsic parameter matrix; and output the second extrinsic parameter matrix as a calibration result of the extrinsic parameter calibration of the vehicle-mounted camera.

10. An electronic device, comprising: comprise: a memory, a processor, and a transceiver; the processor is configured to be coupled with the memory, read and execute instructions in the memory to implement the method steps of any one of claims 1-8; the transceiver is coupled with the processor, and the transceiver is controlled by the processor to perform message transmission and reception.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, when the computer instructions are executed by a computer, the computer instructions cause the computer to execute the instructions of the method of any one of claims 1-8. The computer readable storage medium stores computer instructions, when the computer instructions are executed by a computer, the computer instructions cause the computer to execute the instructions of the method of any one of claims 1-8.

Images