A camera extrinsic parameter calibration method, system, device and storage medium

Abstract

The application discloses a camera external parameter calibration method, comprising the following steps: acquiring a first image collected by a camera and extracting first corner points of a preset reference object from the first image; acquiring an internal parameter matrix, an initial external parameter matrix and a point cloud of the preset reference object of the camera; projecting each point in the point cloud onto a second image with the same size as the first image based on the internal parameter matrix and the initial external parameter matrix and extracting second corner points of the preset reference object from the second image; registering the first corner points and the second corner points to obtain a transformation matrix; and obtaining an external parameter conversion matrix of the camera by using the internal parameter matrix, the initial external parameter matrix and the transformation matrix. The application further discloses a system, a computer device and a readable storage medium. The scheme provided by the application does not need a special calibration board to be deployed on a road by a calibration personnel, a manual intervention is not needed in a calibration process, and the calibration efficiency can be greatly improved and the labor cost can be saved.

Description

Technical Field

[0001] This invention relates to the field of extrinsic parameter calibration, and specifically to a camera extrinsic parameter calibration method, system, device, and storage medium. Background Technology

[0002] Camera extrinsic parameters are a set of data representing the camera's position and attitude in a relative coordinate system. They include the coordinates relative to the coordinate system and the rotations around the three axes of the coordinate system. Therefore, the camera extrinsic parameters can only be determined after the camera is fixed in a relative coordinate system. The most common method for extrinsic parameter calibration is to use a specially designed calibration plate. This method is suitable for simple indoor environments. For cameras installed on the roadside, due to the complex outdoor environment and road safety concerns, it is difficult to calibrate them using a specially designed calibration plate. Summary of the Invention

[0003] In view of this, in order to overcome at least one of the above problems, embodiments of the present invention propose a camera extrinsic parameter calibration method, comprising the following steps:

[0004] Acquire the first image captured by the camera and extract the first corner point of the preset reference object from the first image;

[0005] Obtain the intrinsic parameter matrix, initial extrinsic parameter matrix of the camera, and the point cloud of the preset reference object;

[0006] Based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, each point in the point cloud is projected onto a second image of the same size as the first image, and the second corner point of the preset reference object is extracted from the second image;

[0007] The transformation matrix is ​​obtained by registering the first corner point and the second corner point;

[0008] The camera's extrinsic transformation matrix is ​​obtained using the intrinsic parameter matrix, the initial extrinsic parameter matrix, and the transformation matrix.

[0009] In some embodiments, acquiring a first image captured by a camera and extracting a first corner point of a preset reference object from the first image further includes:

[0010] Place the first image in the HSV color space;

[0011] The first image is filtered according to a preset first color threshold range to obtain a first region image;

[0012] Extract edge contours from the first region and extract a second region image from the first image based on the edge contours;

[0013] The second region image is filtered according to a preset second color threshold range to obtain a third region image from which the first corner point is extracted.

[0014] In some embodiments, filtering the first image according to a preset first color threshold range to obtain a first region image further includes:

[0015] Convert the first region image into a binary image;

[0016] The binary image is subjected to multiple morphological dilation and erosion processes.

[0017] In some embodiments, extracting edge contours from the first region further includes:

[0018] Extract all contours from the first region and use the largest contour as the edge contour.

[0019] In some embodiments, filtering the second region image according to a preset second color threshold range to obtain a third region image and extracting the first corner point from it further includes:

[0020] Convert the third region image into a binary image;

[0021] The first corner point is extracted from the binary image after performing multiple image morphological dilation and erosion operations.

[0022] In some embodiments, obtaining the initial extrinsic parameter matrix of the camera further includes:

[0023] The initial extrinsic parameter matrix is ​​determined based on the camera's pose.

[0024] In some embodiments, obtaining the point cloud of the preset reference object further includes:

[0025] Obtain the first point cloud, which includes multiple point cloud blocks;

[0026] Based on the camera coordinates and the preset boundary angle, multiple point cloud blocks of the first point cloud are filtered to obtain several point cloud blocks, which are then stitched together to obtain the second point cloud.

[0027] The third point cloud is obtained by filtering from the second point cloud based on its altitude.

[0028] Calculate the average density of the third point cloud and use the point cloud with a density greater than the average density as the point cloud of the preset reference object.

[0029] In some embodiments, the process of filtering multiple point cloud blocks of the first point cloud according to the coordinates of the camera and a preset boundary angle to obtain several point cloud blocks, and then stitching them together to obtain a second point cloud, further includes:

[0030] Using the coordinates of the camera as the center, draw a circle with a radius of R1 meters, and enclose a fan-shaped area with ±a° of the direction the camera is facing as the boundary;

[0031] Several point cloud blocks centered within a fan-shaped region are stitched together to obtain a second point cloud.

[0032] In some embodiments, filtering from the second point cloud to obtain a third point cloud based on altitude further includes:

[0033] Determine the altitude range (Z) of the second point in the cloud. min Z max );

[0034] Divide the interval into multiple sub-intervals of a preset size and number each sub-interval in sequence;

[0035] Based on the elevation of each point in the second point cloud and the range of each sub-interval, all points in the second point cloud are grouped, and the two groups with the most and second most points are determined, wherein the number of each group is the same as the number of the sub-interval.

[0036] The elevation range of the third point cloud is determined based on the two groups with the largest and second largest number of clouds to filter out the third point cloud.

[0037] In some embodiments, the elevation range of the third point cloud is determined based on the two most numerous and the two most numerous groups to filter out the third point cloud, further including determining the elevation range (z) of the third point cloud according to the following formula in response to the adjacency of the sub-intervals corresponding to the two groups. min , z max ):

[0038] z min =min(i A i B )*z size +Z min , z max =max(i A i B )*z size +z size +Z min

[0039] Among them, i A The number of the group with the largest number, i B z is the number of the second largest group. size This is the preset size.

[0040] In some embodiments, the elevation range of the third point cloud is determined based on the two most numerous and the two most numerous groups to filter out the third point cloud, further including determining the elevation range (z) of the third point cloud according to the following formula in response to the fact that the ranges of the sub-intervals corresponding to the two groups are not adjacent. min , z max ):

[0041] z min =i A *z size +Z min , z max =i A *z size +z size +Z min

[0042] Among them, i A The number of the group with the largest number, z size This is the preset size.

[0043] In some embodiments, calculating the average density of the third point cloud and using point clouds with density values ​​greater than the average density as the point cloud of the preset reference object further includes:

[0044] The outlier removal algorithm is used to remove noise points in the point cloud of the preset reference object.

[0045] In some embodiments, projecting each point in the point cloud onto a second image of the same size as the first image based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, and extracting the second corner point of the preset reference object from the second image, further includes projecting according to the following formula:

[0046]

[0047] Where, x u and y u For projected coordinates, I M It is a 3x3 intrinsic parameter matrix. CAM is a column vector of all zeros. EX Let x be the initial extrinsic parameter matrix. w y w and z w The coordinates of a point in the point cloud.

[0048] In some embodiments, it also includes:

[0049] Based on the projection coordinates, corresponding mapping points are drawn on the second image, where the pixel value of the mapping points is 255 and the pixel value of the rest of the background is 0.

[0050] The second corner point is extracted from the second image after performing multiple image morphological dilation and erosion operations.

[0051] In some embodiments, the camera's extrinsic transformation matrix is ​​obtained using the intrinsic parameter matrix, the initial extrinsic parameter matrix, and the transformation matrix, further comprising obtaining the camera's extrinsic transformation matrix (R′T′) according to the following formula:

[0052]

[0053] Among them, I M -1 For I M The inverse matrix, F is the transformation matrix, I M It is a 3x3 intrinsic parameter matrix. CAM is a column vector of all zeros. EX This is the initial extrinsic parameter matrix.

[0054] In some embodiments, it also includes:

[0055] The lane lines are used as the preset reference.

[0056] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a camera extrinsic parameter calibration system, comprising:

[0057] The first extraction module is configured to acquire a first image captured by the camera and extract a first corner point of a preset reference object from the first image.

[0058] The acquisition module is configured to acquire the intrinsic parameter matrix, the initial extrinsic parameter matrix of the camera, and the point cloud of the preset reference object;

[0059] The projection module is configured to project each point in the point cloud onto a second image of the same size as the first image based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, and to extract the second corner point of the preset reference object from the second image.

[0060] The registration module is configured to register the first corner point and the second corner point to obtain a transformation matrix;

[0061] The calculation module is configured to obtain the camera's extrinsic transformation matrix using the intrinsic parameter matrix, the initial extrinsic parameter matrix, and the transformation matrix.

[0062] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a computer device, comprising:

[0063] At least one processor; and

[0064] The memory stores a computer program that can run on the processor, which, when executing the program, performs the steps of any of the camera extrinsic calibration methods described above.

[0065] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of any of the camera extrinsic parameter calibration methods described above.

[0066] The present invention has one of the following beneficial technical effects: the proposed solution does not require calibration personnel to deploy special calibration boards on the road, and the calibration process does not require manual intervention; moreover, after the camera equipment is installed on the roadside, the camera extrinsic parameters can be calculated, which is more suitable for the calibration of roadside camera extrinsic parameters, and can greatly improve calibration efficiency and save labor costs. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0068] Figure 1 A schematic flowchart of a camera extrinsic parameter calibration method provided for an embodiment of the present invention;

[0069] Figure 2 A flowchart illustrating a camera extrinsic parameter calibration method provided for an embodiment of the present invention;

[0070] Figure 3 A schematic diagram of a directionally loaded dense point cloud region provided for an embodiment of the present invention;

[0071] Figure 4 A schematic diagram of the camera extrinsic parameter calibration system provided in an embodiment of the present invention;

[0072] Figure 5 A schematic diagram of the structure of a computer device provided for an embodiment of the present invention;

[0073] Figure 6 A schematic diagram of the structure of a computer-readable storage medium provided for an embodiment of the present invention. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0075] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0076] According to one aspect of the present invention, embodiments of the present invention provide a method for calibrating camera extrinsic parameters, such as... Figure 1 As shown, it may include the following steps:

[0077] S1, acquire the first image captured by the camera and extract the first corner point of the preset reference object from the first image;

[0078] S2, obtain the intrinsic parameter matrix of the camera, the initial extrinsic parameter matrix, and the point cloud of the preset reference object;

[0079] S3, based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, project each point in the point cloud onto a second image of the same size as the first image, and extract the second corner point of the preset reference object from the second image;

[0080] S4, register the first corner point and the second corner point to obtain the transformation matrix;

[0081] S5. The camera's extrinsic transformation matrix is ​​obtained using the intrinsic parameter matrix, the initial extrinsic parameter matrix, and the transformation matrix.

[0082] The proposed solution eliminates the need for calibration personnel to deploy specially designed calibration boards on the road, and the calibration process requires no manual intervention. Furthermore, once the camera equipment is installed on the roadside, the camera's extrinsic parameters can be calculated, making it more suitable for calibrating the extrinsic parameters of roadside cameras. This significantly improves calibration efficiency and saves labor costs.

[0083] In some embodiments, lane lines can be used as the preset reference point. Other reference points, such as signs or poles, can also be chosen.

[0084] In some embodiments, S1, acquiring a first image captured by a camera and extracting a first corner point of a preset reference object from the first image, further includes:

[0085] S11, Place the first image in the HSV color space;

[0086] S12, the first image is filtered according to a preset first color threshold range to obtain a first region image;

[0087] S13, extract edge contours from the first region and extract a second region image from the first image based on the edge contours;

[0088] S14, the second region image is filtered according to the preset second color threshold range to obtain the third region image and extract the first corner point from it.

[0089] In some embodiments, S12, filtering the first image according to a preset first color threshold range to obtain a first region image further includes:

[0090] S121, convert the first region image into a binary image;

[0091] S122, perform multiple image morphological dilation and erosion operations on the binary image.

[0092] In some embodiments, S13, extracting edge contours from the first region further includes:

[0093] S131, extract all contours from the first region and take the largest contour as the edge contour.

[0094] In some embodiments, S14, filtering the second region image according to a preset second color threshold range to obtain a third region image and extracting the first corner point from it further includes:

[0095] S141, convert the third region image into a binary image;

[0096] S142, the first corner point is extracted from the binary image after performing multiple image morphological dilation and erosion operations.

[0097] The following section uses lane lines as a preset reference as an example to explain in detail how to extract the first corner point of the preset reference from the first image captured by the camera.

[0098] Specifically, such as Figure 2 As shown, since the camera image cam_image is in RGB format, it can first be converted to the HSV color space. Then, based on a preset first color threshold range (h_road_low, s_road_low, v_road_low) to (h_road_hight, s_road_hight, v_road_hight), the region containing the road (the first region image) is filtered out. This region is then converted to a binary image representation, and after N rounds of image morphological dilation and erosion, the road surface area becomes more coherent. Next, all contours can be extracted using image contour extraction methods, and the largest contour is selected. This contour represents the edge contour of the road. The portion of the camera image falling within this contour is then extracted to obtain the extracted road image road_image (the second region image).

[0099] Similarly, in the HSV color space, the lane line region image (third region image) is filtered out from the road image road_image using a preset second color threshold range. This image is also represented as a binary image. After N rounds of image morphological dilation and erosion, the edges of the lane line region are made smoother, resulting in the lane line binary image line_image. Finally, the first corner point is extracted from the lane line image line_image using an existing corner extraction algorithm.

[0100] The image lane line corner point extraction technology used in this invention can extract lane line related corner points more accurately, avoiding the extraction of too many corner points when extracting corner points globally, thus making corner point matching more accurate and efficient.

[0101] In some embodiments, S2, obtaining the initial extrinsic parameter matrix of the camera further includes:

[0102] The initial extrinsic parameter matrix is ​​determined based on the camera's pose.

[0103] Specifically, after the camera is installed on the roadside, the camera image cam_image is obtained. Since the installation position of the roadside camera is fixed, the relative coordinates of the camera (t) can be roughly obtained. x , t y , t z The camera's external parameter matrix CAM can be calculated by considering the angle γ between the camera and the Y-axis (north direction) in the northeast-north sky coordinate system, and the elevation angle β. The camera is assumed to be horizontal. EX : Where R is a 3x3 rotation matrix and T is a 3x1 translation matrix. The camera intrinsic parameter matrix can be calibrated indoors using existing algorithms to obtain the intrinsic parameter matrix I. M : Where x0 and y0 are the pixel offsets of the optical center in the x and y directions, respectively, and f x / represents the number of pixels in the x-direction, f y / represents the number of pixels in the y-direction.

[0104] In some embodiments, S2, acquiring the point cloud of the preset reference object further includes:

[0105] S21, Obtain the first point cloud, which includes multiple point cloud blocks;

[0106] S22, based on the coordinates of the camera and the preset boundary angle, multiple point cloud blocks of the first point cloud are filtered to obtain several point cloud blocks, which are then spliced ​​together to obtain the second point cloud.

[0107] S23, the third point cloud is obtained by filtering from the second point cloud based on the altitude;

[0108] S24, calculate the average density of the third point cloud and use the point cloud with a density greater than the average density as the point cloud of the preset reference object.

[0109] In some embodiments, S22, filtering multiple point cloud blocks of the first point cloud according to the coordinates of the camera and a preset boundary angle to obtain several point cloud blocks, and then stitching them together to obtain a second point cloud, further includes:

[0110] S221, draw a circle with a radius of R1 meters centered on the coordinates of the camera, and enclose a fan-shaped area with ±a° of the direction the camera is facing as the boundary;

[0111] S222, stitch together several point cloud blocks centered in the fan-shaped area to obtain the second point cloud.

[0112] In some embodiments, S23, filtering from the second point cloud to obtain a third point cloud based on altitude further includes:

[0113] S231, determine the altitude range (Z) of the second point in the cloud. min Z max );

[0114] S232, divide the interval into multiple sub-intervals of a preset size and number each sub-interval in sequence;

[0115] S233, based on the elevation of each point in the second point cloud and the range of each sub-interval, group all points in the second point cloud and determine the two groups with the most and second most points, wherein the number of each group is the same as the number of the sub-interval;

[0116] S234, determine the elevation range of the third point cloud based on the two groups with the largest and second largest number to filter out the third point cloud.

[0117] In some embodiments, S234, determining the elevation range of the third point cloud based on the two groups with the most and second most numbers to filter out the third point cloud further includes determining the elevation range (z) of the third point cloud according to the following formula in response to the adjacent ranges of the sub-intervals corresponding to the two groups. min , z max ):

[0118] z min =min(i A i B )*z size +Z min , z max =max(i Ai B )*z size +z size +Z min

[0119] Among them, i A The number of the group with the largest number, i B z is the number of the second largest group. size This is the preset size.

[0120] In some embodiments, S234, determining the elevation range of the third point cloud based on the two groups with the most and second most numbers to filter out the third point cloud further includes determining the elevation range (z) of the third point cloud according to the following formula in response to the fact that the ranges of the sub-intervals corresponding to the two groups are not adjacent. min , z max ):

[0121] z min =i A *z size +Z min , z max =i A *z size +z size +Z min

[0122] Among them, i A The number of the group with the largest number, z size This is the preset size.

[0123] In some embodiments, S24, calculating the average density of the third point cloud and using point clouds with density values ​​greater than the average density as the point cloud of the preset reference object further includes:

[0124] S241, noise points in the point cloud of the preset reference object are removed using an outlier removal algorithm.

[0125] The following section uses lane lines as a preset reference as an example to explain in detail how to extract the first corner point of the preset reference from the first image captured by the camera.

[0126] Specifically, such as Figure 2 As shown, a dense point cloud is first loaded in a directional manner. The dense point cloud (the first point cloud) consists of a series of points, each with coordinates (x, y, y). i y i , z iDense point clouds are stored in blocks, with each square enclosed by a dashed line representing a block of dense point cloud. To improve the loading and processing speed of dense point clouds, a directional loading scheme can be adopted. The specific method is as follows: A circle with a radius of R1 meters is drawn with the initial camera position as the center. A fan-shaped region is enclosed with the ±a° of the initial camera orientation as the boundary. The system calculates whether the center of the surrounding dense point cloud blocks lies within this fan-shaped region. If it does, the dense point cloud block is loaded. Figure 3 As shown, point cloud blocks A and C need to be loaded, while point cloud block B does not need to be loaded; by splicing all the loaded dense point cloud blocks, the final dense point cloud pointsCloud_raw (the second point cloud) is obtained.

[0127] Next, dense ground point cloud (third point cloud) is extracted from the dense point cloud. After loading the dense point cloud, the ground is extracted. A fast ground point cloud extraction algorithm can be used: based on the characteristics of dense point clouds, within a local area of ​​outdoor roads, the point clouds on the ground are basically at the same elevation, so a statistical method can be used to determine the ground elevation range (Z). min Z max Set the z-axis grid size z size Divide the Z-axis into n = (Z max -Z min ) / z size We first divide the dense point cloud into several intervals, then count the number of points falling within each interval, resulting in an array COUNTS. Each element in the array is (i, count), where i = 0.1...n-1, i represents the index, and count represents the number of points falling within that interval. To avoid the ground spanning exactly two intervals, we sort the array COUNTS by count, and select the two elements A(i, count) with the largest and second largest count values. A count A ), B(i B count B ), when i A i B When adjacent, the Z-interval of the ground point cloud: z min =min(i A i B )*z size +Z min , z max =max(i A i B )*z size +z size +Z min ; when i A i B When they are not adjacent, the Z-interval of the ground point cloud min =i A *z size +Zmin , z max =i A *z size +z size +Z min Determine the ground elevation range (z) min , z max After that, the loaded point cloud is filtered to obtain the dense ground point cloud pointsCloud_ground (the third point cloud).

[0128] Finally, a dense lane line point cloud (point cloud of a preset reference object) is extracted from the dense ground point cloud. Based on the characteristics of the laser point cloud, the point cloud at the lane line has high reflectivity. Therefore, the average point cloud density in pointsCloud_ground is calculated, and points with a density greater than the average are filtered out. Then, an outlier removal algorithm is used to filter out individual noisy points, resulting in the dense lane line point cloud pointsCloud_line.

[0129] In some embodiments, S3, projecting each point in the point cloud onto a second image of the same size as the first image based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, and extracting the second corner point of the preset reference object from the second image, further includes projecting according to the following formula:

[0130]

[0131] Where, x u and y u For projected coordinates, I M It is a 3x3 intrinsic parameter matrix. CAM is a column vector of all zeros. EX Let x be the initial extrinsic parameter matrix. w y w and z w The coordinates of a point in the point cloud.

[0132] In some embodiments, it also includes:

[0133] Based on the projection coordinates, corresponding mapping points are drawn on the second image, where the pixel value of the mapping points is 255 and the pixel value of the rest of the background is 0.

[0134] The second corner point is extracted from the second image after performing multiple image morphological dilation and erosion operations.

[0135] The following section uses lane lines as a preset reference as an example to explain in detail how to project each point in the point cloud onto a second image of the same size as the first image based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, and extract the second corner point of the preset reference from the second image.

[0136] Specifically, such as Figure 2 As shown, based on the principles of pinhole imaging and rigid body transformation, the dense point cloud of lane lines can be projected into a lane line point cloud image (points_image). This allows the points (x, y, z) representing the dense point cloud of lane lines to be represented. w y w , z w Converting 0.05 to pixel positions in an image, the conversion formula is as follows:

[0137]

[0138] The point is then drawn in a blank image of the same size as cam_image, where the background has a pixel value of 0 and the dense point cloud mapping point has a pixel value of 255. That is, the drawn points_image is a binary image with a black background and the mapped lane line is white. Then, the narrow gaps in the binary image are removed by M times of image morphology dilation and erosion.

[0139] Finally, the second corner point in the lane line point cloud image points_image is extracted using an existing corner extraction algorithm.

[0140] The lane corner point extraction technology used in this invention solves the problem of difficult lane corner point extraction in 3D point clouds, and the fast ground point cloud extraction algorithm increases computational efficiency.

[0141] In some embodiments, the second corner point extracted from the lane line point cloud image points_image and the first corner point in the camera image cam_image are registered to obtain a 3x3 single mapping transformation matrix F.

[0142] In some embodiments, the camera's extrinsic transformation matrix is ​​obtained using the intrinsic parameter matrix, the initial extrinsic parameter matrix, and the transformation matrix, further comprising obtaining the camera's extrinsic transformation matrix (R′T′) according to the following formula:

[0143]

[0144] Among them, I M -1 For I M The inverse matrix, F is the transformation matrix, I M It is a 3x3 intrinsic parameter matrix. CAM is a column vector of all zeros. EX This is the initial extrinsic parameter matrix.

[0145] Where R′ is the calibrated 3x3 rotation matrix and T′ is the calibrated 3x1 translation matrix.

[0146] The proposed solution eliminates the need for calibration personnel to deploy specially designed calibration boards on the road, and the calibration process requires no manual intervention. Furthermore, once the camera equipment is installed on the roadside, the camera's extrinsic parameters can be calculated, making it more suitable for calibrating the extrinsic parameters of roadside cameras. This significantly improves calibration efficiency and saves labor costs.

[0147] Based on the same inventive concept, according to another aspect of the present invention, embodiments of the present invention also provide a camera extrinsic parameter calibration system 400, such as... Figure 4 As shown, it includes:

[0148] The first extraction module 401 is configured to acquire a first image captured by the camera and extract a first corner point of a preset reference object from the first image.

[0149] The acquisition module 402 is configured to acquire the intrinsic parameter matrix, the initial extrinsic parameter matrix of the camera, and the point cloud of the preset reference object;

[0150] The projection module 403 is configured to project each point in the point cloud onto a second image of the same size as the first image based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, and to extract the second corner point of the preset reference object from the second image.

[0151] The registration module 404 is configured to register the first corner point and the second corner point to obtain a transformation matrix;

[0152] The calculation module 405 is configured to obtain the camera's extrinsic transformation matrix using the intrinsic parameter matrix, the initial extrinsic parameter matrix, and the transformation matrix.

[0153] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 5 As shown, embodiments of the present invention also provide a computer device 501, comprising:

[0154] At least one processor 520; and

[0155] The memory 510 stores a computer program 511 that can run on the processor. When the processor 520 executes the program, it performs the steps of any of the camera extrinsic calibration methods described above.

[0156] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 6 As shown, embodiments of the present invention also provide a computer-readable storage medium 601, which stores a computer program 610. When the computer program 610 is executed by a processor, it performs the steps of any of the camera extrinsic calibration methods described above.

[0157] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods.

[0158] Furthermore, it should be understood that the computer-readable storage medium (e.g., memory) described herein may be volatile memory or non-volatile memory, or may include both volatile memory and non-volatile memory.

[0159] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0160] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0161] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0162] The embodiment numbers disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0163] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0164] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for calibrating camera extrinsic parameters, characterized in that, Includes the following steps: Acquire the first image captured by the camera and extract the first corner point of a preset reference object from the first image; Obtain the intrinsic parameter matrix, initial extrinsic parameter matrix of the camera, and the point cloud of the preset reference object; Based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, each point in the point cloud is projected onto a second image of the same size as the first image, and the second corner point of the preset reference object is extracted from the second image; The transformation matrix is ​​obtained by registering the first corner point and the second corner point; The camera's extrinsic transformation matrix is ​​obtained using the intrinsic parameter matrix, the initial extrinsic parameter matrix, and the transformation matrix. The extrinsic transformation matrix of the camera is obtained using the intrinsic parameter matrix, the initial extrinsic parameter matrix, and the transformation matrix, further comprising obtaining the extrinsic transformation matrix of the camera according to the following formula. : Among them, I M -1 For I M The inverse matrix, F is the transformation matrix, I M It is a 3x3 intrinsic parameter matrix. CAM is a column vector consisting entirely of zeros. EX Let R′ be the initial extrinsic parameter matrix, where R′ is the calibrated 3x3 rotation matrix and T′ is the calibrated 3x1 translation matrix.

2. The method as described in claim 1, characterized in that, Acquiring a first image captured by the camera and extracting a first corner point of a preset reference object from the first image further includes: Place the first image in the HSV color space; The first image is filtered according to a preset first color threshold range to obtain a first region image, and the first region image is converted into a binary image; Extract all contours from the first region image converted to a binary image and take the largest contour as the edge contour; Extract the second region image from the first image based on the edge contour; The second region image is filtered according to a preset second color threshold range to obtain the third region image; Convert the third region image into a binary image; Extract the first corner point from the third region image, which has been converted into a binary image.

3. The method as described in claim 1, characterized in that, Obtaining the point cloud of the preset reference object further includes: Obtain the first point cloud, which includes multiple point cloud blocks; Based on the camera coordinates and the preset boundary angle, multiple point cloud blocks of the first point cloud are filtered to obtain several point cloud blocks, which are then stitched together to obtain the second point cloud. The third point cloud is obtained by filtering from the second point cloud based on its altitude. Calculate the average density of the third point cloud and use the point cloud with a density greater than the average density as the point cloud of the preset reference object.

4. The method as described in claim 3, characterized in that, Based on the camera coordinates and a preset boundary angle, multiple point cloud blocks of the first point cloud are filtered to obtain several point cloud blocks, which are then stitched together to obtain the second point cloud. This further includes: Using the coordinates of the camera as the center, draw a circle with a radius of R1 meters, and enclose a fan-shaped area with ±a° of the direction the camera is facing as the boundary; Several point cloud blocks centered within a fan-shaped region are stitched together to obtain a second point cloud.

5. The method as described in claim 3, characterized in that, The third point cloud is obtained by filtering from the second point cloud based on altitude, and further includes: Determine the altitude range (Z) of the second point in the cloud. min Z max ); Divide the interval into multiple sub-intervals of a preset size and number each sub-interval in sequence; Based on the elevation of each point in the second point cloud and the range of each sub-interval, all points in the second point cloud are grouped, and the two groups with the most and second most points are determined, wherein the number of each group is the same as the number of the sub-interval. The elevation range of the third point cloud is determined based on the two groups with the largest and second largest number of clouds to filter out the third point cloud.

6. The method as described in claim 5, characterized in that, The elevation range of the third point cloud is determined based on the two groups with the most and second most occurrences to filter out the third point cloud. This further includes determining the elevation range of the third point cloud (z) according to the following formula, in response to the adjacent ranges of the sub-intervals corresponding to the two groups. min , z max ): With min =min(i A ,and B )*With size +Z min ,With max =max(i A ,and B )*With size +with size + Z min Among them, i A The number of the group with the largest number, i B z is the number of the second largest group. size This is the preset size.

7. The method as described in claim 5, characterized in that, The elevation range of the third point cloud is determined based on the two groups with the most and second most occurrences to filter out the third point cloud. This further includes determining the elevation range of the third point cloud (z) according to the following formula, in response to the fact that the sub-intervals corresponding to the two groups are not adjacent. min , z max ): With min = and A *With size +Z min ,With max = and A *With size +with size + Z min Among them, i A The number of the group with the largest number, z size This is the preset size.

8. The method as described in claim 1, characterized in that, Based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, each point in the point cloud is projected onto a second image of the same size as the first image, and the second corner point of the preset reference object is extracted from the second image. This further includes projecting according to the following formula: in, For projected coordinates, I M It is a 3x3 intrinsic parameter matrix. CAM is a column vector consisting entirely of zeros. EX This is the initial extrinsic parameter matrix. The coordinates of the midpoint of the point cloud: Based on the projection coordinates, corresponding mapping points are drawn on the second image, where the pixel value of the mapping points is 255 and the pixel value of the rest of the background is 0. The second corner point is extracted from the second image after performing multiple image morphological dilation and erosion operations.

9. The method according to any one of claims 1-8, characterized in that, Also includes: The lane lines are used as the preset reference.

10. A camera extrinsic parameter calibration system, characterized in that, include: The first extraction module is configured to acquire a first image captured by the camera and extract a first corner point of a preset reference object from the first image; The acquisition module is configured to acquire the intrinsic parameter matrix, the initial extrinsic parameter matrix of the camera, and the point cloud of the preset reference object; The projection module is configured to project each point in the point cloud onto a second image of the same size as the first image based on the intrinsic parameter matrix and the initial extrinsic parameter matrix, and to extract the second corner point of the preset reference object from the second image. The registration module is configured to register the first corner point and the second corner point to obtain a transformation matrix; The calculation module is configured to obtain the camera's extrinsic transformation matrix using the intrinsic parameter matrix, the initial extrinsic parameter matrix, and the transformation matrix; The calculation module is also configured to obtain the extrinsic transformation matrix of the camera according to the following formula. : Among them, I M -1 For I M The inverse matrix, F is the transformation matrix, I M It is a 3x3 intrinsic parameter matrix. CAM is a column vector consisting entirely of zeros. EX Let R′ be the initial extrinsic parameter matrix, where R′ is the calibrated 3x3 rotation matrix and T′ is the calibrated 3x1 translation matrix.

11. A computer device, comprising: At least one processor; as well as A memory storing a computer program executable on the processor, characterized in that the processor executes the program by performing the steps of the method as described in any one of claims 1-9.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it performs the steps of the method as described in any one of claims 1-9.

Images