Multi-camera joint calibration method, device, equipment, medium and product

Abstract

The invention discloses a multi-camera joint calibration method, device and equipment, a medium and a product. Measuring three-dimensional angular point coordinates respectively corresponding to a plurality of calibration plates around the vehicle, calculating according to the plurality of three-dimensional angular point coordinates to obtain a conversion matrix between each slave calibration plate coordinate system and the master calibration plate coordinate system, performing image acquisition on the plurality of calibration plates through a plurality of cameras on the vehicle to obtain a plurality of calibration plate images, and displaying the plurality of calibration plate images in the master calibration plate coordinate system. A calibration plate identifier and a two-dimensional angular point coordinate of the calibration plate are extracted from each calibration plate image, and for each calibration plate identifier, a conversion matrix between a camera coordinate system and a main calibration plate coordinate system is determined according to the two-dimensional angular point coordinate and the three-dimensional angular point coordinate corresponding to the calibration plate identifier; and according to the conversion matrix between the camera coordinate system and the main calibration plate coordinate system, determining a conversion matrix between the camera coordinate system and the main camera coordinate system, and associating the calibration results of the single cameras together in the multi-camera calibration process.

Description

Technical Field

[0001] The present application relates to calibration technology, and in particular to a multi-camera joint calibration method, device, equipment, medium and product. Background Art

[0002] For smart trucks to achieve smart driving on highways, sensors are required to fully and accurately perceive and identify the environment or targets so that the intelligent driving system can make the best decision. Sensor calibration is the basis for the stable operation of the intelligent driving system. It aims to eliminate the errors generated during the sensor installation process and ensure that the sensor can accurately reflect the environmental information it perceives. Taking the camera as an example, multi-camera calibration can determine the external parameters between different cameras, which is crucial to achieve 3D reconstruction and visual measurement between cameras.

[0003] The existing multi-camera calibration technology mainly focuses on the calibration of a single camera, that is, determining the intrinsic and extrinsic parameters of a single camera in sequence. However, in the intelligent driving system of smart trucks, it is usually necessary to process the calibration of multiple cameras, and the existing technology only focuses on the calibration accuracy of a single camera, without considering the calibration accuracy and consistency of the multi-camera system as a whole, that is, it is impossible to associate the calibration results of each camera together to form a unified world coordinate system, which makes it impossible to accurately fuse the information from different cameras in the subsequent 3D reconstruction and target detection process. Summary of the invention

[0004] The present application provides a multi-camera joint calibration method, device, equipment, medium and product to solve the problem that the calibration results of a single camera cannot be associated during the multi-camera calibration process, which leads to the inability to accurately fuse information from different cameras in the subsequent three-dimensional reconstruction and target detection process.

[0005] In a first aspect, an embodiment of the present application provides a multi-camera joint calibration method, comprising:

[0006] Measuring the three-dimensional corner point coordinates corresponding to a plurality of calibration plates located around the vehicle, respectively, so as to calculate a conversion matrix between each slave calibration plate coordinate system and a master calibration plate coordinate system according to the plurality of three-dimensional corner point coordinates; the plurality of calibration plates include a master calibration plate and a slave calibration plate, and the three-dimensional corner point coordinates are the real three-dimensional spatial positions of the corner points of the calibration plates in the coordinate system of the calibration plates themselves;

[0007] Capturing images of the multiple calibration plates by using multiple cameras on the vehicle to obtain multiple calibration plate images, wherein each camera corresponds to one or more calibration plates, and the multiple cameras include a master camera and slave cameras;

[0008] Extracting a calibration plate identifier and two-dimensional corner point coordinates of the calibration plate from each calibration plate image, wherein the two-dimensional corner point coordinates are positions of the corner points in the image;

[0009] For each of the calibration plate identifiers, determining a transformation matrix between a camera coordinate system and a master calibration plate coordinate system according to the two-dimensional corner point coordinates and the three-dimensional corner point coordinates corresponding to the calibration plate identifier, wherein the camera coordinate system includes a master camera coordinate system and a slave camera coordinate system;

[0010] For each of the slave cameras, a transformation matrix between the slave camera coordinate system and the master calibration board coordinate system is determined according to a transformation matrix between the camera coordinate system and the master calibration board coordinate system, wherein the transformation matrix includes a rotation matrix and a translation matrix.

[0011] In a possible implementation, the step of calculating, based on the coordinates of the plurality of three-dimensional corner points, a transformation matrix between each slave calibration plate coordinate system and a master calibration plate coordinate system includes:

[0012] For each corner point of each slave calibration plate, establish a relationship equation between the three-dimensional corner point coordinates of the corner point and the three-dimensional corner point coordinates of the master calibration plate, wherein the unknown parameters of the relationship equation are the conversion matrix and the scale factor between the slave calibration plate coordinate system and the master calibration plate coordinate system, and the scale factor is the ratio of the lengths of the same straight line in the slave calibration plate coordinate system and the master calibration plate coordinate system;

[0013] For each slave calibration board, the four groups of relationship equations corresponding to the slave calibration board are solved to obtain a conversion matrix between the coordinate system of the slave calibration board and the master calibration board.

[0014] In a possible implementation, one of the relationship equations is as follows:

[0015]

[0016] Among them, X, Y, Z are the three-dimensional corner coordinates of a corner point of the calibration plate in the world coordinate system, X0, Y0, Z0 are the three-dimensional corner coordinates of a corner point of the main calibration plate in the world coordinate system, ΔX, ΔY, ΔZ are the translation parameters in the transformation matrix between the calibration plate coordinate system and the main calibration plate coordinate system, θ x ,θ z and θ y is the rotation parameter in the transformation matrix between the coordinate system of the slave calibration plate and the coordinate system of the master calibration plate, and m is the scale factor.

[0017] In a possible implementation, determining the conversion matrix between the camera coordinate system and the main calibration plate coordinate system according to the two-dimensional corner point coordinates, the three-dimensional corner point coordinates, and the conversion matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system includes:

[0018] Performing homogeneous transformation on the two-dimensional corner point coordinates and the three-dimensional corner point coordinates to obtain homogeneous two-dimensional corner point coordinates and homogeneous three-dimensional corner point coordinates respectively;

[0019] The conversion matrix between the camera coordinate system and the main calibration plate coordinate system is determined according to the homogeneous two-dimensional corner point coordinates, the homogeneous three-dimensional corner point coordinates, and the conversion matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system.

[0020] In a possible implementation, determining a transformation matrix between a camera coordinate system and a main calibration plate coordinate system according to the homogeneous two-dimensional corner point coordinates and the homogeneous three-dimensional corner point coordinates corresponding to the calibration plate identifier includes:

[0021] A second relationship equation between the homogeneous two-dimensional corner point coordinates and the homogeneous three-dimensional corner point coordinates is constructed, and the second relationship equation is as follows:

[0022]

[0023] Among them, r1, r2, and r3 are the rotation parameters in the transformation matrix between the camera coordinate system and the main calibration plate coordinate system. is the translation parameter in the transformation matrix between the kth camera coordinate system and the main calibration plate coordinate system, the [u k ,v k ,1] T are the coordinates of the homogeneous two-dimensional corner point, [X k ,Y k ,0,1] T is the coordinates of the homogeneous three-dimensional corner point, K k is the intrinsic parameter matrix of the kth camera, z ck is the scale factor of the kth camera;

[0024] The second relationship equation is solved to obtain a transformation matrix between the camera coordinate system and the main calibration plate coordinate system.

[0025] In a possible implementation, determining the transformation matrix from the camera coordinate system to the main camera coordinate system according to the transformation matrix between the camera coordinate system and the main calibration plate coordinate system includes:

[0026] The transformation matrix between the slave camera coordinate system and the master camera coordinate system is calculated by the following formula:

[0027]

[0028] Among them, R 1k are the kth rotation vectors from the camera coordinate system to the main camera coordinate system, and the t 1k is the translation vector between the kth slave camera coordinate system and the master camera coordinate system, is the rotation vector between the main camera coordinate system and the main calibration plate coordinate system, is the translation vector between the main camera coordinate system and the main calibration plate coordinate system; is the rotation vector between the kth slave camera coordinate system and the master calibration plate coordinate system, is the translation vector between the kth slave camera coordinate system and the master calibration plate coordinate system, where k is a positive integer greater than or equal to 1.

[0029] In a second aspect, the present application provides a multi-camera joint calibration device, comprising:

[0030] A measurement module, used to measure the coordinates of three-dimensional corner points corresponding to multiple calibration plates located around the vehicle;

[0031] A processing module, configured to calculate a conversion matrix between each slave calibration plate coordinate system and a master calibration plate coordinate system according to the plurality of three-dimensional corner point coordinates; the plurality of calibration plates include a master calibration plate and a slave calibration plate, and the three-dimensional corner point coordinates are the real three-dimensional spatial positions of the corner points of the calibration plates in the coordinate system of the calibration plate itself;

[0032] An acquisition module, configured to acquire images of the plurality of calibration plates through a plurality of cameras on the vehicle to obtain a plurality of calibration plate images, wherein each of the cameras corresponds to one or more calibration plates, and the plurality of cameras include a master camera and slave cameras;

[0033] The processing module is further used to extract the calibration plate identification and the two-dimensional corner point coordinates of the calibration plate from each calibration plate image, wherein the two-dimensional corner point coordinates are the positions of the corner points in the image;

[0034] The processing module is further used to determine, for each calibration plate identifier, a conversion matrix between a camera coordinate system and a master calibration plate coordinate system according to the two-dimensional corner point coordinates and the three-dimensional corner point coordinates corresponding to the calibration plate identifier, wherein the camera coordinate system includes a master camera coordinate system and a slave camera coordinate system;

[0035] The processing module is further used to determine, for each of the slave cameras, a transformation matrix between the slave camera coordinate system and the master calibration board coordinate system, wherein the transformation matrix includes a rotation matrix and a translation matrix.

[0036] In a multi-camera joint calibration device provided in this application:

[0037] A processing module is used to establish, for each corner point of each slave calibration plate, a relationship equation between the three-dimensional corner point coordinates of the corner point and the three-dimensional corner point coordinates of the master calibration plate, wherein the unknown parameters of the relationship equation are a conversion matrix and a scale factor between the slave calibration plate coordinate system and the master calibration plate coordinate system, and the scale factor is a ratio of the lengths of the same straight line in the slave calibration plate coordinate system and the master calibration plate coordinate system;

[0038] The processing module is used to solve, for each slave calibration board, the four groups of relationship equations corresponding to the slave calibration board to obtain a transformation matrix between the coordinate system of the slave calibration board and the master calibration board.

[0039] In a multi-camera joint calibration device provided in this application:

[0040]

[0041] Among them, X, Y, Z are the three-dimensional corner coordinates of a corner point of the calibration plate in the world coordinate system, X0, Y0, Z0 are the three-dimensional corner coordinates of a corner point of the main calibration plate in the world coordinate system, ΔX, ΔY, ΔZ are the translation parameters in the transformation matrix between the calibration plate coordinate system and the main calibration plate coordinate system, θ x ,θ z and θ y is the rotation parameter in the transformation matrix between the coordinate system of the slave calibration plate and the coordinate system of the master calibration plate, and m is the scale factor.

[0042] In a multi-camera joint calibration device provided in this application:

[0043] A processing module, used for performing homogeneous transformation on the two-dimensional corner point coordinates and the three-dimensional corner point coordinates to obtain homogeneous two-dimensional corner point coordinates and homogeneous three-dimensional corner point coordinates respectively;

[0044] A processing module is used to determine the transformation matrix between the camera coordinate system and the main calibration plate coordinate system according to the homogeneous two-dimensional corner point coordinates, the homogeneous three-dimensional corner point coordinates, and the transformation matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system.

[0045] In a multi-camera joint calibration device provided in this application:

[0046] A processing module is used to construct a second relationship equation between the homogeneous two-dimensional corner point coordinates and the homogeneous three-dimensional corner point coordinates, wherein the second relationship equation is as follows:

[0047]

[0048] Among them, r1, r2, and r3 are the rotation parameters in the transformation matrix between the camera coordinate system and the main calibration plate coordinate system. is the translation parameter in the transformation matrix between the kth camera coordinate system and the main calibration plate coordinate system, the [u k ,v k ,1] T are the coordinates of the homogeneous two-dimensional corner point, [X k ,Y k ,0,1] T is the coordinates of the homogeneous three-dimensional corner point, K k is the intrinsic parameter matrix of the kth camera, z ck is the scale factor of the kth camera;

[0049] A processing module is used to solve the second relationship equation to obtain a transformation matrix between the camera coordinate system and the main calibration plate coordinate system.

[0050] In a multi-camera joint calibration device provided in this application:

[0051] The processing module is used to calculate the conversion matrix between the slave camera coordinate system and the master camera coordinate system by the following formula:

[0052]

[0053] Among them, R 1k are the kth rotation vectors from the camera coordinate system to the main camera coordinate system, and the t 1k is the translation vector between the kth slave camera coordinate system and the master camera coordinate system, is the rotation vector between the main camera coordinate system and the main calibration plate coordinate system, is the translation vector between the main camera coordinate system and the main calibration plate coordinate system; is the rotation vector between the kth slave camera coordinate system and the master calibration plate coordinate system, is the translation vector between the kth slave camera coordinate system and the master calibration plate coordinate system, where k is a positive integer greater than or equal to 1.

[0054] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0055] Memory stores computer-executable instructions;

[0056] The processor executes the computer-executable instructions stored in the memory to implement a multi-camera joint calibration method provided in the present application.

[0057] In a fourth aspect, the present application provides a computer-readable storage medium, including: the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement a multi-camera joint calibration method provided by the present application.

[0058] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements a multi-camera joint calibration method provided in the present application.

[0059] The present application provides a multi-camera joint calibration method, device, equipment, medium and product, firstly measuring the three-dimensional corner point coordinates corresponding to multiple calibration plates located around the vehicle, so as to calculate the conversion matrix between each slave calibration plate coordinate system and the main calibration plate coordinate system according to the multiple three-dimensional corner point coordinates, then performing image acquisition on the multiple calibration plates through multiple cameras on the vehicle to obtain multiple calibration plate images, extracting the calibration plate identification and the two-dimensional corner point coordinates of the calibration plate from each calibration plate image, the two-dimensional corner point coordinates being the position of the corner point in the image, for each calibration plate identification, according to the two-dimensional corner point coordinates and three-dimensional corner point coordinates corresponding to the calibration plate identification, determining the conversion matrix between the camera coordinate system and the main calibration plate coordinate system, the camera coordinate system includes the main camera coordinate system and the slave camera coordinate system, for each slave camera, according to the conversion matrix between the camera coordinate system and the main calibration plate coordinate system, determining the conversion matrix between the slave camera coordinate system and the main camera coordinate system. Compared with the prior art that can only determine the calibration results of a single camera in sequence, it leads to accurate fusion of information from different cameras in the subsequent three-dimensional reconstruction and target detection process. This application first calculates the transformation matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system, and then determines the transformation matrix between the camera coordinate system and the main calibration plate coordinate system according to the transformation matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system, the two-dimensional corner point coordinates, and the three-dimensional corner point coordinates. Finally, according to the transformation matrix between the camera coordinate system and the main calibration plate coordinate system, the transformation matrix between the slave camera coordinate system and the main camera coordinate system is determined. Finally, the calibration results in the main camera coordinate system can be unified, so that in the subsequent three-dimensional reconstruction and target detection process, the information from different cameras can be accurately integrated. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0061] Figure 1 This is a schematic diagram of the multi-camera joint calibration site;

[0062] Figure 2 is a schematic diagram of the calibration plate;

[0063] Figure 3A schematic diagram of a process flow of a first embodiment of a multi-camera joint calibration method provided in an embodiment of the present application;

[0064] Figure 4 A schematic diagram of a flow chart of a second embodiment of a multi-camera joint calibration method provided in an embodiment of the present application;

[0065] Figure 5 A schematic diagram of a flow chart of a third embodiment of a multi-camera joint calibration method provided in an embodiment of the present application;

[0066] Figure 6 A schematic diagram of a flow chart of a fourth embodiment of a multi-camera joint calibration method provided in an embodiment of the present application;

[0067] Figure 7 A multi-camera joint calibration device provided in an embodiment of the present application;

[0068] Figure 8 An electronic device is provided in an embodiment of the present application.

[0069] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0070] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0071] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0072] In the existing technology, the multi-camera calibration technology mainly focuses on the calibration of a single camera during implementation, without considering calibrating multiple camera systems as a whole, which makes it impossible to associate the calibration results of each camera together to form a unified world coordinate system, and ultimately leads to the accurate fusion of information from different cameras in the subsequent 3D reconstruction and target detection process.

[0073] Based on this, the inventive concept of this application is to provide a multi-camera joint calibration method, which associates the calibration results of multiple cameras together, so that the multiple cameras are finally in the same unified world coordinate system, and then the information from different cameras can be accurately fused in the subsequent three-dimensional reconstruction and target detection process.

[0074] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0075] Figure 1 This is a schematic diagram of the multi-camera joint calibration site. Figure 1 As shown, the square block represents the position of the calibration plate, and the circle represents the camera. The calibration site needs to be determined based on factors such as the size of the vehicle and the camera's field of view. The calibration plate is set toward the vehicle and has a certain height so that the camera can capture the image of the calibration plate and is located at the center of the camera's field of view. The camera includes: a forward long-range camera, a forward medium-range camera, and a forward close-range camera, which respectively capture the long-range image in front of the vehicle, the medium-range image in front, and the forward close-range camera. The left close-range camera and the right close-range camera capture the close-up image of the left side and the close-up image of the right side of the vehicle, respectively. The left medium-range camera and the right medium-range camera capture the left rear image and the right rear image of the vehicle, respectively.

[0076] according to Figure 1 Schematic diagram of multi-camera joint calibration site. Optional, the calibration site size is 22m long and 10m wide, and the calibration plate size is 1400mm long and 1000mm wide. Figure 2 As shown in the figure, it is a schematic diagram of the calibration plate. Calibration plates 0, 11, 23, 35, 47, 59, and 71 in the calibration plate type Apriltag36h11 are selected. The distance of the calibration plate from the ground is determined by the camera installation position. Specifically, the camera can capture the calibration plate and the calibration plate is located at the center of the image.

[0077] This application collects calibration plate images for calibration by installing a forward long-range camera, a forward medium-range camera, and a forward close-range camera, a left close-range camera and a right close-range camera, a left medium-range camera, and a right medium-range camera, respectively covering the long, medium, and short distance ranges as well as the medium and short distance ranges at the side and rear, which can meet the requirements of the intelligent driving system of smart trucks for all-round field of view coverage and improve driving safety and reliability. In addition, Apriltag is used as the calibration plate image to achieve calibration when multiple calibration plate information appears in the field of view of a single camera, and it supports simultaneous calibration of multiple cameras, which greatly saves calibration time and labor costs and improves calibration efficiency.

[0078] Figure 3 A schematic diagram of a flow chart of a first embodiment of a multi-camera joint calibration method provided in an embodiment of the present application, such as Figure 3 As shown, the method comprises the following steps:

[0079] S301: Measure the three-dimensional corner point coordinates corresponding to multiple calibration plates located around the vehicle, and calculate the transformation matrix between each slave calibration plate coordinate system and the main calibration plate coordinate system based on the multiple three-dimensional corner point coordinates; the multiple calibration plates include a main calibration plate and a slave calibration plate, and the three-dimensional corner point coordinates are the real three-dimensional spatial positions of the corner points of the calibration plates in the coordinate system of their own calibration plates.

[0080] In the embodiment of the present application, a total station type electronic tachometer is used to measure the three-dimensional corner point coordinates corresponding to the seven calibration plates located around the vehicle. For each calibration plate, each calibration plate coordinate system is established with its center as the origin. Next, the total station type electronic tachometer is aligned with the four corner points of the calibration plate in turn. For each corner point, the total station will measure its three-dimensional corner point coordinates in the corresponding calibration plate coordinate system. That is, each calibration plate establishes a coordinate system with its center as the origin coordinate, and the three-dimensional corner point coordinates are the real three-dimensional spatial positions of the corner points of the calibration plate in its own calibration plate coordinate system. A calibration plate is randomly selected as the main calibration plate, and the remaining calibration plates are slave calibration plates, and the conversion matrix between each slave calibration plate coordinate system and the main calibration plate coordinate system is calculated.

[0081] S302: Capturing images of multiple calibration plates by using multiple cameras on the vehicle to obtain multiple calibration plate images, where each camera corresponds to one or more calibration plates, and the multiple cameras include a master camera and slave cameras.

[0082] In the embodiment of the present application, the calibration plate is imaged by multiple cameras installed on the vehicle, wherein each camera can capture one calibration plate or multiple calibration plates. For example, when one camera corresponds to one calibration plate, each camera corresponds to one calibration plate image, wherein the camera corresponding to the main calibration plate is the main camera, and the other cameras corresponding to the slave calibration plates are slave cameras.

[0083] S303: extracting the calibration plate identifier and the two-dimensional corner point coordinates of the calibration plate from each calibration plate image, where the two-dimensional corner point coordinates are the positions of the corner points in the image.

[0084] In an embodiment of the present application, when multiple calibration plate images are obtained, the multiple calibration plate images are grayed, and the graying process simplifies the image data, reduces the processing difficulty, and can retain the main features of the image at the same time. Then, adaptive threshold processing is used for the multiple calibration plate images after the graying process to convert them from grayscale images to black and white binary images, that is, to separate the calibration plate from the background in the image captured by the camera, so that the outline of the calibration plate is clearer, which is convenient for subsequent edge detection and contour extraction. Then, on the calibration plate image after the binarization, the edge part of the calibration plate image after the binarization is identified by edge detection technology. Then, through the joint search algorithm, the continuous bright pixels and dark pixels are grouped separately, and then a unique ID is assigned to each group, so that different calibration plate areas are separated from the image, and a unique identifier (ID) is assigned to each calibration plate for subsequent distinction and processing. Subsequently, for each ID area's contour points, they are sorted according to their angles with the area's center of gravity, and straight line fitting is performed. By fitting the edge straight lines of the calibration plate, the shape and position of the calibration plate can be determined more accurately, and then the two-dimensional corner coordinates of the calibration plate can be extracted. Then, the intersection of the fitted straight lines is determined as the vertices of the calibration plate, and the quadrilateral code is extracted based on these vertices. The quadrilateral code is the unique identification code of the calibration plate. The identity of the calibration plate can be confirmed by comparing the codes and used for subsequent calibration and calculation. Finally, the extracted quadrilateral code is compared with the predicted code words to eliminate erroneous candidate quadrilaterals, and a verified and valid tag detection result is output to ensure that the detected calibration plate is correct and valid, thereby obtaining an accurate calibration plate ID number, providing a reliable basis for subsequent camera calibration.

[0085] S304: For each calibration plate identifier, determine a transformation matrix between a camera coordinate system and a main calibration plate coordinate system according to the two-dimensional corner point coordinates and three-dimensional corner point coordinates corresponding to the calibration plate identifier, where the camera coordinate system includes a main camera coordinate system and a slave camera coordinate system.

[0086] In an embodiment of the present application, the transformation matrix between the camera coordinate system and the main calibration plate coordinate system is further determined through each calibration plate identifier obtained as described above, as well as the two-dimensional corner point coordinates and three-dimensional corner point coordinates corresponding to each calibration plate identifier, wherein the camera coordinate system includes the main camera coordinate system and the slave camera coordinate system.

[0087] S305: For each slave camera, determine a transformation matrix between the slave camera coordinate system and the master calibration board coordinate system according to a transformation matrix between the camera coordinate system and the master calibration board coordinate system, where the transformation matrix includes a rotation matrix and a translation matrix.

[0088] In an embodiment of the present application, for each slave camera, the matrix between the slave camera coordinate system and the master camera coordinate system is finally determined based on the obtained transformation matrix between the camera coordinate system and the master calibration plate coordinate system, where the transformation matrix includes: a rotation matrix and a translation matrix.

[0089] In the embodiment of the present application, the three-dimensional corner point coordinates corresponding to the multiple calibration plates located around the vehicle are first measured, so as to calculate the conversion matrix between each slave calibration plate coordinate system and the main calibration plate coordinate system according to the multiple three-dimensional corner point coordinates, and then the multiple calibration plates are imaged by multiple cameras on the vehicle to obtain multiple calibration plate images, and the calibration plate mark and the two-dimensional corner point coordinates of the calibration plate are extracted from each calibration plate image, and the two-dimensional corner point coordinates are the positions of the corner points in the image. For each calibration plate mark, the conversion matrix between the camera coordinate system and the main calibration plate coordinate system is determined according to the two-dimensional corner point coordinates and the three-dimensional corner point coordinates corresponding to the calibration plate mark. The camera coordinate system includes the main camera coordinate system and the slave camera coordinate system. For each slave camera, the conversion matrix between the slave camera coordinate system and the main calibration plate coordinate system is determined according to the conversion matrix between the camera coordinate system and the main calibration plate coordinate system. Compared with the prior art that can only determine the calibration results of a single camera in sequence, the information from different cameras is accurately fused in the subsequent three-dimensional reconstruction and target detection process. This application first calculates the transformation matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system, and then determines the transformation matrix between the camera coordinate system and the main calibration plate coordinate system according to the transformation matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system, the two-dimensional corner point coordinates, and the three-dimensional corner point coordinates. Finally, according to the transformation matrix between the camera coordinate system and the main calibration plate coordinate system, the transformation matrix between the slave camera coordinate system and the main camera coordinate system is determined. Finally, the calibration results in the main camera coordinate system can be unified, so that in the subsequent three-dimensional reconstruction and target detection process, the information from different cameras can be accurately integrated.

[0090] Figure 4 A schematic diagram of a flow chart of a second embodiment of a multi-camera joint calibration method provided in an embodiment of the present application, such as Figure 2 As shown in the above Figure 3 Based on the embodiment shown, a specific implementation method of calculating the conversion matrix between each slave calibration plate coordinate system and the master calibration plate coordinate system according to the coordinates of multiple three-dimensional corner points in step S301 is:

[0091] S401: For each corner point of each slave calibration plate, establish a relationship equation between the three-dimensional corner point coordinates of the corner point and the three-dimensional corner point coordinates of the master calibration plate. The unknown parameters of the relationship equation are the transformation matrix and the scale factor between the slave calibration plate coordinate system and the master calibration plate coordinate system. The scale factor is the ratio of the lengths of the same straight line in the slave calibration plate coordinate system and the master calibration plate coordinate system.

[0092] In the embodiment of the present application, after the three-dimensional corner point coordinates of each calibration plate are measured by the total station type electronic tachometer, a calibration plate is randomly selected as the main calibration plate, and the remaining calibration plates are used as slave calibration plates. Each calibration plate corresponds to four corner points, and each corner point corresponds to a three-dimensional corner point coordinate system. For each corner point of each slave calibration plate, a relationship equation between the three-dimensional corner point coordinates of the corner point and the three-dimensional corner point coordinates of the main calibration plate is established.

[0093] Among them, the unknown parameters of the relationship equation are the transformation matrix and scale factor between the slave calibration plate coordinate system and the master calibration plate coordinate system. The scale factor is the ratio of the lengths of the same straight line in the slave calibration plate coordinate system and the master calibration plate coordinate system. When the sizes of the master calibration plate and the slave calibration plate are the same, the scale factor is 1.

[0094] In one specific embodiment, the relationship equation is:

[0095]

[0096] Among them, X, Y, Z are the three-dimensional corner coordinates of a corner point of the calibration plate in the world coordinate system, X0, Y0, Z0 are the three-dimensional corner coordinates of a corner point of the main calibration plate in the world coordinate system, ΔX, ΔY, ΔZ are the translation parameters in the transformation matrix between the calibration plate coordinate system and the main calibration plate coordinate system, θ x ,θ z and θ y is the rotation parameter in the transformation matrix between the coordinate system of the slave calibration plate and the coordinate system of the master calibration plate, and m is the scale factor.

[0097] S402: For each slave calibration board, four sets of relationship equations corresponding to the slave calibration board are solved to obtain a transformation matrix between the coordinate systems of the slave calibration board and the master calibration board.

[0098] In the embodiment of the present application, the four corner points of each slave calibration board correspond to the four corner points of the master calibration board, so there are four sets of relationship equations for each slave calibration board. Therefore, for each slave calibration board, the four sets of relationship equations corresponding to the slave calibration board are solved to obtain the conversion matrix between each slave calibration board and the master calibration board.

[0099] For example, the method for solving the four sets of relationship equations corresponding to the slave calibration plate is to linearize the four sets of relationship equations corresponding to each slave calibration plate, and then use the least squares method to iteratively solve them, thereby obtaining the estimated values ​​of the translation parameters and rotation parameters in the transformation matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system, and then constructing each transformation matrix from the slave calibration plate to the main calibration plate:

[0100] M 1i =R1i M i +t 1i

[0101] Among them, M 1i is the coordinate of the i-th corner point of the slave calibration plate in the coordinate system of the master calibration plate; M i is the coordinate of the corner point of the ith slave calibration plate in the coordinate system of the ith slave calibration plate; R 1i is the rotation matrix in the transformation matrix corresponding to the i-th calibration plate; t 1i is the translation matrix in the transformation matrix corresponding to the i-th calibration plate.

[0102] In an embodiment of the present application, for each corner point of each slave calibration plate, a relationship equation is established between the three-dimensional corner point coordinates of the corner point and the three-dimensional corner point coordinates of the master calibration plate. The unknown parameters of the relationship equation are the transformation matrix and the scale factor between the slave calibration plate coordinate system and the master calibration plate coordinate system. The scale factor is the ratio of the lengths of the same straight line in the slave calibration plate coordinate system and the master calibration plate coordinate system. Then, for each slave calibration plate, the four sets of relationship equations corresponding to the slave calibration plate are solved to obtain the transformation matrix between the slave calibration plate and the master calibration plate coordinate system, which paves the way for the subsequent solution of the transformation matrix between the camera coordinate system and the master calibration plate coordinate system.

[0103] Figure 5 A schematic diagram of a flow chart of a third embodiment of a multi-camera joint calibration method provided in an embodiment of the present application is shown in FIG. Figure 5 As shown in the above Figure 3 or Figure 4 Based on the illustrated embodiment, a specific implementation of step S304 is as follows:

[0104] S501: Perform homogeneous transformation on the two-dimensional corner point coordinates and the three-dimensional corner point coordinates to obtain homogeneous two-dimensional corner point coordinates and homogeneous three-dimensional corner point coordinates respectively.

[0105] In the embodiment of the present application, the three-dimensional corner point coordinates corresponding to the measured multiple calibration plates are homogeneously transformed to obtain homogeneous three-dimensional corner point coordinates. For example, the seven calibration plates in the embodiment of the present application correspond to four homogeneous three-dimensional corner point coordinates. The two-dimensional corner point coordinates corresponding to the multiple calibration plates captured by the camera are homogeneously transformed to obtain homogeneous two-dimensional corner point coordinates. For example, the seven calibration plates in the embodiment of the present application are respectively imaged by seven cameras, and the seven calibration plate images obtained correspond to four homogeneous two-dimensional corner point coordinates.

[0106] S502: Determine a transformation matrix between the camera coordinate system and the main calibration plate coordinate system according to the homogeneous two-dimensional corner point coordinates and the homogeneous three-dimensional corner point coordinates corresponding to the calibration plate identifier and the transformation matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system.

[0107] In the embodiment of the present application, in order to determine the conversion matrix between the camera coordinate system and the calibration plate coordinate system, a specific implementation of step S502 is:

[0108] Construct a second relationship equation between the homogeneous two-dimensional corner point coordinates and the homogeneous three-dimensional corner point coordinates. The second relationship equation is as follows:

[0109]

[0110] Among them, r1, r2 and r3 represent the rotation parameters in the transformation matrix between the camera coordinate system and the main calibration plate coordinate system; represents the translation parameter in the transformation matrix between the kth camera coordinate system and the main calibration plate coordinate system; [u k ,v k ,1] T Represents the coordinates of a homogeneous two-dimensional corner point; [X k ,Y k ,0,1] T represents the coordinates of a homogeneous three-dimensional corner point; K k represents the intrinsic parameter matrix of the kth camera; Z ck represents the scale factor of the kth camera.

[0111] For example, without the distortion parameter, K k It is expressed as:

[0112]

[0113] Among them, f x represents the horizontal focal length, f y represents the longitudinal focal length, u k , v k Represents the two-dimensional corner point coordinates under camera k.

[0114] The second relationship equation is solved to obtain the transformation matrix between the camera coordinate system and the main calibration plate coordinate system.

[0115] In an embodiment of the present application, after constructing the second relationship equation based on the homogeneous two-dimensional corner point coordinates and the homogeneous three-dimensional corner point coordinates, an optimization algorithm, such as the least squares method, is used to solve the transformation matrix between the camera coordinate system and the main calibration plate coordinate system, that is, to solve the rotation parameters and translation parameters in the transformation matrix.

[0116] In an embodiment of the present application, a homogeneous transformation is performed on the two-dimensional corner point coordinates and the three-dimensional corner point coordinates to obtain homogeneous two-dimensional corner point coordinates and homogeneous three-dimensional corner point coordinates, respectively. The transformation matrix between the camera coordinate system and the main calibration plate coordinate system is determined according to the homogeneous two-dimensional corner point coordinates and the homogeneous three-dimensional corner point coordinates corresponding to the calibration plate identification, and the transformation matrix between the calibration plate coordinate system and the main calibration plate coordinate system. The homogeneous coordinate transformation enables the two-dimensional image coordinates and the three-dimensional space coordinates to be processed in the same mathematical framework, which is convenient for subsequent coordinate transformation and calculation, and finally the transformation matrix between the camera coordinate system and the main calibration plate coordinate system is obtained, which paves the way for finally obtaining the transformation matrix from the camera coordinate system to the main camera coordinate system.

[0117] Figure 6 A flowchart of a fourth embodiment of a multi-camera joint calibration method provided in an embodiment of the present application is shown in FIG. Figure 6 As shown in the above Figure 3 Based on the illustrated embodiment, a specific implementation of step S305 is as follows:

[0118] S401: Calculate the transformation matrix from the camera coordinate system to the main camera coordinate system using the following formula:

[0119]

[0120] In the embodiment of the present application, the transformation matrix between the camera coordinate system and the main camera coordinate system is obtained by solving the transformation matrix between the kth camera coordinate system and the main calibration plate coordinate system, and the transformation matrix between the main calibration plate coordinate system and the main camera coordinate system. That is, the rotation vector between the camera coordinate system and the main camera coordinate system is obtained by solving the transformation matrix between the kth camera coordinate system and the main calibration plate coordinate system, and the rotation vector between the main calibration plate coordinate system and the main camera coordinate system. The translation vector between the camera coordinate system and the main camera coordinate system is obtained by solving the translation vector between the kth camera coordinate system and the main calibration plate coordinate system, and the translation vector between the main calibration plate coordinate system and the main camera coordinate system.

[0121] Among them, R 1k is the kth rotation vector from the camera coordinate system to the main camera coordinate system, t 1k is the translation vector between the kth slave camera coordinate system and the master camera coordinate system, is the rotation vector between the main camera coordinate system and the main calibration plate coordinate system, is the translation vector between the main camera coordinate system and the main calibration plate coordinate system; is the rotation vector between the kth slave camera coordinate system and the master calibration plate coordinate system, is the translation vector between the kth slave camera coordinate system and the master calibration plate coordinate system, where k is a positive integer greater than or equal to 1.

[0122] Figure 7 A multi-camera joint calibration device provided in an embodiment of the present application. A multi-camera joint calibration device provided in the present application includes: a measurement module 71, a processing module 72 and a collection module 73; wherein,

[0123] The measuring module 71 is used to measure the three-dimensional corner point coordinates corresponding to multiple calibration plates located around the vehicle; the processing module 72 is used to calculate the conversion matrix between each slave calibration plate coordinate system and the main calibration plate coordinate system according to the multiple three-dimensional corner point coordinates; the multiple calibration plates include the main calibration plate and the slave calibration plate, and the three-dimensional corner point coordinates are the real three-dimensional spatial positions of the corner points of the calibration plates in the coordinate system of the calibration plate itself; the acquisition module 73 is used to acquire images of the multiple calibration plates through multiple cameras on the vehicle to obtain multiple calibration plate images, each camera corresponds to one or more calibration plates, and the multiple cameras include the main camera and the slave camera; the processing module 72 is also used The processing module 72 is used to extract the calibration plate mark and the two-dimensional corner point coordinates of the calibration plate from each calibration plate image, where the two-dimensional corner point coordinates are the positions of the corner points in the image; the processing module 72 is also used to determine, for each calibration plate mark, the transformation matrix between the camera coordinate system and the main calibration plate coordinate system according to the two-dimensional corner point coordinates and the three-dimensional corner point coordinates corresponding to the calibration plate mark, where the camera coordinate system includes the main camera coordinate system and the slave camera coordinate system; the processing module 72 is also used to determine, for each slave camera, the transformation matrix between the slave camera coordinate system and the main calibration plate coordinate system, where the transformation matrix includes: a rotation matrix and a translation matrix.

[0124] Figure 8 An electronic device provided in an embodiment of the present application includes: at least one processor 81 and a memory 82. The processor 81 and the memory 82 are connected via a bus 83.

[0125] The specific implementation process of the processor 81 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0126] In the above Figure 8In the illustrated embodiment, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0127] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0128] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0129] The present application also provides a computer-readable storage medium having computer-executable instructions stored thereon. When a processor executes the computer-executable instructions, a multi-camera joint calibration method as described above is implemented.

[0130] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general or special purpose computer.

[0131] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0132] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0135] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0136] The present application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement the above-mentioned multi-camera joint calibration method.

[0137] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0138] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0139] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0140] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0141] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0142] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0143] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0145] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A multi-camera joint calibration method, characterized in that: include: Measuring the three-dimensional corner point coordinates corresponding to a plurality of calibration plates located around the vehicle, respectively, so as to calculate a conversion matrix between each slave calibration plate coordinate system and a master calibration plate coordinate system according to the plurality of three-dimensional corner point coordinates; the plurality of calibration plates include a master calibration plate and a slave calibration plate, and the three-dimensional corner point coordinates are the real three-dimensional spatial positions of the corner points of the calibration plates in the coordinate system of the calibration plates themselves; Capturing images of the multiple calibration plates by using multiple cameras on the vehicle to obtain multiple calibration plate images, wherein each camera corresponds to one or more calibration plates, and the multiple cameras include a master camera and slave cameras; Extracting a calibration plate identifier and two-dimensional corner point coordinates of the calibration plate from each calibration plate image, wherein the two-dimensional corner point coordinates are positions of the corner points in the image; For each of the calibration plate identifiers, determining a transformation matrix between a camera coordinate system and a master calibration plate coordinate system according to the two-dimensional corner point coordinates and the three-dimensional corner point coordinates corresponding to the calibration plate identifier, wherein the camera coordinate system includes a master camera coordinate system and a slave camera coordinate system; For each of the slave cameras, a transformation matrix between the slave camera coordinate system and the master calibration board coordinate system is determined according to a transformation matrix between the camera coordinate system and the master calibration board coordinate system, wherein the transformation matrix includes a rotation matrix and a translation matrix.

2. The method according to claim 1, characterized in that The step of calculating the conversion matrix between each slave calibration plate coordinate system and the master calibration plate coordinate system according to the coordinates of the plurality of three-dimensional corner points includes: For each corner point of each slave calibration plate, establish a relationship equation between the three-dimensional corner point coordinates of the corner point and the three-dimensional corner point coordinates of the master calibration plate, wherein the unknown parameters of the relationship equation are the conversion matrix and the scale factor between the slave calibration plate coordinate system and the master calibration plate coordinate system, and the scale factor is the ratio of the lengths of the same straight line in the slave calibration plate coordinate system and the master calibration plate coordinate system; For each slave calibration board, the four groups of relationship equations corresponding to the slave calibration board are solved to obtain a conversion matrix between the coordinate system of the slave calibration board and the master calibration board.

3. The method according to claim 2, characterized in that One such relationship equation is as follows: Among them, X, Y, Z are the three-dimensional corner coordinates of a corner point of the calibration plate in the world coordinate system, X0, Y0, Z0 are the three-dimensional corner coordinates of a corner point of the main calibration plate in the world coordinate system, ΔX, ΔY, ΔZ are the translation parameters in the transformation matrix between the calibration plate coordinate system and the main calibration plate coordinate system, θ x ,θ z and θ y is the rotation parameter in the transformation matrix between the coordinate system of the slave calibration plate and the coordinate system of the master calibration plate, and m is the scale factor.

4. The method according to any one of claims 1 to 3, characterized in that The step of determining the conversion matrix between the camera coordinate system and the main calibration plate coordinate system according to the two-dimensional corner point coordinates, the three-dimensional corner point coordinates, and the conversion matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system, comprises: Performing homogeneous transformation on the two-dimensional corner point coordinates and the three-dimensional corner point coordinates to obtain homogeneous two-dimensional corner point coordinates and homogeneous three-dimensional corner point coordinates respectively; The conversion matrix between the camera coordinate system and the main calibration plate coordinate system is determined according to the homogeneous two-dimensional corner point coordinates, the homogeneous three-dimensional corner point coordinates, and the conversion matrix between the slave calibration plate coordinate system and the main calibration plate coordinate system.

5. The method according to claim 4, characterized in that The step of determining a conversion matrix between a camera coordinate system and a main calibration plate coordinate system according to the homogeneous two-dimensional corner point coordinates and the homogeneous three-dimensional corner point coordinates corresponding to the calibration plate identifier includes: A second relationship equation between the homogeneous two-dimensional corner point coordinates and the homogeneous three-dimensional corner point coordinates is constructed, and the second relationship equation is as follows: Among them, r1, r2, and r3 are the rotation parameters in the transformation matrix between the camera coordinate system and the main calibration plate coordinate system. is the translation parameter in the transformation matrix between the kth camera coordinate system and the main calibration plate coordinate system, the [u k ,v k ,1] T are the coordinates of the homogeneous two-dimensional corner point, [X k ,Y k ,0,1] T is the homogeneous three-dimensional corner point coordinate, K k is the intrinsic parameter matrix of the kth camera, z ck is the scale factor of the kth camera; The second relationship equation is solved to obtain a transformation matrix between the camera coordinate system and the main calibration plate coordinate system.

6. The method according to claim 1, characterized in that The step of determining a transformation matrix from the camera coordinate system to the main camera coordinate system according to the transformation matrix between the camera coordinate system and the main calibration plate coordinate system comprises: The transformation matrix between the slave camera coordinate system and the master camera coordinate system is calculated by the following formula: Among them, R 1k are the kth rotation vectors from the camera coordinate system to the main camera coordinate system, and the t 1k is the translation vector between the kth slave camera coordinate system and the master camera coordinate system, is the rotation vector between the main camera coordinate system and the main calibration plate coordinate system, is the translation vector between the main camera coordinate system and the main calibration plate coordinate system; is the rotation vector between the kth slave camera coordinate system and the master calibration plate coordinate system, is the translation vector between the kth slave camera coordinate system and the master calibration plate coordinate system, where k is a positive integer greater than or equal to 1.

7. A multi-camera joint calibration device, characterized in that: include: A measurement module, used to measure the coordinates of three-dimensional corner points corresponding to multiple calibration plates located around the vehicle; A processing module, configured to calculate a conversion matrix between each slave calibration plate coordinate system and a master calibration plate coordinate system according to the plurality of three-dimensional corner point coordinates; the plurality of calibration plates include a master calibration plate and a slave calibration plate, and the three-dimensional corner point coordinates are the real three-dimensional spatial positions of the corner points of the calibration plates in the coordinate system of the calibration plate itself; An acquisition module, configured to acquire images of the plurality of calibration plates through a plurality of cameras on the vehicle to obtain a plurality of calibration plate images, wherein each of the cameras corresponds to one or more calibration plates, and the plurality of cameras include a master camera and slave cameras; The processing module is further used to extract the calibration plate identification and the two-dimensional corner point coordinates of the calibration plate from each calibration plate image, wherein the two-dimensional corner point coordinates are the positions of the corner points in the image; The processing module is further used to determine, for each calibration plate identifier, a conversion matrix between a camera coordinate system and a master calibration plate coordinate system according to the two-dimensional corner point coordinates and the three-dimensional corner point coordinates corresponding to the calibration plate identifier, wherein the camera coordinate system includes a master camera coordinate system and a slave camera coordinate system; The processing module is further used to determine, for each of the slave cameras, a transformation matrix between the slave camera coordinate system and the master calibration board coordinate system, wherein the transformation matrix includes a rotation matrix and a translation matrix.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 6 when being executed by a processor.

Images