Camera calibration method, control device and calibration system

Abstract

The invention provides a camera calibration method, a control device and a calibration system, relates to the field of camera calibration, and can accurately complete calibration of a camera in various scenes. The method comprises the following steps: acquiring pixel coordinates of a first angular point in an initial calibration image shot by each camera in an initial state; based on the internal parameters of each camera, the pixel coordinates of the first angular point corresponding to each camera and the three-dimensional coordinates of the first angular point on the calibration plate, determining a pose transformation matrix of each camera and the corresponding calibration plate in the initial state; obtaining pixel coordinates of a second angular point in a rotation calibration image shot by each camera after the rotation of the rotation platform; based on the internal parameters of each camera, the pixel coordinates of the second angular point corresponding to each camera and the three-dimensional coordinates of the second angular point on the calibration plate, determining a pose transformation matrix of each camera and the corresponding calibration plate after the rotation of the rotating platform; and determining the pose transformation matrix of any two cameras based on the pose transformation matrixes of the cameras and the corresponding calibration plates before and after the rotation of the rotating platform.

Description

Technical Field

[0001] The application relates to the field of camera calibration, and in particular to a camera calibration method, control device and calibration system. Background Art

[0002] With growing user demands, a single camera with a small field of view can no longer meet the perception requirements of machine vision applications (such as 3D reconstruction, obstacle avoidance, navigation, and simultaneous localization and mapping (SLAM)) in various electronic devices (such as virtual reality (VR) / augmented reality (AR), robots, drones, and autonomous vehicles). Therefore, current machine vision applications require the use of multi-camera perception suites with a larger field of view, consisting of multiple cameras. In order for the multi-camera perception suite to function properly in the desired application, the intrinsic and extrinsic parameters (internal and external) of each camera in the suite must be calibrated in advance.

[0003] In traditional calibration schemes for camera internal and external parameters in multi-view sensing suites, camera external parameter calibration is mostly completed under the premise of a common view area between the cameras. If two cameras in the multi-view sensing suite do not have a common view area, the calibration of the external parameters of these two cameras cannot be completed. In addition, the existing technology for calibrating the internal and external parameters of cameras in a multi-view sensing suite without a common view area has large limitations on the layout positions between different cameras or has low calibration accuracy. Therefore, there is an urgent need for a method with higher applicability and accurate calibration of cameras in a multi-view sensing suite. Summary of the Invention

[0004] The embodiments of the present application provide a camera calibration method, a control device, and a calibration system, which can accurately complete camera calibration in a variety of scenarios.

[0005] In order to achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, embodiments of the present application provide a camera calibration method, which is applied to a calibration system, wherein the calibration system includes a rotating platform, an electronic device equipped with a multi-camera sensing kit, multiple calibration plates, and a control device that establishes a communication connection with both the rotating platform and the electronic device. The electronic device is fixedly mounted on the rotating platform, and the multi-camera sensing kit includes multiple cameras, each of which has a calibration plate fixedly mounted in its shooting area. The method includes:

[0007] The control device obtains the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image captured by each camera in the multi-view perception suite in the initial state; the initial state is the state in which the initial setting of the calibration system is completed;

[0008] The control device determines, in an initial state, a pose transformation matrix between each camera and the calibration plate corresponding to each camera based on the internal parameters of each camera, the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate;

[0009] The control device obtains the pixel coordinates of the second corner point of the calibration pattern in the rotation calibration image captured by each camera when the rotating platform rotates by a preset angle;

[0010] The control device determines, based on the internal parameters of each camera, the pixel coordinates of the second corner point in the rotation calibration image corresponding to each camera, and the three-dimensional coordinates of the second corner point on the corresponding calibration plate, a pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle;

[0011] The control device determines the pose transformation matrix between any two cameras in the multi-view perception kit based on the pose transformation matrix between each camera and the corresponding calibration plate in the initial state, and the pose transformation matrix between each camera and the corresponding calibration plate when the rotating platform rotates by a preset angle.

[0012] Based on the technical solution provided by the embodiment of the present application, by introducing the rotation angle information through the rotating platform, the external parameters (i.e., the pose transformation matrix) of each camera in the multi-eye perception suite can be determined in combination with the pure visual information in the image captured by each camera. For pure visual information, the stability and uniqueness of the feature points contained therein are better, so the calibration scheme can make the external parameters of the camera finally obtained more accurate. In addition, as long as there are enough feature points in the pure visual information in different scenes, the calibration purpose can be successfully completed. This also makes the adaptability of the calibration scheme better. At the same time, since the calibration scheme does not determine the external parameters between cameras (i.e., the pose transformation matrix between different cameras) based on the difference between the images captured by different cameras of the same object, regardless of whether there is a common view area between the different cameras in the multi-eye perception suite, the calibration scheme provided by the present application can smoothly obtain the external parameters between the different cameras in the multi-eye perception suite. Further, the calibration scheme provided by the present application does not have a strong constraint on the position of the calibration plate. It only needs to be adapted according to the layout position of the cameras in the multi-eye perception suite, and the style of the calibration plate is not specifically limited. In summary, the technical solution provided in this application can accurately calibrate the external parameters of the camera in a variety of scenarios (such as production lines or maintenance points), regardless of whether there is a common view area between different cameras in the multi-eye perception kit. Compared with the existing camera calibration solution for multi-eye perception kits, it has better adaptability and stability.

[0013] In a possible design manner of the first aspect, a control device obtains the pixel coordinates of a first corner point of a calibration pattern in an initial calibration image captured by each camera in the multi-eye perception suite in an initial state, including: the control device sends a first instruction to the electronic device; the first instruction is used to instruct the electronic device to control each camera in the multi-eye perception suite to capture and send the captured initial calibration image to the control device; the control device receives the initial calibration image corresponding to each camera from the electronic device, and determines the pixel coordinates of the first corner point of the calibration pattern in each initial calibration image.

[0014] Based on the above technical solution, the control device can complete the control of the camera in the electronic device through instructions, thereby obtaining the initial calibration image taken by each camera in the multi-eye perception kit of the electronic device and the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image, thereby providing a data basis for the execution of subsequent solutions.

[0015] In a possible design manner of the first aspect, the control device determines, in an initial state, a pose transformation matrix between each camera and the calibration plate corresponding to each camera based on the internal parameters of each camera, the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate, including: the control device converts the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera into two-dimensional coordinates in the image coordinate system based on the internal parameters of each camera; the control device determines, in an initial state, a pose transformation matrix between each camera and the calibration plate corresponding to each camera based on the internal parameters of each camera, the two-dimensional coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera in the image coordinate system, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate.

[0016] Based on the above technical solution, a specific algorithm can be used to calculate the pose transformation matrix between each camera in the multi-camera perception kit and the corresponding calibration plate in the initial state, providing a data basis for the execution of subsequent solutions.

[0017] In a possible design manner of the first aspect, the control device obtains the pixel coordinates of the second corner point of the calibration pattern in the rotation calibration image captured by each camera when the rotating platform is rotated by a preset angle, including: the control device sends a second instruction to the rotating platform; the second instruction is used to instruct the rotating platform to rotate by a preset angle; the control device receives a first response from the rotating platform, and in response to the first response, sends a third instruction to the electronic device; the first response is used to indicate that the rotation of the rotating platform is completed; the third instruction is used to instruct the electronic device to control each camera in the multi-eye perception kit to shoot, and send the captured rotation calibration image to the control device; the control device receives the rotation calibration image corresponding to each camera from the electronic device, and determines the pixel coordinates of the second corner point of the calibration pattern in each rotation calibration image.

[0018] Based on the above technical solution, the control device can complete the control of the rotating platform and the electronic device through instructions, thereby obtaining the rotation calibration images taken by each camera in the multi-eye perception kit of the electronic device when the rotating platform rotates to a preset angle, as well as the pixel coordinates of the second corner point of the calibration pattern in the rotation calibration image, thereby providing a data basis for the execution of subsequent solutions.

[0019] In a possible design manner of the first aspect, the control device determines the pose transformation matrix between any two cameras in the multi-camera perception kit based on the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state, and the pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle, including: the control device calculates the pose transformation matrix between the ground and each calibration plate based on the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state, and the pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle; the control device determines the pose transformation matrix between any two calibration plates based on the pose transformation matrix between the ground and each calibration plate; the control device determines the pose transformation matrix between any two calibration plates based on the pose transformation matrix between any two calibration plates, and the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state.

[0020] In this calibration system, the pose transformation matrix between any two cameras can be obtained by converting the pose transformation relationship between each of the two cameras and its corresponding calibration plate using the pose transformation matrix between the calibration plates corresponding to the two cameras. The pose transformation matrix between the calibration plates corresponding to any two cameras can be obtained by converting the pose transformation matrix between the two calibration plates and the ground. Furthermore, since the relative position between the calibration plate and the ground is fixed throughout the calibration system, the pose transformation matrix between the calibration plate and the ground is fixed. That is, the pose transformation matrix between the calibration plate and the ground remains unchanged, regardless of whether the rotating platform rotates before or after. Furthermore, the relative position between the rotating platform and each camera is also fixed, meaning the pose transformation matrix between the rotating platform and the camera remains the same, regardless of whether the rotating platform rotates before or after. Furthermore, the pose transformation matrix between the rotating platform and the camera, whether before or after rotation, can be obtained by converting the pose transformation matrix between the rotating platform and the ground, the pose transformation matrix between the ground and the calibration plate, and the pose transformation matrix between the calibration plate and the corresponding camera. Furthermore, the rotation angle is known for each rotation after the initial state. Therefore, the difference between the pose transformation matrix between the rotating platform and the ground after each rotation and the pose transformation matrix between the rotating platform and the ground before the rotation is only the transformation of the rotation angle mapped into the rotation matrix. Based on the above, the pose transformation matrix between the rotating platform and the ground before and after the rotation can be considered known. In order to obtain the pose transformation matrix between the ground and each calibration plate, it is necessary to obtain the pose transformation matrix between each camera and its corresponding calibration plate before and after the rotation.

[0021] Therefore, after obtaining the pose transformation matrix between each camera and its corresponding calibration plate before and after rotation, we can follow the above logic to sequentially obtain the pose transformation matrix between the ground and each calibration plate, the pose transformation matrix between any two calibration plates, and the pose transformation matrix between any two cameras. Therefore, based on the above scheme, we can successfully derive the pose transformation matrix between any two cameras, that is, the extrinsic parameters between any two cameras in the multi-view perception suite.

[0022] In a possible design manner of the first aspect, the control device calculates a pose transformation matrix between the ground and each calibration plate based on a pose transformation matrix between each camera and the calibration plate corresponding to each camera in an initial state and a pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle, including:

[0023] The control device calculates the pose transformation matrix between the ground and each calibration plate based on the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state and the pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle, using the following formula:

[0024]

[0025] in, is the posture transformation relationship between the rotating platform and the ground in the initial state, is the pose transformation matrix between the ground and the nth calibration plate, is the pose transformation matrix between the nth calibration plate and the nth camera. The calibration plate corresponding to the nth camera is the nth calibration plate. The pose transformation matrix between the rotating platform and the ground when the rotating platform rotates by a preset angle. It is the pose transformation matrix between the nth calibration plate and the nth camera when the rotating platform rotates by a preset angle; the nth calibration plate is one of the calibration plates corresponding to multiple cameras, and the nth camera is any one of the multiple cameras.

[0026] Based on the above technical solution, the pose transformation matrix between the ground and each calibration plate can be calculated based on the conversion relationship between the pose transformation matrices, providing data support for the subsequent calculation of the pose transformation matrix between any two cameras.

[0027] In a possible design manner of the first aspect, the control device determines, based on the internal parameters of each camera, the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate, the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state. The method also includes: the control device determines the internal parameters of each camera in the multi-view perception kit.

[0028] Based on the above technical solution, since the camera's content parameters are needed when calculating the pose transformation matrix between the camera and the camera's corresponding calibration plate, the control device needs to obtain the internal parameters of each camera in advance before calculating the pose transformation matrix, thereby providing strong data support for subsequent calculations.

[0029] In a second aspect, the present application provides a control device, which includes a memory and one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code includes computer commands, and when the computer commands are executed by the processor, the control device executes the camera calibration method provided in the first aspect and any possible design method thereof.

[0030] In a third aspect, the present application provides a computer-readable storage medium, which includes computer commands. When the computer commands are executed on a control device, the control device executes the camera calibration method provided in the first aspect and any possible design thereof.

[0031] In a fourth aspect, the present application provides a computer program product, which, when executed on a control device, enables the control device to execute the camera calibration method provided in the first aspect and any possible design thereof.

[0032] It can be understood that the beneficial effects that can be achieved by the technical solutions provided in the second to fourth aspects mentioned above can be referred to the beneficial effects in the first aspect and any possible design method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of various calibration plates provided in the embodiments of the present application;

[0034] Figure 2 A schematic diagram of the relationship between an image coordinate system and a pixel coordinate system provided in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of the relationship between an image coordinate system and a camera coordinate system provided in an embodiment of the present application;

[0036] Figure 4A schematic diagram of a three-dimensional calibration plate provided in an embodiment of the present application;

[0037] Figure 5 A schematic diagram of the structure of a calibration system provided in an embodiment of the present application;

[0038] Figure 6 A schematic diagram illustrating the principle of a calibration method provided in an embodiment of the present application;

[0039] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0040] Figure 8 A schematic diagram of the hardware structure of a control device provided in an embodiment of the present application;

[0041] Figure 9 A flow chart of a calibration method provided in an embodiment of the present application;

[0042] Figure 10 A schematic diagram of multiple target images provided in an embodiment of the present application;

[0043] Figure 11 Schematic diagram of various coordinate systems in the calibration system provided in the embodiment of this application;

[0044] Figure 12 A schematic diagram of another calibration method provided in an embodiment of the present application;

[0045] Figure 13 A schematic diagram of a flow chart of another calibration method provided in an embodiment of the present application;

[0046] Figure 14 A schematic diagram of the structure of a control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to be limiting of the present application. As used in the specification and appended claims of the present application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that " / " means or, for example, A / B can mean A or B; "and / or" in the text is merely a description of an association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0048] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0049] The terms "first" and "second" in the following embodiments of this application are used for descriptive purposes only and should not be understood as implying or suggesting relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more.

[0050] First, the terms used in this application are explained as follows:

[0051] (1) Calibration plate: The calibration plate is a typical calibration object used for visual measurement. By using a camera to shoot the calibration plate, a specific calibration algorithm can be used to obtain the camera's intrinsic and extrinsic parameters. The calibration plate can include a calibration pattern. Depending on the calibration pattern, there are many types of calibration plates, such as Figure 1 The calibration pattern shown in (a) is a checkerboard calibration plate. Figure 1 The calibration pattern shown in (b) is a dot calibration plate with dots, and Figure 1 The calibration pattern shown in (c) is a cross calibration plate.

[0052] (2) Corner points: Corner points are points that can highlight the characteristics of an image. Corner points often have some significant features that make them easy to identify and locate in the image. In practice, corner point detection is usually used to extract points of interest in an image. These points of interest can be corner points, isolated points with the maximum or minimum intensity of certain attributes, end points of line segments, or points with the maximum local curvature on a curve. It should be noted that there is no clear mathematical definition of corner points, so in practical applications, it is necessary to use specific methods to select appropriate corner points according to specific needs.

[0053] For example, Figure 1 In the checkerboard calibration plate shown in (a), the corner points can be the intersection points of the black and white squares in the checkerboard, or the vertices of each black / white square. Figure 1 In the dot calibration plate shown in (b), the corner point can be the center of each dot. Figure 1 In the cross-calibration plate shown in (c), the corner points can be the intersection points of all straight lines in the chessboard.

[0054] (3) Calibration: In the field of machine vision and image processing, calibration is often used to determine the intrinsic and extrinsic parameters of a camera or other sensor so that pixels in an image can be accurately mapped to their actual positions in three-dimensional space. The calibration process involves photographing a calibration plate of known geometry and calculating the intrinsic and extrinsic parameters of the camera through an algorithm. Calibration is one of the important steps in establishing a high-precision vision system and is crucial for achieving precise measurement, positioning, and reconstruction tasks.

[0055] (4) Camera intrinsic parameters: Camera intrinsic parameters refer to the internal parameters of the camera, which are specifically used to describe the basic configuration and characteristics of the camera optical system. For example, the internal parameters of a camera may include focal length, principal point coordinates, pixel pitch, and distortion coefficient.

[0056] The principal point is also called the intersection of the principal optical axis and the image plane, which is usually the center point of the image. The principal point coordinates may be the coordinates of the principal point in the pixel coordinate system.

[0057] Pixel pitch specifically refers to the physical distance between adjacent pixels on an image sensor.

[0058] Distortion coefficients describe image distortion caused by camera lens imperfections (e.g., lens shape, assembly errors). For example, these coefficients can include radial and tangential distortion coefficients. The radial distortion coefficient primarily characterizes distortion caused by varying magnifications in different parts of the camera lens, while the tangential distortion coefficient primarily characterizes distortion caused by the camera lens not being mounted perpendicular to the imaging plane.

[0059] In some embodiments, the camera intrinsic parameters may also include pixel aspect ratio, sensor size, pixel size, etc.

[0060] (5) Camera extrinsics: Camera extrinsics specifically refer to the external parameters of the camera, and specifically may refer to parameters that describe the position and orientation of the camera in three-dimensional space. In some embodiments, camera extrinsics may refer to the rotational transformation and translational transformation of the camera relative to a certain coordinate system (such as the world coordinate system or the camera coordinate system of another camera). Among them, the rotational transformation can be represented by a rotation matrix, and the translational transformation can be represented by a translation vector. The rotation matrix is ​​usually represented by R, which is a 3×3 matrix. Each column in the rotation matrix is ​​a unit vector, corresponding to the direction of the x-axis, y-axis, and z-axis in the camera coordinate system in the certain coordinate system. The translation vector is usually represented by t, which is a 3×1 vector, which describes the position of the origin of the camera coordinate system in the certain coordinate system.

[0061] In some embodiments, the rotation matrix and the translation vector may form a posture transformation matrix T. Specifically, the composition relationship between the posture transformation matrix and the rotation matrix and the translation vector is as follows:

[0062]

[0063] Among them, 0 is the 3×1 zero vector, and the "1" in the lower right corner of the pose transformation matrix is ​​the scaling factor.

[0064] (6) Pixel coordinate system: The pixel coordinate system is a two-dimensional plane coordinate system, based on which the number of columns and rows of each pixel in an image (or a captured image) can be represented. For example, refer to Figure 2 As shown, the origin of the pixel coordinate system is O s It can be the upper left corner of the image, the horizontal axis O s u axis and ordinate axis O s The v-axis points directly to the right and directly downward, respectively. For a pixel (or pixel point) in the image, its horizontal coordinate in the pixel coordinate system represents its column number, and its vertical coordinate represents its row number. For example, a pixel with coordinates (5,6) in the image is in the 5th column and 6th row of the image.

[0065] (7) Image coordinate system: The image coordinate system is a two-dimensional plane coordinate system. Based on the image coordinate system, the actual position and size of the image in the physical space can be expressed. Figure 2 As shown, the origin of the image coordinate system is O t It can be the intersection of the camera optical axis and the imaging plane, usually the center of the image. t x t Axis and the horizontal axis O of the pixel coordinate system s The u axis is parallel to the vertical axis O of the image coordinate system. t y t Axis and the vertical axis O of the pixel coordinate system s v t Axis parallel.

[0066] In some embodiments, if it is necessary to convert coordinates in the pixel coordinate system into coordinates in the image coordinate system, the conversion may be performed based on camera intrinsic parameters.

[0067] (8) Camera coordinate system: The camera coordinate system is a three-dimensional rectangular coordinate system. Based on the camera coordinate system, the relative position relationship (or posture relationship) between the camera and the object in the three-dimensional space can be expressed. Figure 3 As shown, the camera coordinate system takes the optical center of the camera (lens center) as the origin O c , with the optical axis (the straight line between the optical center and the principal point) as O c z c The three-dimensional rectangular coordinate system established by the axis. In the camera coordinate system, O c x c Axis and the horizontal axis O in the image coordinate system t xt Axis parallel, O c y c Axis and the horizontal axis O in the image coordinate system t y t Axis parallel.

[0068] (9) World Coordinate System: A reference coordinate system is selected in three-dimensional space to describe the position of the camera and the object. This coordinate system can be called the world coordinate system. The world coordinate system is a three-dimensional rectangular coordinate system whose coordinate points can be used to describe the position of any point in space. In some embodiments, the world coordinate system can be freely defined according to actual usage requirements. For example, the origin of the world coordinate system can be a point on the calibration plate, and the three mutually perpendicular coordinate axes can be determined according to the specific position of the calibration plate.

[0069] During the calibration of the camera's internal and external parameters, the relationship between the camera coordinate system and the world coordinate system can be described by the rotation transformation matrix R and the translation vector t (or the pose transformation matrix). Therefore, for a point P in space, the homogeneous coordinates in the world coordinate system and the camera coordinate system are (x1, y1, z1) and (x2, y2, z2), respectively, satisfying the following formula (2):

[0070]

[0071] (10) PNP (perspective-n-point) algorithm: The PNP algorithm is a pose estimation algorithm in computer vision that is used to solve the correspondence between 3D space points and 2D image points. The algorithm can obtain the camera pose and the pose transformation relationship (or spatial relationship or relative position relationship) between the target object and the camera by knowing the coordinate points of the target object (such as a calibration plate) in 3D space and the corresponding pixel coordinates in the image captured by the camera.

[0072] To clarify the internal and external parameters of cameras in a multi-camera perception suite, the following solutions exist in related technologies:

[0073] The first option is to use Figure 4 The stereo calibration plate shown in the figure is used to calibrate the camera's internal and external parameters. However, this solution requires very high manufacturing precision and high cost. It also has certain requirements on the field of view and layout of each camera in the multi-camera perception kit, making it less applicable.

[0074] The second solution is to add an inertial measurement unit (IMU) to the calibration solution. When building the calibration system, a fixed relative position relationship is established between the camera and the IMU in the multi-camera perception kit. Furthermore, the relative position between multiple cameras can be indirectly obtained using the IMU as a medium. However, in this calibration solution, the measurement accuracy of the IMU itself is not high, and the overall solution needs to rely on the IMU's 6 degrees of freedom (6dof) movement, which usually requires a robotic arm or handheld completion, and is not suitable for multi-camera perception kits on medium and large equipment.

[0075] The third solution is traditional binocular camera stereo calibration. In this solution, the two cameras must have a common view area. If the two cameras in the multi-camera perception kit do not have a common view area, this solution cannot be calibrated.

[0076] It can be seen that the existing calibration schemes have certain limitations and cannot be applied to the accurate calibration of camera internal and external parameters in all multi-view perception kits.

[0077] To address the above issues, this application provides a camera calibration solution that can calibrate the internal and external parameters of the camera in the multi-view perception kit. This solution can be applied to Figure 5 In the calibration system shown. Figure 5 As shown, the calibration system includes a rotating platform, an electronic device with a multi-viewing kit, and multiple calibration plates ( Figure 5 The multi-view perception kit includes multiple cameras ( Figure 5 In this example, only camera 1 and camera 2 are used as examples. Each calibration plate corresponds to one camera. For example, calibration plate 1 corresponds to camera 1, and calibration plate 2 corresponds to camera 2.

[0078] In some embodiments, the rotating platform can rotate in a horizontal direction with a rotation accuracy of 0.001°.

[0079] In some embodiments, the electronic device in the present application may only include a multi-eye perception kit, that is, the electronic device may be the multi-eye perception kit itself.

[0080] Reference Figure 5 As shown, the electronic device is fixedly mounted on a rotating platform, which can rotate around a central axis perpendicular to the ground. After the electronic device is fixed, a corresponding calibration plate can be set in the shooting area of ​​each camera in the multi-viewing kit on the electronic device so that each camera can capture a calibration plate. Figure 5As shown in the figure, there can be calibration plate 1 in the shooting area of ​​camera 1, and calibration plate 2 in the shooting area of ​​camera 2. In this way, in the initial state, each camera in the multi-view perception kit can capture the complete calibration pattern on its corresponding calibration plate.

[0081] In addition, the electronic device and the rotating platform can both establish a wired or wireless communication connection with the control device. In some embodiments, each camera in the multi-eye perception kit and the rotating platform can both establish a wired or wireless communication connection with the control device.

[0082] In some embodiments, the control device may also be the electronic device itself. In this case, a limited or wireless communication connection may be established between the rotating platform and the electronic device to facilitate the control device to control the rotating platform.

[0083] Based on the above calibration system, the camera calibration solution provided in this application refers to Figure 6 As shown, first, the internal parameters of each camera in the multi-view perception suite can be determined according to any internal parameter calibration method. Afterwards, the control device can introduce the rotation angle information through the rotating platform. As a result, the pose relationship between the calibration plate and the ground is fixed, the pose relationship between the camera and the rotating platform is fixed, and the pose relationship between the rotating platform and the ground is only determined by the rotation angle. Based on this, the pose relationship between each calibration plate and the corresponding camera before and after the rotating platform is rotated can be solved (specifically, it can be represented by a pose transformation matrix). Based on the pose relationship between each calibration plate and the corresponding camera before and after the rotating platform is rotated, and the pose relationship between the ground and the rotating platform before and after the rotation, the pose relationship between the ground and each calibration plate can be calculated based on the fixed pose relationship between the camera and the rotating platform and the matrix transformation principle, and then the pose relationship between each calibration plate can be calculated. Finally, the pose relationship between each calibration plate is used as an intermediate quantity of the pose transformation between different cameras, and the pose relationship between each camera can be calculated as the external parameter between different cameras in the multi-view perception suite.

[0084] The specific execution of the calibration scheme provided in this application can be completed by the control device in the calibration system.

[0085] As can be seen, in the calibration scheme of the present application, by introducing rotation angle information through the rotating platform, the internal and external parameters of each camera in the multi-view perception suite can be determined in combination with the pure visual information in the images captured by each camera. For pure visual information, the stability and uniqueness of the feature points it contains are relatively good, so this calibration scheme can make the final internal and external parameters of the camera more accurate. In addition, as long as there are enough feature points in the pure visual information, the calibration purpose can be successfully completed in different scenarios. This also makes the calibration scheme more adaptable. At the same time, because the calibration scheme does not determine the external parameters between cameras based on the differences in the images captured by different cameras of the same object, the calibration scheme provided by the present application can successfully obtain the external parameters between different cameras in the multi-view perception suite regardless of whether there is a common view area between different cameras in the multi-view perception suite. Furthermore, the calibration scheme provided by the present application does not have strict constraints on the position of the calibration plate; it only needs to be adapted according to the layout position of the cameras in the multi-view perception suite, and there is no specific limitation on the style of the calibration plate. In summary, the technical solution provided in this application can accurately calibrate the internal and external parameters of the camera in a variety of scenarios (such as production lines or maintenance points), regardless of whether there is a common view area between different cameras in the multi-eye perception kit. Compared with the existing camera calibration solution for multi-eye perception kits, it has higher adaptability.

[0086] The technical solutions provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0087] The technical solution provided in this application can be applied in Figure 5 In the calibration system shown in FIG. 1 , the specific composition of the calibration system can refer to the aforementioned embodiment for Figure 5 The relevant statements will not be repeated here.

[0088] Exemplarily, the electronic devices of the above-mentioned calibration system can be mobile phones, tablet computers, handheld computers, personal computers (PCs), ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) devices, virtual reality (VR) devices, artificial intelligence (AI) devices, wearable devices, vehicle-mounted devices, smart home devices and / or smart city devices, etc., which are equipped with multi-eye perception kits. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device.

[0089] For example, taking the electronic device as a mobile phone, Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown.

[0090] Reference Figure 7 As shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a display 193, a subscriber identification module (SIM) card interface 194, and a camera 195. The sensor module 180 may include a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0091] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0092] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals based on command operation codes and timing signals to complete the control of command retrieval and execution.

[0093] Processor 110 may also include a memory for storing commands and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store commands or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the command or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0094] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0095] The charging management module 140 is used to receive charging input from a power supply device (e.g., a charger, laptop charger, etc.). The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the electronic device.

[0096] While charging the battery 142, the charging management module 140 can also power the electronic device through the power management module 141. Specifically, the battery 142 can be composed of multiple batteries connected in series. The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110.

[0097] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the internal memory 121, the display 193, the camera 195, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery voltage, current, battery cycle count, and battery health status (leakage, impedance). In other embodiments, the power management module 141 can also be provided in the processor 110.

[0098] The external memory interface 120 can be used to connect to an external non-volatile memory device to expand the storage capacity of the electronic device. The external non-volatile memory device communicates with the processor 110 via the external memory interface 120 to implement data storage. For example, files such as music and videos can be stored in the external non-volatile memory device.

[0099] The internal memory 121 may include one or more random access memories (RAMs) and one or more non-volatile memories (NVMs). The RAM can be directly read and written by the processor 110 and can be used to store executable programs (e.g., machine commands) of the operating system or other running programs, as well as user and application data. The NVM can also store executable programs and user and application data, and can be pre-loaded into the RAM for direct reading and writing by the processor 110.

[0100] A touch sensor, also known as a "touch control device," can be provided on the display screen 193. The touch sensor and the display screen 193 form a touch screen, also known as a "touch screen." The touch sensor is used to detect touch operations applied thereto or in the vicinity thereof. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided via the display screen 193. In other embodiments, the touch sensor can also be provided on the surface of the electronic device, at a location different from that of the display screen 193.

[0101] The ambient light sensor is used to sense ambient light brightness. The pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor can be located on display screen 193. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors.

[0102] In some embodiments, the electronic device may include 1 or N cameras 195, where N is a positive integer greater than 1. In an embodiment of the present application, the type of camera 195 can be distinguished based on the hardware configuration and physical location. For example, the camera provided on the side of the display screen 193 of the electronic device can be called a front camera, and the camera provided on the side of the back cover of the electronic device can be called a rear camera; for another example, a camera with a short focal length and a larger viewing angle can be called a wide-angle camera, and a camera with a long focal length and a small viewing angle can be called a normal camera. Among them, the length of the focal length and the size of the viewing angle are relative concepts, and there are no specific parameters to limit them. Therefore, wide-angle cameras and normal cameras are also relative concepts, and can be specifically distinguished based on physical parameters such as focal length and viewing angle.

[0103] In an embodiment of the present application, a combination of multiple cameras included in an electronic device can be called a multi-eye perception kit, and each camera can be called a camera.

[0104] The electronic device implements display functions through a GPU, display screen 193, and an application processor. The GPU is a microprocessor for image processing that connects display screen 193 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program commands to generate or modify display information.

[0105] The electronic device can implement a camera function through an ISP, a camera 195, a video codec, a GPU, a display 193, and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program commands to generate or change display information.

[0106] The ISP processes data fed back by camera 195. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise and brightness. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be incorporated into camera 195. Camera 195 is used to capture still images or video.

[0107] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an electronic device selects a frequency, the DSP performs a Fourier transform on the frequency energy.

[0108] Video codecs are used to compress or decompress digital video. Electronic devices may support one or more video codecs. This allows them to play or record videos in a variety of encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0109] Display screen 193 is used to display images, videos, and the like. Display screen 193 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, the electronic device can include one or N display screens 193, where N is a positive integer greater than one.

[0110] In an embodiment of the present application, the display screen 193 can be used to display pages required by the electronic device (for example, a display page for captured images, etc.), and display images captured by any one or more cameras 195 in the interface.

[0111] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem and baseband processor.

[0112] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in an electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0113] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter, amplify, and perform other processing on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0114] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 193. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0115] The wireless communication module 160 can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0116] In some embodiments, the antenna 1 of the electronic device is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TDSCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a BeiDou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0117] SIM card interface 194 is used to connect a SIM card. A SIM card can be connected to and disconnected from the electronic device by inserting or removing it from the SIM card interface 194. An electronic device may support one or more SIM card interfaces. SIM card interface 194 can support Nano SIM cards, Micro SIM cards, and SIM cards. Multiple cards can be inserted into the same SIM card interface 194 simultaneously. SIM card interface 194 is also compatible with external memory cards. Electronic devices interact with the network through SIM cards to implement functions such as call and data communications. Each SIM card corresponds to one user number.

[0118] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present invention is only a schematic illustration and does not constitute a structural limitation of the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0119] Of course, it is understandable that the above Figure 7 The figure is only an example of the electronic device being a mobile phone. If the electronic device is a tablet computer, handheld computer, PC, PDA, wearable device (such as smart watch, smart bracelet) or other device forms, the structure of the electronic device may include more than Figure 7 The structure shown in the figure is less than Figure 7 More structures are shown in the figure, which are not limited here.

[0120] For example, the control device provided in this application may be a server, a server cluster consisting of multiple servers, or a cloud computing service center, and this application does not impose any specific restrictions on this.

[0121] For example, taking the control device as a server, Figure 8 A schematic diagram of the structure of a server is shown. Figure 8 As shown, the server includes one or more processors 801, a communication line 802, and at least one communication interface ( Figure 8 The example in which the communication interface 803 and a processor 801 are included is merely exemplary), and a memory 804 may be optionally included.

[0122] The processor 801 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program corresponding to the solution of the present application.

[0123] The communication circuit 802 may include a communication bus for communication between different components.

[0124] Communication interface 803 can be a transceiver module for communicating with other devices (e.g., electronic devices equipped with a multi-viewing sensor, rotating platforms, etc.) or communication networks, such as Ethernet, RAN, and wireless local area networks (WLAN). For example, the transceiver module can be a device such as a transceiver or a transceiver. Alternatively, communication interface 803 can be a transceiver circuit located within processor 801, used to implement signal input and output to the processor.

[0125] The memory 804 may be a device having a storage function. For example, it may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a communication line 802. The memory may also be integrated with the processor.

[0126] The memory 804 is used to store computer-executable instructions for executing the solution of the present application, and the execution is controlled by the processor 801. The processor 801 is used to execute the computer-executable instructions stored in the memory 804, thereby implementing the camera calibration method provided in the embodiment of the present application.

[0127] Optionally, the computer-executable instructions in the embodiments of the present application may also be referred to as application code, which is not specifically limited in the embodiments of the present application.

[0128] In a specific implementation, as an embodiment, the processor 801 may include one or more CPUs, such as Figure 8 CPU0 and CPU1 in.

[0129] In a specific implementation, as an embodiment, the server may include multiple processors, such as Figure 8801 and processor 807 in the embodiment. Each of these processors can be a single-core processor or a multi-core processor. The processors herein can include, but are not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, and other types of computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing.

[0130] In a specific implementation, as an embodiment, the server may further include an output device 805 and an input device 806. The output device 805 communicates with the processor 801 and can display information in a variety of ways. For example, the output device 805 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device 806 communicates with the processor 801 and can receive user input in a variety of ways. For example, the input device 806 can be a mouse, a keyboard, a touch screen device, or a sensor device.

[0131] The server can be a general-purpose device or a dedicated device. For example, the server can be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, an embedded device, the terminal device, the network device, or a computer with Figure 8 The embodiment of the present application does not limit the type of server.

[0132] The technical solutions provided in the embodiments of the present application can be implemented in electronic devices and control devices having the above-mentioned hardware architecture, or in a calibration system composed of electronic devices and control devices having the above-mentioned hardware architecture.

[0133] Based on the above Figure 5 The calibration system shown below is combined with Figure 9 As shown, the processing flow of the camera calibration method provided in the embodiment of the present application is introduced. Figure 9 This is a flow chart of a camera calibration method provided in an embodiment of the present application. The calibration method can be executed by a control device in a calibration system. Figure 9As shown, the camera calibration method may include S901-S908:

[0134] S901: The control device determines internal parameters of each camera in the multi-eye perception kit on the electronic device.

[0135] For example, the internal parameters of the camera may include focal length, principal point coordinates, pixel spacing, and distortion coefficient. The definitions of the internal parameters of the camera can refer to the relevant expressions in the explanation of terms in the above embodiments and will not be repeated here.

[0136] In a possible implementation, if the Zhang Zhengyou calibration method is used to calibrate the internal parameters of the camera, S901 may specifically include the following steps:

[0137] S1. The control device controls the rotating platform to rotate multiple times, and controls the target camera to shoot the target calibration plate after each rotation to obtain multiple target images; the target image includes part or all of the target calibration pattern.

[0138] The target camera is one of the multiple cameras included in the multi-camera perception kit, the target calibration plate is the calibration plate corresponding to the target camera, and the target calibration pattern is the calibration pattern on the target calibration plate. For example, the target calibration image can be a checkerboard image, an origin image, a cross image, etc., depending on the target calibration plate.

[0139] In an embodiment of the present application, the control device may control the rotation of the rotating platform by sending multiple rotation instructions to the rotating platform, each of which instructs the rotating platform to rotate by a certain angle. After each rotation instruction is sent and after determining that the rotating platform has completed rotation, a capture instruction may be sent to the target camera. The capture instruction may instruct the target camera to capture a target image and return the target image to the control device.

[0140] In some embodiments, each time the control device sends a rotation instruction to the rotating platform, if a rotation completion response is received from the rotating platform, the rotation of the rotating platform can be determined to be complete. Of course, the control device can also use any other possible method to determine whether the rotating platform has completed rotation, and this application does not impose specific limitations on this.

[0141] In order to obtain sufficient corner information of the target image for further determination of internal parameters, the target calibration pattern included in the target image needs to be as complete as possible. In order to make the target calibration pattern as complete as possible, the angle of each rotation of the rotating platform needs to be very small, so that the target calibration pattern on the target calibration plate can be completely captured by the target camera. Taking the rotating platform rotating three times as an example, the three target images captured by the target camera can be sequentially captured as follows: Figure 10 As shown in (a)-(c).

[0142] Each time the target camera captures a target image, it can send that target image to the control device. The target camera can also send all captured target images to the control device after the control device no longer controls the rotation of the rotating platform. In this case, the capture instruction sent by the control device to the target camera can only be used to instruct the target camera to capture the target image. When the control device no longer needs to control the rotation of the rotating platform, it can send an image acquisition request to the target camera, instructing the target camera to send all target images to the control device. This image acquisition request can then be used to instruct the target camera to send all target images to the control device.

[0143] In some embodiments, the control device may first instruct the target camera to enable the recording function, and then control the rotation of the rotating platform multiple times. When the rotating platform is not required to rotate, a recording request instruction may be sent to the target camera, causing the target camera to send the recorded target video during the rotation process to the control device. This recording request instruction may be used to instruct the target camera to send the recorded video to the control device. After acquiring the target video, the control device may use any possible image recognition algorithm to obtain multiple target images in the target video. Different target images may include different locations of the calibration plate.

[0144] S2. The control device determines the pixel coordinates of multiple corner points in the target calibration pattern in each target image.

[0145] The corner points may be feature points with significant features in the target calibration pattern.

[0146] S2 may specifically include the following steps:

[0147] (1) First, the control device can use any feasible algorithm / method to determine all corner points in the target calibration pattern in the target image.

[0148] In one possible implementation, the corner points of the target calibration pattern in the target image may be determined by the following method:

[0149] In the case where the target calibration image is a checkerboard, determining the corner points of the target calibration pattern includes: setting a sliding window in the target calibration pattern; performing an offset operation on the sliding window, and determining a eigenvalue group for each pixel point in the sliding window. The eigenvalue group includes a first eigenvalue and a second eigenvalue, and the first eigenvalue and the second eigenvalue are used to characterize the grayscale change of the pixel points belonging to the eigenvalue group in two mutually perpendicular directions after the offset operation is performed; the grayscale change direction corresponding to the first eigenvalue is the direction in which the grayscale value changes fastest, and the grayscale change direction corresponding to the second eigenvalue is the direction in which the grayscale value changes least. In the case where the first target eigenvalue is greater than a first preset threshold and the second target eigenvalue is less than a second preset threshold, the pixel points belonging to the eigenvalue group are determined as edge points (i.e., points on the edge of the target calibration pattern); the first preset threshold is greater than the second preset threshold, the first target eigenvalue is the larger of the first eigenvalue and the second eigenvalue, and the second target eigenvalue is the smaller of the first eigenvalue and the second eigenvalue. When both the first eigenvalue and the second eigenvalue are less than the second preset threshold, and the difference between the first eigenvalue and the second eigenvalue is less than the third preset threshold, the pixel point to which the eigenvalue group belongs is determined as an in-grid point (i.e., a point within a small grid in the target calibration pattern). When both the first eigenvalue and the second eigenvalue are greater than the first preset threshold, and the difference between the first eigenvalue and the second eigenvalue is less than the third preset threshold, the pixel point to which the eigenvalue group belongs is determined as a corner point.

[0150] In the case where the target calibration image is a cross pattern, determining the corner points of the target calibration pattern includes: setting a sliding window in the target image; performing an offset operation on the sliding window, and determining a feature value group for each pixel point in the sliding window; the feature value group includes a first feature value and a second feature value. When the first target feature value is greater than a first preset threshold and the second target feature value is less than a second preset threshold, the pixel point belonging to the feature value group is determined as an edge point (i.e., a point on the edge of the target calibration pattern). When both the first feature value and the second feature value are greater than the first preset threshold and the difference between the first feature value and the second feature value is less than a third preset threshold, the pixel point belonging to the feature value group is determined as a corner point.

[0151] It should be noted that in the above method for determining corner points, the target image needs to be converted into a grayscale image first.

[0152] In the case where the target image is an array of dots, extracting the corner points of the target image includes: taking the centers of the dots in the target image as the corner points of the target image.

[0153] Of course, the above-mentioned implementation method of determining the corner points of the target calibration pattern in the target image is only an example. In practice, any other feasible method may be used, and this application does not impose any specific limitation on this.

[0154] (2) Furthermore, too many corner points will lead to excessive computational complexity in the subsequent internal parameter calculation process, and too many corner points will likely result in unreliable, duplicate, or invalid intersections. Therefore, after determining all corner points in the target image and deleting duplicate corner points, any feasible screening conditions can be used to filter out corner points with low reliability. For example, if the distance between a corner point and the target calibration pattern is less than a certain set value, the corner point is filtered out. For another example, if the distance between two corner points is less than a preset threshold, one of the corner points in the chain is deleted.

[0155] Of course, in practice, the diagonal point screening method may be any other possible implementation method, and this application does not impose any specific limitation on this.

[0156] (3) After all corner points in the target calibration pattern are determined, a pixel coordinate system can be established on the target image to obtain the pixel coordinates of each corner point.

[0157] In addition, the actions of establishing the pixel coordinate system and determining the pixel coordinates of the corner points can also be performed simultaneously during (1) execution, that is, the pixel coordinate system will be established at the same time as all the corner points are determined to obtain the pixel coordinates of the corner points.

[0158] S3. The control device calculates the internal parameters of the target camera according to the pixel coordinates of multiple corner points included in the target calibration object in all target images and the three-dimensional coordinates of each corner point in the world coordinate system of the target calibration plate.

[0159] The world coordinate system of the target calibration plate can be customized based on specific needs. Once defined, the world coordinate system will not change when the calibration solution provided by this application is executed. After the world coordinate system is determined, the corresponding three-dimensional coordinates of each point on all calibration plates can be determined. In some embodiments, the world coordinate system can also be referred to as the calibration plate coordinate system.

[0160] When executing S3, the control device can specifically calculate the internal parameters of the target camera based on the pixel coordinates of multiple corner points included in the target calibration object in all target images, and the three-dimensional coordinates of each corner point in the world coordinate system of the target calibration plate, using a minimization reprojection error algorithm.

[0161] Specifically, after obtaining the pixel coordinates and three-dimensional coordinates of the corner points, a matrix equation for the homography matrix can be constructed and solved to obtain the homography matrix. The homography matrix is ​​used to describe the mapping relationship between the world coordinate system and the pixel coordinate system. The rotation matrix and translation vector can then be decomposed from the homography matrix to obtain the rotation matrix and translation vector. The rotation matrix and translation vector are used to characterize the magnitude of the rotation and translation transformations required when transforming the world coordinate system to the pixel coordinate system. Next, based on the rotation matrix and translation vector, the remainder equation for the intrinsic parameter matrix of the target camera can be established. Solving these constraint equations can then yield the intrinsic parameter matrix of the target camera. This intrinsic parameter matrix can include internal parameters such as focal length, principal point coordinates, and pixel spacing. Based on the intrinsic parameter matrix of the target camera, algorithms such as reprojection error can be used to gradually optimize the distortion coefficients. Finally, the intrinsic parameter matrix and distortion coefficients can be continuously adjusted by minimizing the reprojection error, thereby obtaining more accurate internal parameters of the target camera.

[0162] After executing steps S1-S3 for each camera in the multi-camera perception suite, the internal parameters of each camera in the suite can be obtained. Of course, after executing S1, multiple target images corresponding to each camera in the suite can be obtained. Subsequently, steps S2 and S3 can be executed for each camera.

[0163] Based on the technical solutions corresponding to S1-S3 above, the control device can control the rotation of the rotating platform and the camera shooting, thereby quickly obtaining multiple target images for each camera. Based on these target images, the internal parameters of each camera in the multi-view perception suite can be accurately determined.

[0164] It should be noted that the methods for determining camera intrinsic parameters disclosed in S1-S3 above are only examples, and any other feasible methods may be adopted in practice, and this application does not impose any specific restrictions on this.

[0165] In addition, S901 may not be the first step to be executed in the camera calibration solution provided in this application, and may be executed at any other possible time, as long as it is executed before the internal parameters of the camera are needed later (such as step S903).

[0166] Since the calibration scheme provided by this application needs to determine the internal parameters of each camera in the multi-eye perception suite, it is also necessary to determine the external parameters between any two cameras (specifically, the posture transformation matrix). In this calibration system, the posture transformation matrix between any two cameras can be obtained by converting the posture transformation relationship between each camera in the any two cameras and its corresponding calibration plate through the posture transformation matrix between the calibration plates corresponding to the any two cameras. The posture transformation matrix between the calibration plates corresponding to any two cameras can be obtained by converting the posture transformation matrix between the two calibration plates and the ground.

[0167] Furthermore, since the relative position between the calibration plate and the ground is fixed throughout the calibration system, the pose transformation matrix between the calibration plate and the ground is also fixed. That is, the pose transformation matrix between the calibration plate and the ground remains unchanged, regardless of whether the rotating platform is rotated before or after. At the same time, the relative position between the rotating platform and each camera is also fixed, meaning the pose transformation matrix between the rotating platform and the camera remains the same, regardless of whether the rotating platform is rotated before or after.

[0168] Furthermore, whether before or after rotation, the pose transformation matrix between the rotating platform and the camera can be obtained by transforming the pose transformation matrix between the rotating platform and the ground, the pose transformation matrix between the ground and the calibration plate, and the pose transformation matrix between the calibration plate and the corresponding camera. If the three-dimensional coordinate system (specifically, a three-dimensional rectangular coordinate system) between the rotating platform and the ground is set to the same in the initial state, then in the initial state, the rotation matrix in the pose transformation matrix between the rotating platform and the ground is the identity matrix, and the translation vector is 0.

[0169] Furthermore, the rotation angle is known for each rotation after the initial state, so the difference between the pose transformation matrix between the rotating platform and the ground after each rotation and the pose transformation matrix between the rotating platform and the ground before the rotation is only the transformation of the rotation angle mapped in the rotation matrix.

[0170] For example, taking the case where the rotating platform rotates only in the horizontal direction by a preset angle, the positional relationship between the rotating platform and the ground in the three-dimensional rectangular coordinate system in the initial state and the three-dimensional rectangular coordinate system of the rotating platform and the ground after the rotating platform rotates by the preset angle can be referred to as Figure 11As shown. Among them, the Base-Y, Base-X and Base-Z axes are the three axes of the ground coordinate system of the ground (the three-dimensional rectangular coordinate system of the ground), the tt-Y, tt-X and tt-Z axes are the three axes of the rotating platform coordinate system (the three-dimensional rectangular coordinate system of the rotating platform) of the rotating platform in the initial state, and the tt'-Y, tt'-X and tt'-Z axes are the three axes of the rotating platform coordinate system after the rotating platform is rotated by a preset angle. It can be seen that before the rotation, the rotating platform and the ground share a three-dimensional rectangular coordinate system, and the two posture transformation matrices can only include one unit matrix. The horizontal rotation only changes the two axes in the three-dimensional rectangular coordinate system of the rotating platform. Based on the rotation angle, the transformation matrix of the three-dimensional rectangular coordinate system of the rotating platform and the three-dimensional rectangular coordinate system of the ground after the rotation can be easily obtained, and the posture transformation matrix of the rotating platform and the ground after the rotation is obtained.

[0171] In addition, the world coordinate system or the calibration plate coordinate system where the calibration plate is located is not affected by rotation. For the camera coordinate system, the positional relationship between the axis representing the vertical direction and Base-Y and tt-Y is not affected by rotation, and the other two axes are affected by rotation in the same way as the rotating platform coordinate system ( Figure 11 The rotated calibration plate coordinate system is not shown).

[0172] Based on the above, the pose transformation matrices between the rotating platform and the ground before and after rotation can be considered known. To obtain the pose transformation matrices between the ground and each calibration plate, the pose transformation matrices between each camera and its corresponding calibration plate before and after rotation are required.

[0173] After obtaining the pose transformation matrix between each camera and its corresponding calibration plate before and after rotation, the pose transformation matrix between the ground and each calibration plate, the pose transformation matrix between any two calibration plates, and the pose transformation matrix between any two cameras can be obtained in sequence according to the above logic. Using the initial state as the state before rotation is a computationally less complex approach. Therefore, after S901, it is necessary to first determine the pose transformation matrix between each camera in the multi-view perception suite and its corresponding calibration plate in the initial state, i.e., execute S902 and S903.

[0174] S902. In the initial state, the control device controls each camera in the multi-eye perception suite to shoot, obtains an initial calibration image corresponding to each camera, and determines the pixel coordinates of the first corner point of the calibration pattern in each initial calibration image.

[0175] The initial state refers to the state in which the calibration system is initially configured. In this initial state, each camera in the multi-camera perception suite has its entire corresponding calibration plate within its capture area. At this point, the camera is controlled to capture, and the resulting initial calibration image will contain the complete calibration pattern on the calibration plate corresponding to that camera. Alternatively, controlling each camera in the multi-camera perception suite to capture can be accomplished by controlling each camera in the multi-camera perception suite to capture its corresponding calibration plate.

[0176] In some implementations, if, during S901, each camera in the multi-view perception suite captures a target image of the corresponding calibration plate in the initial state, S902 can specifically use the target image of each camera in the initial state as the initial calibration image for each camera. For example, the target image of the first camera in the multi-view perception condition in the initial state can be used as the initial calibration image for the first camera; and the target image of the second camera in the multi-view perception condition in the initial state can be used as the initial calibration image for the second camera.

[0177] The specific implementation of determining the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image can refer to the relevant statements after S2 in the aforementioned embodiment and will not be repeated here. In addition, if during the execution of S901, in the initial state, each camera in the multi-eye perception kit captures the target image of the corresponding calibration plate and returns it to the control device, and the control device determines the pixel coordinates of the corner points of the target calibration pattern in the target image, then S902 can use the pixel coordinates of the corner points of the target calibration pattern in the target image of each camera in the initial state as the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image of each camera. At this time, the corner point of the target calibration pattern is the same as the first corner point of the calibration pattern in the initial calibration image.

[0178] In some embodiments, the control device of the electronic device can be completed through instructions. Based on this, combined with Figure 9 , refer to Figure 12 As shown, S902 may specifically include S1001-S1003:

[0179] S1001: A control device sends a first instruction to an electronic device.

[0180] Among them, the first instruction is used to instruct the electronic device to control each camera in the multi-eye perception kit to take pictures and send the initial calibration image obtained by taking pictures to the control device.

[0181] S1002: The electronic device receives a first instruction, and in response to the first instruction, controls each camera in the multi-eye perception kit to capture an initial calibration image, and sends the initial calibration image captured by each camera to the control device.

[0182] S1003: The control device receives initial calibration images captured by each camera of the electronic device, and determines the pixel coordinates of the first corner point of the calibration pattern in each initial calibration image.

[0183] In S1003 , the specific implementation of the control device determining the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image can refer to the relevant description after S2 in the above embodiment, which will not be repeated here.

[0184] Based on the technical solutions corresponding to the above S1001-S1003, the control device can complete the control of the camera in the electronic device through instructions, thereby obtaining the initial calibration images taken by each camera in the multi-eye perception kit of the electronic device and the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image, thereby providing a data basis for the execution of subsequent solutions.

[0185] S903. The control device determines the pose transformation matrix between each camera in the multi-camera perception suite and the calibration plate corresponding to each camera in the initial state based on the internal parameters of each camera in the multi-camera perception suite, the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate.

[0186] The calibration plate corresponding to the first corner point is the calibration plate associated with the calibration image in the initial calibration image, and is also the calibration plate associated with the camera associated with the initial calibration image. In other words, the first corner point of the calibration pattern in the initial calibration image corresponding to a particular camera is the same as the corner point on the calibration plate associated with that camera. The three-dimensional coordinates of the first corner point on the corresponding calibration plate refer to the coordinates of the first corner point in the world coordinate system (or calibration plate coordinate system) of the calibration plate.

[0187] Specifically, S903 means that for any camera in the multi-view perception suite, the pose transformation matrix between any camera and the calibration plate corresponding to any camera in the initial state will be determined based on the internal parameters of any camera, the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image of any camera, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate.

[0188] Taking the first camera among the multiple cameras included in the multi-camera perception suite as an example, in one possible implementation, S902 may specifically determine the pose transformation matrix between the first camera and the first calibration plate in the initial state using a PNP algorithm based on the internal parameters of the first camera, the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image of the first camera, and the three-dimensional coordinates of the first corner point on the first calibration plate. The first calibration plate is the first calibration plate corresponding to the first camera and also the calibration plate corresponding to the first corner point. The first corner point of the calibration pattern in the initial calibration image is the same as the corner point of the calibration pattern on the first calibration plate.

[0189] Using the PNP algorithm, determining the pose transformation matrix between the first camera and the first calibration plate in the initial state may specifically include the following steps:

[0190] (1) Based on the internal parameters of the first camera, the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image of the first camera are converted into two-dimensional coordinates in the image coordinate system.

[0191] Specifically, the pixel coordinates can be corrected using the distortion coefficients to eliminate the effect of distortion on the coordinates. For example, taking the distortion coefficients including radial distortion coefficients and tangential distortion coefficients as an example, eliminating the effect of distortion can be achieved according to the following formulas (3)-(6):

[0192] x j =x(1+k1r 2 +k2r 4 +k3r 6 );(3)

[0193] y j =y(1+k1r 2 +k2r 4 +k3r 6 ); (4)

[0194] Among them, (x, y) is the pixel coordinate after eliminating radial distortion, (x j , x j ) is the pixel coordinate without eliminating radial distortion, k1, k2, k3 are radial distortion coefficients, r 2 =x 2 +y 2 .

[0195] x q =x+[2p1xy+p2(r 2 +2x 2 )];(5)

[0196] y q =y+[p1(r 2 +2y 2 )+2p2xy];(6)

[0197] Among them, (x, y) is the pixel coordinate after eliminating tangential distortion, (x q , x q ) are the pixel coordinates without eliminating the tangential distortion, and p1 and p2 are the tangential distortion coefficients.

[0198] Then, using the focal length, pixel spacing, and principal point coordinates, the pixel coordinates without distortion are converted into two-dimensional coordinates in the image coordinate system. The conversion process can be performed using the following formulas (7) and (8):

[0199]

[0200] Among them, (x, y) is the two-dimensional coordinate in the converted image coordinate system, (u, v) is the pixel coordinate in the pixel coordinate system, (cx, cy) is the principal point coordinate, f is the focal length, dx is the horizontal pixel spacing, dy is the pixel spacing in the vertical direction, and K is the intrinsic parameter matrix.

[0201] (2) Using the PNP algorithm, the three-dimensional coordinates of the first corner point on the corresponding calibration plate and the two-dimensional coordinates of the first corner point of the calibration pattern in the initial calibration image in the image coordinate system are processed to obtain the pose transformation matrix between the first camera and the first calibration plate in the initial state.

[0202] The PNP algorithm may be any possible PNP algorithm, such as EPnP or P3P.

[0203] After executing the above steps (1) and (2) for each camera in the multi-camera perception suite, the pose transformation matrix between each camera in the multi-camera perception suite and the calibration plate corresponding to each camera in the initial state can be obtained.

[0204] After obtaining the material transformation matrix between each camera and the corresponding calibration plate in the initial state, the material transformation matrix between each camera and the corresponding calibration plate after rotation can be further obtained, that is, executing S904 and S905.

[0205] S904, after the control device controls the rotating platform to rotate by a preset angle based on the initial state, it controls each camera in the multi-eye perception kit to shoot, obtains the rotated calibration image corresponding to each camera, and determines the pixel coordinates of the second corner point of the calibration pattern in the rotated calibration image. In this application, the purpose of rotating the rotating platform is to obtain the image of the corresponding calibration plate captured by each camera after rotation, so in S904, the preset angle should not be too large, for example, it can be 5° or other angles so that after rotation, most or all of the calibration pattern of the calibration plate can be captured by the corresponding camera. In addition, the control device can specifically control the rotating platform to rotate by a preset angle in one direction based on the initial state.

[0206] Based on this, controlling each camera in the multi-eye perception suite to shoot in S904 may be: controlling each camera in the multi-eye perception suite to shoot a calibration plate corresponding to each camera.

[0207] In some embodiments, if during the execution of S901, each camera in the multi-eye perception kit captures a target image of the corresponding calibration plate when the rotating platform is rotated at a small angle (for example, a preset angle), S904 can directly use the target image as the rotation calibration image corresponding to the corresponding camera in S904.

[0208] The pixel coordinates of the second corner point of the calibration pattern in the rotated calibration image can be determined by referring to the relevant description after S2 in the aforementioned embodiment, which will not be repeated here. In addition, if during the execution of S901, each camera in the multi-eye perception kit captures the target image of the corresponding calibration plate while the rotating platform is rotated at a small angle (e.g., a preset angle) and determines the pixel coordinates of the corner point of the target calibration pattern in the target image, then in S904, the pixel coordinates of the corner point of the target calibration pattern in the target image can be used as the pixel coordinates of the second corner point of the calibration pattern in the rotated calibration image corresponding to the corresponding camera.

[0209] In some embodiments, the control device controls the rotating platform and the electronic device by sending corresponding instructions. Based on this, combined with Figure 9 , refer to Figure 12 As shown, S904 may specifically include S1004-S1009:

[0210] S1004: The control device sends a second instruction to the rotating platform.

[0211] The second instruction is used to instruct the rotating platform to rotate by a preset angle based on the initial state, and may specifically instruct the rotating platform to rotate by a preset angle in one direction based on the initial state.

[0212] S1005: The rotating platform receives a second instruction, and rotates by a preset angle based on the initial state in response to the second instruction.

[0213] The subsequent control of the electronic device by the control device needs to be performed after the rotation of the rotating platform is completed. Therefore, after the rotation of the rotating platform is completed, relevant information needs to be sent to the control device to notify the control device that the rotation has been completed. Based on this, S1006 is executed after S1005.

[0214] S1006: When the rotating platform completes rotating the preset angle based on the initial state, a first response is sent to the control device.

[0215] The first response is used to indicate that the rotating platform has completed its rotation.

[0216] S1007: The control device receives the first response, and sends a third instruction to the electronic device in response to the first response.

[0217] Among them, the third instruction is used to instruct the electronic device to control each camera in the multi-eye perception kit to shoot, obtain the rotation calibration image corresponding to each camera, and send the rotation calibration image corresponding to each camera to the control device.

[0218] S1008. The electronic device receives the third instruction, and in response to the third instruction, controls each camera in the multi-eye perception kit to shoot, and sends the rotation calibration image captured by each camera to the control device.

[0219] S1009: The control device receives the rotation calibration images corresponding to the respective cameras of the electronic device, and determines the pixel coordinates of the second corner point of the calibration pattern in each rotation calibration image.

[0220] In S1009 , the specific implementation of the control device determining the pixel coordinates of the second corner point of the calibration pattern in the rotated calibration image can refer to the relevant description after S2 in the above embodiment, which will not be repeated here.

[0221] Based on the technical solutions corresponding to S1004-S1009 above, the control device can complete the control of the rotating platform and the electronic device through instructions, thereby obtaining the rotation calibration images taken by each camera in the multi-eye perception kit of the electronic device when the rotating platform rotates to a preset angle, as well as the pixel coordinates of the second corner point of the calibration pattern in the rotation calibration image, thereby providing a data basis for the execution of subsequent solutions.

[0222] S905. The control device determines the pose transformation matrix between each camera in the multi-eye perception suite and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle based on the internal parameters of each camera in the multi-eye perception suite, the pixel coordinates of the second corner point in the rotation calibration image corresponding to each camera, and the three-dimensional coordinates of the second corner point on the corresponding calibration plate.

[0223] The calibration plate corresponding to the second corner point is the calibration plate associated with the calibration image in the rotated calibration image, and is also the calibration plate associated with the camera associated with the rotated calibration image. In other words, the second corner point of the calibration pattern in the rotated calibration image of a particular camera is the same as the corner point on the calibration plate associated with that camera.

[0224] Specifically, S905 means that for any camera in the multi-view perception kit, the pose transformation matrix between any camera and the calibration plate corresponding to any camera will be determined when the rotating platform rotates a preset angle based on the internal parameters of any camera, the pixel coordinates of the second corner point of the calibration pattern in the rotation calibration image of any camera, and the three-dimensional coordinates of the second corner point on the corresponding calibration plate.

[0225] The specific implementation of S905 can be described in detail after S903 in the above embodiment, and will not be repeated here.

[0226] After obtaining the pose transformation matrices between each camera and its corresponding calibration plate before and after the rotating platform rotates, the control device can further calculate the pose transformation matrices between the ground and each calibration plate based on the fact that the relative positions between the rotating platform and the camera do not change before and after the rotation. This is called S906.

[0227] S906. The control device calculates the pose transformation matrix between the ground and each calibration plate based on the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state and the pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle.

[0228] The following describes the specific implementation of S906, taking the multi-camera perception kit including the first camera and the calibration plate corresponding to the first camera as an example:

[0229] First, since the relative position between the rotating platform and the first camera does not change before and after the rotation, the following formula (9) exists:

[0230]

[0231] in, is the pose transformation matrix between the rotating platform and the first camera in the initial state, The pose transformation matrix between the rotating platform and the first camera when the rotating platform is rotated by a preset angle.

[0232] Secondly, according to the transformation relationship of the posture transformation matrix, the following formula (10) can be obtained:

[0233]

[0234] in, is the posture transformation relationship between the rotating platform and the ground in the initial state, is the pose transformation matrix between the ground and the first calibration plate, is the pose transformation matrix between the first calibration plate and the first camera, The pose transformation matrix between the rotating platform and the ground when the rotating platform rotates by a preset angle. It is the pose transformation matrix between the first calibration plate and the first camera when the rotating platform rotates by a preset angle.

[0235] In order to solve the equation (10) You can multiply both sides of the formula The inverse matrix of , and multiply both sides of the formula by The inverse matrix of , thus obtaining the following formula (11)

[0236]

[0237] In formula (11), Where I is the 3×3 identity matrix; Where R is the rotation matrix obtained based on the preset angle. In addition, and It has been obtained in step S905. Therefore, by solving formula (11), the pose transformation matrix between the ground and the first calibration plate can be obtained.

[0238] Similarly, the pose transformation matrix of the ground and any calibration plate can also be calculated according to the following formula (12):

[0239]

[0240] in, is the pose transformation matrix between the ground and the nth calibration plate, is the pose transformation matrix between the nth calibration plate and the nth camera, and the nth camera corresponds to the nth calibration plate. The pose transformation matrix between the nth calibration plate and the nth camera when the rotating platform is rotated by a preset angle. After obtaining the pose transformation matrix between the ground and all calibration plates, the pose transformation matrix between any two calibration plates can be further obtained. Furthermore, the pose transformation matrix between any two cameras can be combined with the pose transformation matrix between the initial state of each camera and the corresponding calibration plate to obtain the pose transformation matrix between any two cameras as an extrinsic parameter. Therefore, after S906, S907 and S908 are executed.

[0241] S907: The control device determines the pose transformation matrix between any two calibration plates based on the pose transformation matrices between the ground and all calibration plates.

[0242] The following describes the specific implementation of S907, taking the multi-camera perception kit as an example, which includes a first camera and a second camera. The calibration plate corresponding to the first camera is the first calibration plate, and the calibration plate corresponding to the second camera is the second calibration plate.

[0243] According to the transformation principle of the pose transformation matrix, the following formula (13) can be obtained:

[0244]

[0245] in, is the pose transformation matrix between the first calibration plate and the second calibration plate, is the pose transformation matrix between the first calibration plate and the ground, is the pose transformation matrix between the ground and the second calibration plate. In addition,

[0246] In formula (13), It has been calculated in S906, Based on the calculated We can then smoothly derive

[0247] Similarly, for any two calibration plates chartn and chartm, they can be calculated by referring to the following formula (14):

[0248]

[0249] in is the pose transformation matrix between the nth calibration plate and the mth calibration plate, is the pose transformation matrix between the nth calibration plate and the ground, is the pose transformation matrix between the ground and the mth calibration plate. In addition,

[0250] After obtaining the pose transformation matrix between any two calibration plates and the pose transformation matrix between each camera and its corresponding calibration plate, the pose transformation matrix between any two cameras can be obtained based on these data according to the matrix transformation principle. That is, S908 is executed after S907.

[0251] S908. The control device determines the pose transformation matrix between any two cameras based on the pose transformation matrix between the any two calibration plates and the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state.

[0252] The following describes the specific implementation of S908, taking the multi-camera perception kit as an example, which includes a first camera and a second camera. The calibration plate corresponding to the first camera is the first calibration plate, and the calibration plate corresponding to the second camera is the second calibration plate.

[0253] According to the transformation principle of the pose transformation matrix, the following formula (15) can be obtained:

[0254]

[0255] in, is the pose transformation matrix between the first camera and the second camera, is the pose transformation matrix between the first camera and the first calibration plate in the initial state, is the pose transformation matrix between the second calibration plate and the second camera in the initial state. In addition,

[0256] In formula (13), It has been calculated in S906, Based on the calculated We can then smoothly derive

[0257] Similarly, the pose transformation matrix between any two cameras can be calculated using the following formula (16):

[0258]

[0259] in, is the pose transformation matrix between the nth camera and the mth camera, is the pose transformation matrix between the n-th camera and the n-th calibration plate corresponding to the n-th camera, is the pose transformation matrix between the mth calibration plate and the mth camera corresponding to the mth calibration plate.

[0260] In addition, since the relative position relationship between different cameras in the multi-view perception kit is also fixed, in some embodiments, S908 can also determine the pose transformation matrix between any two cameras based on the pose transformation matrix between any two calibration plates and the pose transformation matrix between each camera and the calibration plates corresponding to each camera when the rotating platform rotates by a preset angle.

[0261] At this point, the pose transformation matrix between any two cameras can also be calculated using the following formula (17):

[0262]

[0263] in, The pose transformation matrix between the nth camera and the nth calibration plate corresponding to the nth camera when the rotating platform rotates a preset angle. is the pose transformation matrix between the mth calibration plate and the mth camera corresponding to the mth calibration plate when the rotating platform rotates a preset angle. In addition,

[0264] Based on the technical solution provided by the embodiment of the present application, by introducing rotation angle information through the rotating platform, the internal and external parameters of each camera in the multi-eye perception suite can be determined in combination with the pure visual information in the image captured by each camera. For pure visual information, the stability and uniqueness of the feature points contained therein are better, so the calibration scheme can make the internal and external parameters of the camera finally obtained more accurate. In addition, as long as there are enough feature points in the pure visual information in different scenarios, the calibration purpose can be successfully completed. This also makes the adaptability of the calibration scheme better. At the same time, since the calibration scheme does not determine the external parameters between cameras (i.e., the pose transformation matrix between different cameras) based on the difference between the images taken by different cameras of the same object, the calibration scheme provided by the present application can successfully obtain the external parameters between different cameras in the multi-eye perception suite regardless of whether there is a common view area between different cameras in the multi-eye perception suite. Furthermore, the calibration scheme provided by the present application does not have a strong constraint on the position of the calibration plate. It only needs to be adapted according to the layout position of the cameras in the multi-eye perception suite, and there is no specific limitation on the style of the calibration plate. In summary, the technical solution provided in this application can accurately calibrate the internal and external parameters of the camera in a variety of scenarios (such as production lines or maintenance points), regardless of whether there is a common view area between different cameras in the multi-eye perception kit. Compared with the existing camera calibration solution for multi-eye perception kits, it has higher adaptability.

[0265] For ease of understanding, the following Figure 13 The process of the camera calibration method provided in the embodiment of the present application is schematically illustrated.

[0266] like Figure 13 As shown, the process of the camera calibration method provided in the embodiment of the present application may include S1301-S1305:

[0267] S1301. The control device obtains the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image captured by each camera in the multi-view perception suite in the initial state; the initial state is the state in which the initial setting of the calibration system is completed.

[0268] In the embodiment of the present application, the specific implementation of S1301 can refer to the relevant descriptions of S902 and S1001-S1003 in the aforementioned embodiment, and will not be repeated here.

[0269] S1302. The control device determines the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state based on the internal parameters of each camera, the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate.

[0270] In some embodiments, S1301 may specifically include: the control device converts the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera into two-dimensional coordinates in the image coordinate system based on the internal parameters of each camera; the control device determines the posture transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state based on the internal parameters of each camera, the two-dimensional coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera in the image coordinate system, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate.

[0271] In the embodiment of the present application, the specific implementation of the steps included in S1301 can refer to the relevant description after S903 in the aforementioned embodiment, and will not be repeated here.

[0272] S1303: The control device obtains the pixel coordinates of the second corner point of the calibration pattern in the rotation calibration image captured by each camera when the rotating platform rotates by a preset angle.

[0273] In some embodiments, the specific implementation of S1301 can refer to the relevant descriptions of S904 and S1004-S1009 in the aforementioned embodiments, and will not be repeated here.

[0274] S1304. The control device determines the posture transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle based on the internal parameters of each camera, the pixel coordinates of the second corner point in the rotation calibration image corresponding to each camera, and the three-dimensional coordinates of the second corner point on the corresponding calibration plate.

[0275] In some embodiments, the specific implementation of S1301 can refer to the relevant description of S905 in the aforementioned embodiment, which will not be repeated here.

[0276] S1305. The control device determines the pose transformation matrix between any two cameras in the multi-camera perception suite based on the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state, and the pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle.

[0277] In some embodiments, S1301 may specifically include: the control device calculates the pose transformation matrix between the ground and each calibration plate based on the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state, and the pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle; the control device determines the pose transformation matrix between any two calibration plates based on the pose transformation matrix between the ground and each calibration plate; the control device determines the pose transformation matrix between any two cameras based on the pose transformation matrix between any two calibration plates, and the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state.

[0278] In the embodiment of the present application, the specific implementation of the specific steps included in S1301 can refer to the relevant descriptions of S906-S908 in the aforementioned embodiment, and will not be repeated here.

[0279] In addition, in the embodiment of the present application, before S1302, the camera calibration method further includes: controlling the device to determine the internal parameters of each camera in the multi-eye perception kit.

[0280] Based on the technical solution provided by the embodiment of the present application, by introducing the rotation angle information through the rotating platform, the external parameters (i.e., the pose transformation matrix) of each camera in the multi-eye perception suite can be determined in combination with the pure visual information in the image captured by each camera. For pure visual information, the stability and uniqueness of the feature points contained therein are better, so the calibration scheme can make the external parameters of the camera finally obtained more accurate. In addition, as long as there are enough feature points in the pure visual information in different scenes, the calibration purpose can be successfully completed. This also makes the adaptability of the calibration scheme better. At the same time, since the calibration scheme does not determine the external parameters between cameras (i.e., the pose transformation matrix between different cameras) based on the difference between the images captured by different cameras of the same object, regardless of whether there is a common view area between the different cameras in the multi-eye perception suite, the calibration scheme provided by the present application can smoothly obtain the external parameters between the different cameras in the multi-eye perception suite. Further, the calibration scheme provided by the present application does not have a strong constraint on the position of the calibration plate. It only needs to be adapted according to the layout position of the cameras in the multi-eye perception suite, and the style of the calibration plate is not specifically limited. In summary, the technical solution provided in this application can accurately calibrate the external parameters of the camera in a variety of scenarios (such as production lines or maintenance points), regardless of whether there is a common view area between different cameras in the multi-eye perception kit. Compared with the existing camera calibration solution for multi-eye perception kits, it has better adaptability and stability.

[0281] It is understandable that, in order to realize the above functions, the above control device includes hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0282] The embodiment of the present application can divide the functional modules of the above-mentioned control device according to the above-mentioned method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical functional division. In actual implementation, there may be other division methods.

[0283] In the case of dividing each functional module into corresponding functional modules, refer to Figure 14 As shown, the embodiment of the present application further provides a control device, which includes multiple sound-generating devices. The control device may include: an acquisition module 1401 and a processing module 1402.

[0284] Acquisition module 1401 is configured to acquire the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image captured by each camera in the multi-camera perception suite in the initial state; the initial state is the state in which the calibration system is initially configured. Processing module 1402 is configured to determine the pose transformation matrix between each camera and the corresponding calibration plate in the initial state based on the internal parameters of each camera, the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera acquired by acquisition module 1401, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate. Acquisition module 1401 is further configured to acquire the pixel coordinates of the second corner point of the calibration pattern in the rotated calibration image captured by each camera when the rotating platform is rotated by a preset angle. Processing module 1402 is further configured to determine the pose transformation matrix between each camera and the corresponding calibration plate when the rotating platform is rotated by a preset angle based on the internal parameters of each camera, the pixel coordinates of the second corner point in the rotated calibration image corresponding to each camera acquired by acquisition module 1401, and the three-dimensional coordinates of the second corner point on the corresponding calibration plate. The processing module 1402 is also used to determine the pose transformation matrix between any two cameras in the multi-view perception kit based on the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state, and the pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle.

[0285] Furthermore, the cooperation between acquisition module 1401 and processing module 1402 can implement any step of the camera calibration method provided in the aforementioned embodiments, and will not be further elaborated here. Regarding the electronic device in the aforementioned embodiments, the specific manner in which each module performs operations has been described in detail in the embodiment of the camera calibration method in the aforementioned embodiments, and will not be further elaborated here. The related beneficial effects can also be referred to the related beneficial effects of the aforementioned camera calibration method, and will not be further elaborated here.

[0286] The present application also provides a control device, which includes: a memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the control device executes the camera calibration method provided in the above embodiment. The specific structure of the control device can be referred to Figure 8 The structure of the control device shown in .

[0287] An embodiment of the present application further provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on a control device, the control device executes the camera calibration method provided in the aforementioned embodiment.

[0288] An embodiment of the present application further provides a computer program product, which includes executable instructions. When the computer program product is run on a control device, the control device executes the camera calibration method provided in the aforementioned embodiment.

[0289] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0290] In the several embodiments provided in this application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0291] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0292] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0293] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0294] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A camera calibration method, characterized in that: Applied to a calibration system, the calibration system includes a rotating platform, an electronic device equipped with a multi-eye perception kit, multiple calibration plates, and a control device that establishes a communication connection with the rotating platform and the electronic device, the electronic device is fixedly mounted on the rotating platform, the multi-eye perception kit includes multiple cameras, and a calibration plate is fixedly mounted in the shooting area of ​​each camera; the method includes: The control device obtains the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image captured by each camera in the multi-view perception suite in the initial state; the initial state is the state in which the initial setting of the calibration system is completed; The control device determines, in the initial state, a pose transformation matrix between each camera and the calibration plate corresponding to each camera based on internal parameters of each camera, pixel coordinates of a first corner point of a calibration pattern in an initial calibration image corresponding to each camera, and three-dimensional coordinates of the first corner point on the corresponding calibration plate; The control device obtains the pixel coordinates of the second corner point of the calibration pattern in the rotation calibration image captured by each camera when the rotating platform is rotated by a preset angle; The control device determines, based on internal parameters of each camera, pixel coordinates of a second corner point in a rotation calibration image corresponding to each camera, and three-dimensional coordinates of the second corner point on a corresponding calibration plate, a pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle; The control device determines the pose transformation matrix between any two cameras in the multi-camera perception kit based on the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state, and the pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle.

2. The method according to claim 1, characterized in that The control device obtains the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image captured by each camera in the multi-view perception suite in the initial state, including: The control device sends a first instruction to the electronic device; the first instruction is used to instruct the electronic device to control each camera in the multi-eye perception kit to shoot, and send the captured initial calibration image to the control device; The control device receives an initial calibration image corresponding to each camera of the electronic device, and determines the pixel coordinates of a first corner point of a calibration pattern in each of the initial calibration images.

3. The method according to claim 1 or 2, characterized in that The control device determines, in the initial state, a pose transformation matrix between each camera and the calibration plate corresponding to each camera based on internal parameters of each camera, pixel coordinates of a first corner point of a calibration pattern in an initial calibration image corresponding to each camera, and three-dimensional coordinates of the first corner point on a corresponding calibration plate, including: The control device converts the pixel coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera into two-dimensional coordinates in the image coordinate system based on the internal parameters of each camera; The control device determines the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state based on the internal parameters of each camera, the two-dimensional coordinates of the first corner point of the calibration pattern in the initial calibration image corresponding to each camera in the image coordinate system, and the three-dimensional coordinates of the first corner point on the corresponding calibration plate.

4. The method according to any one of claims 1 to 3, characterized in that The control device obtains the pixel coordinates of the second corner point of the calibration pattern in the rotation calibration image captured by each camera when the rotating platform rotates by a preset angle, including: The control device sends a second instruction to the rotating platform; the second instruction is used to instruct the rotating platform to rotate a preset angle; The control device receives a first response from the rotating platform and, in response to the first response, sends a third instruction to the electronic device; the first response is used to indicate that the rotating platform has completed rotation; the third instruction is used to instruct the electronic device to control each camera in the multi-eye perception kit to capture and send the captured rotation calibration image to the control device; The control device receives the rotation calibration images corresponding to the respective cameras from the electronic device, and determines the pixel coordinates of the second corner point of the calibration pattern in each of the rotation calibration images.

5. The method according to any one of claims 1 to 4, characterized in that: The control device determines a pose transformation matrix between any two cameras in the multi-camera perception suite based on a pose transformation matrix between each camera and a calibration plate corresponding to each camera in the initial state, and a pose transformation matrix between each camera and a calibration plate corresponding to each camera when the rotating platform rotates by a preset angle, including: The control device calculates a pose transformation matrix between the ground and each calibration plate based on a pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state and a pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle; The control device determines the pose transformation matrix between any two calibration plates according to the pose transformation matrix between the ground and each calibration plate; The control device determines the pose transformation matrix between any two cameras based on the pose transformation matrix between any two calibration plates and the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state.

6. The method according to any one of claims 1 to 5, characterized in that The control device calculates a pose transformation matrix between the ground and each calibration plate based on a pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state and a pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle, including: The control device calculates the pose transformation matrix between the ground and each calibration plate based on the pose transformation matrix between each camera and the calibration plate corresponding to each camera in the initial state and the pose transformation matrix between each camera and the calibration plate corresponding to each camera when the rotating platform rotates by a preset angle, using the following formula: in, is the posture transformation relationship between the rotating platform and the ground in the initial state, is the pose transformation matrix between the ground and the nth calibration plate, is the pose transformation matrix between the nth calibration plate and the nth camera, and the calibration plate corresponding to the nth camera is the nth calibration plate, is the posture transformation matrix between the rotating platform and the ground when the rotating platform rotates by a preset angle, The nth calibration plate is a pose transformation matrix between the nth calibration plate and the nth camera when the rotating platform is rotated by a preset angle; the nth calibration plate is one of the calibration plates corresponding to the multiple cameras, and the nth camera is any one of the multiple cameras.

7. The method according to any one of claims 1 to 6, characterized in that Before the control device determines, in the initial state, a pose transformation matrix between each camera and the calibration plate corresponding to each camera based on internal parameters of each camera, pixel coordinates of a first corner point of a calibration pattern in an initial calibration image corresponding to each camera, and three-dimensional coordinates of the first corner point on the corresponding calibration plate, the method further includes: The control device determines the internal parameters of each camera in the multi-eye perception kit.

8. A control device, characterized in that: include: A memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, causes the control device to perform the camera calibration method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The method comprises computer instructions, which, when executed on a control device, enable the control device to execute the camera calibration method according to any one of claims 1 to 7.

10. A calibration system, characterized in that: The system comprises a rotating platform, an electronic device equipped with a multi-eye perception kit, a plurality of calibration plates, and a control device that establishes a communication connection with the rotating platform and the electronic device; the electronic device is fixedly mounted on the rotating platform, the multi-eye perception kit includes a plurality of cameras, and a calibration plate is fixedly mounted in the shooting area of ​​each camera; The control device is configured to execute the camera calibration method according to any one of claims 1 to 7.

Images