Image acquisition device and rotating shaft calibration method and device, and storage medium

Abstract

This application provides a calibration method, apparatus, and storage medium for an image acquisition device and a rotation axis. In this embodiment, the image acquisition device and the rotation axis are rigidly connected. Images containing a calibration plate are acquired at two rotation angles, and image features are extracted. Based on the initial pose information of the calibration plate and the initial pose information of the rotation axis and the image acquisition device, the original features of the calibration plate are projected onto the image to obtain projection features. A reprojection error is established based on the projection features and image features, and an error function is constructed. The error function is iteratively optimized until convergence, yielding the optimized relative pose information of the rotation axis and the image acquisition device. This method fully utilizes the motion characteristics of the image acquisition device, establishing a reprojection error based on projection features and image features, and constructing an error function, which can improve the stability of the algorithm and the accuracy of the image acquisition device and rotation axis calibration.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a calibration method, apparatus and storage medium for an image acquisition device and a rotating axis. Background Technology

[0002] Calibration refers to the precise measurement of the relative pose parameters between individual sensors or between a sensor and a rotation axis. The calibration results between sensors and the rotation axis can be applied to many aspects, such as floor plan generation, point cloud registration, model accuracy, and texture mapping. For example, in the field of panoramic image stitching technology, VR acquisition devices are used to collect indoor environmental information. The image acquisition device is rigidly connected to a rotation axis. When acquiring environmental images indoors, the rotation axis is driven to rotate, causing the image acquisition device to collect environmental images from different angles. These images are then fused together to obtain the overall indoor environmental image. Throughout this process, the relative transformation relationship between the image acquisition device and the rotation axis needs to be obtained, i.e., the calibration of the image acquisition device and the rotation axis.

[0003] In existing technologies, the angle and dimensions between the image acquisition device and the rotation axis need to be set manually. The relative transformation relationship between the image acquisition device and the rotation axis is obtained based on this angle and dimensions. However, the accuracy of the relative transformation relationship obtained in this way is low, which affects the accuracy of image fusion. Summary of the Invention

[0004] This application provides a method, apparatus, and storage medium for calibrating an image acquisition device and a rotating axis, thereby improving the accuracy of calibration between the image acquisition device and the rotating axis.

[0005] This application provides a calibration method for an image acquisition device and a rotation axis, wherein the image acquisition device and the rotation axis are rigidly connected and both are located within a target spatial object. The method includes:

[0006] S1. When the rotation axis is at rotation angle A1 and rotation angle A2, real scene images P1 and P2 containing the calibration board are acquired by the image acquisition device respectively. The calibration board has at least four original features F. Each original feature F has a corresponding image feature in real scene image P1 and real scene image P2. The calibration board has target pose information.

[0007] S2. Based on at least four original features F and their corresponding image features in the real scene image, perform pose estimation on the calibration board and the image acquisition device to obtain the relative pose information T0 between the calibration board and the image acquisition device.

[0008] S3. Based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation angle of the rotation axis and the relative pose information T0, determine the initial pose information Pb of the calibration plate.

[0009] S4. Based on the initial pose information Pb of the calibration board, project the original target feature F0 in the calibration board onto the real scene image P1 and the real scene image P2 respectively to obtain the projected feature f3 and the projected feature f4. The original target feature F0 is any original feature in the calibration board.

[0010] S5. Based on the target image feature f10 corresponding to the original target feature F0 in the real scene image P1, and the target image feature f20, projection feature f3 and projection feature f4 corresponding to the original target feature F0 in the real scene image P2, respectively establish the reprojection error e1 = f10 - f3 and the reprojection error e2 = f20 - f4 corresponding to the original target feature F0;

[0011] S6. Based on the sum of the squares of the reprojection errors e1 and e2, establish the error function to be optimized. Iterate and optimize the error function until convergence to obtain the target relative pose information between the image acquisition device and the rotation axis after optimization, so as to realize the calibration between the image acquisition device and the rotation axis.

[0012] This application embodiment also provides a calibration device for an image acquisition device and a rotation axis. The image acquisition device and the rotation axis are rigidly connected and both are located in a target space object. The device includes: an acquisition module, an estimation module, a determination module, a projection module, a construction module, and an optimization module.

[0013] The acquisition module is used to acquire real-scene images P1 and P2 containing the calibration board based on the image acquisition device when the rotation axis is at rotation angle A1 and rotation angle A2 respectively. The calibration board has at least four original features F, and each original feature F has a corresponding image feature in real-scene image P1 and real-scene image P2. The target pose information of the calibration board is also included.

[0014] The estimation module is used to estimate the pose of the calibration board and the image acquisition device based on at least four original features F and their corresponding image features in the real scene image, so as to obtain the relative pose information T0 between the calibration board and the image acquisition device.

[0015] The determination module is used to determine the initial pose information Pb of the calibration plate based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation angle of the rotation axis and the relative pose information T0.

[0016] The projection module is used to project the original target feature F0 in the calibration board onto the real scene image P1 and the real scene image P2 respectively, based on the initial pose information Pb of the calibration board, to obtain the projected feature f3 and the projected feature f4. The original target feature F0 is any original feature in the calibration board.

[0017] The construction module is used to establish the reprojection error e1 = f10 - f3 and the reprojection error e2 = f20 - f4 corresponding to the original feature F0 in the real image P1, and the target image feature f20, projection feature f3 and projection feature f4 corresponding to the original feature F0 in the real image P2, respectively.

[0018] The optimization module is used to establish an error function to be optimized based on the sum of the squares of the reprojection errors e1 and e2. The error function to be optimized is iteratively optimized until convergence, so as to obtain the target relative pose information between the image acquisition device and the rotation axis after optimization, so as to realize the calibration between the image acquisition device and the rotation axis.

[0019] This application embodiment also provides an image acquisition device and a calibration device for a rotating axis, including: a memory and a processor; the memory for storing a computer program; and the processor, coupled to the memory, for executing the computer program to implement the steps in the image acquisition device and rotating axis calibration method provided in this application embodiment.

[0020] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the calibration method for the image acquisition device and the rotation axis provided in this application.

[0021] In this embodiment, the image acquisition device is rigidly connected to the rotation axis. Images containing the calibration plate are acquired at two rotation angles, and image features are extracted. Based on the initial pose information of the calibration plate and the initial pose information of the rotation axis and the image acquisition device, the original features of the calibration plate are projected onto the image to obtain projection features. A reprojection error is established based on the projection features and image features, and an error function is constructed. The error function is iteratively optimized until convergence, resulting in the optimized relative pose information of the rotation axis and the image acquisition device. This approach fully utilizes the motion characteristics of the image acquisition device. Establishing a reprojection error and constructing an error function based on projection features and image features improves the stability of the algorithm and the accuracy of the calibration of the image acquisition device and the rotation axis. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0023] Figure 1 A schematic flowchart illustrating the calibration method for an image acquisition device and a rotation axis provided in an exemplary embodiment of this application;

[0024] Figure 2A schematic diagram of the structure of the image acquisition device and the calibration device for the rotating axis provided in an exemplary embodiment of this application;

[0025] Figure 3 A schematic diagram of the structure of the image acquisition device and the calibration device for the rotating axis provided in an exemplary embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0028] To address the issue of low accuracy in determining the calibration parameters between the image acquisition device and the rotation axis, in this embodiment of the application,

[0029] The following describes in detail a solution provided by an embodiment of this application, with reference to the accompanying drawings.

[0030] Figure 1 This is a flowchart illustrating a calibration method for an image acquisition device and a rotation axis, provided as an exemplary embodiment of this application. The image acquisition device and the rotation axis are rigidly connected and both are located within a target spatial object. Figure 1 As shown, the method includes:

[0031] S1. When the rotation axis is at rotation angle A1 and rotation angle A2, real scene images P1 and P2 containing the calibration board are acquired by the image acquisition device respectively. The calibration board has at least four original features F. Each original feature F has a corresponding image feature in real scene image P1 and real scene image P2. The calibration board has target pose information.

[0032] S2. Based on at least four original features F and their corresponding image features in the real scene image, perform pose estimation on the calibration board and the image acquisition device to obtain the relative pose information T0 between the calibration board and the image acquisition device.

[0033] S3. Based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation angle of the rotation axis and the relative pose information T0, determine the initial pose information Pb of the calibration plate.

[0034] S4. Based on the initial pose information Pb of the calibration board, project the original target feature F0 in the calibration board onto the real scene image P1 and the real scene image P2 respectively to obtain the projected feature f3 and the projected feature f4. The original target feature F0 is any original feature in the calibration board.

[0035] S5. Based on the target image feature f10 corresponding to the original target feature F0 in the real scene image P1, and the target image feature f20, projection feature f3 and projection feature f4 corresponding to the original target feature F0 in the real scene image P2, respectively establish the reprojection error e1 = f10 - f3 and the reprojection error e2 = f20 - f4 corresponding to the original target feature F0;

[0036] S6. Based on the sum of the squares of the reprojection errors e1 and e2, establish the error function to be optimized. Iterate and optimize the error function until convergence to obtain the target relative pose information between the image acquisition device and the rotation axis after optimization, so as to realize the calibration between the image acquisition device and the rotation axis.

[0037] In this embodiment, the image acquisition device and the rotation axis are located within a target spatial object in physical space. Physical space refers to a specific spatial region containing multiple spatial objects; in other words, multiple spatial objects constitute the physical space. For example, physical space refers to a house, and the multiple spatial objects included in the house could be a kitchen, bedroom, living room, or bathroom, etc. The target spatial object can be any one of these multiple spatial objects.

[0038] In this embodiment, the image acquisition device can be a fisheye camera or the like. The image acquisition device is rigidly connected to the rotation axis and rotates with the rotation axis. During the rotation of the rotation axis, the image acquisition device acquires the real-world environment in the target space object. For example, the real-world image can be a fisheye image.

[0039] In practical applications, considering factors such as equipment tolerances, material deformation, or measurement errors, the initial calibration parameters between the image acquisition device and the rotating axis are not accurate, which may affect the subsequent image fusion results. In this embodiment, a calibration plate is used to assist in the calibration between the image acquisition device and the rotating axis. The calibration plate is a flat plate with a fixed-spacing pattern array, such as a checkerboard calibration plate or a circular calibration plate. Regardless of the type of calibration plate, it includes original features. For example, the original features on the checkerboard can be the corner points where black and white blocks meet, and the original features on the circular calibration plate can be the point where the center of the circle is located.

[0040] In this embodiment, regarding step S1, the calibration board, the image acquisition device, and the rotation axis are all located within the target space object. The calibration board is located within the target space object and possesses target pose information, enabling the image acquisition device to acquire a real-world image containing the calibration board. The target pose information may include a target position and a target angle. The target position is the location of the calibration board within the target space object, and the target angle is the angle of the calibration board within the target space object. When the calibration board is at this target position and at this target angle, the image acquisition device can acquire a real-world image containing the calibration board. This angle corresponds to a three-degree-of-freedom spatial rotation.

[0041] In this scenario, when the calibration board possesses target pose information within the target space object, and the rotation axis is at rotation angle A1, the image acquisition device acquires a real-world image P1 containing the calibration board. Maintaining the target pose information of the calibration board, the rotation axis drives the image acquisition device to rotate. When the rotation axis is at rotation angle A2, the image acquisition device acquires a real-world image P2 containing the calibration board. The difference between rotation angles A1 and A2 is not limited; it applies wherever the calibration board possesses target pose information and the image acquisition device can acquire a real-world image containing the calibration board. Preferably, the difference between rotation angles A1 and A2 is between 40 and 60 degrees. Each original feature F corresponds to an image feature in both real-world image P1 and real-world image P2. For subsequent calculations, a certain number of original features are required. Preferably, the number of original features on the calibration board is at least four, and correspondingly, the original feature F also has at least four corresponding image features in real-world image P1. Among them, the original feature F has world coordinates (i.e. standard coordinates) in three-dimensional space, and the image feature has image coordinates in the image coordinate system.

[0042] Optionally, a feature detection function can be used to detect image feature f1 corresponding to the original features in real-world image P1, and image feature f2 corresponding to the original features in real-world image P2. For example, the findChessboardCorners() function can be used to detect the inner corner positions of the chessboard grid, and the inner corner positions detected in real-world image P1 can be used as image feature f1, and the inner corner positions detected in real-world image P2 can be used as image feature f2.

[0043] In this embodiment, for step S2, the pose of the calibration board and the image acquisition device can be estimated based on at least four original features F on the calibration board and their corresponding image features in the real-world image, to obtain the relative pose information T0 between the calibration board and the image acquisition device. The pose estimation method can be the Perspective-n-Point (PnP) algorithm, which solves for the correspondence between the original features and the image features, i.e., the relative pose information between the calibration board and the image acquisition device.

[0044] Optionally, based on at least four original features F and their corresponding image features f1 in the real-scene image P1, pose estimation is performed on the calibration board and the image acquisition device to obtain relative pose information T1; the relative pose information T1 is used as the relative pose information T0 between the calibration board and the image acquisition device; or based on at least four original features F and their corresponding image features f2 in the real-scene image P2, pose estimation is performed on the calibration board and the image acquisition device to obtain relative pose information T2, and the relative pose information T2 is used as the relative pose information T0 between the calibration board and the image acquisition device.

[0045] In this embodiment, there is initial pose information P0 between the image acquisition device and the rotation axis. This initial pose information P0 can be obtained by manual measurement or calculated according to the dimensional relationship and relative positional relationship between the image acquisition device and the rotation axis. The initial pose information P0 is the initial value of the four degrees of freedom to be determined.

[0046] For step S3, based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation angle of the rotation axis and the relative pose information T0, the initial pose information Pb of the calibration plate is determined.

[0047] Optionally, based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation matrix R1 corresponding to the rotation angle A1, and the relative pose information T1, the initial pose information Pb of the calibration plate is determined, where Pb = inverse(P0*R1)*T1, inverse(·) represents inverse transformation, and "*" represents matrix multiplication. If the rotation axis is considered as the Y-axis of the standard coordinate system, the rotation angle A1 can be transformed into a rotation matrix R1 for a rigid rotation around the Y-axis with a single degree of freedom. Alternatively, based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation matrix R2 corresponding to the rotation angle A2, and the relative pose information T2, the initial pose information Pb of the calibration plate is determined, where Pb = inverse(P0*R2)*T2, inverse(·) represents inverse transformation, and "*" represents matrix multiplication. Where the rotation axis is considered as the Y-axis of the standard coordinate system, the rotation angle A2 is a rotation matrix R2 for a rigid rotation around the Y-axis with a single degree of freedom.

[0048] For step S4, based on the initial pose information Pb of the calibration board and the rotation matrix R1, the original target feature F0 in the calibration board is projected onto the real scene image P1 to obtain the projected feature f3; based on the initial pose information Pb of the calibration board and the rotation matrix R2, the original target feature F0 in the calibration board is projected onto the real scene image P2 to obtain the projected feature f4. The original target feature F0 is any original feature in the calibration board.

[0049] Optionally, based on the initial pose information Pb of the calibration board, the initial pose information P0 of the image acquisition device and the rotation axis, and the rotation angle A1, the original target feature F0 in the calibration board is projected onto the real-world image P1 to obtain the projected feature f3 = K*(P0*R1)*Pb*F0, where K is the intrinsic parameter of the image acquisition device, "*" indicates matrix multiplication, and R1 is the rotation matrix corresponding to the rotation angle A1; based on the initial pose information Pb of the calibration board, the initial pose information P0 of the image acquisition device and the rotation axis, and the rotation angle A2, the original target feature F0 in the calibration board is projected onto the real-world image P2 to obtain the projected feature f4 = K*(P0*R2)*Pb*F0, where R2 is the rotation matrix corresponding to the rotation angle A2.

[0050] For step S5, based on the target image features f10 and f20, projection features f3 and f4 corresponding to the original target feature F0, the reprojection errors e1 = f10 - f3 and e2 = f20 - f4 corresponding to the original target feature F0 are established respectively.

[0051] For step S6, an error function to be optimized is established based on the sum of the squares of the reprojection errors e1 and e2. For example, the sum of the squares of the reprojection errors e1 and e2 can be directly used as an error term, and the error function includes this error term. The initial pose information P0 and Pb contained in the error function are iteratively optimized until convergence, resulting in the optimized target relative pose information between the image acquisition device and the rotation axis, thereby achieving calibration between the image acquisition device and the rotation axis. The iterative optimization of the error function can be achieved by constructing a least-squares problem to obtain the optimized target relative pose information between the image acquisition device and the rotation axis.

[0052] In this embodiment, the image acquisition device is rigidly connected to the rotation axis. Images containing the calibration plate are acquired at two rotation angles, and image features are extracted. Based on the initial pose information of the calibration plate and the initial pose information of the rotation axis and the image acquisition device, the original features of the calibration plate are projected onto the image to obtain projection features. A reprojection error is established based on the projection features and image features, and an error function is constructed. With the goal of the error function value being less than a set error threshold, the initial pose information is iteratively optimized to obtain the optimized relative pose information of the rotation axis and the image acquisition device. This approach fully utilizes the motion characteristics of the image acquisition device, establishing a reprojection error based on projection features and image features, and constructing an error function, which can improve the stability of the algorithm and the accuracy of the calibration of the image acquisition device and the rotation axis.

[0053] In an optional embodiment, to improve the accuracy of the calibration parameters, i.e., the relative pose information between the image acquisition device and the rotation axis, a reprojection error can be established for each original feature. An error term corresponding to each original feature is established based on the reprojection error. The error terms corresponding to multiple original features are accumulated to obtain the error function to be optimized, thereby optimizing the error function. The method provided in this application embodiment further includes: traversing at least four original features contained in the calibration board, taking each original feature as a target original feature, and obtaining at least four target original features; for each target original feature, executing steps S4-S5 to obtain the reprojection error e1 and reprojection error e2 corresponding to each target original feature; correspondingly, the specific implementation of establishing the error function to be optimized based on the sum of the squares of the reprojection errors e1 and e2 in step S6 is as follows: taking the sum of the squares of the reprojection errors e1 and e2 corresponding to each target original feature as the error term corresponding to that target original feature; accumulating the error terms corresponding to the at least four target original features to obtain the error function to be optimized.

[0054] In an optional embodiment, based on the traversal of the original features, the target pose information of the calibration board can be changed so that the calibration board covers as many positions as possible in the real-world images P1 and P2. For example, the pose information of the calibration board can be adjusted, such as adjusting the position information or angle information of the calibration board individually, or adjusting the position and angle of the calibration board simultaneously, to obtain target pose information for multiple calibration boards; for the target pose information of each calibration board, the rotation angle of the rotation axis can be adjusted so that the image acquisition device can acquire real-world images P1 and P2 containing the calibration board at rotation angles R1 and R2 respectively, thereby improving the calibration accuracy.

[0055] Specifically, the calibration board has multiple target pose information within the target space object. The multiple target pose information are traversed, and steps S1-S6 are executed for each target pose information. During each execution of steps S4-S6, before executing step S4, at least four original features contained in the calibration board are traversed; each original feature is used as a target original feature, resulting in at least four target original features; for each target original feature, steps S4-S5 are executed to obtain the reprojection error e1 and reprojection error e2 corresponding to each target original feature; for step S6, the reprojection errors e1 and e2 corresponding to each target original feature are... The sum of squares is used as the error term corresponding to the original feature of the target; the error terms corresponding to at least four original features of the target are accumulated to obtain the sub-function to be optimized for each target pose information; the sub-functions to be optimized corresponding to multiple target pose information are accumulated to obtain the error function to be optimized; with the value of the error function to be optimized being less than the set error threshold as the objective, the initial pose information P0 and the initial pose information Pb are iteratively optimized to obtain the target relative pose information between the image acquisition device and the rotation axis after optimization, so as to realize the calibration between the image acquisition device and the rotation axis.

[0056] It should be noted that the initial pose information P0 of the image acquisition device and the rotation axis remains unchanged, but the position of the calibration plate changes, requiring the introduction of new initial pose information Pb for the calibration plate. The initial pose information P0 has 4 degrees of freedom, namely X-axis rotation, Z-axis rotation, X-axis translation, and Z-axis translation, while the other two degrees of freedom (Y-axis translation and Y-axis rotation) are 0, which is equivalent to treating the rotation axis as the Y-axis of the standard coordinate system. Pb has 6 degrees of freedom for unconstrained rigid motion. Therefore, when optimizing the error function, the total number of variables to be calculated for n sets of data is 6n+4.

[0057] In an optional embodiment, when the rotating axis is rigidly connected to the image acquisition device, the rotating axis can also be rigidly connected to the lidar acquisition device. The device consisting of the rotating axis, the image acquisition device, and the lidar acquisition device can be referred to as a VR acquisition device. The installation positions of the image acquisition device and the lidar acquisition device are not limited. For example, the image acquisition device and the lidar acquisition device can have a certain angle in the horizontal direction, such as 90 degrees, 180 degrees, or 270 degrees, and a certain distance in the vertical direction, such as 0 cm, 1 cm, or 5 cm. Both the image acquisition device and the lidar acquisition device are rigidly connected to the rotating axis, which is connected to a motor. The rotating axis can rotate 360 ​​degrees, driving the image acquisition device and the lidar acquisition device to rotate. During the rotation of the rotating axis, the lidar acquisition device acquires the 3D point cloud dataset corresponding to the target spatial object, and the image acquisition device acquires the image of the target spatial object.

[0058] In this embodiment, the calibration scheme for the VR acquisition device is divided into three types: ① calibration between the lidar acquisition device and the rotating axis; ② calibration between the image acquisition device and the rotating axis (i.e., calibration between the lidar acquisition device and the rotating axis). Figure 1 (The scheme shown); ③ Calibration between the lidar acquisition device and the image acquisition device. Among them, the calibration processes of ① and ② are independent of each other, while ③ requires the calibration results of ① and ②.

[0059] Among them, ③ mainly transforms the target point cloud data collected by the lidar acquisition device into the image coordinate system to obtain the projected point cloud data, and aligns the projected point cloud data with the target real scene image collected by the image acquisition device. The projected point cloud data and the target real scene image only need to be rotated or translated relative to the rotation axis, that is, the alignment is achieved in two degrees of freedom (Y-axis rotation and Y-axis translation). Alignment means that the feature points in the projected point cloud data correspond to the image feature points in the real scene image.

[0060] Specifically:

[0061] Step A1: Assume the LiDAR acquisition device has been calibrated to the rotation axis (i.e., ① is complete), and the calibration between the image acquisition device and the rotation axis is also complete (i.e., ② is complete). The term to be determined is the relative pose information between the LiDAR acquisition device and the image acquisition device, i.e., the rigid transformation T of the point cloud coordinate system relative to the image coordinate system. Since both the point cloud coordinate system and the image coordinate system are based on the rotation axis (e.g., the Y-axis), this rigid transformation T includes two degrees of freedom. These two degrees of freedom are represented by a rotation matrix (Y-axis rotation) and a translation matrix (Y-axis translation). These two degrees of freedom can be assigned values ​​through interactive operations, such as assigning values ​​to the rotation matrix and translation matrix through interactive methods such as keyboard, mouse, or touch screen.

[0062] Step A2: Based on the assignment results of Step 1, the rigid transformation T formed by the combination of the two-degree-of-freedom rotation matrix and translation matrix can be obtained. That is, in response to the interactive operation, the relative pose information between the lidar acquisition device and the image acquisition device is continuously assigned.

[0063] Step A3: Based on the assigned relative pose information, and combined with the target relative pose information between the rotation axis and the image acquisition device, project the target point cloud data onto the image coordinate system to obtain projected point cloud data. For example, based on the target relative pose information P between the image acquisition device and the rotation axis, construct a projection matrix K*P*T, and project the target point cloud data onto the image coordinate system to obtain projected point cloud data. K is the camera intrinsic parameter.

[0064] Among them, target point cloud data is a feature that is easy to visualize. For example, it can be the extracted corner / line features, or the color result of mapping the intensity / depth values ​​of all points to a heatmap.

[0065] Step A4: Continuously assign values ​​to the relative pose information between the LiDAR acquisition device and the image acquisition device through interactive operations until the projected point cloud data is aligned with the target real-world image. For example, alignment can be achieved by mapping point cloud line features to image edges, and intensity / depth values ​​to the surface distribution of objects in the image. Once the projected point cloud data is aligned with the target real-world image, the assigned values ​​to the relative pose information between the LiDAR acquisition device and the image acquisition device are used as the target relative pose information between the LiDAR acquisition device and the image acquisition device.

[0066] Regarding the embodiments provided in this application Figure 1 The detailed implementation methods and beneficial effects of each step in the method shown have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0067] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps S1 to S6 can be device A; or the execution subject of steps S1 and S3 can be device A, and the execution subject of steps S4 to S6 can be device B; and so on.

[0068] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as S1, S2, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0069] In addition to providing method embodiments, this application also provides an image acquisition device and a calibration device for a rotation axis. The process of the image acquisition device and the calibration device for a rotation axis provided in this application embodiment will be described below.

[0070] Figure 2 This is a schematic diagram of the structure of an image acquisition device and a calibration device for a rotation axis, provided as an exemplary embodiment of this application. The image acquisition device and the rotation axis are rigidly connected and both are located within a target spatial object. Figure 2 The device includes: an acquisition module 21, an estimation module 22, a determination module 23, a projection module 24, a construction module 25, and an optimization module 26;

[0071] The acquisition module 21 is used to acquire real-scene images P1 and P2 containing the calibration board based on the image acquisition device when the rotation axis is at rotation angle A1 and rotation angle A2. The calibration board has at least four original features F, and each original feature F corresponds to an image feature in real-scene image P1 and real-scene image P2. The calibration board target pose information is also acquired.

[0072] The estimation module 22 is used to estimate the pose of the calibration board and the image acquisition device based on at least four original features F and their corresponding image features in the real scene image, so as to obtain the relative pose information T0 between the calibration board and the image acquisition device.

[0073] The determination module 23 is used to determine the initial pose information Pb of the calibration plate based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation angle of the rotation axis and the relative pose information T0.

[0074] The projection module 24 is used to perform projection operations. Specifically, according to the initial pose information Pb of the calibration board, the original target feature F0 in the calibration board is projected onto the real scene image P1 and the real scene image P2 respectively to obtain the projection feature f3 and the projection feature f4. The original target feature F0 is any original feature in the calibration board.

[0075] The construction module 25 is used to perform construction operations. Specifically, based on the target image feature f10 corresponding to the original target feature F0 in the real scene image P1, and the target image feature f20, projection feature f3 and projection feature f4 corresponding to the original target feature F0 in the real scene image P2, the reprojection error e1 = f10 - f3 and the reprojection error e2 = f20 - f4 corresponding to the original target feature F0 are established respectively.

[0076] The optimization module 26 is used to establish an error function to be optimized based on the sum of the squares of the reprojection errors e1 and e2, and to iteratively optimize the error function until convergence, so as to obtain the target relative pose information between the image acquisition device and the rotation axis after optimization, so as to realize the calibration between the image acquisition device and the rotation axis.

[0077] In an optional embodiment, the calibration device for the image acquisition device and the rotation axis further includes: a traversal module; the traversal module is used to traverse at least four original features contained in the calibration board, taking each original feature as a target original feature, to obtain at least four target original features; for each target original feature, the projection module and the construction module respectively perform projection operation and construction operation to obtain the reprojection error e1 and reprojection error e2 corresponding to each target original feature.

[0078] Accordingly, when the optimization module establishes the error function to be optimized based on the sum of the squares of the reprojection errors e1 and e2, it specifically uses the following steps: taking the sum of the squares of the reprojection errors e1 and e2 corresponding to each original feature of the target as the error term corresponding to that original feature; and summing the error terms corresponding to at least four original features of the target to obtain the error function to be optimized.

[0079] In an optional embodiment, the calibration device for the image acquisition device and the rotation axis further includes:

[0080] The calibration board has multiple target pose information in the target space object. The traversal module iterates through these multiple target pose information. For each target pose information of the calibration board, the acquisition module 21, estimation module 22, determination module 23, projection module 24, construction module 25, and optimization module 26 sequentially execute the required actions. Before the projection module performs the projection operation, at least four original features contained in the calibration board are traversed, and each original feature is used as a target original feature, resulting in at least four target original features. For each target original feature, the projection module performs a projection operation, and the construction module performs a construction operation, obtaining the reprojection error e1 and reprojection error e2 corresponding to each target original feature. The optimization module uses the sum of the squares of the reprojection errors e1 and e2 corresponding to each target original feature as the error term corresponding to that target original feature. The error terms corresponding to the at least four target original features are accumulated to obtain the sub-function to be optimized for each target pose information. The sub-functions to be optimized corresponding to multiple target pose information are accumulated to obtain the error function to be optimized.

[0081] In an optional embodiment, the estimation module is specifically used to: estimate the pose of the calibration board and the image acquisition device based on at least four original features F and their corresponding image features f1 in the real scene image P1 to obtain relative pose information T1; and use the relative pose information T1 as the relative pose information T0 between the calibration board and the image acquisition device; or estimate the pose of the calibration board and the image acquisition device based on at least four original features F and their corresponding image features f2 in the real scene image P2 to obtain relative pose information T2, and use the relative pose information T2 as the relative pose information T0 between the calibration board and the image acquisition device.

[0082] Optionally, the determining module is specifically used to: determine the initial pose information Pb of the calibration plate based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation matrix R1 corresponding to the rotation angle A1, and the relative pose information T1, where Pb = inverse(P0*R2)*T2, inverse(·) represents inverse transformation, and "*" represents matrix multiplication; or determine the initial pose information Pb of the calibration plate based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation matrix R2 corresponding to the rotation angle A2, and the relative pose information T2, where Pb = inverse(P0*R2)*T2, inverse(·) represents inverse transformation, and "*" represents matrix multiplication.

[0083] In an optional embodiment, the projection module is specifically used to: project the original target feature F0 in the calibration board onto the real scene image P1 based on the initial pose information Pb of the calibration board, the initial pose information P0 of the image acquisition device and the rotation axis, and the rotation angle A1, to obtain the projected feature f3 = K*(P0*R1)*Pb*F0, where K is the intrinsic parameter of the image acquisition device, "*" indicates matrix multiplication, and R1 is the rotation matrix corresponding to the rotation angle A1; and project the original target feature F0 in the calibration board onto the real scene image P2 based on the initial pose information Pb of the calibration board, the initial pose information P0 of the image acquisition device and the rotation axis, and the rotation angle A2, to obtain the projected feature f4 = K*(P0*R2)*Pb*F0, where R2 is the rotation matrix corresponding to the rotation angle A2.

[0084] In one optional embodiment, the rotation axis is rigidly connected to the lidar acquisition device. When the image acquisition device acquires a real-world image of the target, the lidar acquisition device acquires target point cloud data corresponding to the real-world image. The calibration device for the image acquisition device and the rotation axis further includes: an assignment module, a projection module, and a processing module. The assignment module is used to continuously assign values ​​to the relative pose information between the lidar acquisition device and the image acquisition device in response to interactive operations. The projection module is used to project the target point cloud data onto the image coordinate system based on the assigned relative pose information and the target relative pose information between the rotation axis and the image acquisition device to obtain projected point cloud data until the projected point cloud data is aligned with the target real-world image. The processing module is used to use the assigned values ​​of the relative pose information between the lidar acquisition device and the image acquisition device as the target relative pose information between the lidar acquisition device and the image acquisition device when the projected point cloud data is aligned with the target real-world image.

[0085] For a detailed description of the implementation method and effect of the calibration device for the image acquisition device and the rotating axis, please refer to the foregoing embodiments, which will not be repeated here.

[0086] Figure 3This is a schematic diagram illustrating the structure of an image acquisition device and a calibration device for a rotating axis, provided as an exemplary embodiment of this application. The image acquisition device and the rotating axis are rigidly connected and both are located within a target spatial object. Figure 3 As shown, the calibration device includes a memory 34 and a processor 35.

[0087] Memory 34 is used to store computer programs and can be configured to store various other data to support operation on the image acquisition device and the calibration device for the rotary axis. Examples of this data include instructions for any application or method used to operate on the image acquisition device and the calibration device for the rotary axis.

[0088] The processor 35 is coupled to the memory 34 and is used to execute the computer program in the memory 34 for performing the following operations:

[0089] Acquisition operation: When the rotation axis is at rotation angle A1 and rotation angle A2, the real scene image P1 and real scene image P2 containing the calibration board are acquired by the image acquisition device respectively. The calibration board has at least four original features F, and each original feature F has a corresponding image feature in the real scene image P1 and real scene image P2. The calibration board has target pose information.

[0090] Estimation operation: Based on at least four original features F and their corresponding image features in the real scene image, the pose of the calibration board and the image acquisition device is estimated to obtain the relative pose information T0 between the calibration board and the image acquisition device.

[0091] Determine the operation: Based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation angle of the rotation axis and the relative pose information T0, determine the initial pose information Pb of the calibration plate;

[0092] Projection operation: Based on the initial pose information Pb of the calibration board, the original target feature F0 in the calibration board is projected onto the real scene image P1 and the real scene image P2 respectively to obtain the projected feature f3 and the projected feature f4. The original target feature F0 is any original feature in the calibration board.

[0093] Construction operation: Based on the target image feature f10 corresponding to the original target feature F0 in the real scene image P1, and the target image feature f20, projection feature f3 and projection feature f4 corresponding to the original target feature F0 in the real scene image P2, respectively establish the reprojection error e1 = f10 - f3 and the reprojection error e2 = f20 - f4 corresponding to the original target feature F0;

[0094] Optimization operation: Based on the sum of the squares of the reprojection errors e1 and e2, an error function to be optimized is established. The error function to be optimized is iteratively optimized until convergence, so as to obtain the target relative pose information between the image acquisition device and the rotation axis after optimization, so as to realize the calibration between the image acquisition device and the rotation axis.

[0095] Optionally, the processor 35 is further configured to: traverse at least four original features contained in the calibration board, take each original feature as a target original feature, and obtain at least four target original features; for each target original feature, perform a projection operation and a construction operation to obtain the reprojection error e1 and reprojection error e2 corresponding to each target original feature;

[0096] When the processor 35 establishes the error function to be optimized based on the sum of the squares of the reprojection errors e1 and e2, it specifically uses the following steps: taking the sum of the squares of the reprojection errors e1 and e2 corresponding to each original feature of the target as the error term corresponding to that original feature; and summing the error terms corresponding to at least four original features of the target to obtain the error function to be optimized.

[0097] Further optionally, the calibration board has multiple target pose information in the target space object; when the processor traverses at least four original features contained in the calibration board, it is specifically used to: traverse multiple target pose information, and perform the above-mentioned acquisition operation, estimation operation, determination operation, projection operation, construction operation and optimization operation for each target pose information of the calibration board; before performing the projection operation, traverse at least four original features contained in the calibration board;

[0098] When processor 35 accumulates the error terms corresponding to at least four original target features to obtain the error function to be optimized, it specifically performs the following: accumulating the error terms corresponding to at least four original target features to obtain the sub-function to be optimized for each target pose information; and accumulating the sub-functions to be optimized corresponding to multiple target pose information to obtain the error function to be optimized.

[0099] In an optional embodiment, when performing the estimation operation, the processor 35 is specifically configured to: perform pose estimation on the calibration board and the image acquisition device based on at least four original features F and their corresponding image features f1 in the real scene image P1 to obtain relative pose information T1; use the relative pose information T1 as the relative pose information T0 between the calibration board and the image acquisition device; or perform pose estimation on the calibration board and the image acquisition device based on at least four original features F and their corresponding image features f2 in the real scene image P2 to obtain relative pose information T2, and use the relative pose information T2 as the relative pose information T0 between the calibration board and the image acquisition device.

[0100] Optionally, when performing the determination operation, the processor 35 is specifically used to: determine the initial pose information Pb of the calibration plate based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation matrix R1 corresponding to the rotation angle A1, and the relative pose information T1, where Pb = inverse(P0*R2)*T2, inverse(·) represents inverse transformation, and "*" represents matrix multiplication; or determine the initial pose information Pb of the calibration plate based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation matrix R2 corresponding to the rotation angle A2, and the relative pose information T2, where Pb = inverse(P0*R2)*T2, inverse(·) represents inverse transformation, and "*" represents matrix multiplication.

[0101] In an optional embodiment, when the processor 35 performs the projection operation, it is specifically used to: project the original target feature F0 in the calibration board onto the real scene image P1 according to the initial pose information Pb of the calibration board, the initial pose information P0 of the image acquisition device and the rotation axis, and the rotation angle A1, to obtain the projection feature f3 = K*(P0*R1)*Pb*F0, where K is the intrinsic parameter of the image acquisition device, "*" indicates matrix multiplication, and R1 is the rotation matrix corresponding to the rotation angle A1;

[0102] Based on the initial pose information Pb of the calibration board, the initial pose information P0 of the image acquisition device and the rotation axis, and the rotation angle A2, the original target feature F0 in the calibration board is projected onto the real scene image P2 to obtain the projected feature f4 = K*(P0*R2)*Pb*F0, where R2 is the rotation matrix corresponding to the rotation angle A2.

[0103] In an optional embodiment, the rotation axis is rigidly connected to the lidar acquisition device. When the image acquisition device acquires a real-world image of the target, the lidar acquisition device acquires target point cloud data corresponding to the real-world image of the target. The processor 35 is further configured to: continuously assign values ​​to the relative pose information between the lidar acquisition device and the image acquisition device in response to interactive operations; based on the assigned relative pose information, combined with the target relative pose information between the rotation axis and the image acquisition device, project the target point cloud data onto the image coordinate system to obtain projected point cloud data, until the projected point cloud data is aligned with the target real-world image; when the projected point cloud data is aligned with the target real-world image, the assigned values ​​to the relative pose information between the lidar acquisition device and the image acquisition device are used as the target relative pose information between the lidar acquisition device and the image acquisition device.

[0104] Furthermore, such as Figure 3 As shown, the image acquisition device and the calibration device for the rotating axis also include: communication component 36, display 37, power supply component 38, audio component 39 and other components. Figure 3The diagram only shows some components and does not imply that the image acquisition device and the calibration device for the rotating axis only include... Figure 3 The components shown. Additionally... Figure 3 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the product form of the image acquisition device and the calibration device for the rotating axis.

[0105] The detailed implementation methods and beneficial effects of the image acquisition device and the rotation axis calibration device provided in the embodiments of this application have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0106] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described functions. Figure 1 The steps in the method shown.

[0107] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0108] The aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices. The device containing the communication components can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0109] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0110] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0111] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0112] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0113] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0116] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0117] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0118] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0119] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0120] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A calibration method for an image acquisition device and a rotation axis, wherein the image acquisition device and the rotation axis are rigidly connected and both are located within a target spatial object, characterized in that... The method includes: S1. When the rotation axis is at rotation angle A1 and rotation angle A2, the image acquisition device acquires real scene image P1 and real scene image P2 containing the calibration plate respectively. The calibration plate has at least four original features F. Each original feature F corresponds to an image feature in the real scene image P1 and the real scene image P2. The calibration plate has target pose information. S2. Based on the at least four original features F and their corresponding image features in the real scene image, perform pose estimation on the calibration board and the image acquisition device to obtain the relative pose information T0 between the calibration board and the image acquisition device. S3. Based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation angle of the rotation axis and the relative pose information T0, determine the initial pose information Pb of the calibration plate; S4. Based on the initial pose information Pb of the calibration board, project the original target feature F0 in the calibration board onto the real scene image P1 and the real scene image P2 respectively to obtain the projection feature f3 and the projection feature f4. The original target feature F0 is any one of the original features in the calibration board. S5. Based on the target image feature f10 corresponding to the original target feature F0 in the real scene image P1, and the target image feature f20, the projection feature f3, and the projection feature f4 corresponding to the real scene image P2, respectively establish the reprojection error e1 = f10 - f3 and the reprojection error e2 = f20 - f4 corresponding to the original target feature F0; S6. Based on the sum of the squares of the reprojection error e1 and the reprojection error e2, establish an error function to be optimized, and iteratively optimize the error function to be optimized until convergence, so as to obtain the target relative pose information between the image acquisition device and the rotation axis after optimization, so as to realize the calibration between the image acquisition device and the rotation axis.

2. The method according to claim 1, characterized in that, The method further includes: The calibration board contains at least four original features, and each original feature is used as a target original feature to obtain at least four target original features. For each original feature of the target, execute steps S4-S5 to obtain the reprojection error e1 and reprojection error e2 corresponding to each original feature of the target. Step S6 establishes the error function to be optimized based on the sum of the squares of the reprojection error e1 and the reprojection error e2, including: The sum of the squares of the reprojection errors e1 and e2 corresponding to each original feature of the target is taken as the error term corresponding to the original feature of the target. The error terms corresponding to the at least four original target features are summed to obtain the error function to be optimized.

3. The method according to claim 2, characterized in that, The calibration board contains multiple target pose information within the target space object; the calibration board is traversed by at least four original features, including: The plurality of target pose information is traversed, and steps S1-S6 are executed for each target pose information of the calibration board; Before performing step S4, at least four original features contained in the calibration board are traversed; The error terms corresponding to the at least four original target features are summed to obtain the error function to be optimized, including: The error terms corresponding to the at least four original features of the targets are summed to obtain the sub-function to be optimized for each target pose information; The error sub-functions corresponding to the multiple target pose information are accumulated to obtain the error function to be optimized.

4. The method according to claim 1, characterized in that, Based on the at least four original features F and their corresponding image features in the real-world image, pose estimation is performed on the calibration board and the image acquisition device to obtain the relative pose information T0 between the calibration board and the image acquisition device, including: Based on the at least four original features F and their corresponding image features f1 in the real scene image P1, pose estimation is performed on the calibration board and the image acquisition device to obtain relative pose information T1; the relative pose information T1 is used as the relative pose information T0 between the calibration board and the image acquisition device. or Based on the at least four original features F and their corresponding image features f2 in the real scene image P2, pose estimation is performed on the calibration board and the image acquisition device to obtain relative pose information T2, and the relative pose information T2 is used as the relative pose information T0 between the calibration board and the image acquisition device.

5. The method according to claim 4, characterized in that, Based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation angle of the rotation axis, and the relative pose information T0, the initial pose information Pb of the calibration plate is determined, including: Based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation matrix R1 corresponding to the rotation angle A1, and the relative pose information T1, the initial pose information Pb of the calibration plate is determined, where Pb = inverse(P0*R2)*T2, inverse(·) represents inverse transformation, and "*" represents matrix multiplication; or Based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation matrix R2 corresponding to the rotation angle A2, and the relative pose information T2, the initial pose information Pb of the calibration plate is determined, where Pb = inverse(P0*R2)*T2, inverse(·) represents inverse transformation, and "*" represents matrix multiplication.

6. The method according to claim 1, characterized in that, Based on the initial pose information Pb of the calibration board, the original target features F0 in the calibration board are projected onto the real-world images P1 and P2 respectively, to obtain projection features f3 and f4, including: Based on the initial pose information Pb of the calibration board, the initial pose information P0 of the image acquisition device and the rotation axis, and the rotation angle A1, the original target feature F0 in the calibration board is projected onto the real scene image P1 to obtain the projected feature f3 = K*(P0*R1)*Pb*F0, where K is the intrinsic parameter of the image acquisition device, "*" indicates matrix multiplication, and R1 is the rotation matrix corresponding to the rotation angle A1; Based on the initial pose information Pb of the calibration board, the initial pose information P0 of the image acquisition device and the rotation axis, and the rotation angle A2, the original target feature F0 in the calibration board is projected onto the real scene image P2 to obtain the projected feature f4 = K*(P0*R2)*Pb*F0, where R2 is the rotation matrix corresponding to the rotation angle A2.

7. The method according to claim 1, characterized in that, The rotation axis is rigidly connected to the lidar acquisition device. When the image acquisition device acquires a real-world image of the target, the lidar acquisition device acquires target point cloud data corresponding to the real-world image of the target. The method further includes: In response to interactive operations, the relative pose information between the lidar acquisition device and the image acquisition device is continuously assigned values; based on the assigned relative pose information, combined with the target relative pose information between the rotation axis and the image acquisition device, the target point cloud data is projected onto the image coordinate system to obtain projected point cloud data, until the projected point cloud data is aligned with the target real-world image; When the projected point cloud data is aligned with the target real-world image, the value assigned to the relative pose information between the lidar acquisition device and the image acquisition device is used as the target relative pose information between the lidar acquisition device and the image acquisition device.

8. A calibration device for an image acquisition device and a rotating axis, characterized in that, The image acquisition device is rigidly connected to the rotation axis and both are located within the target space object. The device includes: an acquisition module, an estimation module, a determination module, a projection module, a construction module, and an optimization module. The acquisition module is used to acquire real-scene images P1 and P2 containing the calibration plate respectively based on the image acquisition device when the rotation axis is at rotation angle A1 and rotation angle A2. The calibration plate has at least four original features F, and each original feature F corresponds to an image feature in the real-scene image P1 and the real-scene image P2. The calibration plate has target pose information. The estimation module is used to perform pose estimation on the calibration board and the image acquisition device based on the at least four original features F and their corresponding image features in the real scene image, so as to obtain the relative pose information T0 between the calibration board and the image acquisition device. The determining module is used to determine the initial pose information Pb of the calibration plate based on the initial pose information P0 of the image acquisition device and the rotation axis, the rotation angle of the rotation axis and the relative pose information T0. The projection module is used to project the original target feature F0 in the calibration board onto the real scene image P1 and the real scene image P2 respectively, to obtain the projected feature f3 and the projected feature f4. The original target feature F0 is any one of the original features in the calibration board. The construction module is used to establish reprojection errors e1 = f10 - f3 and e2 = f20 - f4 corresponding to the original target feature F0 in the real scene image P1, and the original target feature F0 in the real scene image P2, the projection feature f3 and the projection feature f4, respectively. The optimization module is used to establish an error function to be optimized based on the sum of the squares of the reprojection error e1 and the reprojection error e2, and to iteratively optimize the error function to obtain the target relative pose information between the image acquisition device and the rotation axis after optimization, so as to realize the calibration between the image acquisition device and the rotation axis.

9. A calibration device for an image acquisition device and a rotating axis, characterized in that, include: Memory and processor; The memory is used to store a computer program; the processor, coupled to the memory, is used to execute the computer program to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-7.

Images