Calibration method, electronic device and storage medium

Abstract

The present application provides a calibration method, an electronic device and a storage medium. The method includes: obtaining multiple frames of original point clouds, each frame of the original point cloud corresponds to a rotation angle; aligning the multiple frames of original point clouds to the same world coordinate system to obtain a target point cloud; determining the three-dimensional coordinate point corresponding to the preset coordinate point according to the position and posture of each frame of the target point cloud; determining the rotation coordinate point of each three-dimensional coordinate point in the rotation coordinate system based on multiple three-dimensional coordinate points and the rotation angle corresponding to each three-dimensional coordinate point; determining the initial transformation parameters between the rotation coordinate system and the world coordinate system by aligning each three-dimensional coordinate point with the corresponding rotation coordinate point; using the initial transformation parameters, aligning each frame of the original point cloud with the corresponding target point cloud to obtain the target transformation parameters. Using the above method, the target transformation parameters between the rotation coordinate system and the world coordinate system can be determined, so that the point cloud data can be converted to the camera coordinate system for coloring according to the target transformation parameters.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning technology, and in particular to a calibration method, electronic equipment and storage medium. Background Art

[0002] In order to improve the visualization and realism of point cloud data, it is often necessary to color the point cloud data based on the image data. Before coloring the point cloud data, it is necessary to convert the point cloud data to the coordinate system corresponding to the visual sensor based on the conversion relationship between the point cloud sensor (e.g., radar) that collects the point cloud data and the visual sensor (e.g., camera) that collects the image data, to ensure the coloring effect.

[0003] In some application scenarios, it is often necessary to rotate the point cloud sensor to adjust the observation angle or cover a wider scanning area. However, this rotation will change the spatial relationship between the point cloud sensor and the visual sensor, causing the originally fixed conversion relationship between the point cloud sensor and the visual sensor to become unfixed, making it difficult to convert the point cloud data to the coordinate system corresponding to the visual sensor (for example, the camera coordinate system).

[0004] If the point cloud data cannot be converted to the coordinate system corresponding to the visual sensor, it will be difficult to accurately color the point cloud. Summary of the invention

[0005] In view of the above, it is necessary to provide a calibration method, an electronic device and a storage medium that can solve the technical problem of difficulty in accurately coloring the point cloud due to the difficulty in converting the point cloud data to the coordinate system corresponding to the visual sensor.

[0006] On the one hand, the present application provides a calibration method, which includes: obtaining multiple frames of original point clouds scanned by a point cloud sensor, wherein the point cloud sensor is configured to be rotated by a rotating mechanism, and each frame of the original point cloud corresponds to a rotation angle of the point cloud sensor, aligning the multiple frames of original point clouds to the same world coordinate system, and obtaining the target point cloud corresponding to each frame of the original point cloud in the world coordinate system, determining each three-dimensional coordinate point corresponding to a preset coordinate point in the world coordinate system according to the posture of each frame of the target point cloud, determining the rotation coordinate point corresponding to each three-dimensional coordinate point in the rotation coordinate system corresponding to the rotation mechanism based on multiple three-dimensional coordinate points and the rotation angle of the original point cloud corresponding to each three-dimensional coordinate point, determining the initial transformation parameters between the rotation coordinate system and the world coordinate system by aligning each three-dimensional coordinate point with the corresponding rotation coordinate point, and using the initial transformation parameters, aligning each frame of the original point cloud with the corresponding target point cloud to obtain the target transformation parameters between the rotation coordinate system and the world coordinate system.

[0007] In some embodiments of the present application, each rotation coordinate point has a corresponding rotation coordinate, and each three-dimensional coordinate point is determined based on the rotation angle of the original point cloud corresponding to each three-dimensional coordinate point. The corresponding rotation coordinate point in the rotation coordinate system corresponding to the rotation mechanism includes: fitting a position distribution trajectory in the world coordinate system according to the multiple three-dimensional coordinate points, and determining the rotation coordinate of the rotation coordinate point corresponding to each three-dimensional coordinate point according to the rotation angle corresponding to each three-dimensional coordinate point and the parameters of the position distribution trajectory.

[0008] In some embodiments of the present application, the shape of the position distribution trajectory is a circle, and determining the rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point according to the rotation angle corresponding to each three-dimensional coordinate point and the parameters of the position distribution trajectory includes: determining an initial rotation coordinate point and other rotation coordinate points except the initial rotation coordinate point from the rotation coordinate points, determining the rotation coordinates of the initial rotation coordinate point according to the radius of the position distribution trajectory, and determining the rotation coordinates of each rotation coordinate point according to the rotation matrix of the rotation coordinates of the initial rotation coordinate point and the rotation angle corresponding to each rotation coordinate point among the other rotation coordinate points.

[0009] In some embodiments of the present application, determining the initial transformation parameters between the rotating coordinate system and the world coordinate system by aligning each three-dimensional coordinate point with the corresponding rotating coordinate point includes: constructing a function equation about the world coordinates of each three-dimensional coordinate point, the rotating coordinates of the rotating coordinate point corresponding to each three-dimensional coordinate point, and the initial transformation parameters, and solving the function equation according to the world coordinates of each three-dimensional coordinate point and the rotating coordinates of the rotating coordinate point corresponding to each three-dimensional coordinate point to obtain the initial transformation parameters.

[0010] In some embodiments of the present application, the method also includes: acquiring point cloud data obtained by the point cloud sensor scanning the target scene, and image data obtained by the visual sensor photographing the target scene, wherein the point cloud data corresponds to a rotation angle of the point cloud sensor, and according to the calibration parameters of the visual sensor, the target transformation parameters and the rotation angle corresponding to the point cloud data, converting the point cloud data from an initial coordinate system to a camera coordinate system, and coloring the point cloud data in the camera coordinate system based on the image data and the calibration parameters.

[0011] In some embodiments of the present application, converting the point cloud data from the initial coordinate system to the camera coordinate system according to the calibration parameters of the visual sensor, the target transformation parameters and the rotation angle corresponding to the point cloud data includes: using the target transformation parameters and the rotation matrix corresponding to the rotation angle of the point cloud data to convert the point cloud data from the initial coordinate system to the world coordinate system, and using the external parameters in the calibration parameters to convert the point cloud data located in the world coordinate system to the camera coordinate system, wherein the external parameters are the conversion relationship between the world coordinate system and the camera coordinate system.

[0012] In some embodiments of the present application, the method of converting the point cloud data from the initial coordinate system to the world coordinate system using the target transformation parameters and the rotation matrix of the rotation angle corresponding to the point cloud data includes: calculating the target pose according to the target transformation parameters and the rotation matrix of the rotation angle corresponding to the point cloud data, and converting the point cloud data from the initial coordinate system to the world coordinate system according to the target pose.

[0013] In some embodiments of the present application, the coloring processing of the point cloud data located in the camera coordinate system based on the image data and the calibration parameters includes: determining the target image data corresponding to the point cloud data based on the matching of the image data and the point cloud data, projecting the point cloud data located in the camera coordinate system to the target image data using the internal parameters in the calibration parameters, determining the pixel position corresponding to the target image data of each point in the point cloud data, and coloring the corresponding point in the camera coordinate system according to the color corresponding to the pixel position.

[0014] The present application provides a calibration device, the device comprising: an acquisition module, used to acquire multiple frames of original point clouds scanned by a point cloud sensor, wherein the point cloud sensor is configured to be driven to rotate by a rotating mechanism, and each frame of the original point cloud corresponds to a rotation angle of the point cloud sensor; an alignment module, used to align the multiple frames of original point clouds to the same world coordinate system, and obtain the target point cloud corresponding to each frame of the original point cloud in the world coordinate system; a determination module, used to determine each three-dimensional coordinate point corresponding to a preset coordinate point in the world coordinate system according to the position and posture of each frame of the target point cloud; the determination module is further used to determine the rotation coordinate point corresponding to each three-dimensional coordinate point in the rotation coordinate system corresponding to the rotation mechanism based on multiple three-dimensional coordinate points and the rotation angle of the original point cloud corresponding to each three-dimensional coordinate point; the determination module is further used to determine the initial transformation parameters between the rotation coordinate system and the world coordinate system by aligning each three-dimensional coordinate point with the corresponding rotation coordinate point; the alignment module is further used to align each frame of the original point cloud with the corresponding target point cloud using the initial transformation parameters, and obtain the target transformation parameters between the rotation coordinate system and the world coordinate system.

[0015] On the other hand, the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the calibration method.

[0016] On the other hand, the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the calibration method when executed by a processor in an electronic device.

[0017] In the calibration method provided in the embodiment of the present application, the point cloud sensor is driven to rotate by a rotating mechanism, which can expand the scanning field angle of the point cloud sensor, thereby improving the scanning efficiency. By aligning the multiple frames of original point clouds to the same world coordinate system, not only the position and posture of the target point cloud can be obtained, but also the conversion reference can be unified, so that the point cloud data can be converted to the camera coordinate system according to the conversion relationship between the world coordinate system and the camera coordinate system. Since the three-dimensional coordinate point is obtained by converting the preset coordinate point through the position and posture of the target point cloud, the three-dimensional coordinate point can reflect the position and posture of the corresponding target point cloud. Since each three-dimensional coordinate point can reflect the position and posture of the corresponding target point cloud, the distance between the rotation coordinate point corresponding to the three-dimensional coordinate point and the origin of the rotation coordinate system can be determined according to the multiple three-dimensional coordinate points. Since each rotation angle is the angle difference between the angle corresponding to the initial position of the point cloud sensor and the angle corresponding to the rotated position, the trigonometric function value of the rotation angle can reflect the distance between the rotation coordinate point and the origin of the rotation coordinate system in multiple directions (for example, horizontal and vertical directions). Therefore, according to the rotation angle corresponding to the multiple three-dimensional coordinate points and each three-dimensional coordinate point, the rotation coordinate corresponding to the three-dimensional coordinate point can be quickly determined, so that the corresponding rotation coordinate point can be determined. Since the three-dimensional coordinate point is located in the world coordinate system and the rotation coordinate point is located in the rotation coordinate system, the initial transformation parameters between the rotation coordinate system and the world coordinate system can be quickly determined by aligning each three-dimensional coordinate point with the corresponding rotation coordinate point. By using the initial transformation parameters, the initial transformation parameters can be optimized by aligning the original point cloud of each frame with the corresponding target point cloud, so that the accuracy of the target transformation parameters can be improved. According to the target transformation parameters, the point cloud data can be accurately converted to the world coordinate system, and then converted to the camera coordinate system, so as to facilitate the coloring of the point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of a calibration method provided in one embodiment of the present application.

[0019] Figure 2 It is a scanning schematic diagram of the original point cloud provided in one embodiment of the present application.

[0020] Figure 3 It is a schematic diagram of a location distribution trajectory provided by an embodiment of the present application.

[0021] Figure 4 It is a schematic diagram of a rotating coordinate system provided in an embodiment of the present application.

[0022] Figure 5 This is a flowchart of a method for coloring point cloud data provided by an embodiment of the present application.

[0023] Figure 6It is a flowchart of a method for converting point cloud data provided by an embodiment of the present application.

[0024] Figure 7 It is a functional module diagram of a calibration device provided in one embodiment of the present application.

[0025] Figure 8 It is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0026] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0027] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. The following embodiments and features in the embodiments may be combined with each other without conflict.

[0028] The embodiment of the present application provides a calibration method, which can determine the target transformation parameters between the rotating coordinate system and the world coordinate system, so that the point cloud data can be converted to the camera coordinate system according to the target transformation parameters to facilitate coloring of the point cloud data.

[0029] The calibration method provided in the embodiment of the present application can be applied to one or more electronic devices, which may be computers, mobile phones, laptops, servers and other devices, wherein the server may be a cloud server or a server cluster. The present application does not limit the type of electronic device.

[0030] like Figure 1 FIG. 1 is a flow chart of a calibration method provided by an embodiment of the present application. According to different requirements, the order of each step in the flow chart can be adjusted according to actual requirements, and some steps can be omitted. The calibration method is applied to electronic devices, such as Figure 8 An electronic device 10 is shown.

[0031] S11, obtaining multiple frames of original point clouds scanned by a point cloud sensor, wherein the point cloud sensor is configured to be rotated by a rotating mechanism, and each frame of the original point cloud corresponds to a rotation angle of the point cloud sensor.

[0032] In some embodiments of the present application, the point cloud sensor may include but is not limited to: radar, depth sensor, stereo vision sensor and ultrasonic sensor. Among them, the radar may be a laser radar, the depth sensor may be a time of flight (TOF) sensor, a spot time of flight (Spot TOF) and an infrared structured light sensor, etc., the stereo vision sensor may be a binocular stereo camera, a three-dimensional camera, etc., and the ultrasonic sensor may be an ultrasonic rangefinder. The point cloud sensor can be connected to the electronic device for communication, or the point cloud sensor can be integrated into the electronic device, so that the electronic device can obtain multiple frames of original point clouds obtained by scanning and detecting point cloud data. Among them, the point cloud sensor can communicate with the electronic device through a wired connection or a wireless connection. The present application does not limit the connection method between the point cloud sensor and the electronic device.

[0033] Multiple frames of original point clouds can be obtained by scanning or detecting a scanning scene through a point cloud sensor. This application does not limit the type of scene.

[0034] The point cloud sensor can be located above the rotating mechanism so that the point cloud sensor can be driven to rotate by the rotating mechanism. The rotating mechanism can be a rotating table, also known as a pan-tilt table, which can achieve 360° rotation, thereby driving the point cloud sensor to perform a panoramic scan of the scanning scene. After each rotation driven by the rotating mechanism, the point cloud sensor can scan the scanning scene once to obtain a frame of original point cloud. Through multiple scans, multiple frames of original point cloud can be obtained, and each frame of original point cloud has a corresponding rotation angle. The rotation angle can be the angle difference between the angle corresponding to the initial position of the point cloud sensor and the angle corresponding to the rotated position.

[0035] For example, if the point cloud sensor is a radar, forward and backward scanning can be achieved, and the radar frame can be used to represent the original point cloud, such as Figure 2 FIG. 1 is a schematic diagram of scanning the original point cloud provided by an embodiment of the present application. Figure 2In the figure, the pink dotted line can represent the scanning field angle of the radar for the first scan. By scanning the scanning scene for the first time, the scene information within the corresponding scanning field angle can be captured, so as to obtain radar frame 1 as the first frame of original point cloud. The blue dotted line can represent the scanning field angle of the radar for the second scan. By scanning the scanning scene for the second time, the scene information within the corresponding scanning field angle can be captured, so as to obtain radar frame 2 as the second frame of original point cloud. The green dotted line can represent the scanning field angle of the radar for the third scan. By scanning the scanning scene for the third time, the scene information within the corresponding scanning field angle can be captured, so as to obtain radar frame 3 as the third frame of original point cloud. Figure 2 The scanning method of multiple frames of original point clouds shown is only an example and is not limited to this in actual applications.

[0036] Considering that the scanning field of view of the point cloud sensor is limited, for example, the field of view of the radar in the Z-axis direction is often no more than 40°, the limited field of view will lead to poor scanning efficiency of the point cloud sensor. Therefore, in this embodiment, the point cloud sensor is driven to rotate by the rotating mechanism, so that the scanning field of view of the point cloud sensor can be expanded to 360°×360°, achieving panoramic scanning coverage, thereby improving scanning efficiency.

[0037] S12, aligning the multiple frames of original point clouds to the same world coordinate system, and obtaining a target point cloud corresponding to each frame of the original point cloud in the world coordinate system.

[0038] In some embodiments of the present application, the world coordinate system can be customized, and the present application does not limit this. For example, in order to quickly achieve point cloud alignment, the original coordinate system corresponding to the first frame of the original point cloud can be determined as the world coordinate system. Among them, the original coordinate system corresponding to the first frame of the original point cloud can be a three-dimensional coordinate system constructed according to the original position of the point cloud sensor, and the original position can be a position that has not been rotated.

[0039] In some embodiments of the present application, the electronic device can use the Point Cloud Library (PCL) to align / register each frame of the original point cloud to the same world coordinate system to obtain the target point cloud. The point cloud library may include a point cloud alignment algorithm, such as an iterative closest point (ICP) algorithm and a point cloud alignment algorithm based on a normal distribution transform (NDT). Through the point cloud alignment algorithm in the point cloud library, the electronic device can align the multiple frames of original point clouds to the same world coordinate system, thereby obtaining the target point cloud corresponding to each frame of the original point cloud in the world coordinate system.

[0040] Among them, through the point cloud alignment algorithm in the point cloud library, the electronic device can align the multiple frames of original point clouds to the same world coordinate system. The calculation process of the point cloud alignment algorithm in the relevant technology can be referred to.

[0041] Considering that point cloud coloring processing often requires converting the point cloud to the camera coordinate system corresponding to the visual sensor (for example, a camera), and the point cloud sensor is rotated by the rotating mechanism, the conversion relationship between the point cloud sensor and the visual sensor will be not fixed. Therefore, in this embodiment, multiple frames of original point clouds are aligned to the same world coordinate system, which can unify the conversion reference so that the point cloud data can be converted to the camera coordinate system according to the conversion relationship between the world coordinate system and the camera coordinate system.

[0042] S13, determining each three-dimensional coordinate point corresponding to the preset coordinate point in the world coordinate system according to the position and posture of the target point cloud in each frame.

[0043] In some embodiments of the present application, the pose of each frame of the target point cloud can be a 4x4 matrix, including a rotation matrix and a translation vector. The preset coordinate point can be a coordinate point point.

[0044] In some embodiments of the present application, a three-dimensional coordinate point may have a corresponding world coordinate in a world coordinate system, and the electronic device may determine, based on the pose of the target point cloud in each frame, each three-dimensional coordinate point corresponding to a preset coordinate point in the world coordinate system, including: using the pose of the target point cloud in each frame to calculate the coordinates of the preset coordinate point, and obtaining the world coordinates corresponding to each three-dimensional coordinate point in the world coordinate system.

[0045] For example, the method of calculating the coordinates of the preset coordinate points by using the pose of the target point cloud in each frame can refer to the following formula (1):

[0046] ; (1)

[0047] in, Indicates The world coordinates of the three-dimensional coordinate points, Indicates the coordinates of the preset coordinate point. Indicates The rotation matrix in the pose of the frame target point cloud, Indicates The translation vector in the pose of the frame target point cloud, Indicates the number of all rotated coordinate points or all 3D coordinate points or all original point clouds or all target point clouds.

[0048] In this embodiment, since the three-dimensional coordinate point is obtained by converting the preset coordinate point through the position and posture of the target point cloud, the three-dimensional coordinate point can reflect the position and posture of the corresponding target point cloud.

[0049] S14, based on the plurality of three-dimensional coordinate points and the rotation angle of the original point cloud corresponding to each three-dimensional coordinate point, determining a rotation coordinate point corresponding to each three-dimensional coordinate point in the rotation coordinate system corresponding to the rotation mechanism.

[0050] In some embodiments of the present application, each rotation coordinate point has a corresponding rotation coordinate, and the electronic device determines each three-dimensional coordinate point based on multiple three-dimensional coordinate points and the rotation angle of the original point cloud corresponding to each three-dimensional coordinate point, and the corresponding rotation coordinate point in the rotation coordinate system corresponding to the rotation mechanism includes: fitting a position distribution trajectory in the world coordinate system according to the multiple three-dimensional coordinate points, and determining the rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point according to the rotation angle corresponding to each three-dimensional coordinate point and the parameters of the position distribution trajectory.

[0051] Among them, the position distribution trajectory can be fitted by a fitting algorithm based on the position and posture of the three-dimensional coordinate points, and the present application does not limit the type of the fitting algorithm. For example, the fitting algorithm can be the least squares method. The position distribution trajectory is used to indicate or describe the distribution shape of multiple three-dimensional coordinate points. When the point cloud sensor performs a panoramic scan on the scanning scene, the shape of the position distribution trajectory is circular, and the parameters of the position distribution trajectory may include the radius of the position distribution trajectory.

[0052] For example, Figure 3 , which is a schematic diagram of a location distribution trajectory provided by an embodiment of the present application. Figure 3 In , a circle composed of multiple points represents the position distribution trajectory, where r represents the radius of the position distribution trajectory. It can represent the first three-dimensional coordinate point, Represents the second three-dimensional coordinate point, Can represent three-dimensional coordinate points The corresponding rotation angle of the original point cloud, Can be expressed The corresponding rotation angle of the original point cloud.

[0053] In this embodiment, since the three-dimensional coordinate points can reflect the position and posture of the corresponding target point cloud, the position distribution trajectory can be accurately fitted according to the three-dimensional coordinate points, so that the position distribution trajectory can accurately reflect the scanning trajectory of the point cloud sensor.

[0054] In some embodiments of the present application, the electronic device can determine the initial rotation coordinate point from all the rotation coordinate points based on the three-dimensional coordinate point corresponding to the original point cloud of the first frame, and construct a rotation coordinate system with the preset position point as the origin, the rotation axis of the rotating mechanism as the Z axis, the direction from the origin to the initial rotation coordinate point as the X axis direction, and the direction perpendicular to the X axis and the Z axis as the Y axis direction.

[0055] The rotation coordinate point corresponding to the three-dimensional coordinate point corresponding to the original point cloud of the first frame can be used as the initial rotation coordinate point. For example, Figure 3 The rotation coordinate point corresponding to the first three-dimensional coordinate point shown is used as the initial rotation coordinate point. Each frame of the original point cloud has a corresponding scanning time. The first frame of the original point cloud can be determined based on the scanning time of the original point cloud. The preset position point can be customized or can be the center coordinate of the position distribution trajectory.

[0056] For example, Figure 4 , which is a schematic diagram of a rotating coordinate system provided by an embodiment of the present application. Figure 4 In the rotation coordinate system, the Z axis is the rotation axis L, the X axis is the direction from the origin to the initial rotation coordinate point, and the Y axis is perpendicular to the X axis and the Z axis. Figure 4 The rotating coordinate system shown is only an example and is not limited to this in practical applications.

[0057] Exemplarily, if the position distribution trajectory is a circle, the electronic device determines the rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point according to the rotation angle corresponding to each three-dimensional coordinate point and the parameters of the position distribution trajectory, including: determining other rotation coordinate points except the initial rotation coordinate point from the rotation coordinate points, determining the rotation coordinates of the initial rotation coordinate point according to the radius of the position distribution trajectory, and determining the rotation coordinates of each rotation coordinate point according to the rotation matrix of the rotation coordinates of the initial rotation coordinate point and the rotation angle corresponding to each rotation coordinate point among the other rotation coordinate points.

[0058] Among them, since the rotation coordinate system is the X-axis, the initial rotation coordinate point is located on the X-axis, so the rotation coordinate of the initial rotation coordinate point can be expressed as , where r represents the radius of the position distribution trajectory.

[0059] The rotation angle corresponding to each rotating coordinate point may be the rotation angle corresponding to the original point cloud to which the target point cloud corresponding to the three-dimensional coordinate point corresponding to the rotating coordinate point belongs. The rotation matrix corresponding to the rotating coordinate point may be a linear transformation matrix of the corresponding rotation angle rotated around the rotation axis of the rotating mechanism. Among them, according to the rotation angle corresponding to each three-dimensional coordinate point, the method for determining the rotation matrix corresponding to the rotating coordinate point corresponding to the three-dimensional coordinate point may refer to the method of calculating the rotation matrix according to the rotation angle and the direction vector of the rotation axis using the axis-angle representation of the rotation matrix in the related art, wherein the direction vector of the rotation axis may be .

[0060] The calculation method of the rotation coordinates of each of the other rotation coordinate points can refer to the following formula (2):

[0061] ; (2)

[0062] in, represents the rotation coordinates of the initial rotation coordinate point, Indicates the other rotation coordinate points The rotation coordinates of the rotation coordinate points, Indicates the other rotation coordinate points The rotation matrix of the rotation angle corresponding to the rotation coordinate points, Indicates the number of all rotated coordinate points or all 3D coordinate points or all original point clouds or all target point clouds.

[0063] In this embodiment, since each rotation angle is the angle difference between the angle corresponding to the initial position of the point cloud sensor and the angle corresponding to the rotated position, and the rotation matrix of the rotation angle corresponding to the rotation coordinate point is a linear transformation matrix of the rotation angle corresponding to the rotation around the rotation axis of the rotating mechanism, the rotation coordinates of each of the other rotation coordinate points can be accurately determined based on the rotation coordinates of the initial rotation coordinate point and the rotation matrix of the rotation angle corresponding to each of the other rotation coordinate points.

[0064] S15, determining initial transformation parameters between the rotating coordinate system and the world coordinate system by aligning each of the three-dimensional coordinate points with the corresponding rotating coordinate point.

[0065] In some embodiments of the present application, the electronic device determines the initial transformation parameters between the rotating coordinate system and the world coordinate system by aligning each of the three-dimensional coordinate points with the corresponding rotating coordinate points, including: constructing a function equation about the world coordinates of each of the three-dimensional coordinate points, the rotation coordinates of the rotating coordinate points corresponding to each of the three-dimensional coordinate points, and the initial transformation parameters, solving the function equation according to the world coordinates of each of the three-dimensional coordinate points and the rotation coordinates of the rotating coordinate points corresponding to each of the three-dimensional coordinate points to obtain the initial transformation parameters.

[0066] The initial transformation parameter may be a 4x4 matrix, including a rotation matrix and a translation vector. For example, the function equation may refer to the following formula (3):

[0067] ; (3)

[0068] in, Indicates The rotation coordinates of the rotation coordinate points, represents the rotation matrix in the initial transformation parameters, represents the translation vector in the initial transformation parameters, Indicates The world coordinates of the three-dimensional coordinate points, Indicates the number of all rotated coordinate points or all 3D coordinate points or all original point clouds or all target point clouds.

[0069] The electronic device can solve formula (3) according to the world coordinates of each three-dimensional coordinate point and the rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point, complete the alignment of the three-dimensional coordinate point with the corresponding rotation coordinate point, and obtain the initial transformation parameters.

[0070] In this embodiment, since the rotation angle is the angle difference between the angle corresponding to the initial position of the point cloud sensor and the angle corresponding to the position after rotation, the rotation matrix corresponding to the rotation angle is a linear transformation matrix of the corresponding rotation angle around the rotation axis, so through the rotation matrix, the corresponding relationship between the rotation coordinates of the initial rotation coordinate point and the rotation coordinates of each of the other rotation coordinate points can be constructed. Since the initial transformation parameters are used to represent the transformation relationship between the rotating coordinate system and the world coordinate system, the corresponding relationship between the world coordinates of each three-dimensional coordinate point and the rotation coordinates of the corresponding rotating coordinate point can be constructed according to the initial transformation relationship. Through the above-mentioned corresponding relationship, the initial transformation parameters between the rotating coordinate system and the world coordinate system can be quickly determined.

[0071] S16, using the initial transformation parameters, aligning the original point cloud of each frame with the corresponding target point cloud to obtain the target transformation parameters between the rotation coordinate system and the world coordinate system.

[0072] In some embodiments of the present application, the electronic device may align the original point cloud of each frame to the corresponding target point cloud using a point cloud alignment algorithm (e.g., an ICP algorithm) according to the initial transformation parameters to obtain the target transformation parameters. Wherein, according to the initial transformation parameters, the original point cloud of each frame is aligned to the corresponding target point cloud using a point cloud alignment algorithm, and the target transformation parameters can be obtained by referring to the operation process of the point cloud alignment algorithm in the related art.

[0073] In this embodiment, by utilizing the initial transformation parameters to align the original point cloud of each frame with the corresponding target point cloud, the initial transformation parameters can be optimized, thereby improving the accuracy of the target transformation parameters.

[0074] In the calibration method provided in the embodiment of the present application, the point cloud sensor is driven to rotate by a rotating mechanism, which can expand the scanning field angle of the point cloud sensor, thereby improving the scanning efficiency. By aligning the multiple frames of original point clouds to the same world coordinate system, not only the position and posture of the target point cloud can be obtained, but also the conversion reference can be unified, so that the point cloud data can be converted to the camera coordinate system according to the conversion relationship between the world coordinate system and the camera coordinate system. Since the three-dimensional coordinate point is obtained by converting the preset coordinate point through the position and posture of the target point cloud, the three-dimensional coordinate point can reflect the position and posture of the corresponding target point cloud. Since each three-dimensional coordinate point can reflect the position and posture of the corresponding target point cloud, the distance between the rotation coordinate point corresponding to the three-dimensional coordinate point and the origin of the rotation coordinate system can be determined according to the multiple three-dimensional coordinate points. Since each rotation angle is the angle difference between the angle corresponding to the initial position of the point cloud sensor and the angle corresponding to the rotated position, the trigonometric function value of the rotation angle can reflect the distance between the rotation coordinate point and the origin of the rotation coordinate system in multiple directions (for example, horizontal and vertical directions). Therefore, according to the rotation angle corresponding to the multiple three-dimensional coordinate points and each three-dimensional coordinate point, the rotation coordinate corresponding to the three-dimensional coordinate point can be quickly determined, so that the corresponding rotation coordinate point can be determined. Since the three-dimensional coordinate point is located in the world coordinate system and the rotation coordinate point is located in the rotation coordinate system, the initial transformation parameters between the rotation coordinate system and the world coordinate system can be quickly determined by aligning each three-dimensional coordinate point with the corresponding rotation coordinate point. By using the initial transformation parameters, the initial transformation parameters can be optimized by aligning the original point cloud of each frame with the corresponding target point cloud, so that the accuracy of the target transformation parameters can be improved. According to the target transformation parameters, the point cloud data can be accurately converted to the world coordinate system, and then converted to the camera coordinate system, so as to facilitate the coloring of the point cloud data.

[0075] In some embodiments of the present application, since the target transformation parameter is the conversion relationship between the rotating coordinate system and the world coordinate system, each frame of point cloud data obtained by scanning the point cloud sensor can be converted to the world coordinate system through the target transformation parameter, and then each frame of point cloud data can be converted to the camera coordinate system according to the conversion relationship between the world coordinate system and the camera coordinate system, thereby realizing the coloring of the point cloud. Figure 5 , which is a flow chart of a method for coloring point cloud data provided in one embodiment of the present application.

[0076] S21, acquiring point cloud data obtained by scanning the target scene with a point cloud sensor, and image data obtained by photographing the target scene with a visual sensor, wherein the point cloud data corresponds to a rotation angle at which the point cloud sensor is driven to rotate.

[0077] In some embodiments of the present application, the present application does not limit the type of target scene. The point cloud data may be one frame or multiple frames, and the scanning method of the point cloud data may refer to the description of the acquisition method of the original point cloud in step S11.

[0078] Visual sensors are devices that can acquire image data, including but not limited to: cameras and webcams. This application does not limit the specific type of visual sensors.

[0079] In some embodiments of the present application, the electronic device may communicate with the visual sensor via a wired connection or a wireless connection, and the present application does not limit the communication connection method between the electronic device and the visual sensor.

[0080] S22, converting the point cloud data from the initial coordinate system to the camera coordinate system according to the calibration parameters of the visual sensor, the target transformation parameters and the rotation angle corresponding to the point cloud data.

[0081] In some embodiments of the present application, the calibration parameters may include intrinsic parameters and external parameters, wherein the internal parameter is the conversion relationship between the camera coordinate system corresponding to the visual sensor and the image coordinate system or pixel coordinate system corresponding to the image captured by the visual sensor, and the external parameter is the conversion relationship between the camera coordinate system corresponding to the visual sensor and the above-mentioned world coordinate system.

[0082] S23, based on the image data and the calibration parameters, coloring the point cloud data in the camera coordinate system.

[0083] In some embodiments of the present application, based on the image data and the calibration parameters, the point cloud data located in the camera coordinate system is colorized, including: based on the matching of the image data and the point cloud data, the target image data corresponding to the point cloud data is determined, and the point cloud data located in the camera coordinate system is projected onto the target image data using the internal parameters in the calibration parameters, and the pixel position corresponding to each point in the point cloud data in the target image data is determined, and the corresponding point in the camera coordinate system is colorized according to the color corresponding to the pixel position in the target image data.

[0084] The electronic device can use a feature matching algorithm to match the image data with the point cloud data, thereby determining the target image data corresponding to the point cloud data. The feature matching algorithm includes but is not limited to: nearest neighbor matching algorithm, random sample consensus (RANSAC) algorithm, and neural network-based matching algorithm.

[0085] In this embodiment, the target image data to which the point cloud data belongs can be accurately determined by the feature matching algorithm. Since the internal parameter of the calibration parameter is the conversion relationship between the camera coordinate system and the image coordinate system or the pixel coordinate system, the point cloud data can be accurately projected to the target image data through the internal parameter, so that the color corresponding to each point in the point cloud data can be accurately determined, and then the accurate coloring of the point cloud data can be achieved.

[0086] like Figure 6 , which is a flow chart of a method for converting point cloud data provided in one embodiment of the present application.

[0087] S221, transforming the point cloud data from the initial coordinate system to the world coordinate system using the target transformation parameters and the rotation matrix of the rotation angle corresponding to the point cloud data.

[0088] In some embodiments of the present application, the initial coordinate system of the point cloud data may be a coordinate system constructed based on the position of the point cloud sensor when scanning the point cloud data. The point cloud data may include multiple points with three-dimensional coordinates, each of which is a three-dimensional coordinate in the initial coordinate system.

[0089] In some embodiments of the present application, the electronic device may calculate the target pose based on the target transformation parameters and the rotation matrix corresponding to the point cloud data, and convert the point cloud data from the initial coordinate system to the world coordinate system based on the target pose.

[0090] The target pose may include a rotation matrix and a translation vector. For example, the target pose may be calculated by referring to formulas (4) to (5):

[0091] ; (4)

[0092] ; (5)

[0093] in, represents the rotation matrix in the target pose, represents the inverse matrix of the rotation matrix in the target transformation parameters, The rotation matrix representing the rotation angle corresponding to the point cloud data, represents the translation vector in the target pose, Represents the translation vector in the target transformation parameters.

[0094] In this embodiment, since the target transformation parameter is the conversion relationship between the rotating coordinate system and the world coordinate system, the rotation matrix of the rotation angle corresponding to the point cloud data is a linear transformation matrix of the corresponding rotation angle around the rotation axis. Therefore, through the target transformation parameter and the rotation matrix of the rotation angle corresponding to the point cloud data, the point cloud data can be quickly and accurately converted from the rotating coordinate system to the world coordinate system.

[0095] S222, using the external parameters in the calibration parameters, convert the point cloud data in the world coordinate system to the camera coordinate system.

[0096] In some embodiments of the present application, external parameters may include a rotation matrix and a translation vector. Therefore, the electronic device can calculate the world coordinates of each point in the point cloud data in the world coordinate system according to the external parameters in the calibration parameters to obtain the camera coordinates of each point in the point cloud data in the camera coordinate system.

[0097] Among them, the method of calculating the world coordinates of each point in the point cloud data located in the world coordinate system according to the external parameters in the calibration parameters can refer to the above formula (1), and this application will not repeat it again.

[0098] In this embodiment, since the external parameters of the calibration parameters are the conversion relationship between the camera coordinate system and the above-mentioned world coordinate system, the point cloud data can be accurately converted from the world coordinate system to the camera coordinate system according to the external parameters in the calibration parameters, so as to facilitate coloring of the point cloud data in the camera coordinate system.

[0099] like Figure 7 , which is a functional module diagram of a calibration device provided in an embodiment of the present application. The calibration device 11 includes an acquisition module 110, an alignment module 111 and a determination module 112. The module / unit referred to in the present application refers to a Figure 8 A series of computer readable instruction segments acquired by the processor 103 in the embodiment and capable of completing fixed functions, which are stored in Figure 8 In the memory 102 in the present embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.

[0100] An acquisition module 110 is used to acquire multiple frames of original point clouds scanned by a point cloud sensor, wherein the point cloud sensor is configured to be rotated by a rotating mechanism, and each frame of the original point cloud corresponds to a rotation angle of the point cloud sensor;

[0101] The alignment module 111 is used to align the multiple frames of original point clouds to the same world coordinate system to obtain the target point cloud corresponding to each frame of the original point cloud in the world coordinate system.

[0102] The determination module 112 is used to determine each three-dimensional coordinate point corresponding to the preset coordinate point in the world coordinate system according to the position and posture of the target point cloud in each frame.

[0103] The determination module 112 is further used to determine the rotation coordinate point corresponding to each three-dimensional coordinate point in the rotation coordinate system corresponding to the rotation mechanism based on the plurality of three-dimensional coordinate points and the rotation angle of the original point cloud corresponding to each three-dimensional coordinate point.

[0104] In some embodiments of the present application, each rotation coordinate point has a corresponding rotation coordinate, and the determination module 112 is further used to fit a position distribution trajectory in the world coordinate system according to the multiple three-dimensional coordinate points, and determine the rotation coordinate of the rotation coordinate point corresponding to each three-dimensional coordinate point according to the rotation angle corresponding to each three-dimensional coordinate point and the parameters of the position distribution trajectory.

[0105] In some embodiments of the present application, the shape of the position distribution trajectory is a circle, and the determination module 112 is also used to determine the initial rotation coordinate point and other rotation coordinate points except the initial rotation coordinate point from the rotation coordinate points, determine the rotation coordinates of the initial rotation coordinate point according to the radius of the position distribution trajectory, and determine the rotation coordinates of each rotation coordinate point according to the rotation matrix of the rotation coordinates of the initial rotation coordinate point and the rotation angle corresponding to each rotation coordinate point in the other rotation coordinate points.

[0106] The determination module 112 is further configured to determine initial transformation parameters between the rotation coordinate system and the world coordinate system by aligning each of the three-dimensional coordinate points with the corresponding rotation coordinate point.

[0107] In some embodiments of the present application, the determination module 112 is also used to construct a function equation about the world coordinates of each three-dimensional coordinate point, the rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point, and the initial transformation parameters, and solve the function equation according to the world coordinates of each three-dimensional coordinate point and the rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point to obtain the initial transformation parameters.

[0108] The alignment module 111 is further used to align the original point cloud of each frame with the corresponding target point cloud using the initial transformation parameters, and obtain the target transformation parameters between the rotation coordinate system and the world coordinate system.

[0109] In some embodiments of the present application, the calibration device further includes a coloring module ( Figure 7 Not shown), a coloring module is used to obtain point cloud data obtained by the point cloud sensor scanning the target scene, and image data obtained by the visual sensor shooting the target scene, wherein the point cloud data corresponds to a rotation angle of the point cloud sensor, and according to the calibration parameters of the visual sensor, the target transformation parameters and the rotation angle corresponding to the point cloud data, the point cloud data is converted from an initial coordinate system to a camera coordinate system, and based on the image data and the calibration parameters, the point cloud data located in the camera coordinate system is colored.

[0110] In some embodiments of the present application, the shading module is also used to transform the point cloud data from the initial coordinate system to the world coordinate system using the target transformation parameters and the rotation matrix of the rotation angle corresponding to the point cloud data, and to transform the point cloud data located in the world coordinate system to the camera coordinate system using the external parameters in the calibration parameters, wherein the external parameters are the transformation relationship between the world coordinate system and the camera coordinate system.

[0111] In some embodiments of the present application, the coloring module is also used to determine the target image data corresponding to the point cloud data based on the matching of the image data and the point cloud data, project the point cloud data located in the camera coordinate system to the target image data using the internal parameters in the calibration parameters, determine the pixel position corresponding to the target image data for each point in the point cloud data, and color the corresponding point in the camera coordinate system according to the color corresponding to the pixel position.

[0112] like Figure 8 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 10 can be a computer, a mobile phone, a tablet computer, a notebook computer, a server, etc. The specific type of the electronic device 10 is not limited in any way in the embodiment of the present application.

[0113] exist Figure 8 In the embodiment, the electronic device 10 may include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104 and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102 and the input / output interface 104 through the bus 105.

[0114] The communication module 101 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more wired communication solutions such as universal serial bus (USB), controller area network bus (CAN, Controller Area Network). The wireless communication module may provide one or more wireless communication solutions such as wireless fidelity (wireless fidelity, Wi-Fi), Bluetooth (Bluetooth, BT), mobile communication network, frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR), etc.

[0115] The memory 102 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 103, and can be used to store executable programs (such as machine instructions) of other running programs, and can also be used to store user and application data. The random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0116] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 103. The non-volatile memory may include a disk storage device and a flash memory.

[0117] The memory 102 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include multiple instructions, and when the multiple instructions are executed by the processor 103, the calibration method executed on the electronic device 10 can be implemented.

[0118] In other embodiments, Figure 8 The electronic device 10 shown also includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 10 .

[0119] The processor 103 may include one or more processing units, for example, the processor 103 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0120] The processor 103 provides computing and control capabilities. For example, the processor 103 is used to execute a computer program stored in the memory 102 to implement the above-mentioned calibration method.

[0121] The input / output interface 104 is used to provide a channel for user input or output. For example, the input / output interface 104 can be used to connect various input and output devices, such as a mouse, keyboard, touch device, display screen, etc., so that the user can enter information or visualize information.

[0122] The bus 105 is at least used to provide a channel for mutual communication among the communication module 101 , the memory 102 , the processor 103 , and the input / output interface 104 in the electronic device 10 .

[0123] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0124] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the methods in the above-mentioned embodiments of the present application.

[0125] The computer-readable storage medium may be an internal memory of the electronic device described in the above embodiment, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device.

[0126] In some embodiments, the computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the electronic device, etc.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0128] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0130] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present application is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0131] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in this application can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A calibration method, characterized in that: The method comprises: Acquire multiple frames of original point clouds scanned by a point cloud sensor, wherein the point cloud sensor is configured to be rotated by a rotating mechanism, and each frame of the original point cloud corresponds to a rotation angle of the point cloud sensor; Aligning the multiple frames of original point clouds to the same world coordinate system, and obtaining the target point cloud corresponding to each frame of the original point cloud in the world coordinate system; According to the pose of the target point cloud in each frame, determine each three-dimensional coordinate point corresponding to the preset coordinate point in the world coordinate system; Based on the plurality of three-dimensional coordinate points and the rotation angle of the original point cloud corresponding to each three-dimensional coordinate point, determining the rotation coordinate point corresponding to each three-dimensional coordinate point in the rotation coordinate system corresponding to the rotation mechanism; Determining initial transformation parameters between the rotating coordinate system and the world coordinate system by aligning each of the three-dimensional coordinate points with the corresponding rotating coordinate points; The initial transformation parameters are used to align the original point cloud of each frame with the corresponding target point cloud to obtain target transformation parameters between the rotating coordinate system and the world coordinate system.

2. The calibration method according to claim 1, characterized in that: Each rotation coordinate point has a corresponding rotation coordinate, and each three-dimensional coordinate point is determined based on the rotation angle of the original point cloud corresponding to the plurality of three-dimensional coordinate points and each three-dimensional coordinate point. The corresponding rotation coordinate points in the rotation coordinate system corresponding to the rotation mechanism include: Fitting a position distribution trajectory in the world coordinate system according to the multiple three-dimensional coordinate points; The rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point are determined according to the rotation angle corresponding to each three-dimensional coordinate point and the parameters of the position distribution trajectory.

3. The calibration method according to claim 2, characterized in that: The shape of the position distribution trajectory is a circle, and determining the rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point according to the rotation angle corresponding to each three-dimensional coordinate point and the parameters of the position distribution trajectory includes: Determine an initial rotation coordinate point and other rotation coordinate points except the initial rotation coordinate point from the rotation coordinate points; Determining the rotation coordinates of the initial rotation coordinate point according to the radius of the position distribution trajectory; The rotation coordinates of each rotation coordinate point are determined according to the rotation coordinates of the initial rotation coordinate point and the rotation matrix of the rotation angle corresponding to each rotation coordinate point among the other rotation coordinate points.

4. The calibration method according to claim 3, characterized in that: The step of determining the initial transformation parameters between the rotating coordinate system and the world coordinate system by aligning each of the three-dimensional coordinate points with the corresponding rotating coordinate points comprises: Constructing a function equation about the world coordinates of each three-dimensional coordinate point, the rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point, and the initial transformation parameter; The function equation is solved according to the world coordinates of each three-dimensional coordinate point and the rotation coordinates of the rotation coordinate point corresponding to each three-dimensional coordinate point to obtain the initial transformation parameters.

5. The calibration method according to claim 1, characterized in that: The method further comprises: Acquire point cloud data obtained by scanning a target scene with the point cloud sensor, and image data obtained by photographing the target scene with a visual sensor, wherein the point cloud data corresponds to a rotation angle at which the point cloud sensor is driven to rotate; Converting the point cloud data from an initial coordinate system to a camera coordinate system according to the calibration parameters of the visual sensor, the target transformation parameters, and a rotation angle corresponding to the point cloud data; Based on the image data and the calibration parameters, the point cloud data in the camera coordinate system is colored.

6. The calibration method according to claim 5, characterized in that: The step of converting the point cloud data from an initial coordinate system to a camera coordinate system according to the calibration parameters of the visual sensor, the target transformation parameters, and a rotation angle corresponding to the point cloud data includes: The point cloud data is transformed from the initial coordinate system to the world coordinate system using the target transformation parameter and a rotation matrix of a rotation angle corresponding to the point cloud data; The point cloud data in the world coordinate system is converted to the camera coordinate system by using an external parameter in the calibration parameter, wherein the external parameter is a conversion relationship between the world coordinate system and the camera coordinate system.

7. The calibration method according to claim 6, characterized in that: The step of converting the point cloud data from the initial coordinate system to the world coordinate system by using the target transformation parameter and a rotation matrix corresponding to the rotation angle of the point cloud data comprises: Calculating the target pose according to the target transformation parameters and the rotation matrix of the rotation angle corresponding to the point cloud data; The point cloud data is converted from the initial coordinate system to the world coordinate system according to the target pose.

8. The calibration method according to claim 5, characterized in that: The coloring process of the point cloud data in the camera coordinate system based on the image data and the calibration parameters includes: Based on the matching of the image data and the point cloud data, determining the target image data corresponding to the point cloud data; Using the internal parameters in the calibration parameters, projecting the point cloud data in the camera coordinate system onto the target image data, and determining the pixel position of each point in the point cloud data corresponding to the target image data; According to the color corresponding to the pixel position, the corresponding point in the camera coordinate system is colored.

9. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the calibration method as claimed in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor in an electronic device, the calibration method according to any one of claims 1 to 8 is implemented.

Images