LiDAR calibration methods, devices, electronic equipment and readable storage media
By determining the total transformation matrix between the radar coordinate system and the quay crane coordinate system, the problem of pose calibration of lidar and container crane was solved, achieving high-precision pose calibration and improving the automation and safety of port operations.
Patent Information
- Application Number
- CN202311669050.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-06
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-12-06
AI Technical Summary
How to achieve pose calibration between LiDAR and container crane to ensure the accuracy and safety of automated scanning systems.
By obtaining point cloud data, the total transformation matrix between the radar coordinate system and the quay crane coordinate system is determined. Using the geometric characteristics of the ground, crane lifting equipment, and gantry column in the point cloud data, the first, second, and third transformation relationships are calculated to realize the transformation from the radar coordinate system to the quay crane coordinate system.
It achieves high-precision pose calibration between lidar and crane spreader, supports the normal use of automated scanning systems, and improves the efficiency and safety of port operations.
Smart Images

Figure CN117741625B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of lidar, and more specifically, to a lidar calibration method, apparatus, electronic device, and readable storage medium. Background Technology
[0002] Container cranes at port terminals are specialized equipment used for loading and unloading container ships. They are typically installed on the quayside of port terminals. Also known as quay cranes, gantry cranes, or dock cranes, the loading and unloading capacity and speed of quay container cranes directly determine the terminal's operational productivity.
[0003] Automated scanning provides accurate information on vessel hold positions or truck locations by scanning vessels or trucks in the waiting area below container cranes, providing crucial data support for the automated scheduling of port information systems.
[0004] LiDAR, as a key sensor in automated scanning systems, is used to sense the position of ship holds / container trucks and ensure the safety of automated control. However, after the LiDAR is installed on a container crane, its pose calibration is required for it to function properly. Therefore, how to achieve pose calibration between the LiDAR and the container crane is a problem that needs to be considered. Summary of the Invention
[0005] The purpose of this invention is to provide a lidar calibration method, device, electronic device, and readable storage medium that can determine the total transformation matrix from the radar coordinate system to the quay crane coordinate system, thereby achieving pose calibration between the radar and the crane lifting device.
[0006] The embodiments of the present invention can be implemented as follows: In a first aspect, the present invention provides a lidar calibration method, comprising: Point cloud data is obtained by scanning the ground, crane spreader and main body of the crane with a lidar mounted on the quay crane trolley within the field of view using a lidar. Based on the point cloud data, a first transformation relationship, a second transformation relationship, and a third transformation relationship are determined; wherein, the first transformation relationship, the second transformation relationship, and the third transformation relationship respectively reflect the transformation relationship between the radar position and the hoist position in a first direction, a second direction, and a third direction in three-dimensional space; Based on the first transformation relationship, the second transformation relationship, and the third transformation relationship, the overall transformation relationship between the radar coordinate system and the quay crane coordinate system is determined.
[0007] In an optional implementation, the step of determining the first transformation relationship, the second transformation relationship, and the third transformation relationship based on the point cloud data includes: The first transformation relationship is determined based on the ground point cloud in the point cloud data; The point cloud data is transformed using the first transformation relationship to obtain the first transformed point cloud data; The second transformation relationship is determined based on the correlation point clouds of two reference planes in the first transformed point cloud data; the reference plane is a plane orthogonal to the ground. The first transformed point cloud data is transformed using the second transformation relationship to obtain the second transformed point cloud data; The third transformation relationship is determined based on the crane lifting point cloud in the second transformed point cloud data.
[0008] In an optional implementation, the step of determining the first transformation relationship based on the ground point cloud in the point cloud data includes: At least three ground point clouds are sampled from the point cloud data; Based on the at least three ground point clouds, fit the plane equation of the ground; Based on the plane equation of the ground, the normal vector of the ground and the height value between the lidar and the ground are obtained; The first rotation matrix required to convert the ground normal vector into the Z-axis unit normal vector of the quay bridge coordinate system; The first transformation relationship is determined based on the first rotation matrix and the height value between the lidar and the ground.
[0009] In an optional implementation, the step of determining the second transformation relationship based on the correlated point clouds of two reference planes in the first transformed point cloud data includes: Multiple related point clouds of two reference planes are sampled from the first converted point cloud data; the two reference planes are the inner sides of the two large vehicle columns represented in the first converted point cloud data, and the ground and the large vehicle track on the ground are perpendicular to the inner sides of the two large vehicle columns. Based on multiple related point clouds of the two reference planes, the plane equations of the two reference planes are fitted respectively. The equation of the intermediate plane is obtained by averaging the plane equations of the two reference planes with parameters. The Z-direction of the intermediate plane equation is set to zero to obtain the plane equation of the translation plane; Based on the plane equation of the translation plane, the normal vector and the offset in the Y direction of the translation plane are obtained; The second rotation matrix required to convert the normal vector of the translation plane into the unit normal vector of the Y-axis of the quay bridge coordinate system; The second transformation relationship is determined based on the second rotation matrix and the Y-direction offset of the translation plane.
[0010] In an optional implementation, the step of determining the third transformation relationship based on the crane spreader point cloud in the second transformed point cloud data includes: Segment all the lifting point clouds of the crane lifting device from the second transformed point cloud data; After setting the Z-axis coordinate value of each of the lifting device point clouds to 0, grid downsampling processing is performed to obtain multiple uniformly distributed sampling point clouds; Determine the centroid coordinates of the multiple sampling point clouds; The third transformation relationship is determined based on the centroid coordinates.
[0011] In an optional implementation, the step of determining the overall transformation relationship between the radar coordinate system and the quay crane coordinate system based on the first transformation relationship, the second transformation relationship, and the third transformation relationship includes: The transformation matrices of the first transformation relation, the second transformation relation, and the third transformation relation are multiplied sequentially to obtain the total transformation relation.
[0012] In a second aspect, the present invention provides a lidar calibration device, comprising: The acquisition module is used to obtain point cloud data, which is obtained by the lidar mounted on the quay crane trolley of the quay crane scanning the ground, crane spreader and trolley column within the field of view. The processing module is used to determine a first transformation relationship, a second transformation relationship, and a third transformation relationship based on the point cloud data; wherein the first transformation relationship, the second transformation relationship, and the third transformation relationship respectively reflect the transformation relationship between the radar position and the hoist position in a first direction, a second direction, and a third direction in three-dimensional space; The merging module is used to determine the overall transformation relationship between the radar coordinate system and the quay crane coordinate system based on the first transformation relationship, the second transformation relationship, and the third transformation relationship.
[0013] In an optional implementation, the processing module is specifically used for: The first transformation relationship is determined based on the ground point cloud in the point cloud data; The point cloud data is transformed using the first transformation relationship to obtain the first transformed point cloud data; The second transformation relationship is determined based on the correlation point clouds of two reference planes in the first transformed point cloud data; the reference plane is a plane orthogonal to the ground. The first transformed point cloud data is transformed using the second transformation relationship to obtain the second transformed point cloud data; The third transformation relationship is determined based on the crane lifting point cloud in the second transformed point cloud data.
[0014] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory stores a software program, and when the electronic device is running, the processor executes the software program to implement the lidar calibration method as described in the first aspect above.
[0015] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the lidar calibration method described in the first aspect.
[0016] Compared with existing technologies, embodiments of the present invention provide a lidar calibration method, apparatus, electronic device, and readable storage medium. First, point cloud data from lidar scanning is obtained. This point cloud data is obtained by the lidar mounted on the quay crane trolley scanning the ground, crane spreader, and trolley column within its field of view. Next, based on the point cloud data, a first transformation relationship, a second transformation relationship, and a third transformation relationship are determined. Finally, based on the first, second, and third transformation relationships, a total transformation relationship is determined. Since the first, second, and third transformation relationships respectively reflect the transformation relationships of the radar position and the spreader position in a first direction, a second direction, and a third direction in three-dimensional space, the final determined total transformation relationship can realize the conversion from the lidar coordinate system to the quay crane coordinate system, thereby facilitating the pose calibration between the lidar and the crane spreader. Attached Figure Description
[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a schematic diagram of the installation of a lidar on a quay crane, provided as an embodiment of the present invention.
[0019] Figure 2 This is one of the flowcharts illustrating a lidar calibration method provided in an embodiment of the present invention.
[0020] Figure 3This is a schematic diagram showing the positions of the quay crane trolley and the crane lifting device when collecting point cloud data, as provided in an embodiment of the present invention.
[0021] Figure 4 This is a second schematic flowchart of a lidar calibration method provided in an embodiment of the present invention.
[0022] Figure 5 This is a schematic diagram illustrating a coordinate transformation in the Z direction, provided as an embodiment of the present invention.
[0023] Figure 6 This is a schematic diagram of a transformation of the intermediate coordinate system 1 in the Y direction, provided as an embodiment of the present invention.
[0024] Figure 7 This is a schematic diagram of a lidar calibration device provided in an embodiment of the present invention.
[0025] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0027] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0028] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0029] Furthermore, the terms "first" and "second" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.
[0030] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.
[0031] The application scenarios of the embodiments of the present invention will be introduced first.
[0032] Please see Figure 1 , Figure 1 This is a schematic diagram of the installation of a lidar on a quay crane, provided as an embodiment of the present invention. Figure 1 The illustrated quay crane includes: a quay crane trolley that can move along trolley tracks, trolley tracks mounted on the quay crane trolley, a quay crane trolley that can move along the trolley tracks, and a liftable crane spreader suspended from the bottom right side of the quay crane trolley by multiple slings. This crane spreader is used to clamp containers to achieve container loading and unloading operations.
[0033] The quay crane consists of four main columns (A1B1, D1C1, A2B2, and D2C2) and wheel sets on the left and right sides of the bottom. The rectangular area enclosed by the left-side columns A1B1 and D1C1 is called the left portal frame, and the rectangular area enclosed by the right-side columns A2B2 and D2C2 is called the right portal frame.
[0034] In this invention, the lidar is installed on the bottom left side of the quay crane trolley, with the radar's scanning window facing downwards. For example... Figure 1 When the quay crane trolley is located at the far left of the trolley track and the crane spreader is placed on the ground, a radar coordinate system can be established using the center of the lidar at this time as the origin. The coordinate system of the quay crane can be established by taking the projection point of the crane spreader center on the ground at this moment as the origin. Alternatively, the radar coordinate system and the quay crane coordinate system can also be established when the quay crane trolley is located at the far right of the trolley track. The principle of establishment is similar and will not be elaborated here.
[0035] It should be noted that, Figure 1 The example shown is merely a schematic illustration; the structure of the quay crane, the installation location of the lidar, and the radar's field of view are not necessarily indicative of the actual situation. Figure 1 The above is a limitation.
[0036] The lidar calibration method provided in this invention can be applied to electronic devices, which may be, but are not limited to, computer equipment, servers, or crane controllers of quay cranes.
[0037] The following describes the radar coordinate system With the coordinate system of the quay bridge The calibration process for the transformation relationship between them is described in detail.
[0038] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a lidar calibration method provided in an embodiment of the present invention. The method includes the following steps: S101~S103: S101, Obtain point cloud data.
[0039] In this embodiment, the point cloud data is obtained by scanning the ground, crane spreader, and main column within the field of view using a lidar mounted on the quay crane trolley.
[0040] Optionally, the crane can be moved to a specific position on the trolley track and its spreader adjusted to a specific height in the air, ensuring that the lidar's field of view meets preset acquisition conditions. These conditions could be: the lidar's field of view simultaneously includes the ground, the two main columns of the quay crane, and the spreader in the air. Under these conditions, the lidar scans the ground, the crane spreader, and the two main columns within its field of view to obtain a single frame of point cloud data required for calibration.
[0041] S102. Based on point cloud data, determine the first transformation relationship, the second transformation relationship, and the third transformation relationship.
[0042] In this embodiment, the first transformation relationship, the second transformation relationship, and the third transformation relationship can respectively reflect the transformation relationship of the radar position and the lifting position in the first direction, the second direction, and the third direction in three-dimensional space.
[0043] S103. Based on the first transformation relationship, the second transformation relationship, and the third transformation relationship, determine the overall transformation relationship between the radar coordinate system and the quay crane coordinate system.
[0044] It is understandable that the overall transformation relationship can be described using matrix M. Subsequently, the crane controller can use matrix M to transform the object scanned by the lidar from the radar coordinate system to the quay crane coordinate system.
[0045] The lidar calibration method provided in this invention first obtains the point cloud data required for lidar scanning calibration; then, based on the point cloud data, determines a first transformation relationship, a second transformation relationship, and a third transformation relationship; finally, based on the first transformation relationship, the second transformation relationship, and the third transformation relationship, determines the overall transformation relationship. Since the first transformation relationship, the second transformation relationship, and the third transformation relationship can respectively reflect the transformation relationship between the radar position and the crane position in a first direction, a second direction, and a third direction in three-dimensional space, the finally determined overall transformation relationship can realize the transformation from the radar coordinate system to the quay crane coordinate system, thereby achieving the pose calibration between the radar and the crane spreader.
[0046] Optional, please refer to Figure 3 , Figure 3 The image shows an example of the specific position of the quay crane trolley on the trolley track and the specific height of the crane hoist when collecting the point cloud data required for calibration. Figure 3In the middle, the ground, the crane lifting equipment, and the two large trolley pillars of the left gantry of the quay crane are just within the radar's field of view, that is, the field of view of the lidar meets the preset acquisition conditions.
[0047] It should be noted that, Figure 3 The example shown is merely one example. This embodiment of the invention does not limit the specific position of the quay crane trolley or the specific height of the crane lifting device when the preset data collection conditions are met.
[0048] In the optional implementation, the first, second, and third directions can be the Z, Y, and X directions in three-dimensional space, respectively. Utilizing the scene's geometric characteristics, the transformation matrices of the first, second, and third transformation relationships can be calculated sequentially. Correspondingly, in... Figure 2 Based on this, please refer to Figure 4 The sub-steps of step S102 above may include S1021 to S1025: S1021. Based on the ground point cloud in the point cloud data, determine the first transformation relationship.
[0049] In this embodiment, the first transformation matrix representing the first transformation relationship can be determined using the ground point cloud in the point cloud data. , It can reflect the rotation matrix in the Z direction from the radar coordinate system to the quay crane coordinate system, as well as the offset in the Z direction from the radar coordinate system to the quay crane coordinate system. .exist Figure 1 In the middle, offset This reflects the distance between the origins of the two coordinate systems in the Z direction of the quay crane coordinate system (i.e., the height of the lidar above the ground).
[0050] Optionally, the sub-steps of step S1021 may include S10211 to S10215: S10211. Sample at least three ground point clouds from the point cloud data.
[0051] S10212. Fit the plane equation of the ground based on at least three ground point clouds.
[0052] It is understandable that three three-dimensional points can define a plane. Therefore, at least three ground point clouds can be sampled from point cloud data to fit the plane equation of the ground.
[0053] In this embodiment, a frame of point cloud data It is a set of several three-dimensional points, and the coordinates of the three-dimensional points reflect the position of the three-dimensional points. Figure 1 The location is shown in the radar coordinate system. Therefore, the determined plane equation of the ground can be expressed as follows: (1) Equation (1) reflects the ground's position. Figure 1 The position of the radar coordinate system shown.
[0054] S10213. Based on the plane equation of the ground, obtain the normal vector of the ground and the height value between the lidar and the ground.
[0055] In this embodiment, it can be seen from the above equation (1) that the normal vector of the ground in the radar coordinate system is... The height value between the lidar and the ground is That is, the Z-direction offset between the radar coordinate system and the quay crane coordinate system is .
[0056] S10214. Calculate the first rotation matrix required to convert the ground normal vector into the Z-axis unit normal vector of the quay crane coordinate system.
[0057] In this embodiment, the unit normal vector of the Z-axis of the quay crane coordinate system is The normal vector of the ground in the radar coordinate system Unit normal vector of the Z-axis transformed to the quay crane coordinate system Required first rotation matrix It can be as follows: (2) It should be noted that the process of calculating the rotation matrix of the normal vector in two different coordinate systems is existing technology and will not be elaborated here.
[0058] S10215. Based on the first rotation matrix and the height value between the lidar and the ground, determine the first transformation relationship.
[0059] In this embodiment, based on the first rotation matrix and the height value between the lidar and the ground. The first transformation matrix representing the first transformation relationship is obtained. It can be: (3) The above steps S10211~S10215 are for obtaining the first transformation matrix. The detailed process.
[0060] S1022. The point cloud data is transformed using the first transformation relationship to obtain the first transformed point cloud data.
[0061] It's understandable; the first transformation matrix can be used. The first relay coordinate system can be obtained by transforming the radar coordinate system.
[0062] In this embodiment, the first transformation matrix can be used. Point cloud data Each 3D point in the image is transformed to obtain the first transformed point cloud data. In this context, the coordinates of each 3D point in the first transformed point cloud data reflect its position in the first intermediate coordinate system.
[0063] See optional examples. Figure 5 Using the first rotation matrix The radar coordinate system can be rotated around the origin along the Z-axis until its Z-axis direction aligns with that of the quay crane coordinate system. After rotation, it can be translated along the negative Z-axis direction. This will give us the first intermediate coordinate system, i.e. Figure 5 The intermediate coordinate system 1 is shown.
[0064] S1023. Determine the second transformation relationship based on the related point clouds of the two reference planes in the first transformed point cloud data.
[0065] In this embodiment, the reference plane can be a plane orthogonal to the ground.
[0066] Optionally, when acquiring point cloud data, the reference plane in the radar field of view can correspond to one of the four sides of the trolley column that is perpendicular to the ground and the trolley track. For example, in Figure 5 In the middle, the two selected reference planes are the inner sides of the left door frame of the main vehicle column A1B1 and the main vehicle column D1C1, which are perpendicular to the ground and the main vehicle track.
[0067] Optionally, the sub-steps of step S1023 may include S10231 to S10235: S10231. Sample multiple related point clouds of two reference planes from the first transformed point cloud data.
[0068] In this embodiment, the two selected reference planes can be the inner surfaces of the two large vehicle columns represented in the first converted point cloud data, and the ground and the large vehicle track on the ground are both perpendicular to the inner surfaces of the two large vehicle columns.
[0069] S10232. Based on multiple related point clouds of two reference planes, fit the plane equations of the two reference planes respectively.
[0070] In this embodiment, multiple related point clouds of two reference planes can be sampled from the first transformed point cloud data, and the plane equations of the two reference planes obtained by fitting them can be as follows: (4) (5) It can be understood that expressions (4) and (5) can respectively reflect the positions of the two reference planes in the first intermediate coordinate system.
[0071] In the optional examples, refer to Figure 5 As shown, expression (4) can reflect the position of the inner side of the left door frame where the main vehicle column A1B1 is located in the central coordinate system 1, and expression (5) can reflect the position of the inner side of the left door frame where the main vehicle column D1C1 is located in the central coordinate system 1. This example is only for illustration and is not intended to be limiting.
[0072] S10233. The equation of the intermediate plane is obtained by averaging the plane equations of the two reference planes with parameters.
[0073] In this embodiment, the intermediate plane equation obtained based on expressions (4) and (5) is as follows: (6) It can be understood that expression (6) can represent the position of the intermediate plane of the two reference planes in the first intermediate coordinate system.
[0074] Combination Figure 5 As shown in the example, expression (6) represents: Figure 5 The marked intermediate plane is located in the transfer coordinate system 1. Ideally, this intermediate plane is perpendicular to the ground and the trolley track, dividing the left portal frame of the quay crane into two symmetrical halves.
[0075] S10234. Set the Z-direction of the intermediate plane equation to zero to obtain the plane equation of the translation plane.
[0076] In this embodiment, since the Z-direction transformation relationship from the radar coordinate system to the quay crane coordinate system has been calibrated in step S1021 above, there is no need to focus on the Z-axis component at this time. It is necessary to set the Z-direction of the intermediate plane equation to zero, and the plane equation of the translation plane can be as follows: (7) Combination Figure 5 As illustrated in the example, ideally, after setting the Z-direction of the intermediate plane equation to zero, the resulting translation plane equation will be consistent with the intermediate plane equation, i.e. Figure 5 The marked intermediate plane is also the translation plane.
[0077] S10235. Based on the plane equation of the translation plane, obtain the normal vector of the translation plane and the offset in the Y direction.
[0078] Combination Figure 5 As shown in the example and from the above equation (7), the normal vector of the translation plane in the intermediate coordinate system 1 is... The offset in the Y direction determined based on the translation plane is .
[0079] S10236. The second rotation matrix required to convert the normal vector of the translation plane into the unit normal vector of the Y-axis of the quay crane coordinate system.
[0080] In this embodiment, the unit normal vector of the Y-axis of the quay crane coordinate system is .
[0081] Combination Figure 5 As shown in the example, the normal vector of the translation plane in the intermediate coordinate system 1 is... Unit normal vector of the Y-axis transformed to the quay crane coordinate system Required second rotation matrix It can be as follows: (8) S10237. Determine the second transformation relationship based on the second rotation matrix and the Y-direction offset of the translation plane.
[0082] In this embodiment, based on the second rotation matrix and the Y-direction offset corresponding to the translation plane The second transformation matrix representing the second transformation relationship is obtained. It can be: (9) The above steps S10231~S10235 are for obtaining the second transformation matrix. The detailed process.
[0083] S1024. The first transformed point cloud data is transformed using the second transformation relationship to obtain the second transformed point cloud data.
[0084] It's understandable; the second transformation matrix can be used. The second intermediate coordinate system can be obtained by transforming the first intermediate coordinate system.
[0085] In this embodiment, the second transformation matrix is used. The first converted point cloud data Each 3D point in the data is transformed to obtain the second transformed point cloud data. In this context, the coordinates of each 3D point in the second transformed point cloud data reflect its position in the second intermediate coordinate system.
[0086] See optional examples. Figure 5 Using the second rotation matrix The transfer coordinate system 1 can be rotated around the origin until its Y-axis aligns with the Y-axis of the quay crane coordinate system. After rotation, it can be translated along the positive Y-axis. This will give us the second intermediate coordinate system, i.e. Figure 6 The intermediate coordinate system 2 is shown.
[0087] S1025. Based on the crane lifting point cloud in the second transformed point cloud data, determine the third transformation relationship.
[0088] Assume that the plane that divides the left door frame of the quay crane into two symmetrical parts is called the cutting plane, and that the cutting plane is perpendicular to the ground.
[0089] In an ideal installation scenario, if both the installation point of the lidar and the center point of the crane lifting device are located on the cutting plane, then the second transformation matrix mentioned above contains the complete Y-direction offset.
[0090] However, in actual installation, the installation point of the lidar and the center point of the crane lifting device may not be located on the cutting plane. Figure 5 (The marked intermediate plane is the cutting plane). In this case, the second transformation matrix only determines a portion of the Y-direction offset. Therefore, in step S1025, in addition to calibrating the X-direction offset, it is also necessary to calibrate another portion of the Y-direction offset.
[0091] Correspondingly, the sub-steps of step S1025 may include S10251 to S10254: S10251. Segment all the lifting point clouds of the crane lifting device from the second transformed point cloud data.
[0092] In this embodiment, all lifting device point clouds belonging to the crane lifting device can be segmented from the second transformed point cloud data.
[0093] S10252. After setting the Z-axis coordinate value of each lifting tool point cloud to 0, perform grid downsampling processing to obtain multiple evenly distributed sampling point clouds.
[0094] In this embodiment, combined with Figure 6 As shown in the example, the Z-axis coordinate value of each spreader point cloud is set to 0, which means that each spreader point cloud is projected onto the transfer coordinate system 2. The planar data is used to obtain the projected point cloud for each spreader point cloud. Downsampling is performed by meshing all projected point clouds and then selecting one projected point cloud from each mesh as the sampled point cloud, thus obtaining multiple evenly distributed sampled point clouds.
[0095] S10253. Determine the centroid coordinates of multiple sampling point clouds.
[0096] In this embodiment, the centroid coordinates are .
[0097] Under the ideal installation conditions described above .
[0098] In the actual installation scenario described above, for example, combined with Figure 6 As shown in the example, the centroid coordinates of the calculated cloud of multiple sampling points can be point K. In the intermediate coordinate system 2, the coordinates of point K are... ,at this time Not zero.
[0099] S10254. Determine the third transformation relationship based on the centroid coordinates.
[0100] In this embodiment, the first transformation matrix is used. Transforming the radar coordinate system yields the first relay coordinate system, which is then used in conjunction with the second transformation matrix. Transforming the first transit coordinate system yields the second transit coordinate system. The transition from the second transit coordinate system to the quay crane coordinate system requires only translation, not rotation. Correspondingly, the third transformation matrix, representing the third transformation relationship, is obtained based on the centroid coordinates. It can be as follows: (10) Combination Figure 6 As shown in the example, it can be seen that the transfer coordinate system 2 to the quay crane coordinate system does not require rotation, but only translation.
[0101] It is understandable that although step S10236 above determines a portion of the Y-axis offset (used in the transformation process from transit coordinate system 1 to transit coordinate system 2), since the installation point of the lidar and the center point of the crane lifting device are not within the cutting plane, the transit coordinate system 1 is transformed based on the second transformation matrix to obtain transit coordinate system 2. At this point, the Y-axis offset in the second transformation matrix can only ensure that the Z-axis of transit coordinate system 2 is within the cutting plane, but the Z-axis of the quay crane coordinate system is not within the cutting plane. Therefore, for... Figure 6 The intermediate coordinate system 2, in addition to needing to translate along the positive X-axis direction It also needs to be translated along the positive Y-axis. Only then can it coincide with the coordinate system of the quay crane.
[0102] For optional implementation methods, please refer to [link / reference]. Figure 4 The sub-steps of step S103 above may include: S1031. Multiply the transformation matrices of the first transformation relation, the second transformation relation, and the third transformation relation in sequence to obtain the total transformation relation.
[0103] In this embodiment, the total transformation matrix characterizing the positive transformation relationship is... = It can be understood that the total transformation matrix reflects the pose relationship between the radar coordinate system and the quay crane coordinate system. By using the total transformation matrix to rotate and translate the radar coordinate system in the X, Y, and Z directions, the transformation from the radar coordinate system to the quay crane coordinate system can be achieved.
[0104] It should be noted that the execution order of each step in the above method embodiments is not limited to that shown in the attached figures, and the execution order of each step shall be subject to the actual application situation.
[0105] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: First, in this invention, when the quay crane trolley moves to one side of the trolley track, a radar coordinate system can be established based on the installation position of the lidar, and a quay crane coordinate system can be established based on the center point of the crane lifting device when it is lowered to the ground. Secondly, in this invention, the point cloud data required for calibration is collected when the quay crane trolley is at a specific position on the trolley track that meets the preset acquisition conditions, and the crane's lifting device is suspended at a specific height that meets the preset acquisition conditions. In this invention, the selected markers for calibration are: the ground portion, the inner side of the portal frame containing the two quay crane columns, and the lifting device in the air. Therefore, the preset acquisition conditions mean that the lidar's field of view simultaneously includes the ground portion, the two quay crane columns, and the lifting device in the air. Third, considering the geometric positional characteristics of the LiDAR and crane spreader in the application scenario, a simple, effective, and high-precision LiDAR calibration method is provided: First, the Z-direction transformation relationship between the radar coordinate system and the quay crane coordinate system is calibrated using ground point cloud data. Then, the Y-direction transformation relationship between the radar coordinate system and the quay crane coordinate system is calibrated using the point cloud data of the inner side of the gantry frame containing the two quay crane columns from the first transformed point cloud data. This Y-direction transformation relationship includes at least a portion of the Y-direction offset. Finally, the X-direction offset and another portion of the Y-direction offset (ideally, the Y-direction offset is 0) between the radar coordinate system and the quay crane coordinate system are calibrated using the spreader point cloud data corresponding to the crane spreader from the second transformed point cloud data. This calibration process determines the total transformation matrix between the radar coordinate system and the quay crane coordinate system, enabling the crane controller to automate operations based on LiDAR, thereby promoting port automation and improving port efficiency and safety.
[0106] In order to perform the corresponding steps in the above method embodiments and various possible implementations, an implementation of a lidar calibration device is given below.
[0107] Please see Figure 7 , Figure 7 A schematic diagram of the structure of a lidar calibration device provided in an embodiment of the present invention is shown. The lidar calibration device 200 includes: an acquisition module 210, a processing module 220, and a merging module 230.
[0108] The acquisition module 210 is used to acquire point cloud data, which is obtained by the LiDAR on the quay crane trolley scanning the ground, crane spreader and trolley column within the field of view. The processing module 220 is used to determine a first transformation relationship, a second transformation relationship, and a third transformation relationship based on point cloud data; wherein the first transformation relationship and the second transformation relationship reflect the transformation relationship between the radar position and the hoist position in a first direction, a second direction, and a third direction in three-dimensional space, respectively; The merging module 230 is used to determine the overall transformation relationship between the radar coordinate system and the quay crane coordinate system based on the first transformation relationship, the second transformation relationship and the third transformation relationship.
[0109] Optionally, the processing module 220 is specifically used for: determining a first transformation relationship based on the ground point cloud in the point cloud data; transforming the point cloud data using the first transformation relationship to obtain first transformed point cloud data; determining a second transformation relationship based on the related point clouds of two reference planes in the first transformed point cloud data; the reference planes are planes orthogonal to the ground; transforming the first transformed point cloud data using the second transformation relationship to obtain second transformed point cloud data; and determining a third transformation relationship based on the lifting point cloud of the crane lifting device in the second transformed point cloud data.
[0110] Those skilled in the art will clearly understand that the acquisition module 210, processing module 220, and merging module 230 can be used to implement the above steps S101, S102, S103, and their respective sub-steps. For the sake of convenience and brevity, the specific working process of the lidar calibration device 200 described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0111] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 300 includes a processor 310, a memory 320, and a bus 330, with the processor 310 connected to the memory 320 via the bus 330.
[0112] The memory 320 can be used to store software programs, such as the software program corresponding to the lidar calibration device 200 provided in the embodiments of the present invention. The processor 310 executes various functional applications and data processing by running the software program stored in the memory 320 to implement the lidar calibration method provided in the embodiments of the present invention.
[0113] The memory 320 may be, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), FLASH (Flash Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.
[0114] The processor 310 can be an integrated circuit chip with signal processing capabilities. The processor 310 can be a general-purpose processor, including: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be: DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0115] Understandable. Figure 8 The structure shown is for illustrative purposes only; the electronic device 300 may also include components that are more advanced than those shown. Figure 8 The more or fewer components shown, or having the same Figure 8 The different configurations shown. Figure 8 The components shown can be implemented using hardware, software, or a combination thereof.
[0116] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the lidar calibration method disclosed in the above embodiments. The computer-readable storage medium can be, but is not limited to, various media capable of storing program code, such as a USB flash drive, portable hard drive, ROM, RAM, PROM, EPROM, EEPROM, FLASH disk, or optical disk.
[0117] In summary, this invention provides a lidar calibration method, apparatus, electronic device, and readable storage medium. First, point cloud data from a lidar scan is obtained. This point cloud data is obtained by the lidar mounted on the quay crane trolley scanning the ground, crane spreader, and trolley column within its field of view. Next, based on the point cloud data, a first transformation relationship, a second transformation relationship, and a third transformation relationship are determined. Finally, based on the first, second, and third transformation relationships, a total transformation relationship is determined. Since the first, second, and third transformation relationships respectively reflect the transformation relationships of the radar position and the spreader position in a first direction, a second direction, and a third direction in three-dimensional space, the final determined total transformation relationship can realize the conversion from the lidar coordinate system to the quay crane coordinate system, thereby achieving pose calibration between the lidar and the crane spreader.
[0118] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A lidar calibration method, characterized in that, include: Point cloud data is obtained by scanning the ground, crane spreader and main body of the crane with a lidar mounted on the quay crane trolley within the field of view using a lidar. Based on the ground point cloud in the point cloud data, determine the first transformation relationship; The point cloud data is transformed using the first transformation relationship to obtain the first transformed point cloud data; Multiple related point clouds of two reference planes are sampled from the first converted point cloud data; the two reference planes are the inner sides of the two large vehicle columns represented in the first converted point cloud data, and the ground and the large vehicle track on the ground are perpendicular to the inner sides of the two large vehicle columns. Based on multiple related point clouds of the two reference planes, the plane equations of the two reference planes are fitted respectively. The equation of the intermediate plane is obtained by averaging the plane equations of the two reference planes with parameters. The Z-direction of the intermediate plane equation is set to zero to obtain the plane equation of the translation plane; Based on the plane equation of the translation plane, the normal vector and the offset in the Y direction of the translation plane are obtained; The second rotation matrix required to convert the normal vector of the translation plane into the unit normal vector of the Y-axis of the quay crane coordinate system; wherein the quay crane coordinate system is established with the projection point of the crane spreader center on the ground as the origin; The second transformation relationship is determined based on the second rotation matrix and the Y-direction offset of the translation plane; The first transformed point cloud data is transformed using the second transformation relationship to obtain the second transformed point cloud data; Based on the crane lifting point cloud in the second transformed point cloud data, a third transformation relationship is determined; wherein, the first transformation relationship, the second transformation relationship, and the third transformation relationship reflect the transformation relationship between the radar position and the lifting position in the Z, Y, and X directions in three-dimensional space, respectively; Based on the first transformation relationship, the second transformation relationship, and the third transformation relationship, the overall transformation relationship between the radar coordinate system and the quay crane coordinate system is determined.
2. The method according to claim 1, characterized in that, The step of determining the first transformation relationship based on the ground point cloud in the point cloud data includes: At least three ground point clouds are sampled from the point cloud data; Based on the at least three ground point clouds, fit the plane equation of the ground; Based on the plane equation of the ground, the normal vector of the ground and the height value between the lidar and the ground are obtained; The first rotation matrix required to convert the ground normal vector into the Z-axis unit normal vector of the quay bridge coordinate system; The first transformation relationship is determined based on the first rotation matrix and the height value between the lidar and the ground.
3. The method according to claim 1, characterized in that, The step of determining the third transformation relationship based on the crane spreader point cloud in the second transformed point cloud data includes: Segment all the lifting point clouds of the crane lifting device from the second transformed point cloud data; After setting the Z-axis coordinate value of each of the lifting device point clouds to 0, grid downsampling processing is performed to obtain multiple uniformly distributed sampling point clouds; Determine the centroid coordinates of the multiple sampling point clouds; The third transformation relationship is determined based on the centroid coordinates.
4. The method according to claim 1, characterized in that, The step of determining the overall transformation relationship between the radar coordinate system and the quay crane coordinate system based on the first transformation relationship, the second transformation relationship, and the third transformation relationship includes: The transformation matrices of the first transformation relation, the second transformation relation, and the third transformation relation are multiplied sequentially to obtain the total transformation relation.
5. A lidar calibration device, characterized in that, include: The acquisition module is used to obtain point cloud data, which is obtained by the lidar mounted on the quay crane trolley of the quay crane scanning the ground, crane spreader and trolley column within the field of view. Processing module, used for: Based on the ground point cloud in the point cloud data, determine the first transformation relationship; The point cloud data is transformed using the first transformation relationship to obtain the first transformed point cloud data; Multiple related point clouds of two reference planes are sampled from the first converted point cloud data; the two reference planes are the inner sides of the two large vehicle columns represented in the first converted point cloud data, and the ground and the large vehicle track on the ground are perpendicular to the inner sides of the two large vehicle columns. Based on multiple related point clouds of the two reference planes, the plane equations of the two reference planes are fitted respectively. The equation of the intermediate plane is obtained by averaging the plane equations of the two reference planes with parameters. The Z-direction of the intermediate plane equation is set to zero to obtain the plane equation of the translation plane; Based on the plane equation of the translation plane, the normal vector and the offset in the Y direction of the translation plane are obtained; The second rotation matrix required to convert the normal vector of the translation plane into the unit normal vector of the Y-axis of the quay crane coordinate system; wherein the quay crane coordinate system is established with the projection point of the crane spreader center on the ground as the origin; The second transformation relationship is determined based on the second rotation matrix and the Y-direction offset of the translation plane; The first transformed point cloud data is transformed using the second transformation relationship to obtain the second transformed point cloud data; Based on the crane lifting point cloud in the second transformed point cloud data, a third transformation relationship is determined; wherein, the first transformation relationship, the second transformation relationship, and the third transformation relationship reflect the transformation relationship between the radar position and the lifting position in the Z, Y, and X directions in three-dimensional space, respectively; The merging module is used to determine the overall transformation relationship between the radar coordinate system and the quay crane coordinate system based on the first transformation relationship, the second transformation relationship, and the third transformation relationship.
6. An electronic device, characterized in that, include: The electronic device includes a memory and a processor, wherein the memory stores a software program, and the processor executes the software program to implement the lidar calibration method as described in any one of claims 1-4 when the electronic device is running.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the lidar calibration method according to any one of claims 1-4.
Citation Information
Patent Citations
Gantry crane lifting appliance laser radar installation and calibration method and system
CN116106868A
Three-dimensional laser radar calibration method and device
CN116973893A
