Calibration method and device for excavator sensor, excavator and storage medium

Abstract

The invention provides an excavator sensor calibration method and device, an excavator and a storage medium, the action mode of a mechanical arm in the calibration process is that one joint is a moving joint, other joints are static joints, and first pose data of the moving joint in an excavator coordinate system are determined when the mechanical arm acts according to the action mode; second pose data of moving joints in a sensor coordinate system are fitted through point cloud collected by a distance measuring sensor, and the conversion relation between the sensor coordinate system and an excavator coordinate system is determined through the first pose data and the second pose data. According to the method, on the one hand, extra targets do not need to be carried and installed, manual intervention of the calibration process is not needed, the calibration efficiency is improved, on the other hand, errors caused by target installation are avoided, calibration is carried out through multiple different movement joints, and the calibration precision of the sensor is improved.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent construction machinery technology, and in particular to a calibration method, device, excavator, and storage medium for excavator sensors. Background Technology

[0002] With the continuous improvement of the intelligence level of construction machinery, the automation and unmanned operation technology of excavators has become the key direction of industry development. In the process of automated excavation, sensors (laser sensors, millimeter-wave radar, ultrasonic sensors, etc.) play a crucial role, mainly used for core functions such as environmental perception, obstacle detection, operation trajectory planning, and autonomous control.

[0003] The accuracy and reliability of a sensor are highly dependent on the accuracy of its external parameter calibration, such as... Figure 1 As shown, when calibrating the sensors on the excavator, a calibration ball needs to be suspended on the bucket as a target. The sensor is calibrated by using this target to first classify the sensor as an external parameter of the excavator body.

[0004] The above-mentioned method of using a suspended calibration ball as a target requires carrying the calibration ball and spending time installing it. The calibration accuracy depends on the installation accuracy, which means that the entire calibration process needs to rely on external objects, resulting in low efficiency and difficulty in guaranteeing calibration accuracy. Summary of the Invention

[0005] This disclosure provides a calibration method, apparatus, excavator, and storage medium for excavator sensors, to solve the problems of low calibration efficiency and difficulty in ensuring calibration accuracy caused by the need to carry and install targets to calibrate excavator sensors.

[0006] In a first aspect, this disclosure provides a calibration method for excavator sensors, wherein the excavator's robotic arm includes multiple joints, and the excavator sensor calibration method includes:

[0007] The excavator's robotic arm is controlled to move according to at least one preset set of action modes, wherein during the calibration process, one joint is a moving joint and the other joints are stationary joints;

[0008] When the robotic arm moves in the manner described, the first pose data of the motion joint relative to the excavator coordinate system is determined.

[0009] At least two frames of point cloud data are collected during the movement of the joint using a ranging sensor.

[0010] The second pose data of the motion joint relative to the sensor coordinate system is determined based on at least two frames of point cloud;

[0011] The transformation relationship between the sensor coordinate system and the excavator coordinate system is determined using the first pose data and the second pose data of multiple motion joints.

[0012] Secondly, this disclosure provides a calibration device for excavator sensors, wherein the excavator's robotic arm includes multiple joints, and the excavator sensor calibration method includes:

[0013] The robotic arm control module is used to control the robotic arm of the excavator to move according to at least one set of preset action modes, wherein during the calibration process, one joint is a moving joint and the other joints are stationary joints;

[0014] The first pose data determination module is used to determine the first pose data of the motion joint relative to the excavator coordinate system when the robotic arm moves in the manner described.

[0015] The point cloud acquisition module is used to acquire at least two frames of point cloud data during the movement of the joint using a ranging sensor.

[0016] The second pose data determination module is used to determine the second pose data of the motion joint relative to the sensor coordinate system based on at least two frames of point cloud.

[0017] The transformation relationship determination module is used to determine the transformation relationship between the sensor coordinate system and the excavator coordinate system using the first pose data and the second pose data of multiple motion joints.

[0018] Thirdly, this disclosure provides an excavator, characterized in that the excavator includes computer equipment and sensors, the computer equipment comprising:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the excavator sensor calibration method described in the first aspect of this disclosure.

[0022] Fourthly, this disclosure provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the excavator sensor calibration method described in the first aspect of this disclosure.

[0023] In this embodiment of the disclosure, during the calibration process, the robotic arm operates with one joint as a moving joint and the other joints as stationary joints. When the robotic arm moves according to at least one set of motion patterns, the first pose data of the moving joint relative to the excavator coordinate system is determined, and the second pose data of the moving joint relative to the sensor coordinate system is fitted by at least two frames of point cloud data collected by the ranging sensor. The transformation relationship between the sensor coordinate system and the excavator coordinate system is determined by using the first pose data and the second pose data of the moving joints. This realizes the use of each joint in the robotic arm as a moving joint as a target, and the determination of the coordinate transformation relationship by fitting the second pose data and the first pose data of the moving joints by collecting point cloud data. On the one hand, by using different joints on the robotic arm as targets, there is no need to carry or install additional targets, and no need for manual intervention in the calibration process, which improves the calibration efficiency. On the other hand, it avoids the error introduced by target installation, and the calibration accuracy of the sensor is improved by calibrating through multiple different moving joints.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the existing technology of calibrating by suspending a calibration ball on an excavator;

[0027] Figure 2 This is a flowchart of a calibration method for excavator sensors provided in an embodiment of this disclosure;

[0028] Figure 3 This is a structural diagram of an excavator;

[0029] Figure 4 This is a flowchart of a calibration method for excavator sensors provided in an embodiment of this disclosure;

[0030] Figure 5 This is a schematic diagram of the structure of a calibration device for an excavator sensor provided in an embodiment of this disclosure;

[0031] Figure 6 This is a schematic diagram of the structure of the excavator provided in the embodiments of this disclosure. Detailed Implementation

[0032] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present disclosure, and not all embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0033] Figure 2 This is a flowchart illustrating a calibration method for an excavator sensor according to an embodiment of this disclosure. This embodiment is applicable to calibrating a distance measuring sensor on an excavator to determine the transformation relationship between the distance measuring sensor's coordinate system and the excavator's coordinate system. This method can be executed by a calibration device for the excavator sensor, which can be implemented in hardware and / or software and can be configured in a computer device. Figure 2 As shown, the calibration method for the excavator's sensors includes:

[0034] S201. Control the excavator's robotic arm to move according to at least one preset set of motion modes, wherein during the calibration process, one joint is a moving joint and the other joints are stationary joints.

[0035] The excavator in this embodiment can be a manned excavator or an unmanned excavator. The excavator's robotic arm includes multiple joints and is equipped with a ranging sensor, such as... Figure 3 The schematic diagram of the excavator shown includes a robotic arm that can include a boom 1, a stick 2, and a bucket 3. The excavator is also equipped with a ranging sensor 4, which can be a sensor such as a lidar, millimeter-wave radar, or ultrasonic radar. The robotic arm can include joints P1, P2, and P3, where joints P1, P2, and P3 can be rotating axes. The robotic arm can move by the coordinated rotation of joints P1, P2, and P3. For example, when joint P1 rotates, it can lift or lower the boom 1; when joint P2 rotates, it can control the stick 2 to extend or retract; and when joint P3 rotates, it can control the bucket 3 to scoop or unload material.

[0036] The action mode is used to indicate the state of each joint during the calibration process. Each action mode defines a moving joint and at least one stationary joint. The moving joint can be a joint that can translate and rotate during the calibration process, and the stationary joint can be a joint that remains stationary during the calibration process, such as a joint that stops rotating. In this embodiment, multiple action modes can be predefined, and the moving joints in each action mode are different.

[0037] Since each set of actions defines moving joints and stationary joints, when controlling the excavator to execute each set of actions, the stationary joints can be kept stationary, such as stopping rotation, and the moving joints can be controlled to move, so that the robotic arm can move.

[0038] S202. When the robotic arm moves according to the action mode, determine the first position data of the moving joint relative to the excavator coordinate system.

[0039] In this embodiment, the kinematic parameters of the robotic arm can be calibrated in advance. For example, the origin P0 of the excavator coordinate system is defined on the base. Then, the distance between two adjacent points in the origin P0, joint P1, joint P2 and joint P3 are measured in sequence, and the coordinate transformation relationship from joint P1 to origin P0, from joint P2 to joint P1 and from joint P3 to joint P2 is calibrated.

[0040] When the moving joint moves and the stationary joint remains stationary, the first pose data of the moving joint relative to the origin P0 of the excavator coordinate system can be calculated based on the pre-calibrated kinematic parameters, the current rotation angle of the stationary joint, the rotation angle of the moving joint, and the distance between adjacent joints.

[0041] S203. Collect at least two frames of point cloud data during joint motion using a ranging sensor.

[0042] During the movement of the joint, the ranging sensor can be controlled to collect point clouds every time the joint rotates by a preset angle, or the ranging sensor can be controlled to collect point clouds at a preset frame rate to obtain at least two frames of point clouds.

[0043] Taking a ranging sensor as an example of a lidar, during the movement of the joint, the lidar can be controlled to scan the robotic arm and its environment at a preset frame rate to obtain at least two frames of point cloud.

[0044] S204. Determine the second pose data of the motion joint relative to the sensor coordinate system based on at least two frames of point cloud.

[0045] The point cloud collected by the ranging sensor includes the point cloud of the robotic arm and the environmental point cloud of the environment in which the robotic arm is located. The environmental point cloud can be filtered out from the collected point cloud or the robotic arm point cloud can be directly extracted after identifying the robotic arm point cloud. Then, the motion joint is fitted by at least two frames of robotic arm point cloud when the motion joint is rotated to different rotation angles, so as to determine the second pose data of the motion joint relative to the sensor coordinate system.

[0046] S205. The transformation relationship between the sensor coordinate system and the excavator coordinate system is determined by using the first pose data and the second pose data of multiple motion joints.

[0047] In this embodiment, the transformation relationship between the sensor coordinate system and the excavator coordinate system can be a homogeneous transformation matrix. Sensor calibration is equivalent to solving this homogeneous transformation matrix. The third pose data of the moving joint relative to the excavator coordinate system can be calculated by using this homogeneous transformation matrix and the second pose data. The error between the third pose data and the first pose data is calculated. Solving for the homogeneous transformation matrix that minimizes the error yields the transformation relationship between the sensor coordinate system and the excavator coordinate system.

[0048] In this embodiment, the robotic arm is configured to have one joint as a moving joint and the others as stationary joints. When the robotic arm moves according to the specified motion, the first pose data of the moving joint relative to the excavator coordinate system is determined, and the second pose data of the moving joint relative to the sensor coordinate system is fitted by at least two frames of point cloud data collected by a ranging sensor. The transformation relationship between the sensor coordinate system and the excavator coordinate system is determined by using the first pose data and the second pose data of multiple moving joints. This allows each joint in the robotic arm to be used as a target, and the coordinate transformation relationship is determined by fitting the second pose data and the first pose data of the moving joints with point cloud data. On the one hand, using different joints on the robotic arm body as targets eliminates the need to carry or install additional targets and eliminates the need for manual intervention in the calibration process, thus improving calibration efficiency. On the other hand, it avoids errors introduced by target installation, and the calibration accuracy of the sensor is improved by calibrating through multiple different moving joints.

[0049] Figure 4 A flowchart illustrating an optional embodiment of the present disclosure provides a method for calibrating excavator sensors, as shown below. Figure 4 As shown, the calibration method for the excavator's sensors includes:

[0050] S401. Set at least one set of motion modes for the excavator's robotic arm, wherein during the calibration process, one joint is a moving joint and the other joints are stationary joints.

[0051] like Figure 3 As shown, in the excavator of this embodiment, the robotic arm may include a boom 1, a stick 2 and a bucket 3, and the joints on the robotic arm may include joint P1, joint P2 and joint P3, wherein joint P1, joint P2 and joint P3 may be rotation axes.

[0052] The action mode is used to indicate the state of each joint during the calibration process. Each action mode defines one moving joint and at least one stationary joint. The moving joint can be a joint that needs to move during the calibration process, and the stationary joint can be a joint that remains stationary during the calibration process. In this embodiment, multiple action modes can be defined, and the moving joints in each action mode are different.

[0053] In one embodiment, the excavator's robotic arm includes a boom joint, a stick joint, and a bucket joint, such as... Figure 3 As shown, the boom joint is joint P1, the stick joint is joint P2, and the bucket joint is joint P3. Taking the setting of three sets of action modes as an example, the action mode setting process is as follows:

[0054] The first set of action modes is set as follows: the boom joint (joint P2) and bucket joint (joint P3) are set as stationary joints, and the boom joint (joint P1) is set as a moving joint.

[0055] The second set of action modes is set as follows: the boom joint (joint P1) and bucket joint (joint P3) are set as stationary joints, and the stick joint (joint P2) is set as a moving joint.

[0056] The third set of action modes is set as follows: the boom joint (joint P1) and stick joint (joint P2) are set as stationary joints, and the bucket joint (joint P3) is set as a moving joint.

[0057] Of course, the above example only uses three joints. In practical applications, when the robotic arm includes more joints, more sets of motion methods can be set up by referring to the above example.

[0058] S402, Controlling the stationary joints in the excavator's robotic arm to stop rotating, and controlling the rotation of the moving joints.

[0059] In this embodiment, the excavator controls the robotic arm's movements sequentially according to each set of action methods. Specifically, taking joint movement as rotation as an example, a moving joint refers to a joint that needs to rotate, while a stationary joint can refer to a joint that is prohibited from rotating but can translate. The stationary joint can be controlled to stop rotating, and the moving joint can be controlled to rotate. Taking the first set of action methods set above as an example, such as... Figure 3 As shown, the stick joint (joint P2) and bucket joint (joint P3) can be controlled to stop rotating, while the boom joint (joint P1) can be controlled to rotate. That is, the entire robotic arm moves through the boom 1, which drives the stick 2 and bucket 3 to translate.

[0060] S403. When the robotic arm moves according to the action mode, determine the rotation angles of the stationary joints and the moving joints.

[0061] An angle sensor (such as a rotary encoder) can also be provided at each joint in this embodiment. The angle sensor is used to detect the current rotation angle of each joint. The rotation angle of each joint can be detected by the angle sensor. Taking the first set of action modes mentioned above as an example, although the stick joint (joint P2) and the bucket joint (joint P3) do not rotate, the current rotation angle (the angle of rotation relative to the reference starting angle) of the stick joint (joint P2) and the bucket joint (joint P3) can still be detected by the angle sensor. The rotation angle of the boom joint (joint P1) can also be detected by the angle sensor during the rotation of the boom joint (joint P1).

[0062] S404. Calculate the first pose data of the moving joint relative to the excavator coordinate system based on the rotation angle, the preset distance between each joint, and the pre-calibrated kinematic parameters.

[0063] like Figure 3 As shown, the origin of the excavator's coordinate system is defined as P0, and the origins of the coordinate systems of each joint are P1, P2, and P3, respectively. Let denote the homogeneous transformation matrix from coordinate system i to coordinate system j. Then, let denote the homogeneous transformation matrix from the coordinate system containing the bucket joint (joint P3) to the excavator coordinate system. for:

[0064] ;

[0065] in, It is a 3×3 rotation matrix. The translation matrix is ​​3×1. Similarly, the homogeneous transformation matrices from the coordinate system of the stick joint to the coordinate system of the excavator can be obtained. The homogeneous transformation matrix from the coordinate system of the boom joint to the coordinate system of the excavator. .

[0066] Based on the above formula, the first pose data of a certain joint i on the robotic arm in the excavator coordinate system can be obtained. as follows:

[0067] ;

[0068] Let i represent the direction vector of joint i in the excavator coordinate system. Represents the displacement of joint i in the excavator coordinate system, where and The transformation matrix from the coordinate system of joint i to the coordinate system of the excavator can be calculated using the formula.

[0069] Taking the boom joint (joint P1) and stick joint (joint P2) of the third set of actions as stationary joints and the bucket joint (joint P3) as a moving joint, the rotation angles of the boom joint (joint P1), stick joint (joint P2), and bucket joint (joint P3), the distance from the boom joint (joint P1) to the stick joint (joint P2), and the distance from the stick joint (joint P2) to the bucket joint (joint P3) of the third set of actions can be input into formula (1) to obtain the first position data of joint P3 in the excavator coordinate system. Similarly, the first pose data of joint P2 in the excavator coordinate system can be calculated. The first pose data of joint P1 in the excavator coordinate system .

[0070] S405. During the rotation of the moving joint, at least two frames of point cloud are obtained by collecting point cloud data according to a preset frame rate using a ranging sensor.

[0071] During the rotation of each joint, the ranging sensor can be controlled to collect point clouds at a pre-configured frame rate to obtain at least two frames of point clouds. Each frame of point cloud includes the point cloud of the robotic arm and the point cloud of the environment in which the robotic arm is located.

[0072] S406. Remove the background point cloud from each frame of point cloud to obtain the robotic arm point cloud.

[0073] In one embodiment, the point cloud of the robotic arm can be extracted from a pre-configured robotic arm recognition model.

[0074] In another alternative embodiment, static background point clouds can be filtered using background subtraction. Specifically, the point cloud can first be converted into a voxel mesh or elevation map, and then the difference image D(x,y) can be obtained by calculating the difference between the current frame I(x,y) and the static background B(x,y). For example, the absolute difference D(x,y)=|I(x,y)-B(x,y)| can be used, or the squared difference D(x,y)=(I(x,y)-B(x,y))×(I(x,y)-B(x,y)) can be used to obtain the difference image. The difference image is then binarized and a threshold is set. When the pixel value of a pixel in the difference image D(x,y) is greater than the threshold, it is determined to be a point of a dynamic object; when it is less than the threshold, it is determined to be a static background. Finally, the pixels corresponding to static pixels are filtered out from the difference image, and the remaining pixels are used to construct a point cloud to obtain the point cloud of the robotic arm.

[0075] Of course, those skilled in the art can also use other background recognition algorithms to filter the background point cloud from the point cloud to obtain the robotic arm point cloud.

[0076] S407. Use at least two frames of robotic arm point cloud to fit the second pose data of the moving joints relative to the sensor coordinate system.

[0077] In an optional embodiment, the rotation angle of the motion joint when acquiring each frame of point cloud can be determined, and the second pose data of the motion joint relative to the sensor coordinate system can be fitted by at least two frames of robotic arm point cloud and the rotation angle corresponding to each frame of robotic arm point cloud.

[0078] Specifically, the second pose data of the moving joint relative to the sensor coordinate system is: To fit the rotation axis of the motion joint i in the sensor coordinate system, let p be the three-dimensional point on the robotic arm before the motion joint i rotates. pre Rotation angle of joint i The same three-dimensional point after that is p after The relationship between the two is:

[0079] p after =T i ×p pre ;

[0080] ;

[0081] Where e is a natural constant, representing the coordinates of multiple 3D points on the robotic arm before and after the rotation of joint i, as well as the rotation angle of joint i. Substituting into the above formula, the pose of multiple rotation axes can be solved. pose for multiple rotation axes After fitting, it is used as the second pose data of the rotational joint i.

[0082] In another alternative embodiment, the second pose data of the moving joint i can also be estimated based on a point cloud registration method (such as ICP (Iterative ClosestPoint)). .

[0083] S408. Calculate the third pose data of the motion joint using the second pose data of each motion joint and the target homogeneous transformation matrix. The target homogeneous transformation matrix is ​​the transformation matrix between the sensor coordinate system and the excavator coordinate system.

[0084] Let the transformation matrix from the sensor coordinate system to the excavator coordinate system be: The third pose data of joint i is .

[0085] S409. Construct an error loss function using the first and third pose data of each joint.

[0086] In one embodiment, the error loss function is as follows:

[0087] ;

[0088] Represents the error function. This represents the first pose data of the moving joint i.

[0089] S410. Solve for the target homogeneous transformation matrix when the error loss function is minimized using the first and third pose data of multiple motion joints.

[0090] Specifically, paired data from multiple sets of different motion joints i can be used. The homogeneous transformation matrix that minimizes the error loss function is: The transformation matrix is This represents the transformation relationship from the sensor coordinate system to the excavator coordinate system. The solution algorithm can be linear least squares, nonlinear least squares, etc.; this embodiment does not impose any restrictions on the solution algorithm.

[0091] In this embodiment, the robotic arm's movement is configured such that one joint is a moving joint and the others are stationary joints. After setting multiple sets of movement modes, when the robotic arm moves according to the movement mode, the first pose data of the rotating joint relative to the excavator coordinate system is calculated using the excavator's kinematic parameters. After removing the background point cloud to obtain the robotic arm point cloud, the second pose data of the moving joint relative to the sensor coordinate system is fitted using at least two frames of the robotic arm point cloud. An error loss function is constructed with the transformation matrix between the sensor coordinate system and the excavator coordinate system as variables. The target homogeneous transformation matrix that minimizes the error loss function is solved using the first pose data and third pose data of multiple moving joints. This achieves the goal of setting each joint in the robotic arm as a moving joint as a target, and determining the coordinate transformation relationship by fitting the second pose data and first pose data of the moving joints through point cloud acquisition. On the one hand, using different joints on the robotic arm body as targets eliminates the need to carry and install additional targets, and eliminates the need for manual intervention in the calibration process, thus improving calibration efficiency. On the other hand, it avoids the errors introduced by target installation, and the calibration accuracy of the sensor is improved by calibrating through multiple different moving joints.

[0092] Figure 5 This is a schematic diagram of a calibration device for an excavator sensor provided in an embodiment of this disclosure. Figure 5 As shown, the calibration device for the excavator sensor includes:

[0093] The robotic arm control module 501 is used to control the robotic arm of the excavator to move according to at least one set of preset action modes, wherein one joint is a moving joint and the other joints are stationary joints during the calibration process;

[0094] The first pose data determination module 502 is used to determine the first pose data of the motion joint relative to the excavator coordinate system when the robotic arm moves in the manner described.

[0095] The point cloud acquisition module 503 is used to acquire at least two frames of point cloud during the movement of the joint via a ranging sensor.

[0096] The second pose data determination module 504 is used to determine the second pose data of the motion joint relative to the sensor coordinate system based on at least two frames of point cloud.

[0097] The transformation relationship determination module 505 is used to determine the transformation relationship between the sensor coordinate system and the excavator coordinate system using the first pose data and the second pose data of the multiple motion joints.

[0098] Figure 6 A schematic diagram of an excavator 60 that can be used to implement embodiments of the present disclosure is shown. The excavator 60 includes a robotic arm, sensors, and a computer device 601, which is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, mainframe computers, smartphones, tablets, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0099] like Figure 6 As shown, computer device 601 includes at least one processor 6011 and a memory, such as read-only memory (ROM) 6012 and random access memory (RAM) 6013, communicatively connected to the at least one processor 6011. The memory stores computer programs executable by the at least one processor. The processor 6011 can perform various appropriate actions and processes based on the computer program stored in the ROM 6012 or loaded into the RAM 6013 from storage unit 6018. The RAM 6013 may also store various programs and data required for the operation of computer device 601. The processor 6011, ROM 6012, and RAM 6013 are interconnected via bus 6014. An input / output (I / O) interface 6015 is also connected to bus 6014.

[0100] Multiple components in computer device 601 are connected to I / O interface 6015, including: input unit 6016, such as a ranging sensor; output unit 6017, such as various types of displays, speakers, etc.; storage unit 6018, such as a disk, optical disk, etc.; and communication unit 6019, such as a network interface card, modem, wireless transceiver, etc. Communication unit 6019 allows computer device 601 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0101] Processor 6011 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 6011 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 6011 performs the various methods and processes described above, such as the calibration method for excavator sensors.

[0102] The calibration methods for excavator sensors include:

[0103] The excavator's robotic arm is controlled to move according to at least one preset set of action modes, wherein during the calibration process, one joint is a moving joint and the other joints are stationary joints;

[0104] When the robotic arm moves in the manner described, the first pose data of the motion joint relative to the excavator coordinate system is determined.

[0105] At least two frames of point cloud data are collected during the movement of the joint using a ranging sensor.

[0106] The second pose data of the motion joint relative to the sensor coordinate system is determined based on at least two frames of point cloud;

[0107] The transformation relationship between the sensor coordinate system and the excavator coordinate system is determined using the first pose data and the second pose data of multiple motion joints.

[0108] Optionally, before controlling the excavator's robotic arm to move according to at least one preset set of actions, the method further includes:

[0109] At least one set of motion modes is set for the excavator's robotic arm.

[0110] Optionally, the excavator's robotic arm includes a boom joint, a stick joint, and a bucket joint, and at least one set of operating modes is provided for the excavator's robotic arm, including:

[0111] The first set of action modes is set as follows: the stick joint and the bucket joint are set as stationary joints, and the boom joint is set as a moving joint;

[0112] The second set of action modes is set as follows: the boom joint and the bucket joint are set as stationary joints, and the stick joint is set as a moving joint;

[0113] The third set of action modes is set as follows: the boom joint and the stick joint are set as stationary joints, and the bucket joint is set as a moving joint.

[0114] Optionally, controlling the excavator's robotic arm to move according to at least one preset set of actions includes:

[0115] Control the stationary joints in the excavator's robotic arm to stop rotating, and control the rotation of the moving joints.

[0116] Optionally, when the robotic arm moves according to the described action mode, determining the first pose data of the motion joint relative to the excavator coordinate system includes:

[0117] When the robotic arm moves in the manner described, the rotation angles of the stationary joint and the moving joint are determined.

[0118] The first pose data of the moving joint relative to the excavator coordinate system is calculated based on the rotation angle, the preset distance between each joint, and the pre-calibrated kinematic parameters.

[0119] Optionally, at least two frames of point cloud data are acquired using a ranging sensor during the movement of the joint, including:

[0120] During the rotation of the joint, at least two frames of point cloud are obtained by the ranging sensor according to a preset frame rate.

[0121] Optionally, determining the second pose data of the motion joint relative to the sensor coordinate system based on at least two frames of point cloud includes:

[0122] The robotic arm point cloud is obtained by removing the background point cloud from each frame of the point cloud;

[0123] The second pose data of the motion joint relative to the sensor coordinate system is fitted using at least two frames of the robotic arm point cloud.

[0124] Optionally, the second pose data of the motion joint relative to the sensor coordinate system is fitted using at least two frames of the robotic arm point cloud, including:

[0125] Determine the rotation angle of the motion joint when acquiring each frame of point cloud;

[0126] The second pose data of the motion joint relative to the sensor coordinate system is fitted by at least two frames of robotic arm point cloud and the rotation angle corresponding to each frame of robotic arm point cloud.

[0127] Optionally, the transformation relationship between the sensor coordinate system and the excavator coordinate system is determined using the first pose data and the second pose data of multiple motion joints, including:

[0128] The third pose data of each joint is calculated using the second pose data of each joint and the target homogeneous transformation matrix, where the target homogeneous transformation matrix is ​​the transformation matrix between the sensor coordinate system and the excavator coordinate system.

[0129] An error loss function is constructed using the first pose data and the third pose data of each motion joint;

[0130] The target homogeneous transformation matrix is ​​calculated by using the first pose data and the third pose data of multiple motion joints to minimize the error loss function.

[0131] In some embodiments, the excavator sensor calibration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 6018. In some embodiments, part or all of the computer program may be loaded and / or installed on computer device 601 via ROM 6012 and / or communication unit 6019. When the computer program is loaded into RAM 6013 and executed by processor 6011, one or more steps of the excavator sensor calibration method described above may be performed. Alternatively, in other embodiments, processor 6011 may be configured to perform the excavator sensor calibration method by any other suitable means (e.g., by means of firmware).

[0132] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] Computer programs used to implement the methods of this disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of this disclosure, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0135] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0136] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0137] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0138] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0139] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A calibration method for excavator sensors, characterized in that, The excavator's robotic arm includes multiple joints, and the excavator's sensor calibration methods include: The excavator's robotic arm is controlled to move according to at least one preset set of action modes, wherein during the calibration process, one joint is a moving joint and the other joints are stationary joints; When the robotic arm moves in the manner described, the first pose data of the motion joint relative to the excavator coordinate system is determined. At least two frames of point cloud data are collected during the movement of the joint using a ranging sensor. The second pose data of the motion joint relative to the sensor coordinate system is determined based on at least two frames of point cloud; The transformation relationship between the sensor coordinate system and the excavator coordinate system is determined using the first pose data and the second pose data of multiple motion joints.

2. The method according to claim 1, characterized in that, Before controlling the excavator's robotic arm to move according to at least one preset set of actions, the method further includes: At least one set of motion modes is set for the excavator's robotic arm.

3. The method according to claim 2, characterized in that, The excavator's robotic arm includes a boom joint, a stick joint, and a bucket joint. At least one set of operating modes is configured for the excavator's robotic arm, including: The first set of action modes is set as follows: the stick joint and the bucket joint are set as stationary joints, and the boom joint is set as a moving joint; The second set of action modes is set as follows: the boom joint and the bucket joint are set as stationary joints, and the stick joint is set as a moving joint; The third set of action modes is set as follows: the boom joint and the stick joint are set as stationary joints, and the bucket joint is set as a moving joint.

4. The method according to claim 1, characterized in that, Controlling the excavator's robotic arm to move according to at least one preset set of actions, including: Control the stationary joints in the excavator's robotic arm to stop rotating, and control the rotation of the moving joints.

5. The method according to claim 1, characterized in that, When the robotic arm moves according to the described action method, the first pose data of the motion joint relative to the excavator coordinate system is determined, including: When the robotic arm moves in the manner described, the rotation angles of the stationary joint and the moving joint are determined. Based on the rotation angle, the preset distance between each joint, and the pre-calibrated kinematic parameters, the first pose data of the moving joint relative to the excavator coordinate system is calculated.

6. The method according to claim 1, characterized in that, At least two frames of point cloud data are acquired using a ranging sensor during the movement of the joint, including: During the rotation of the joint, at least two frames of point cloud are obtained by the ranging sensor according to a preset frame rate.

7. The method according to claim 1, characterized in that, Determining the second pose data of the motion joint relative to the sensor coordinate system based on at least two frames of point cloud data includes: The robotic arm point cloud is obtained by removing the background point cloud from each frame of the point cloud; The second pose data of the motion joint relative to the sensor coordinate system is fitted using at least two frames of the robotic arm point cloud.

8. The method according to claim 7, characterized in that, Fitting the second pose data of the motion joint relative to the sensor coordinate system using at least two frames of robotic arm point cloud data includes: Determine the rotation angle of the motion joint when acquiring each frame of point cloud; The second pose data of the motion joint relative to the sensor coordinate system is fitted by at least two frames of robotic arm point cloud and the rotation angle corresponding to each frame of robotic arm point cloud.

9. The method according to any one of claims 1-8, characterized in that, Determining the transformation relationship between the sensor coordinate system and the excavator coordinate system using the first pose data and the second pose data of multiple motion joints includes: The third pose data of the motion joint is calculated using the second pose data of each motion joint and the target homogeneous transformation matrix, where the target homogeneous transformation matrix is ​​the transformation matrix between the sensor coordinate system and the excavator coordinate system. An error loss function is constructed using the first pose data and the third pose data of each motion joint; The target homogeneous transformation matrix is ​​calculated by using the first pose data and the third pose data of multiple motion joints to minimize the error loss function.

10. A calibration device for an excavator sensor, characterized in that, The excavator's robotic arm includes multiple joints, and the excavator's sensor calibration device includes: The robotic arm control module is used to control the robotic arm of the excavator to move according to at least one set of preset action modes, wherein during the calibration process, one joint is a moving joint and the other joints are stationary joints; The first pose data determination module is used to determine the first pose data of the motion joint relative to the excavator coordinate system when the robotic arm moves in the manner described. The point cloud acquisition module is used to acquire at least two frames of point cloud data during the movement of the joint using a ranging sensor. The second pose data determination module is used to determine the second pose data of the motion joint relative to the sensor coordinate system based on at least two frames of point cloud. The transformation relationship determination module is used to determine the transformation relationship between the sensor coordinate system and the excavator coordinate system using the first pose data and the second pose data of multiple motion joints.

11. An excavator, characterized in that, The excavator includes computer equipment and sensors, the computer equipment including: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the excavator sensor calibration method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the calibration method for the excavator sensor as described in any one of claims 1-9.

Images