External parameter calibration method, device, computer-readable storage medium and robot

Abstract

The present application belongs to the field of robot technology, and in particular, relates to an external parameter calibration method, device, computer-readable storage medium and robot. The method includes: obtaining first point cloud data, and determining z-axis translation parameters based on the first point cloud data; wherein the first point cloud data is the point cloud data of the ground collected by the robot's depth camera; obtaining second point cloud data and third point cloud data, and determining x-axis translation parameters, y-axis translation parameters and rotation parameters based on the second point cloud data and the third point cloud data; wherein the second point cloud data is the point cloud data of the vertically intersecting wall collected by the depth camera, and the third point cloud data is the point cloud data of the vertically intersecting wall collected by the robot's laser radar. Through the present application, external parameter calibration can be performed only by relying on the ground and the wall without the aid of specific calibration objects, which is extremely convenient, conducive to large-scale promotion and use, and meets the needs of actual use.

Description

Technical Field

[0001] The present application belongs to the field of robotics technology, and in particular, relates to an external parameter calibration method, device, computer-readable storage medium and robot. Background Art

[0002] In mobile robot applications, due to the limited viewing angle of the laser radar, a depth camera is usually required to assist in the detection of obstacles. However, the coordinate systems of the depth camera and the laser radar are not coincident, so external parameter calibration is required, that is, the translation parameters of the x-axis, y-axis, and z-axis, as well as the pitch angle, roll angle, and roll angle, etc. are calibrated to convert the detection data of the laser radar and the depth camera into a unified coordinate system. However, in the prior art, external parameter calibration usually requires the use of specific calibration objects such as calibration plates, which is inconvenient, not conducive to large-scale promotion and use, and difficult to meet the needs of actual use. Summary of the invention

[0003] In view of this, the embodiments of the present application provide an external parameter calibration method, device, computer-readable storage medium and robot to solve the problem that the existing external parameter calibration method requires the use of specific calibration objects such as calibration plates, which is inconvenient, not conducive to large-scale promotion and use, and difficult to meet the needs of actual use.

[0004] A first aspect of an embodiment of the present application provides an external parameter calibration method, which may include:

[0005] Acquire first point cloud data, and determine a z-axis translation parameter according to the first point cloud data; wherein the first point cloud data is point cloud data of the ground collected by a depth camera of the robot;

[0006] Acquire second point cloud data and third point cloud data; wherein the second point cloud data is the point cloud data of the vertically intersecting wall collected by the depth camera, and the third point cloud data is the point cloud data of the vertically intersecting wall collected by the laser radar of the robot; determine the second fitting plane and the third fitting plane of the vertically intersecting wall according to the second point cloud data; determine the pitch angle and the roll angle according to the second fitting plane and the third fitting plane; project the second point cloud data to the plane where the third point cloud data is located to obtain the fourth point cloud data; determine the x-axis translation parameter, the y-axis translation parameter and the yaw angle according to the third point cloud data and the fourth point cloud data.

[0007] In a specific implementation of the first aspect, determining the z-axis translation parameter according to the first point cloud data may include:

[0008] Perform ground detection according to the first point cloud data;

[0009] If the ground detection fails, the angle of the depth camera is adjusted to perform ground detection again until the ground detection succeeds;

[0010] If the ground detection is successful, the z-axis translation parameter is determined according to the first point cloud data.

[0011] In a specific implementation of the first aspect, performing ground detection according to the first point cloud data may include:

[0012] Determine a first fitting plane of the ground according to the first point cloud data;

[0013] Calculating the number of ground points in the first point cloud data and the variance of the distances from each point in the first point cloud data to the first fitting plane; wherein the ground point is a point whose distance from the first fitting plane is less than a preset distance threshold;

[0014] If the number of the ground points is greater than a preset number threshold, and the variance is less than a preset variance threshold, then it is determined that the ground detection is successful;

[0015] If the number of the ground points is less than or equal to the number threshold, or the variance is greater than or equal to the variance threshold, it is determined that the ground detection fails.

[0016] In a specific implementation manner of the first aspect, determining an x-axis translation parameter, a y-axis translation parameter, and a yaw angle according to the third point cloud data and the fourth point cloud data may include:

[0017] The third point cloud data and the fourth point cloud data are matched with each other using a random sampling consensus algorithm to obtain the x-axis translation parameter, the y-axis translation parameter and the yaw angle.

[0018] In a specific implementation manner of the first aspect, after determining the x-axis translation parameter, the y-axis translation parameter, and the yaw angle according to the third point cloud data and the fourth point cloud data, the method may further include:

[0019] The x-axis translation parameter, the y-axis translation parameter and the yaw angle are corrected using a normal distribution transformation algorithm to obtain a corrected x-axis translation parameter, a corrected y-axis translation parameter and a corrected yaw angle.

[0020] In a specific implementation of the first aspect, after correcting the x-axis translation parameter, the y-axis translation parameter, and the yaw angle using a normal distribution transformation algorithm to obtain the corrected x-axis translation parameter, the corrected y-axis translation parameter, and the corrected yaw angle, the method may further include:

[0021] Performing three-dimensional expansion on the third point cloud data to obtain fifth point cloud data;

[0022] The corrected x-axis translation parameter, the corrected y-axis translation parameter, the corrected yaw angle, the pitch angle and the roll angle are optimized according to the fifth point cloud data to obtain optimized x-axis translation parameters, optimized y-axis translation parameters and optimized rotation parameters.

[0023] A second aspect of an embodiment of the present application provides an external parameter calibration device, which may include:

[0024] A ground calibration module, used to obtain first point cloud data and determine a z-axis translation parameter according to the first point cloud data; wherein the first point cloud data is point cloud data of the ground collected by the robot's depth camera;

[0025] A wall calibration module, used to obtain second point cloud data and third point cloud data; wherein the second point cloud data is the point cloud data of the vertically intersecting wall collected by the depth camera, and the third point cloud data is the point cloud data of the vertically intersecting wall collected by the laser radar of the robot; determine the second fitting plane and the third fitting plane of the vertically intersecting wall according to the second point cloud data; determine the pitch angle and the roll angle according to the second fitting plane and the third fitting plane; project the second point cloud data to the plane where the third point cloud data is located to obtain the fourth point cloud data; determine the x-axis translation parameter, the y-axis translation parameter and the yaw angle according to the third point cloud data and the fourth point cloud data.

[0026] In a specific implementation of the second aspect, the ground calibration module may include:

[0027] A ground detection unit, used for performing ground detection according to the first point cloud data;

[0028] The z-axis translation parameter determination unit is used to adjust the angle of the depth camera and re-perform ground detection if the ground detection fails, until the ground detection succeeds; if the ground detection succeeds, determine the z-axis translation parameter according to the first point cloud data.

[0029] In a specific implementation of the second aspect, the ground detection unit can be specifically used to: determine a first fitting plane of the ground based on the first point cloud data; calculate the number of ground points in the first point cloud data, and the variance of the distance from each point in the first point cloud data to the first fitting plane; wherein the ground point is a point whose distance from the first fitting plane is less than a preset distance threshold; if the number of ground points is greater than a preset number threshold, and the variance is less than a preset variance threshold, then the ground detection is determined to be successful; if the number of ground points is less than or equal to the number threshold, or the variance is greater than or equal to the variance threshold, then the ground detection is determined to have failed.

[0030] In a specific implementation of the second aspect, the second external parameter determination unit can be specifically used to: use a random sampling consensus algorithm to perform point cloud matching on the third point cloud data and the fourth point cloud data to obtain the x-axis translation parameter, the y-axis translation parameter and the yaw angle.

[0031] In a specific implementation of the second aspect, the wall calibration module may further include:

[0032] The external parameter correction unit is used to correct the x-axis translation parameter, the y-axis translation parameter and the yaw angle using a normal distribution transformation algorithm to obtain a corrected x-axis translation parameter, a corrected y-axis translation parameter and a corrected yaw angle.

[0033] In a specific implementation of the second aspect, the wall calibration module may further include:

[0034] The external parameter optimization unit is used to perform three-dimensional expansion on the third point cloud data to obtain fifth point cloud data; according to the fifth point cloud data, the corrected x-axis translation parameter, the corrected y-axis translation parameter, the corrected yaw angle, the pitch angle and the roll angle are optimized to obtain optimized x-axis translation parameters, optimized y-axis translation parameters and optimized rotation parameters.

[0035] A third aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned external parameter calibration methods are implemented.

[0036] The fourth aspect of an embodiment of the present application provides a robot, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned external parameter calibration methods when executing the computer program.

[0037] A fifth aspect of an embodiment of the present application provides a computer program product. When the computer program product runs on a robot, the robot executes the steps of any one of the above-mentioned external parameter calibration methods.

[0038] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application obtain first point cloud data, and determine z-axis translation parameters based on the first point cloud data; wherein the first point cloud data is the point cloud data of the ground collected by the robot's depth camera; obtain second point cloud data and third point cloud data, and determine x-axis translation parameters, y-axis translation parameters and rotation parameters based on the second point cloud data and the third point cloud data; wherein the second point cloud data is the point cloud data of the vertically intersecting wall collected by the depth camera, and the third point cloud data is the point cloud data of the vertically intersecting wall collected by the robot's laser radar. Through the embodiments of the present application, external parameter calibration can be performed only by relying on the ground and the wall without the aid of specific calibration objects such as calibration plates, which is extremely convenient, conducive to large-scale promotion and use, and meets the needs of actual use. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 This is a flow chart of an embodiment of an external parameter calibration method in an embodiment of the present application;

[0041] Figure 2 is a schematic diagram of projecting the second point cloud data onto a target two-dimensional plane;

[0042] Figure 3 Schematic diagram of point cloud matching;

[0043] Figure 4 Schematic diagram of the final calibration result;

[0044] Figure 5 This is a structural diagram of an embodiment of an external parameter calibration device in an embodiment of the present application;

[0045] Figure 6 This is a schematic block diagram of a robot in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0048] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0049] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0050] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0051] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0052] See also Figure 1 , an embodiment of an external parameter calibration method in the embodiment of the present application may include:

[0053] Step S101: Acquire first point cloud data, and determine a z-axis translation parameter according to the first point cloud data.

[0054] The first point cloud data is point cloud data of the ground collected by the robot's depth camera.

[0055] In a specific implementation of an embodiment of the present application, considering that due to viewing angle and scene issues, some parts of the ground may not be flat enough, in this case, the robot can be controlled to rotate to adjust the angle of the depth camera until a relatively flat ground is found at a certain angle.

[0056] Specifically, the robot may obtain the first point cloud data collected by the depth camera at an initial angle, and perform ground detection according to the first point cloud data.

[0057] When performing ground detection, the robot can use the Random Sample Consensus (RANSAC) algorithm to determine the fitting plane of the ground based on the first point cloud data. For the sake of distinction, it is recorded here as the first fitting plane. For example, three non-collinear points can be randomly selected from the first point cloud data to form a plane, and then the distance from each point in the first point cloud data to the plane is calculated, and the points with a distance less than a preset distance threshold are marked as the inner points of the plane, and the number of inner points of the plane is counted, and then the above process is repeated several times, and finally the plane with the largest number of inner points is determined as the first fitting plane. The specific value of the distance threshold can be set according to actual conditions, and the embodiment of the present application does not specifically limit it. For example, it can be set to 1 cm, 3 cm, 5 cm, and so on.

[0058] After determining the first fitting plane, the number of ground points in the first point cloud data and the variance of the distances from each point in the first point cloud data to the first fitting plane can be calculated; wherein the ground point is a point whose distance from the first fitting plane is less than the distance threshold.

[0059] If the number of ground points is greater than a preset number threshold, and the variance is less than a preset variance threshold, the ground detection is determined to be successful; if the number of ground points is less than or equal to the number threshold, or the variance is greater than or equal to the variance threshold, the ground detection is determined to have failed. The specific value of the number threshold can be set according to actual conditions, and the embodiment of the present application does not specifically limit it. For example, it can be set to 70%, 80%, 90%, etc. of the total number of points in the first point cloud data. The specific value of the variance threshold can be set according to actual conditions, and the embodiment of the present application does not specifically limit it. For example, it can be set to 2, 3, 5, etc.

[0060] If the ground detection fails, the angle of the depth camera can be adjusted to re-perform ground detection until the ground detection is successful; if the ground detection is successful, the z-axis translation parameter, that is, the distance between the depth camera and the first fitting plane, can be determined based on the first point cloud data.

[0061] Preferably, when the depth camera collects point cloud data, it can collect the point cloud data multiple times at each angle, and use the average of the point cloud data collected multiple times as the first point cloud data, so as to improve the reliability of the data.

[0062] Step S102: Acquire the second point cloud data and the third point cloud data, and determine the x-axis translation parameter, the y-axis translation parameter and the rotation parameter according to the second point cloud data and the third point cloud data.

[0063] The second point cloud data is the point cloud data of the vertically intersecting wall collected by the depth camera. When collecting point cloud data, the viewing angle of the depth camera should be directed to the angle bisector of the two walls as much as possible. The third point cloud data is the point cloud data of the vertically intersecting wall collected by the robot's laser radar. Generally, the laser radar can be a single-line laser radar, which mainly performs planar scanning.

[0064] In the embodiment of the present application, the robot can use the RANSAC algorithm to determine the two fitting planes of the perpendicularly intersecting wall according to the second point cloud data. For the sake of distinction, they are respectively recorded as the second fitting plane and the third fitting plane. The specific plane fitting method can refer to the specific description in step S101, which will not be repeated here.

[0065] After determining the second fitting plane and the third fitting plane, the robot can determine the pitch angle (picth) and the roll angle (roll) according to the second fitting plane and the third fitting plane. Specifically, the normal vector of the second fitting plane and the normal vector of the third fitting plane are calculated respectively, the two normal vectors are cross-multiplied, the result is recorded as the gravity vector, and the posture rotation matrix can be solved according to the projection component of the gravity vector, so as to determine the pitch angle and the roll angle.

[0066] Preferably, after completing the solution, the solved attitude rotation matrix can be further verified by converting the normal vector of the second fitting plane and the normal vector of the third fitting plane respectively. If the converted z-axis component is less than a preset threshold, it means that the solution result is basically accurate and the verification is passed.

[0067] After the verification is passed, the second point cloud data may be projected onto a target two-dimensional plane to obtain fourth point cloud data, wherein the target two-dimensional plane is the plane where the third point cloud data is located. Figure 2 What is shown is a schematic diagram of projecting the second point cloud data onto the target two-dimensional plane, in which the smaller points are the second point cloud data, and the larger points are the fourth point cloud data obtained after projection.

[0068] After the projection is completed, the x-axis translation parameter, the y-axis translation parameter and the yaw angle (yaw) can be determined according to the third point cloud data and the fourth point cloud data. Specifically, in the embodiment of the present application, the RANSAC algorithm can be used to perform point cloud matching on the third point cloud data and the fourth point cloud data to obtain the x-axis translation parameter, the y-axis translation parameter and the yaw angle.

[0069] Preferably, in a specific implementation of an embodiment of the present application, considering that the point noise after projection is large, the result obtained is only a rough value, so the normal distribution transform (NDT) algorithm can be further used to match and refine the obtained rough value, and the previous rough value is used as the initial value of the NDT matching, and the NDT matching is performed to obtain the corrected x-axis translation parameter, the corrected y-axis translation parameter and the corrected yaw angle. Figure 3 The figure shows a schematic diagram of point cloud matching, in which the upper point is the third point cloud data, and the lower point is the fourth point cloud data. After point cloud matching and correction, the fourth point cloud data can basically overlap with the third point cloud data.

[0070] Preferably, in a specific implementation of the embodiment of the present application, in order to further reduce the error, under the premise that the wall is vertical and smooth, the third point cloud data can be three-dimensionally expanded, and the third point cloud data can be translated up and down to expand into a three-dimensional point cloud, which is recorded as the fifth point cloud data. Then, the previously obtained external parameters are further optimized according to the plane constraints provided by the fifth point cloud data, and the residual between the corrected second point cloud data and the fifth point cloud data is calculated, and then the previously obtained external parameters are optimized by the LM (Levenberg-Marquardt) algorithm, and the z-axis translation parameters are kept fixed during the optimization process to obtain the optimized external parameters. Figure 4 The final calibration result is shown, from which it can be seen that the point cloud data collected by the depth camera can basically overlap with the point cloud data collected by the lidar.

[0071] In summary, the embodiment of the present application obtains the first point cloud data, and determines the z-axis translation parameter based on the first point cloud data; wherein the first point cloud data is the point cloud data of the ground collected by the robot's depth camera; obtains the second point cloud data and the third point cloud data, and determines the x-axis translation parameter, the y-axis translation parameter and the rotation parameter based on the second point cloud data and the third point cloud data; wherein the second point cloud data is the point cloud data of the vertically intersecting wall collected by the depth camera, and the third point cloud data is the point cloud data of the vertically intersecting wall collected by the robot's laser radar. Through the embodiment of the present application, there is no need to resort to specific calibration objects such as calibration plates, and external parameter calibration can be performed only by relying on the ground and walls. It is extremely convenient, conducive to large-scale promotion and use, and meets the needs of actual use.

[0072] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0073] Corresponding to an external parameter calibration method described in the above embodiment, Figure 5 A structural diagram of an embodiment of an external parameter calibration device provided in an embodiment of the present application is shown.

[0074] In this embodiment, an external parameter calibration device may include:

[0075] A ground calibration module 501 is used to obtain first point cloud data and determine a z-axis translation parameter according to the first point cloud data; wherein the first point cloud data is point cloud data of the ground collected by a depth camera of the robot;

[0076] The wall calibration module 502 is used to obtain the second point cloud data and the third point cloud data, and determine the x-axis translation parameter, the y-axis translation parameter and the rotation parameter according to the second point cloud data and the third point cloud data; wherein the second point cloud data is the point cloud data of the vertically intersecting wall collected by the depth camera, and the third point cloud data is the point cloud data of the vertically intersecting wall collected by the laser radar of the robot.

[0077] In a specific implementation of the embodiment of the present application, the ground calibration module may include:

[0078] A ground detection unit, used for performing ground detection according to the first point cloud data;

[0079] The z-axis translation parameter determination unit is used to adjust the angle of the depth camera and re-perform ground detection if the ground detection fails, until the ground detection succeeds; if the ground detection succeeds, determine the z-axis translation parameter according to the first point cloud data.

[0080] In a specific implementation method of an embodiment of the present application, the ground detection unit can be specifically used to: determine a first fitting plane of the ground based on the first point cloud data; calculate the number of ground points in the first point cloud data, and the variance of the distance from each point in the first point cloud data to the first fitting plane; wherein the ground point is a point whose distance from the first fitting plane is less than a preset distance threshold; if the number of ground points is greater than a preset number threshold, and the variance is less than a preset variance threshold, then the ground detection is determined to be successful; if the number of ground points is less than or equal to the number threshold, or the variance is greater than or equal to the variance threshold, then the ground detection is determined to have failed.

[0081] In a specific implementation of the embodiment of the present application, the wall calibration module may include:

[0082] A wall fitting unit, used for determining a second fitting plane and a third fitting plane of the vertically intersecting wall according to the second point cloud data;

[0083] A first external parameter determination unit, configured to determine a pitch angle and a roll angle according to the second fitting plane and the third fitting plane;

[0084] A two-dimensional projection unit, used for projecting the second point cloud data onto a target two-dimensional plane to obtain fourth point cloud data; wherein the target two-dimensional plane is the plane where the third point cloud data is located;

[0085] The second external parameter determination unit is used to determine an x-axis translation parameter, a y-axis translation parameter and a yaw angle according to the third point cloud data and the fourth point cloud data.

[0086] In a specific implementation of an embodiment of the present application, the second external parameter determination unit can be specifically used to: use a random sampling consensus algorithm to perform point cloud matching on the third point cloud data and the fourth point cloud data to obtain the x-axis translation parameter, the y-axis translation parameter and the yaw angle.

[0087] In a specific implementation of the embodiment of the present application, the wall calibration module may further include:

[0088] The external parameter correction unit is used to correct the x-axis translation parameter, the y-axis translation parameter and the yaw angle using a normal distribution transformation algorithm to obtain a corrected x-axis translation parameter, a corrected y-axis translation parameter and a corrected yaw angle.

[0089] In a specific implementation of the embodiment of the present application, the wall calibration module may further include:

[0090] The external parameter optimization unit is used to perform three-dimensional expansion on the third point cloud data to obtain fifth point cloud data; according to the fifth point cloud data, the corrected x-axis translation parameter, the corrected y-axis translation parameter, the corrected yaw angle, the pitch angle and the roll angle are optimized to obtain optimized x-axis translation parameters, optimized y-axis translation parameters and optimized rotation parameters.

[0091] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0092] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0093] Figure 6 A schematic block diagram of a robot provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0094] like Figure 6 As shown, the robot 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, the steps in the above-mentioned embodiments of the external parameter calibration method are implemented, for example Figure 1 Alternatively, when the processor 60 executes the computer program 62, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 5 The functions of modules 501 to 502 are shown.

[0095] Exemplarily, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 62 in the robot 6.

[0096] Those skilled in the art will understand that Figure 6 It is only an example of the robot 6 and does not constitute a limitation of the robot 6. The robot 6 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the robot 6 may also include input and output devices, network access devices, buses, etc.

[0097] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0098] The memory 61 may be an internal storage unit of the robot 6, such as a hard disk or memory of the robot 6. The memory 61 may also be an external storage device of the robot 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the robot 6. Further, the memory 61 may include both an internal storage unit and an external storage device of the robot 6. The memory 61 is used to store the computer program and other programs and data required by the robot 6. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0099] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0100] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0101] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0102] In the embodiments provided in the present application, it should be understood that the disclosed devices / robots and methods can be implemented in other ways. For example, the device / robot embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0104] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0105] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0106] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for calibrating an external parameter, characterized in that: include: Acquire first point cloud data, and determine a z-axis translation parameter according to the first point cloud data; wherein the first point cloud data is point cloud data of the ground collected by a depth camera of the robot; Acquire second point cloud data and third point cloud data; wherein the second point cloud data is the point cloud data of the vertically intersecting wall collected by the depth camera, and the third point cloud data is the point cloud data of the vertically intersecting wall collected by the laser radar of the robot; determine the second fitting plane and the third fitting plane of the vertically intersecting wall according to the second point cloud data; determine the pitch angle and the roll angle according to the second fitting plane and the third fitting plane; project the second point cloud data to the plane where the third point cloud data is located to obtain the fourth point cloud data; determine the x-axis translation parameter, the y-axis translation parameter and the yaw angle according to the third point cloud data and the fourth point cloud data.

2. The external parameter calibration method according to claim 1, characterized in that: The determining of the z-axis translation parameter according to the first point cloud data includes: Perform ground detection according to the first point cloud data; If the ground detection fails, the angle of the depth camera is adjusted to perform ground detection again until the ground detection succeeds; If the ground detection is successful, the z-axis translation parameter is determined according to the first point cloud data.

3. The external parameter calibration method according to claim 2, characterized in that: The performing ground detection according to the first point cloud data includes: Determine a first fitting plane of the ground according to the first point cloud data; Calculating the number of ground points in the first point cloud data and the variance of the distances from each point in the first point cloud data to the first fitting plane; wherein the ground point is a point whose distance from the first fitting plane is less than a preset distance threshold; If the number of ground points is greater than a preset number threshold, and the variance is less than a preset variance threshold, then it is determined that the ground detection is successful; If the number of the ground points is less than or equal to the number threshold, or the variance is greater than or equal to the variance threshold, it is determined that the ground detection fails.

4. The external parameter calibration method according to any one of claims 1 to 3, characterized in that: The determining of the x-axis translation parameter, the y-axis translation parameter and the yaw angle according to the third point cloud data and the fourth point cloud data comprises: The third point cloud data and the fourth point cloud data are matched with each other using a random sampling consensus algorithm to obtain the x-axis translation parameter, the y-axis translation parameter and the yaw angle.

5. The external parameter calibration method according to any one of claims 1 to 3, characterized in that: After determining the x-axis translation parameter, the y-axis translation parameter and the yaw angle according to the third point cloud data and the fourth point cloud data, the method further includes: The x-axis translation parameter, the y-axis translation parameter and the yaw angle are corrected using a normal distribution transformation algorithm to obtain a corrected x-axis translation parameter, a corrected y-axis translation parameter and a corrected yaw angle.

6. The external parameter calibration method according to claim 5, characterized in that: After correcting the x-axis translation parameter, the y-axis translation parameter and the yaw angle by using a normal distribution transformation algorithm to obtain the corrected x-axis translation parameter, the corrected y-axis translation parameter and the corrected yaw angle, the method further includes: Performing three-dimensional expansion on the third point cloud data to obtain fifth point cloud data; The corrected x-axis translation parameter, the corrected y-axis translation parameter, the corrected yaw angle, the pitch angle and the roll angle are optimized according to the fifth point cloud data to obtain optimized x-axis translation parameters, optimized y-axis translation parameters and optimized rotation parameters.

7. An external parameter calibration device, characterized in that: include: A ground calibration module, used to obtain first point cloud data and determine a z-axis translation parameter according to the first point cloud data; wherein the first point cloud data is point cloud data of the ground collected by the robot's depth camera; A wall calibration module, used to obtain second point cloud data and third point cloud data; wherein the second point cloud data is the point cloud data of the vertically intersecting wall collected by the depth camera, and the third point cloud data is the point cloud data of the vertically intersecting wall collected by the laser radar of the robot; determine the second fitting plane and the third fitting plane of the vertically intersecting wall according to the second point cloud data; determine the pitch angle and the roll angle according to the second fitting plane and the third fitting plane; project the second point cloud data to the plane where the third point cloud data is located to obtain the fourth point cloud data; determine the x-axis translation parameter, the y-axis translation parameter and the yaw angle according to the third point cloud data and the fourth point cloud data.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the external parameter calibration method according to any one of claims 1 to 6 are implemented.

9. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the external parameter calibration method according to any one of claims 1 to 6 are implemented.

Images