Camera coordinate transformation method, calibration environment correction method and robot

Abstract

The present application discloses a camera coordinate transformation method, a calibration environment correction method, and a robot. The camera coordinate transformation method includes: obtaining point cloud data of the calibration environment where the camera is located; performing plane segmentation and plane fitting on the point cloud data to obtain three point cloud planes that intersect with each other; taking one of the point cloud planes as a standard plane, calculating the intersection direction vector of the standard plane and the other point cloud plane, and determining three standard planes with the intersection direction vector and the standard plane; constructing a world coordinate system with the three standard planes; and calculating the camera's posture transformation matrix in the world coordinate system. The method of the present application can calculate mutually perpendicular standard world coordinate systems, avoiding the use of a calibration environment with deviations as a world coordinate system, thereby accurately solving the depth camera's external parameters and solving the orthogonal posture transformation matrix, thereby improving the accuracy of calibration.

Description

Technical Field

[0001] The present application belongs to the field of sensor environment calibration technology, and specifically relates to a camera coordinate transformation method, a calibration environment correction method, and a robot. Background Art

[0002] Multi-sensor calibration for mobile robots is a topic of considerable concern in both industry and academia. RGDB depth cameras are widely used. Due to installation errors and motion pose variations, it is necessary to accurately determine the depth camera's extrinsic parameters to unify the sensor data into a common coordinate system for precise positioning and navigation.

[0003] However, in the real world, due to factors such as calibration environment processing errors, installation errors, and data acquisition errors, the obtained planar point cloud data cannot meet the requirements of mutual perpendicularity, the solved attitude rotation matrix is ​​not completely orthogonal, and the camera's external parameters are not calculated accurately. Summary of the Invention

[0004] The present application provides a camera coordinate transformation method, a calibration environment correction method, and a robot to solve the technical problem that the calibration environment cannot satisfy the requirement that the planes are perpendicular to each other.

[0005] To solve the above technical problems, a technical solution adopted in this application is: a camera coordinate transformation method, including: obtaining point cloud data of a calibration environment where the camera is located; performing plane segmentation and plane fitting on the point cloud data to obtain three point cloud planes that intersect with each other; taking one of the point cloud planes as a standard plane, calculating the intersection direction vector of the standard plane and another point cloud plane, and determining three standard planes using the intersection direction vector and the standard plane; constructing a world coordinate system using the three standard planes; and calculating the camera's posture transformation matrix in the world coordinate system.

[0006] According to one embodiment of the present application, taking one of the point cloud planes as a standard plane, calculating the intersection direction vector of the standard plane and another point cloud plane, and determining three standard planes using the intersection direction vector and the standard plane include: taking the first point cloud plane as the first standard plane, calculating the normal vector of the first standard plane; determining the intersection direction vector of the first point cloud plane and the second point cloud plane according to the plane equations of the first point cloud plane and the second point cloud plane, and using it as the normal vector of the third standard plane to obtain the third standard plane; cross-multiplying the normal vector of the first standard plane and the normal vector of the third standard plane to obtain the normal vector of the second standard plane to obtain the second standard plane.

[0007] According to one embodiment of the present application, constructing a world coordinate system using the three standard planes includes: establishing a world coordinate system using the intersection of the three point cloud planes as the origin and the intersection line of the three standard planes as the coordinate axis.

[0008] To solve the above technical problems, another technical solution adopted in the present application is: a calibration environment correction method, the method comprising: calculating the deflection angle of each point cloud plane and the corresponding standard plane, the point cloud plane comprising three point cloud planes intersecting with each other obtained based on plane segmentation and plane fitting of the point cloud data of the calibration environment where the camera is located, the standard plane comprising one of the point cloud planes as a standard plane, calculating the intersection direction vector of the standard plane and an adjacent point cloud plane, and three standard planes determined by the intersection direction vector and the standard plane; in response to the deflection angle being greater than a threshold, correcting the calibration environment.

[0009] According to one embodiment of the present application, one of the point cloud planes is taken as a standard plane, and the intersection direction vector of the standard plane and an adjacent point cloud plane is calculated. The three standard planes determined by the intersection direction vector and the standard plane include: taking the first point cloud plane as the first standard plane; according to the plane equations of the first point cloud plane and the second point cloud plane, determining the intersection direction vector of the first point cloud plane and the second point cloud plane, and using it as the normal vector of the third standard plane to obtain the third standard plane.

[0010] According to one embodiment of the present application, one of the point cloud planes is taken as a standard plane, and the intersection direction vector of the standard plane and an adjacent point cloud plane is calculated. The three standard planes determined by the intersection direction vector and the standard plane include: calculating the normal vector of the first standard plane, cross-multiplying the normal vector of the first standard plane and the normal vector of the third standard plane to obtain the normal vector of the second standard plane, so as to obtain the second standard plane.

[0011] According to one embodiment of the present application, the calculation of the deflection angle between each point cloud plane and the corresponding standard plane includes: calculating the angle between the normal vector of the third standard plane and the normal vector of the third point cloud plane, as the deflection angle between the third point cloud plane and the third standard plane; calculating the angle between the normal vector of the second standard plane and the normal vector of the second point cloud plane, as the deflection angle between the second point cloud plane and the second standard plane.

[0012] To solve the above technical problems, another technical solution adopted in this application is: a robot comprising a camera and a processor, wherein the processor is coupled to the camera to implement the above-mentioned camera coordinate transformation method or calibration environment correction method.

[0013] To solve the above technical problems, another technical solution adopted in this application is: a computer-readable storage medium having program data stored thereon, which, when executed by a processor, implements any of the above-mentioned camera coordinate transformation methods or calibration environment correction methods.

[0014] To solve the above technical problems, another technical solution adopted in this application is: a computer program product, including a computer program, which implements any of the above-mentioned camera coordinate transformation methods or calibration environment correction methods when executed by a processor.

[0015] The beneficial effects of the present application are as follows: the method of the present application can not only calculate a completely vertical standard world coordinate system, but also avoid using a calibration environment with deviations as the world coordinate system, thereby accurately solving the depth camera external parameters and solving a completely orthogonal pose transformation matrix, thereby improving the accuracy of the calibration. At the same time, the deviation of the world coordinate system in the actual calibration environment can be solved by assuming that the three planes in the world coordinate system are perpendicular to each other (vertical surfaces), thereby correcting the calibration environment and improving the accuracy of the calibration. The present method is simple to calculate and can quickly calculate the deflection between the standard world coordinate system and the calibration environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0017] Figure 1 1 is a flow chart of an embodiment of a coordinate transformation method for a camera of the present application;

[0018] Figure 2 is a schematic diagram of calibrating the environment and the world coordinate system in an embodiment of the coordinate transformation method of a camera of the present application;

[0019] Figure 3 1 is a schematic diagram of the coordinate system transformation relationship in an embodiment of the coordinate transformation method of a camera of the present application;

[0020] Figure 4 This is a flow chart of an embodiment of a calibration environment correction method of the present application;

[0021] Figure 5 This is a schematic diagram of the framework of a robot embodiment of the present application;

[0022] Figure 6 This is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application;

[0023] Figure 7 1 is a schematic diagram of a frame of an embodiment of a coordinate transformation device for a camera of the present application;

[0024] Figure 8 It is a schematic diagram of the framework of an embodiment of the calibration environment correction device of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] See also Figure 1 and Figure 2 , Figure 1 1 is a flow chart of an embodiment of a camera coordinate transformation method of the present application. Figure 2 is a schematic diagram of calibrating the environment and the world coordinate system in an embodiment of the coordinate transformation method of a camera of the present application; Figure 3 This is a schematic diagram of the coordinate system transformation relationship in an embodiment of the coordinate transformation method of the camera of the present application.

[0027] An embodiment of the present application provides an environmental deviation compensation method, comprising the following steps:

[0028] S11: Obtain point cloud data of the calibration environment where the camera is located.

[0029] The calibration environment is pre-built and includes three intersecting planes. It should be noted that the calibration environment is often built with the three planes perpendicular to each other, but in practice, these planes often cannot be perpendicular to each other. Therefore, the method of this application can detect the deflection angle of the calibration environment plane. Point cloud data is data collected using an acquisition device, which records various physical parameters of a finite surface at discrete points. The camera is a depth camera.

[0030] In some embodiments, obtaining point cloud data of the calibration environment in which the camera is located includes: obtaining point cloud data of the calibration environment using a depth camera. In other embodiments, other devices such as a laser scanner may also be used to obtain point cloud data of the calibration environment.

[0031] S12: Perform plane segmentation and plane fitting on the point cloud data to obtain three point cloud planes that intersect with each other.

[0032] Before performing plane segmentation and plane fitting on the point cloud data, the point cloud data obtained in step S11 can be denoised. During the process of acquiring point cloud data, due to human disturbance or defects in the acquisition equipment itself, the generated data often contains noise, resulting in a certain deviation between the obtained point cloud data and the actual object. Therefore, before performing relevant digital geometry processing and application on the point cloud data, it can be denoised and smoothed. Denoising can effectively eliminate noise points, make the reconstructed surface smooth, and maintain the original topological and geometric characteristics of the sampled surface.

[0033] Specifically, point cloud data denoising methods include bilateral filtering, Gaussian filtering, binning denoising, KD-Tree, straight-through filtering, and random sampling consistency filtering. You can choose the appropriate method for point cloud data denoising based on your specific situation. Point cloud denoising can eliminate outliers and facilitate the acquisition of point cloud planes.

[0034] The purpose of point cloud segmentation is to extract different objects in the point cloud, thereby achieving the goal of divide and conquer, highlighting key points, and processing them separately. In some embodiments, plane segmentation of point cloud data includes using the RanSaC algorithm (random sampling consistency) to perform plane segmentation on the point cloud data. The RanSaC algorithm is a classic algorithm used in data processing, and its function is to extract specific components in objects in the presence of large amounts of noise.

[0035] The point cloud data can be segmented into point cloud planes using the RanSaC algorithm. Of course, in other embodiments, other algorithms can be selected to segment the point cloud data into point cloud planes.

[0036] Furthermore, based on the assumption that X=0, Y=0, and Z=0, plane fitting and matching are performed on the segmented point cloud planes to obtain three point cloud planes that intersect with each other.

[0037] Specifically, the general expression of the plane equation is Ax+By+Cz+D=0. After constructing the coefficient matrix, the least squares method can be used for fitting to finally obtain the plane equations of the three point cloud planes.

[0038] S13: Taking one of the point cloud planes as a standard plane, calculating the intersection direction vector of the standard plane and the other point cloud plane, and determining three standard planes based on the intersection direction vector and the standard plane.

[0039] In some embodiments, taking one of the point cloud planes as a standard plane, calculating the intersection direction vector of the standard plane and another point cloud plane, and determining three standard planes using the intersection direction vector and the standard plane includes:

[0040] The first point cloud plane is taken as the first standard plane, and the normal vector of the first standard plane is calculated.

[0041] According to the plane equations of the first point cloud plane and the second point cloud plane, a direction vector of an intersection line of the first point cloud plane and the second point cloud plane is determined and used as a normal vector of the third standard plane to obtain the third standard plane.

[0042] The normal vector of the second standard plane is obtained by cross-producting the normal vector of the first standard plane and the normal vector of the third standard plane to obtain the second standard plane.

[0043] Specifically, if Figure 2 As shown in the figure, the solid plane is the calibration environment, that is, the three point cloud planes, the dotted plane is the standard plane, and the dotted coordinate system is the standard world coordinate system. Taking the point cloud planes π1, π2 and π3 as an example, the first point cloud plane π1, the second point cloud plane π2 and the third point cloud plane π3 are combined to solve the plane intersection o w (x o ,y o ,z o ), take the intersection as the origin, and use the first point cloud plane π1 as the reference, that is, as the first standard plane, to determine the approximate direction of the world coordinate axis. The first point cloud plane π1 is the first standard plane, that is, the z=0 plane in the world coordinate system, passing through the origin o w The normal vector of the first standard plane (x z ,y z ,z z ) is the Z-axis direction.

[0044] According to the plane equation of the first point cloud plane π1 and the plane equation of the second point cloud plane π2, solve the intersection direction vector (x x ,y x ,z x ), is the normal vector of the x=0 plane in the world coordinate system. x ,y x ,z x ) is used as the normal vector of the third standard plane to calculate the plane equation of the third standard plane. At this time, the intersection direction vector (x x ,y x ,z x ) should be the X-axis direction of the world coordinate system. Since the normal vector of the third standard plane lies in the first standard plane, that is, it is perpendicular to the normal vector of the first standard plane, the first standard plane and the third standard plane are perpendicular.

[0045] The normal vector (x z ,y z ,z z ) and the normal vector of the third standard plane (x x ,y x ,zx ) cross product to obtain the normal vector (x y ,y y ,z y ), which is the normal vector to the y=0 plane of the world coordinate system, to obtain the second standard plane. Since the normal vector of the second standard plane is obtained by cross-producting the normal vectors of the first and third standard planes, the normal vector of the second standard plane is perpendicular to the normal vectors of the first and third standard planes. Therefore, the first, second, and third standard planes are mutually perpendicular.

[0046] S14: Construct the world coordinate system using three standard planes.

[0047] The three standard planes obtained in the above steps are perpendicular to each other in pairs. By constructing a world coordinate system with the three standard planes, a completely vertical standard world coordinate system can be obtained.

[0048] Specifically, constructing a world coordinate system with three standard planes includes taking the intersection of the three point cloud planes as the origin of the world coordinate system, that is, taking the intersection of the three standard planes as the origin of the world coordinate system, and establishing the world coordinate system with the intersection line of the three standard planes as the coordinate axis of the world coordinate system, and the three coordinate axes of the world coordinate system are perpendicular to each other.

[0049] S15: Calculate the camera's pose transformation matrix in the world coordinate system.

[0050] Pose represents position and attitude. Any rigid body can use position and attitude to accurately and uniquely represent its position state in the world coordinate system (OXYZ). In some embodiments, the camera has a camera coordinate system. The camera coordinate system is only the coordinates under a single camera. All cameras need to be converted to the same coordinate system, that is, the world coordinate system. It needs to be rotated and translated, so as to define the pose of the camera in the world coordinate system by a position and rotation, and the pose transformation matrix represents the rotation and translation process. The specific calculation method of the pose transformation matrix can be calculated by any method in the prior art, and is not limited here. By calculating the pose transformation matrix of the camera in the world coordinate system, the coordinates of a point in the camera coordinate system are given, and multiplied by this pose transformation matrix, the coordinates of the point in the world coordinate system can be obtained.

[0051] The following provides a specific implementation method of the pose transformation matrix in the world coordinate system, such as Figure 3 As shown, the world coordinate system F is established with three mutually perpendicular planes W , mobile robot coordinate system F R , camera installation coordinate system F C , depth camera coordinate system F D .

[0052] T WD =T WR *T RC *T CD

[0053] The plane point cloud data obtained by the depth camera is calculated to obtain T WD , T WR is the robot’s position in the world coordinate system, T RC is the camera pose in the robot coordinate system, T CD The depth camera coordinate system is the pose in the camera installation coordinate system.

[0054] The world coordinate system calculated in this application, each plane equation can be expressed by its normal vector and distance as (N x ,d x )(N y ,d y )(N z ,d z ).

[0055] The point in the camera coordinate system is transformed into the world coordinate system to satisfy the homogeneous transformation matrix P W =T WD P D , that is, the pose transformation matrix, is as follows:

[0056]

[0057] The pose transformation homogeneous coordinate system can be expressed by the plane equation:

[0058]

[0059] The rotation and displacement parameters from the camera to the world coordinate system can be calculated based on the pose transformation matrix.

[0060] In the method of the present application, a standard world coordinate system is obtained through the above steps, avoiding the error of using a calibration environment with deviations as the world coordinate system, so that the depth camera extrinsic parameters can be accurately solved, and a completely orthogonal posture transformation matrix can be solved, thereby improving the accuracy of calibration.

[0061] The following implementation method verifies the accuracy of the calibration environment correction method in this application. Specifically, continue to refer to Figure 3 According to Table 1 below, the camera is rotated about the axis ZYX relative to the robot coordinate system by (15°, 15°, -30°). The robot is located at (1.95, 1.95, 0) in the world coordinate system, and the camera is located at (0, -0.3, 0.55) in the robot coordinate system.

[0062] Table 1: External parameter estimation

[0063] Reference pose Estimated pose error x 1.625 1.62495 0.003％ y 1.925 1.92506 0.003％ z 0.55 0.550852 0.15％ α 15° 15.052° 0.34％ β 15° 15.003° 0.02％ γ -30° -29.997° 0.01％

[0064] As can be seen from the table above, without the need for a completely vertical calibration environment, the method of this application can also achieve the effect of extrinsic calibration by acquiring depth point cloud data, with a small calibration error, and can accurately solve the RGBD camera extrinsic parameters. The calibration environment correction method of this application can also accurately detect the calibration environment error, thereby effectively solving the deviation of the world coordinate system in the actual calibration environment, facilitating further correction of the calibration environment and improving calibration accuracy.

[0065] See also Figure 4 , Figure 4 It is a flowchart of an embodiment of the calibration environment correction method of the present application.

[0066] S21: Calculate the deflection angle between the point cloud plane and the corresponding standard plane.

[0067] The point cloud planes comprise three intersecting point cloud planes obtained by plane segmentation and plane fitting based on the point cloud data of the calibration environment in which the camera is located. The point cloud data of the calibration environment can be acquired using a depth camera. The point cloud data acquisition process and the specific point cloud plane calculation process can be referenced to the corresponding steps in the above embodiment and will not be further described here.

[0068] The standard planes include one of the point cloud planes as a standard plane, calculating the intersection direction vector between the standard plane and an adjacent point cloud plane, and determining three standard planes based on the intersection direction vector and the standard plane. The specific calculation process of the standard planes can be referred to the corresponding steps in the above embodiment and will not be repeated here.

[0069] Calculating the deflection between the point cloud plane and the corresponding standard plane includes calculating the deflection between the first point cloud plane and the first standard plane, calculating the deflection between the second point cloud plane and the second standard plane, and calculating the deflection between the third point cloud plane and the third standard plane.

[0070] Calculating the deviation angle between the first point cloud plane and the first standard plane includes: since the first point cloud plane is used as a reference, that is, the first point cloud plane coincides with the first standard plane, and therefore there is no deviation angle between the first point cloud plane and the first standard plane.

[0071] Calculating the deflection angle between the second point cloud plane π2 and the second standard plane includes calculating the angle between the normal vector of the second standard plane and the normal vector of the second point cloud plane π2 as the deflection angle between the second point cloud plane π2 and the second standard plane.

[0072] Specifically, the normal vector (x y ,y y ,z y) should be the Y-axis direction of the world coordinate system. By calculating the angle between the normal vector of the actual second point cloud plane π2 and the normal vector of the second standard plane, the inclination angle of the second point cloud plane π2 can be calculated. The normal vector of the second point cloud plane π2 can be obtained through the plane equation of the second point cloud plane π2. Specifically, Formula, the normal vector of the second point cloud plane π2 and the normal vector of the second standard plane are substituted into the calculation as a and b respectively.

[0073] Calculating the deflection between the third point cloud plane π3 and the third standard plane includes calculating an angle between a normal vector of the third standard plane and a normal vector of the third point cloud plane π3 as the deflection between the third point cloud plane π3 and the third standard plane.

[0074] Specifically, the normal vector (x x ,y x ,z x ) should be the X-axis direction of the world coordinate system. Calculate the angle between the normal vector of the actual third point cloud plane π3 and the normal vector of the third standard plane to calculate the inclination angle of the third point cloud plane π3. The normal vector of the third point cloud plane π3 can be obtained through the plane equation of the third point cloud plane π3. Specifically, Formula, the normal vector of the third point cloud plane π3 and the normal vector of the third standard plane are substituted into the calculation as a and b respectively.

[0075] S22: In response to the deflection angle being greater than the threshold, correct the calibration environment.

[0076] After calculating the deflection angles of the point cloud plane and the corresponding standard plane, the deflection angles of the three point cloud planes need to be compared with the corresponding thresholds, including comparing the deflection angle of the first point cloud plane with the threshold, comparing the deflection angle of the second point cloud plane with the threshold, and comparing the deflection angle of the third point cloud plane with the threshold. Since the three planes cannot be completely perpendicular to each other during the actual calibration environment construction process, the threshold is a reasonable deviation range for the calibration environment construction, and the specific threshold can be adjusted according to actual conditions. In response to the deflection angle being greater than the threshold, the calibration environment error is large and the calibration environment needs to be re-corrected. Specifically, the calibration environment can be re-built by a third party, or the calibration environment can be re-corrected by yourself.

[0077] In addition, in response to the deflection angle of each point cloud plane being less than or equal to the corresponding threshold, the deflection angle of each point cloud plane is within a reasonable range, and the original calibration environment is retained.

[0078] The method of the embodiment of the present application not only calculates a completely vertical standard world coordinate system, but also avoids using a calibration environment with deviations as the world coordinate system, thereby solving a completely orthogonal pose transformation matrix and improving the accuracy of the calibration. At the same time, it can also solve the deviation of the world coordinate system in the actual calibration environment by assuming that the three planes in the world coordinate system are perpendicular to each other (vertical planes), thereby correcting the calibration environment and improving the accuracy of the calibration.

[0079] See also Figure 5 , Figure 5 It is a schematic diagram of the framework of an embodiment of the robot of the present application.

[0080] Another embodiment of the present application includes a robot 30, including a camera 31 and a processor 32, wherein the camera 31 and the processor 32 are coupled to implement the coordinate transformation method or calibration environment correction of the camera of any of the above embodiments. Specifically, the camera 31 obtains point cloud data of the calibration environment in which the camera 31 (i.e., the robot 30) is located, and the processor 32 performs plane segmentation and plane fitting on the point cloud data to obtain three point cloud planes that intersect with each other; the processor 32 uses one of the point cloud planes as a standard plane, calculates the intersection direction vector of the standard plane and the other point cloud plane, and determines the three standard planes using the intersection direction vector and the standard plane; the processor 32 constructs a world coordinate system using the three standard planes; the processor 32 calculates the pose transformation matrix of the camera 31 (i.e., the robot 30) in the world coordinate system.

[0081] Furthermore, the processor 32 calculates the deflection angle between the point cloud plane and the corresponding standard plane; and corrects the calibration environment in response to the deflection angle being greater than a threshold.

[0082] The calibration environment is pre-built and includes three intersecting planes. It should be noted that the calibration environment is often built with the three planes perpendicular to each other, but in practice, these planes often do not meet the requirement of being perpendicular to each other. Therefore, the method of this application can detect the deflection angle of the planes in this calibration environment. Camera 31 is a depth camera. Robot 30 can be a home service robot, etc.

[0083] In some embodiments, obtaining point cloud data of the calibration environment where the camera 31 is located includes: obtaining point cloud data of the calibration environment using a depth camera. In other embodiments, other devices such as a laser scanner can also be used to obtain point cloud data of the calibration environment.

[0084] The robot 30 of the embodiment of the present application can not only calculate a completely vertical standard world coordinate system, but also avoid using a calibration environment with deviations as a world coordinate system, so that the depth camera external parameters can be accurately solved, and a completely orthogonal posture transformation matrix can be solved, thereby improving the accuracy of the calibration. At the same time, the deviation of the world coordinate system in the actual calibration environment can be solved by assuming that the three planes in the world coordinate system are perpendicular to each other (vertical surfaces), and then the calibration environment can be corrected to improve the accuracy of the calibration. This method is simple to calculate and can quickly calculate the deflection of the standard world coordinate system and the calibration environment.

[0085] See also Figure 6 , Figure 6 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application.

[0086] Yet another embodiment of the present application provides a computer-readable storage medium 40 on which program data 41 is stored. When the program data 41 is executed by a processor, the coordinate transformation method or calibration environment correction method of the camera of any of the above embodiments is implemented.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0088] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0089] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0090] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 40. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium 40, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium 40 includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0091] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in each embodiment of the camera coordinate transformation method or the calibration environment correction method are implemented.

[0092] In one embodiment, a computer program product or program is provided. The computer program product or program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the camera coordinate transformation method or calibration environment correction method described above.

[0093] like Figure 7 As shown, another embodiment of the present application provides a camera coordinate transformation device 50. The camera coordinate transformation device 50 can be implemented as a software module or a hardware module, or a combination of the two as part of a computer device. The camera coordinate transformation device 50 specifically includes: an acquisition device 51, a plane processing device 52, and a posture transformation device 53. The acquisition device 51 acquires point cloud data of the calibration environment in which the camera is located. The plane processing device 52 performs plane segmentation and plane fitting on the point cloud data to obtain three point cloud planes that intersect with each other. The plane processing device 52 uses one of the point cloud planes as a standard plane, calculates the intersection direction vector between the standard plane and another point cloud plane, and determines three standard planes using the intersection direction vector and the standard plane. The posture transformation device 53 constructs a world coordinate system using the three standard planes and calculates the posture transformation matrix of the camera in the world coordinate system. The camera coordinate transformation device 50 of the present application can implement the camera coordinate transformation method in any of the above embodiments, and will not be described in detail here.

[0094] The device 50 of the present application can calculate a completely vertical standard world coordinate system, avoiding using a calibration environment with deviations as the world coordinate system, so that the depth camera external parameters can be accurately solved and a completely orthogonal posture transformation matrix can be solved, thereby improving the accuracy of calibration.

[0095] like Figure 8 As shown, another embodiment of the present application provides a calibration environment correction device 60, including a plane processing device 61, a deflection calculation device 62 and a judgment device 63. The plane processing device 61 performs plane segmentation and plane fitting on the calibration point cloud data where the camera is located to obtain three point cloud planes that intersect with each other. The plane processing device 61 also uses one of the point cloud planes as a standard plane, calculates the intersection direction vector between the standard plane and another point cloud plane, and determines the three standard planes based on the intersection direction vector and the standard plane. The plane processing device 61 also constructs a world coordinate system based on the three standard planes and calculates the position transformation matrix of the camera in the world coordinate system. The deflection calculation device 62 calculates the deflection between the point cloud plane and the corresponding standard plane. The judgment device 63 determines that the deflection is greater than a threshold, that is, the calibration environment needs to be corrected. Specifically, the calibration environment can be rebuilt by a third party, or the calibration environment can be re-corrected by itself. The calibration environment correction device 60 of the present application can implement the calibration environment correction method in any of the above embodiments, which will not be repeated here.

[0096] The device 60 of the present application can solve the deviation of the world coordinate system in the actual calibration environment by assuming that the three planes in the world coordinate system are perpendicular to each other (the vertical surfaces are perpendicular), and then correct the calibration environment to improve the accuracy of the calibration.

[0097] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A calibration environment correction method, characterized in that: The method comprises: Calculating the deflection angle between each point cloud plane and the corresponding standard plane, wherein the point cloud planes include three point cloud planes intersecting with each other obtained by plane segmentation and plane fitting based on the point cloud data of the calibration environment where the camera is located, and the standard planes include taking one of the point cloud planes as a standard plane, calculating the intersection direction vector between the standard plane and an adjacent point cloud plane, and determining three standard planes based on the intersection direction vector and the standard plane; In response to the deflection angle being greater than a threshold, correcting the calibration environment; Wherein, taking one of the point cloud planes as a standard plane, calculating the intersection direction vector of the standard plane and an adjacent point cloud plane, and the three standard planes determined by the intersection direction vector and the standard plane include: Taking the first point cloud plane as the first standard plane; determining the intersection direction vector of the first point cloud plane and the second point cloud plane according to the plane equations of the first point cloud plane and the second point cloud plane, and using the intersection direction vector as the normal vector of the third standard plane, so as to obtain the third standard plane; Calculating a normal vector of the first standard plane, and cross-multiplying the normal vector of the first standard plane with the normal vector of the third standard plane to obtain a normal vector of a second standard plane, so as to obtain the second standard plane; The calculation of the deflection angle between each point cloud plane and the corresponding standard plane includes: The angle between the normal vector of each point cloud plane and the normal vector of the corresponding standard plane is calculated as the deflection angle between each point cloud plane and the corresponding standard plane.

2. A camera coordinate transformation method, characterized in that: In the calibration environment correction method according to claim 1, the camera coordinate transformation method includes: Get the point cloud data of the calibration environment where the camera is located; Performing plane segmentation and plane fitting on the point cloud data to obtain three point cloud planes that intersect with each other; Taking one of the point cloud planes as a standard plane, calculating the intersection direction vector of the standard plane and another point cloud plane, and determining three standard planes based on the intersection direction vector and the standard plane; Constructing a world coordinate system using the three standard planes; Calculate the camera's pose transformation matrix in the world coordinate system.

3. The camera coordinate transformation method according to claim 2, wherein: The step of taking one of the point cloud planes as a standard plane, calculating the intersection direction vector of the standard plane and another point cloud plane, and determining three standard planes using the intersection direction vector and the standard plane comprises: Taking the first point cloud plane as a first standard plane, calculating a normal vector of the first standard plane; Determine, according to the plane equations of the first point cloud plane and the second point cloud plane, a direction vector of an intersection line between the first point cloud plane and the second point cloud plane, and use the direction vector as a normal vector of a third standard plane to obtain the third standard plane; The normal vector of the second standard plane is obtained by cross-producting the normal vector of the first standard plane and the normal vector of the third standard plane to obtain the second standard plane.

4. The camera coordinate transformation method according to claim 2, wherein: The constructing of the world coordinate system using the three standard planes includes: A world coordinate system is established with the intersection of the three point cloud planes as the origin and the intersection of the three standard planes as the coordinate axis.

5. A robot, characterized in that: It comprises a camera and a processor, wherein the processor is coupled to the camera to implement the calibration environment correction method according to claim 1 or the camera coordinate transformation method according to any one of claims 2 to 4.

6. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the calibration environment correction method according to claim 1 or the camera coordinate transformation method according to any one of claims 2 to 4 is implemented.

7. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the calibration environment correction method according to claim 1 or the camera coordinate transformation method according to any one of claims 2 to 4 is implemented.

Images