Camera installation and calibration tool method and equipment based on depth information
Through the camera installation calibration method of depth information, the optimal rotation and translation of the camera is calculated using depth images and RANSAC algorithm, solving the complexity and cost problems of camera installation error detection and calibration in the prior art, and achieving fast and accurate calibration Improve efficiency.
Patent Information
- Application Number
- CN202410787977.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-18
- Publication Date
- 2024-11-22
AI Technical Summary
Existing robot camera installation calibration tools rely on auxiliary tools, which are complex and costly, making it difficult to quickly and accurately detect and calibrate camera installation errors, affecting the robot's running accuracy.
Using the camera installation calibration method based on depth information, the depth images of the depth camera and reference robot of the robot to be calibrated are obtained, and the optimal rotation and translation are calculated using filtering and RANSAC algorithms to achieve accurate calibration of the camera, reducing the dependence on auxiliary tools. .
The calibration process is simplified, the operation difficulty and cost of workers is reduced, calibration efficiency and operability is improved, and calibration can be completed in one minute, suitable for industrial environments.
Smart Images

Figure CN119006606A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robots, and in particular to a camera installation and calibration tooling method and equipment based on depth information. Background Art
[0002] In recent years, with the development of technologies such as big data, artificial intelligence, and 5G, the field of robotics has also been booming. Robots have also expanded from simple assembly line operations to autonomous movement in unknown environments and more complex functions. Sensors, as the "eyes" of robots, play a vital role in enabling robots to achieve autonomous movement.
[0003] According to the difference in algorithms, the sensors used by robots can be mainly divided into two categories: laser sensors and visual sensors. Among them, visual sensors can be divided into monocular cameras, binocular cameras and depth cameras according to different structures. Among them, monocular cameras cannot directly measure scale information due to their simple structure, and are prone to problems such as loss of tracking when the robot moves quickly; binocular cameras have limited depth information distance due to camera baseline problems, and the amount of calculation is greatly increased due to the increase in cameras, which increases the load on the processor. Compared with monocular and binocular cameras, depth cameras use structured light to measure actual distance to obtain more accurate depth information. They are simple to operate and have high accuracy, and are widely used in the current robotics field.
[0004] Due to the importance of depth sensors in the entire robot system, accurate installation is the key to achieving precise robot operation. We also call the installation parameters between the camera coordinate system and the robot coordinate system the camera's external parameters. The wrong camera installation position may cause the image captured by the camera to be different from the actual scene, and may also introduce distortion to distort the image, increase the difficulty of later external parameter calibration, and even cause the robot's operation accuracy to decrease or even fail.
[0005] However, due to the errors in robot mold manufacturing and worker assembly, the actual robot camera installation angle often has a large error from the theoretical value, so how to effectively detect the camera installation error and calibrate it is a problem that must be effectively solved. For existing calibration tools, most of them require the use of some auxiliary tools such as calibration plates, ArUco tags, calibration balls, etc. during the calibration process, which also increases the difficulty of calibration for workers. Summary of the invention
[0006] An object of the present invention is to provide a camera installation and calibration tooling method and equipment based on depth information.
[0007] According to one aspect of the present invention, a method for installing and calibrating a camera based on depth information is provided, the method comprising:
[0008] Obtain an original depth image of a stationary rigid object taken by a depth camera of the robot to be calibrated at a preset position, and obtain a target depth image of the same stationary rigid object taken by a properly installed depth camera of a reference robot at the same preset position;
[0009] Performing a filtering operation on the original depth image to obtain a filtered original depth image; performing a filtering operation on the target depth image to obtain a filtered target depth image;
[0010] According to the corresponding matching points in the unit matrix of the filtered original depth image and the target depth image, the optimal rotation and optimal translation of the depth camera of the robot to be calibrated are calculated.
[0011] Furthermore, in the above method, filtering is performed on the original depth image to obtain a filtered original depth image; filtering is performed on the target depth image to obtain a filtered target depth image, including:
[0012] Projecting the depth pixel points of the original depth image into an original 3D point cloud according to the intrinsic parameters of the depth camera of the robot to be calibrated;
[0013] The RANSAC algorithm is used to remove points in the original 3D point cloud whose z-axis depth is greater than a preset depth threshold to obtain the filtered original 3D point cloud.
[0014] Furthermore, in the above method, the depth pixel points of the original depth image are projected into the original 3D point cloud according to the intrinsic parameters of the depth camera of the robot to be calibrated.
[0015] The formula for projection to the original 3D point cloud is as follows:
[0016]
[0017] y=(uc y )*z / f y
[0018] Among them, x, y are the x-axis and y-axis coordinates of the 3D point cloud;
[0019] v and u are the x-axis and y-axis coordinates on the depth pixel map plane, and z is the depth value of the corresponding pixel position captured by the depth camera;
[0020] The internal parameters of the depth camera include: the horizontal focal length of the camera f x , the vertical focal length of the camera is f y , the optical center displacement c in the x-axis direction of the imaging plane x , the optical center displacement c in the y-axis direction of the imaging plane y .
[0021] Furthermore, in the above method, the RANSAC algorithm is used to remove points whose z-axis depth in the original 3D point cloud is greater than a preset depth threshold to obtain a filtered original 3D point cloud, including:
[0022] Use the RANSAC algorithm to train the final point model corresponding to the original 3D point cloud;
[0023] Based on the final point model, points in the original 3D point cloud whose z-axis depth is greater than a preset depth threshold are removed to obtain a filtered original 3D point cloud.
[0024] Furthermore, in the above method, the RANSAC algorithm is used to train the final point model corresponding to the original 3D point cloud, including:
[0025] The RANSAC algorithm repeatedly selects a set of random points in the original 3D point cloud as hypothetical inliers.
[0026] Set an initial point model that is suitable for all assumed in-library points in the original 3D point cloud;
[0027] Use the initial point model to test other points in the original 3D point cloud. If a certain other point also applies to the initial point model, then add the point to the assumed local point.
[0028] If there are points greater than a preset number threshold that are classified as hypothetical inliers, the model is re-estimated based on all current hypothetical inliers to obtain the optimal point model corresponding to the original 3D point cloud;
[0029] The optimal point model is evaluated by the number of points within the optimal point hypothesis and the error rate of the model to obtain the final point model.
[0030] Furthermore, in the above method, according to the corresponding matching points in the unit matrix of the filtered original depth image and the target depth image, the optimal rotation and optimal translation of the installation position of the depth camera of the robot to be calibrated are calculated, including:
[0031] Step S31, according to the corresponding matching points in the unit matrix of the filtered original depth image and the target depth image, calculate the transformation matrix from the filtered original depth image to the filtered target depth image, transform the filtered original depth image based on the transformation matrix, and obtain the projection of the origin cloud of the filtered original depth image in the target point cloud coordinate system of the filtered target depth image;
[0032] Step S32, based on the projection, searching in the target point cloud for the point closest to each sampling point in the projection, and obtaining a matching point pair between the target point and the origin;
[0033] Step S33, when the distance between the target point and the origin of a certain matching point pair is greater than a preset distance threshold, the matching point pair is considered to be an unreasonable matching point pair, and the unreasonable matching point pair needs to be eliminated to obtain the remaining reasonable matching point pairs, that is, the matching point pairs that are not eliminated;
[0034] Step S34, based on the remaining reasonable matching point pairs, solve the optimal rotation R * , t * and the optimal translation R * , t * , so that the loss loss is minimized.
[0035] Furthermore, in the above method, step S34, based on the remaining reasonable matching point pairs, solve the optimal rotation R * , t * and the optimal translation R * , t * , to minimize the loss, and also include:
[0036] After the preset number of iterations of steps S31 to S34 is executed cyclically, or the optimal rotation R calculated in the previous iteration is * , t * and the optimal translation R * , t * The optimal rotation R calculated in this iteration * , t * and the optimal translation R * , t * If the difference is less than the preset threshold, it is judged that the iteration has converged, and the optimal rotation R calculated by this iteration is * , t * and the optimal translation R * , t * , as the final optimal rotation R * , t * and optimal translation.
[0037] Furthermore, in the above method, the calculation formula of loss Loss is as follows:
[0038]
[0039] in, and is the origin cloud P s and the target point cloud P t The corresponding point pairs in R * , t * It is the rotation error and translation error of the installation position of the depth camera of the robot to be calibrated, and i represents the serial number of a point pair;
[0040] and Represent the centroid of the source point cloud and the target point cloud respectively, let make This is a 3×3 matrix. Perform SVD decomposition on H to get H=U∑V T , then the closed form solution of the ICP problem is as follows:
[0041] The optimal rotation is:
[0042] R * =VU T ,
[0043] The optimal translation is:
[0044]
[0045] According to another aspect of the present invention, a computer-readable medium is further provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement any of the above methods.
[0046] According to another aspect of the present invention, there is also provided a device for information processing at a network device end, the device comprising a memory for storing computer program instructions and a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute any of the methods described above.
[0047] Compared with the prior art, the present invention is a calibration scheme that only relies on the depth information of the camera. It only needs to mark the camera sampling and taking pictures at the selected calibration location, ensure that the marked place is open, and the characteristics of the reference calibration object are obvious. Among them, no special calibration plate is required, which greatly reduces the calibration cost. Only by comparing the pictures taken by an accurately installed depth camera, the calibration external parameters of the camera to be calibrated can be accurately and quickly calculated. For each robot to be calibrated, only one image needs to be sampled for calibration, which improves the efficiency of calibration, reduces the calibration difficulty of workers, and improves operability. Actual use shows that the scheme can complete calibration within one minute, with fast speed and strong robustness, which greatly improves the calibration efficiency in industrial environments. The present invention is mainly used in the field of robot sensor installation accuracy calibration. The installation accuracy calibration is carried out using the depth sensor information installed by the robot itself, without relying on additional sensor input, to achieve a calibration tool with high accuracy and strong operability. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:
[0049] Figure 1A schematic diagram showing a depth image of a three-dimensional rigid object chair according to an embodiment of the present invention;
[0050] Figure 2 A flowchart of a method for installing and calibrating a camera based on depth information according to an embodiment of the present invention is shown.
[0051] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION
[0052] The present invention is further described in detail below in conjunction with the accompanying drawings.
[0053] In a typical configuration of the present application, the terminal, the device of the service network and the trusted party all include one or more processors (CPU), input / output interfaces, network interfaces and memories.
[0054] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0055] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.
[0056] like Figure 1 and 2 As shown, the present invention provides a camera installation and calibration tooling method based on depth information, the method comprising steps S1 to S3:
[0057] Step S1, obtaining an original depth image of a stationary rigid object taken by a depth camera of the robot to be calibrated at a preset position, and obtaining a target depth image of the same stationary rigid object taken by a properly installed depth camera of a reference robot at the same preset position;
[0058] Here, before starting the calibration, we need a robot with qualified installation as a reference robot. Place the reference robot and the robot to be calibrated in the same position to shoot the same rigid object, and take out two depth camera pictures to install and calibrate. It should be noted that during the shooting process, it is necessary to ensure that the two robots are in the same place and the rigid object cannot be moved. In the actual factory environment, as long as the calibration location and rigid object are arranged, the reference robot picture does not need to be taken every time and can be reused. That is, for each robot to be calibrated, only one depth image needs to be taken at a fixed location. This greatly reduces the difficulty of operation and calibration cost for workers.
[0059] Step S2, filtering the original depth image to obtain a filtered original depth image; filtering the target depth image to obtain a filtered target depth image;
[0060] Here, the system only needs depth map information but not RGB camera information to complete the calibration. For each robot to be calibrated, if we have obtained the original depth image and the target depth image of the reference robot according to the above instructions, we can let the algorithm perform the calibration operation.
[0061] Preferably, performing a filtering operation on the original depth image to obtain a filtered original depth image; or performing a filtering operation on the target depth image to obtain a filtered target depth image, including:
[0062] Step S21, projecting the depth pixel points of the original depth image into an original 3D point cloud according to the intrinsic parameters of the depth camera of the robot to be calibrated;
[0063] Preferably, the formula for projecting the original 3D point cloud is as follows:
[0064]
[0065] Among them, x, y are the x-axis and y-axis coordinates of the 3D point cloud;
[0066] v and u are the x-axis and y-axis coordinates on the depth pixel map plane, and z is the depth value of the corresponding pixel position captured by the depth camera;
[0067] The internal parameters of the depth camera include: the horizontal focal length of the camera f x , the vertical focal length of the camera is f y , the optical center displacement c in the x-axis direction of the imaging plane x , the optical center displacement c in the y-axis direction of the imaging plane y ;
[0068] In step S22, the RANSAC algorithm is used to remove points in the original 3D point cloud whose z-axis depth is greater than a preset depth threshold to obtain a filtered original 3D point cloud.
[0069] Here, the point cloud can be filtered to remove background noise points and reduce the impact of noise on the calibration algorithm. Since incorrect correspondences will have a negative impact on the estimation of the final rigid body transformation matrix, they must be removed. We use the z-axis depth of the 3D point cloud and the RANSAC (Random Sample Consensus) algorithm to filter the point cloud. By removing points that are too far in z-axis depth, the background point cloud can be effectively removed.
[0070] More preferably, using the RANSAC algorithm, removing points whose z-axis depth in the original 3D point cloud is greater than a preset threshold, to obtain a filtered original 3D point cloud, including:
[0071] In step S221, the final point model corresponding to the original 3D point cloud is trained using the RANSAC algorithm, and points whose z-axis depth in the original 3D point cloud is greater than a preset threshold are removed through the optimal point model to obtain the filtered original 3D point cloud.
[0072] Preferably, step S221 includes:
[0073] Use the RANSAC algorithm to train the final point model corresponding to the original 3D point cloud;
[0074] Based on the final point model, points whose z-axis depth in the original 3D point cloud is greater than a preset depth threshold are removed to obtain a filtered original 3D point cloud;
[0075] Preferably, the RANSAC algorithm is used to train the final point model corresponding to the original 3D point cloud, including:
[0076] Step S2211, the RANSAC algorithm repeatedly selects a set of random points in the original 3D point cloud as hypothetical inliers;
[0077] Here, in our algorithm, the input of RANSAC is a set of 3D point clouds, and RANSAC achieves the goal by repeatedly selecting a set of random subsets in the data to be considered as inliers. The selected subsets are assumed to be inliers;
[0078] Specifically, both the original 3D point cloud and the target 3D point cloud need to be filtered, using different corresponding point models. This embodiment takes the final point model corresponding to the original 3D point cloud as an example. The training method of the final point model corresponding to the target 3D point cloud is similar and will not be repeated here. In addition, each time a new robot to be detected is detected, a new corresponding final point model needs to be trained;
[0079] Step S2212, setting an initial point model, which is suitable for all assumed local points in the original 3D point cloud;
[0080] Here, there is an initial model adapted to the assumed inliers, and all unknown model parameters can be calculated from the assumed inliers;
[0081] Step S2213, using the initial point model to test other points in the original 3D point cloud, if any other point is also applicable to the initial point model, then the point is added to the assumed local point;
[0082] Specifically, the obtained preliminary point model is used to test all other 3D points in the original 3D point cloud. If a certain 3D point is suitable for the estimated preliminary point model, it is considered to be a hypothesized inlier point.
[0083] In step S2214, if there are points greater than a preset number threshold that are classified as hypothetical in-library points, the model is re-estimated based on all current hypothetical in-library points to obtain the optimal point model corresponding to the original 3D point cloud.
[0084] Here, if enough 3D points are classified as hypothetical inliers, the estimated model parameters are reasonable enough. Then, because the initial model is only estimated by the initial hypothetical inliers, it is necessary to use all the hypothetical inliers that satisfy this point model to re-estimate the point model and obtain the optimized point model.
[0085] Step S2215, evaluate the optimal point model by the number of points in the optimal point model and the error rate of the model to obtain the final point model.
[0086] Step S3, calculating the optimal rotation and optimal translation of the depth camera of the robot to be calibrated according to the corresponding matching points in the unit matrix of the original depth image after filtering and the target depth image.
[0087] Preferably, step S3, calculating the optimal rotation and optimal translation of the installation position of the depth camera of the robot to be calibrated according to the corresponding matching points in the unit matrix of the filtered original depth image and the target depth image, comprises:
[0088] Step S31, according to the corresponding matching points in the unit matrix of the original depth image after filtering and the target depth image, calculate the transformation matrix from the original depth image after filtering to the target depth image after filtering, transform the original depth image after filtering based on the transformation matrix, and obtain the projection of the origin cloud of the original depth image after filtering in the target point cloud coordinate system of the target depth image after filtering;
[0089] Step S32, based on the projection, searching in the target point cloud for the point closest to each sampling point in the projection, and obtaining a matching point pair between the target point and the origin, that is, a corresponding relationship between the target point and the origin;
[0090] Step S33: when the distance between the target point and the origin of a certain matching point pair is greater than a preset distance threshold, the matching point pair is considered to be an unreasonable matching point pair, and the unreasonable matching point pair needs to be eliminated to obtain the remaining reasonable matching point pairs, that is, the matching point pairs that are not eliminated;
[0091] Step S34, based on the remaining reasonable matching point pairs, solve the optimal rotation R * , t * and the optimal translation R * , t * , so that the loss loss is minimized;
[0092] Here, we can minimize the loss and solve the current optimal transformation, R * , t * Minimize loss;
[0093] Step S35: after the preset number of iterations of steps S31 to S34 are executed cyclically, or the optimal rotation R calculated in the previous iteration is * , t * and the optimal translation R * , t * The optimal rotation R calculated in this iteration * , t * and the optimal translation R * , t * If the difference is less than the preset threshold, it is judged that the iteration has converged, and the optimal rotation R calculated by this iteration is * , t * and the optimal translation R * , t * , as the final optimal rotation R * , t * and optimal translation.
[0094] Here, the filtered point cloud can be matched to calculate the rigid transformation matrix. Matching can be divided into fine matching and coarse matching. Coarse matching refers to a relatively rough matching when the transformation between the two point clouds is completely unknown. The main purpose is to provide a better initial value for the fine matching.
[0095] Since the installation error generally does not exceed ten degrees and the translation does not exceed 5 cm, which is far below the robustness limit of the algorithm, we can consider the rough matching transformation matrix to be the unit matrix of the point cloud image. The fine matching criterion is to give an initial transformation and further optimize it to obtain a more accurate transformation. The system uses the ICP algorithm to complete the fine matching calculation.
[0096] Preferably, the calculation formula of loss Loss is as follows:
[0097]
[0098] in, and is the origin cloud P s and the target point cloud P t The corresponding point pairs in R * , t * That is, the rotation error and translation error of the installation position of the depth camera of the robot to be calibrated. i represents the serial number of a point pair, such as 20 point pairs, which is 1-20.
[0099] For the point-to-point ICP problem, the optimal transformation has a closed form solution, which can be calculated using SVD decomposition. When the corresponding relationship of the points is known, let and Represent the centroid of the source point cloud and the target point cloud respectively, let make This is a 3×3 matrix. Perform SVD decomposition on H to get H=U∑V T , then the closed form solution of the ICP problem is as follows:
[0100] The optimal rotation is:
[0101] R * =VU T
[0102] The optimal translation is:
[0103]
[0104] Among them, R * , t * This is the rotation error and translation error of the camera installation position of the calibration robot. Workers can calculate the error according to the calibrated parameter R * , t * Reinstall the depth camera or calculate new depth camera extrinsics.
[0105] The present invention only relies on the depth map information of the depth camera for calibration. Most of the current camera calibration tools need to use the camera RGB information for calibration. Calibration using only depth information can meet the calibration requirements of depth cameras that do not contain RGB lenses. The present invention does not require additional auxiliary tools, relies on three-dimensional rigid body structure information, and realizes manual and target-free rapid calibration. It is easy to get started and helps to quickly implement deployment and worker calibration in the factory. The present invention does not require additional auxiliary tools: the calibration process does not require additional auxiliary tools. It only needs to place a three-dimensional rigid body with obvious features in front of the robot camera and keep the background empty, because three-dimensional rigid bodies can be seen everywhere in life, such as factory cars, tables and chairs, wardrobes, etc. Therefore, compared with the need for special calibration plates and other auxiliary tools, the calibration cost and operation difficulty are greatly reduced. As shown in the figure below, the chair is a three-dimensional rigid body target with rich features in space. It can be used for camera installation calibration.
[0106] According to another aspect of the present invention, a computer-readable medium is further provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement any of the above methods.
[0107] According to another aspect of the present invention, there is also provided a device for information processing at a network device end, the device comprising a memory for storing computer program instructions and a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute any of the methods described above.
[0108] In summary, the present invention is a calibration scheme that only relies on the depth information of the camera. It is only necessary to mark the camera sampling and taking pictures at the selected calibration location, ensure that the marked place is open, and the characteristics of the reference calibration object are obvious. Among them, no special calibration plate is required, which greatly reduces the calibration cost. Only by comparing the pictures taken by an accurately installed depth camera, the calibration external parameters of the camera to be calibrated can be accurately and quickly calculated. For each robot to be calibrated, only one image needs to be sampled for calibration, which improves the efficiency of calibration, reduces the calibration difficulty of workers, and improves operability. Actual use shows that the scheme can complete calibration within one minute, with fast speed and strong robustness, which greatly improves the calibration efficiency in industrial environments. The present invention is mainly used in the field of robot sensor installation accuracy calibration. The installation accuracy calibration is carried out using the depth sensor information installed by the robot itself, without relying on additional sensor input, to achieve a calibration tool with high accuracy and strong operability.
[0109] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
[0110] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present invention (including related data structures) can be stored in a computer readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and the like. In addition, some steps or functions of the present invention can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.
[0111] In addition, a part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. The program instruction for calling the method of the present invention may be stored in a fixed or removable recording medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium, and / or stored in a working memory of a computer device that runs according to the program instruction. Here, according to an embodiment of the present invention, a device is included, the device including a memory for storing computer program instructions and a processor for executing program instructions, wherein, when the computer program instruction is executed by the processor, the device is triggered to run the method and / or technical solution based on the aforementioned multiple embodiments of the present invention.
[0112] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.
Claims
1. A camera installation and calibration tooling method and device based on depth information, wherein: The method includes: Obtain an original depth image of a stationary rigid object taken by a depth camera of the robot to be calibrated at a preset position, and obtain a target depth image of the same stationary rigid object taken by a properly installed depth camera of a reference robot at the same preset position; Performing a filtering operation on the original depth image to obtain a filtered original depth image; performing a filtering operation on the target depth image to obtain a filtered target depth image; According to the corresponding matching points in the unit matrix of the filtered original depth image and the target depth image, the optimal rotation and optimal translation of the depth camera of the robot to be calibrated are calculated.
2. The camera installation and calibration tooling method based on depth information as claimed in claim 1, wherein: Performing a filtering operation on the original depth image to obtain a filtered original depth image; The target depth image is filtered to obtain a filtered target depth image, including: Projecting the depth pixel points of the original depth image into an original 3D point cloud according to the intrinsic parameters of the depth camera of the robot to be calibrated; The RANSAC algorithm is used to remove points in the original 3D point cloud whose z-axis depth is greater than a preset depth threshold to obtain the filtered original 3D point cloud.
3. The camera installation and calibration tooling method based on depth information as claimed in claim 1, wherein: According to the intrinsic parameters of the depth camera of the robot to be calibrated, the depth pixels of the original depth image are projected into the original 3D point cloud. The formula for projection to the original 3D point cloud is as follows: y=(u-c y )*z / f y Among them, x, y are the x-axis and y-axis coordinates of the 3D point cloud; v and u are the x-axis and y-axis coordinates on the depth pixel map plane, and z is the depth value of the corresponding pixel position captured by the depth camera; The internal parameters of the depth camera include: the horizontal focal length of the camera f x , the vertical focal length of the camera is f y , the optical center displacement c in the x-axis direction of the imaging plane x , the optical center displacement c in the y-axis direction of the imaging plane y .
4. The camera installation and calibration tooling method based on depth information as claimed in claim 2, wherein: Using the RANSAC algorithm, remove points in the original 3D point cloud whose z-axis depth is greater than the preset depth threshold to obtain the filtered original 3D point cloud, including: Use the RANSAC algorithm to train the final point model corresponding to the original 3D point cloud; Based on the final point model, points in the original 3D point cloud whose z-axis depth is greater than a preset depth threshold are removed to obtain a filtered original 3D point cloud.
5. The camera installation and calibration tooling method based on depth information as claimed in claim 4, wherein: Using the RANSAC algorithm, the final point model corresponding to the original 3D point cloud is trained, including: The RANSAC algorithm repeatedly selects a set of random points in the original 3D point cloud as hypothetical inliers. Set an initial point model that is suitable for all assumed in-library points in the original 3D point cloud; Use the initial point model to test other points in the original 3D point cloud. If a certain other point also applies to the initial point model, then add the point to the assumed local point. If there are points greater than a preset number threshold that are classified as hypothetical inliers, the model is re-estimated based on all current hypothetical inliers to obtain the optimal point model corresponding to the original 3D point cloud; The optimal point model is evaluated by the number of points within the optimal point hypothesis and the error rate of the model to obtain the final point model.
6. The camera installation and calibration tooling method based on depth information as claimed in claim 1, wherein: According to the corresponding matching points in the unit matrix of the filtered original depth image and the target depth image, the optimal rotation and optimal translation of the installation position of the depth camera of the robot to be calibrated are calculated, including: Step S31, according to the corresponding matching points in the unit matrix of the filtered original depth image and the target depth image, calculate the transformation matrix from the filtered original depth image to the filtered target depth image, transform the filtered original depth image based on the transformation matrix, and obtain the projection of the origin cloud of the filtered original depth image in the target point cloud coordinate system of the filtered target depth image; Step S32, based on the projection, searching in the target point cloud for the point closest to each sampling point in the projection, and obtaining a matching point pair between the target point and the origin; Step S33, when the distance between the target point and the origin of a certain matching point pair is greater than a preset distance threshold, the matching point pair is considered to be an unreasonable matching point pair, and the unreasonable matching point pair needs to be eliminated to obtain the remaining reasonable matching point pairs, that is, the matching point pairs that are not eliminated; Step S34, based on the remaining reasonable matching point pairs, solve the optimal rotation R * , t * and the optimal translation R * , t * , so that the loss loss is minimized.
7. The camera installation and calibration tooling method based on depth information as claimed in claim 6, wherein: Step S34, based on the remaining reasonable matching point pairs, solve the optimal rotation R * , t * and the optimal translation R * , t * , to minimize the loss, and also include: After the preset number of iterations of steps S31 to S34 is executed cyclically, or the optimal rotation R calculated in the previous iteration is * , t * and the optimal translation R * , t * The optimal rotation R calculated in this iteration * , t * and the optimal translation R * , t * If the difference is less than the preset threshold, it is judged that the iteration has converged, and the optimal rotation R calculated by this iteration is * , t * and the optimal translation R * , t * , as the final optimal rotation R * , t * and optimal translation.
8. The camera installation and calibration tooling method based on depth information as claimed in claim 6, wherein: The calculation formula of loss Loss is as follows: in, and is the origin cloud P s and the target point cloud P t The corresponding point pairs in R * , t * It is the rotation error and translation error of the installation position of the depth camera of the robot to be calibrated, and i represents the serial number of a point pair; and Represent the centroid of the source point cloud and the target point cloud respectively, let make This is a 3×3 matrix. Perform SVD decomposition on H to get H=U∑V T , then the closed form solution of the ICP problem is as follows: The optimal rotation is: R * =VU T , The optimal translation is:
9. A computer-readable medium having computer-readable instructions stored thereon, wherein the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1 to 8.
10. A device for information processing at a network device end, the device comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein: When the computer program instructions are executed by the processor, the device is triggered to execute the method according to any one of claims 1 to 8.
