Calibration method, device and system, readable storage medium and chip
By obtaining and using the characteristic parameters of the calibration of the plane working robot and the internal and external parameters of the camera, projecting and adjusting, the distortion problem caused by the small optical axis angle in the calibration of the plane working robot is solved, and the calibration accuracy and accuracy are improved.
Patent Information
- Application Number
- CN202311781476.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-22
- Publication Date
- 2024-11-12
AI Technical Summary
During the calibration process of planar working robots, due to the small angle between the optical axis and the calibration plate, large distortions are easily generated, resulting in failed calibration or poor recognition effect. It is difficult for the existing technology to effectively solve this problem.
A calibration method is adopted to obtain the characteristic parameters of the target calibration pattern, including the center coordinates and dimension parameters of the feature point, and project using the camera's internal parameters and distortion parameters to generate the actual calibration pattern, and adjust it according to the external parameters of the plane working robot. The position of the camera device to ensure that the camera device is correctly aligned with the calibration plate and realize plane coordinate calibration.
It improves the accuracy of feature point coordinate extraction, avoids the problem of calibration pattern deformation caused by distance and distortion, and enhances the accuracy of subsequent measurement and control.
Smart Images

Figure CN118941638A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of robot calibration, and in particular to a calibration method, a calibration device, a calibration system, a readable storage medium and a chip. Background Art
[0002] Planar working robots need to calibrate their built-in cameras in order to complete various tasks. In related technologies, the calibration patterns are generally regular patterns. After the plane working robot is calibrated, it has a good recognition effect only when the camera optical axis and the calibration plate plane are at a vertical or nearly vertical viewing angle. Once the angle between the optical axis and the calibration plate is small, it will produce large distortion, or even fail to calibrate at all. Summary of the invention
[0003] In order to solve or improve at least one of the above technical problems, an object of the present invention is to provide a calibration method.
[0004] Another object of the present invention is to provide a calibration device.
[0005] Another object of the present invention is to provide a calibration system.
[0006] Another object of the present invention is to provide a readable storage medium.
[0007] Another object of the present invention is to provide a chip.
[0008] To achieve the above-mentioned purpose, the first aspect of the present invention provides a calibration method for a calibration system, the calibration system comprising a planar working robot and a camera device arranged on the planar working robot, the calibration method comprising: acquiring a target calibration pattern; determining characteristic parameters of the target calibration pattern, the characteristic parameters comprising center coordinate parameters and size parameters of characteristic points; projecting a target calibration plate according to the characteristic parameters to generate an actual calibration pattern on the target calibration plate; adjusting the position of the camera device according to the external parameter parameters of the planar working robot; and performing plane coordinate calibration on the camera device.
[0009] The calibration method provided by the present invention is mainly used for a calibration system composed of a plane working robot and a camera device. The camera device can use a low-cost monocular camera. The main goal of this scheme is to generate a calibration pattern for monocular camera plane coordinate estimation to improve the accuracy of feature point coordinate extraction under the condition of distortion and distance. Specifically, the target calibration pattern is first obtained, and the target calibration pattern is usually a specially designed calibration plate with known features. Then the characteristic parameters of the target calibration pattern are determined, and the characteristics of the target calibration pattern are extracted, usually including the center coordinate parameters and size parameters of the feature points. These parameters provide key information about the calibration pattern. The target calibration pattern is then projected onto the target calibration plate using the characteristic parameters to generate an actual calibration pattern. This process can consider the intrinsic parameters and distortion parameters of the camera to obtain an accurate projection. Finally, the position of the camera device is adjusted according to the external parameter parameters of the plane working robot, wherein the external parameter parameters include the posture information of the robot, that is, the position and direction of the robot. In this step, the position of the camera device is adjusted according to these external parameter parameters to ensure that the camera device is correctly aligned with the calibration plate. Finally, calibration is performed, that is, the plane coordinate calibration of the camera device is performed. This can include determining the intrinsic parameters of the camera and calibrating the camera coordinate system with the working coordinate system of the planar working robot so that subsequent measurement and control can be carried out accurately.
[0010] It can be understood that in this solution, according to the working characteristics of the robot, the basic graphics of the calibration pattern, such as circles, grids, etc., are determined to design an ideal calibration plate. Then the camera is regarded as a projector, and the approximate internal and external parameters of the camera are used to perform projection transformation on the ideal calibration plate to generate a calibration plate with a projection effect. Since distortion is taken into account in the projection process, the actual imaging captured by the camera is closer to the ideal calibration plate. This method avoids the problem of calibration pattern deformation caused by distance and distortion, thereby improving the accuracy of subsequent feature point coordinate extraction.
[0011] In addition, the above technical solution provided by the present invention may also have the following additional technical features:
[0012] In some technical schemes, optionally, the target calibration plate is projected according to the characteristic parameters to generate an actual calibration pattern on the target calibration plate, specifically including: obtaining the distance parameter between the camera device and the calibration plane; determining a first dimension perpendicular to the optical axis direction and a second dimension parallel to the optical axis direction in the calibration area of the target calibration plate; determining the imaging pixel coordinates on the target calibration plate according to the distance parameter, the first dimension and the second dimension; projecting according to the imaging pixel coordinates, and connecting the foreground points generated after the projection according to the flood filling method and performing inversion processing.
[0013] In this technical solution, in the process of projecting the target calibration plate according to the characteristic parameters, first, it is necessary to determine the distance parameter between the camera device and the calibration plane. This parameter is used to correct the projection because the distance affects the relationship between the pixel coordinates and the actual coordinates. On the target calibration plate, it is necessary to determine the calibration area for projection. The size of this area includes a first size perpendicular to the optical axis direction and a second size parallel to the optical axis direction. These sizes will help determine the size and shape of the actual calibration pattern to be generated on the calibration plate. Using the distance parameter, the first size and the second size, the imaging pixel coordinates of the actual calibration pattern generated on the calibration plate can be calculated. These coordinates will determine the position and shape on the image. Specifically, the horizontal coordinate in the imaging pixel coordinate can be determined according to the distance parameter and the first size, and the vertical coordinate in the imaging pixel coordinate can be determined according to the distance parameter and the second size. Using the calculated imaging pixel coordinates, the actual calibration pattern is projected onto the calibration plate. It can be understood that this projection takes into account the distance parameter to ensure that the generated pattern is consistent with the shape of the actual calibration plate. Finally, in order to ensure that the generated calibration pattern is continuous and can be accurately measured, the foreground points can be connected using the flood filling method. Additionally, it may be necessary to invert the generated pattern so that subsequent measurements can be made accurately.
[0014] In general, the purpose of this solution is to generate an accurate calibration pattern for use in subsequent calibration processes. This calibration pattern will help the robot system accurately identify and measure the target calibration plate, thereby improving the positioning and measurement accuracy of the system.
[0015] In some technical solutions, optionally, before projecting the target calibration plate according to the characteristic parameters, it also includes: obtaining distortion parameters of the camera device; determining an edge point set of the target calibration pattern according to the characteristic parameters; and performing a dedistortion operation on points in the edge point set according to the distortion parameters.
[0016] In this technical solution, before the calibration plate is projected, it also involves obtaining the distortion parameters of the camera device and performing a dedistortion operation. Specifically, the distortion parameters are first obtained from the camera device. These parameters generally include radial distortion and tangential distortion. Radial distortion is caused by the non-ideal lens shape, while tangential distortion is caused by the imperfect installation of the camera lens. The characteristic parameters (i.e., the center coordinate parameters and size parameters mentioned above) are then used to determine the edge point set of the target calibration pattern. These edge points are the outer contour of the calibration pattern. Finally, the obtained distortion parameters are used to perform a dedistortion operation on each point in the edge point set. This operation will correct the distortion and make the positions of the edge points closer to their positions in the absence of distortion.
[0017] This solution enables the feature points on the target calibration plate to be more accurately mapped into the image so that subsequent projection and measurement are accurate. Distortion correction can significantly improve the quality of the image, thereby improving the accuracy of calibration.
[0018] In some technical solutions, optionally, the camera device is placed according to the external parameters of the planar working robot, specifically including: setting the target calibration plate in the framing direction of the camera device; acquiring image information of the camera device; determining the calibration deviation according to the image information; and adjusting the target calibration plate according to the size of the calibration deviation.
[0019] In this technical solution, in the process of adjusting the position of the camera device by the external parameter parameters of the plane working robot, the target calibration plate is set in the viewfinder direction of the camera device, that is, the target calibration plate is placed in a position visible to the camera device to ensure that the camera can see the calibration plate. The camera device is started to acquire an image of the calibration plate. This usually requires setting appropriate exposure time, focal length and other parameters on the camera to ensure image clarity and visibility. After acquiring the image, computer vision technology is used to analyze the image to determine the position and orientation of the calibration plate relative to the camera device. This can include detecting feature points on the calibration plate, such as corner points, and then calculating their relative positions. These feature points are usually known, so they can be used to determine the calibration deviation. Finally, based on the analysis results of the calibration deviation, it can be determined whether the position and orientation of the calibration plate are correct. If there is a deviation, the position of the calibration plate needs to be adjusted to ensure that its position within the camera field of view is accurate. This usually involves translating and rotating the calibration plate to correct the deviation.
[0020] Through these steps, the position of the camera device can be calibrated according to the external parameters to ensure that the camera can accurately see the target calibration plate. This is a critical step in the camera calibration process because the correct camera position is very important for subsequent calibration and measurement.
[0021] In some technical solutions, optionally, the target calibration pattern includes a plurality of circular feature points arranged in an array, the point spacing and radius of each row of circular feature points are the same, and the point spacing and radius of different rows of circular feature points are different.
[0022] In this technical solution, the feature points are arranged in multiple rows and columns to form a grid-like structure. Each row contains a set of circular feature points. Within each row, the distance between the circular feature points is equal, which means that they are evenly distributed in the same interval. In addition, the radius of these circular feature points is also the same, which helps to maintain their uniformity. Although the feature points within each row have the same point spacing and radius, the feature points between different rows can have different spacing and radius. This means that the calibration pattern has vertical changes, and the point spacing and radius may gradually increase or decrease.
[0023] This regular arrangement and variation helps the calibration algorithm detect and identify these feature points in the image. When the calibration plate is imaged with the camera, the regularity and variation of these feature points can be used to determine the internal and external parameters of the camera for calibration. Such calibration patterns are often used for camera calibration, calibration of machine vision systems, or for ranging and navigation in robots and autonomous navigation systems.
[0024] A second aspect of the present invention provides a calibration device for a calibration system, the calibration system comprising a plane working robot and a camera device arranged on the plane working robot, the calibration device comprising: an acquisition module for acquiring a target calibration pattern; a parameter determination module for determining characteristic parameters of the target calibration pattern, the characteristic parameters comprising center coordinate parameters and size parameters of characteristic points; a projection module for projecting a target calibration plate according to the characteristic parameters to generate an actual calibration pattern on the target calibration plate; a calibration module for adjusting the position of the camera device according to the external parameter parameters of the plane working robot, and performing plane coordinate calibration on the camera device.
[0025] According to the technical solution of the calibration device of the present invention, it mainly includes an acquisition module, a parameter determination module, a projection module and a calibration module, which is mainly used for a calibration system composed of a plane working robot and a camera device. The camera device can use a low-cost monocular camera. The main goal of this solution is to generate a calibration pattern for monocular camera plane coordinate estimation to improve the accuracy of feature point coordinate extraction under the condition of considering distortion and distance. Specifically, first obtain a target calibration pattern, which is usually a specially designed calibration plate with known features. Then determine the characteristic parameters of the target calibration pattern, extract the characteristics of the target calibration pattern, usually including the center coordinate parameters and size parameters of the feature points. These parameters provide key information about the calibration pattern. Then use the characteristic parameters to project the target calibration pattern onto the target calibration plate to generate an actual calibration pattern. This process can consider the intrinsic parameters and distortion parameters of the camera to obtain accurate projection. Finally, according to the external parameter parameters of the plane working robot, the position of the camera device is adjusted, wherein the external parameter parameters include the posture information of the robot, that is, the position and direction of the robot. In this step, the position of the camera device is adjusted according to these external parameters to ensure that the camera device is correctly aligned with the calibration plate. Finally, calibration is performed, that is, the plane coordinate calibration of the camera device. This can include determining the intrinsic parameters of the camera and calibrating the camera coordinate system with the working coordinate system of the plane working robot so that subsequent measurement and control can be carried out accurately.
[0026] A third aspect of the present invention provides a calibration system, comprising: any calibration device as described above; a planar working robot, comprising a mobile device and a camera device disposed on the mobile device.
[0027] On the other hand, the present invention provides a calibration system, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor, which implements the steps of the calibration method in any of the above technical solutions when the program or instruction is executed by the processor.
[0028] A fourth aspect of the present invention provides a readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the calibration method in any of the above technical solutions are implemented.
[0029] A fifth aspect of the present invention provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the steps of the calibration method in any of the above technical solutions.
[0030] Additional aspects and advantages of the technical solutions of the present invention will become apparent in the following description or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 A schematic diagram showing a calibration method according to an embodiment of the present invention is shown;
[0032] Figure 2 A schematic diagram showing a calibration method according to an embodiment of the present invention is shown;
[0033] Figure 3 A schematic diagram showing a calibration method according to an embodiment of the present invention is shown;
[0034] Figure 4 A schematic diagram showing a calibration method according to an embodiment of the present invention is shown;
[0035] Figure 5 A schematic structural diagram of a calibration device according to an embodiment of the present application is shown;
[0036] Figure 6 A schematic diagram of a calibration system according to an embodiment of the present invention is shown;
[0037] Figure 7 A schematic diagram of a calibration system according to an embodiment of the present invention is shown;
[0038] Figure 8 It shows the effect diagram after the inversion process is performed according to one embodiment of the present invention;
[0039] Fig. 9 It shows the effect diagram of an ideal calibration plate according to one embodiment of the present invention;
[0040] Fig.10 A schematic diagram showing a calibration method according to an embodiment of the present invention is shown;
[0041] Fig.11 A schematic diagram of a calibration method according to an embodiment of the present invention is shown.
[0042] in, Figures 5 to 7 The corresponding relationship between the reference numerals and the component names is as follows:
[0043] 100: calibration system; 102: plane working robot; 1022: moving device; 1024: camera device; 200: calibration device; 202: acquisition module; 204: parameter determination module; 206: projection module; 208: calibration module; 212: memory; 214: processor. DETAILED DESCRIPTION
[0044] In order to more clearly understand the above-mentioned purposes, features and advantages of the embodiments of the present invention, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0045] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the embodiments of the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present application is not limited to the specific embodiments disclosed below.
[0046] Refer to the following Figures 1 to 11 The present invention describes a calibration method, a calibration device, a calibration system, a readable storage medium, and a chip provided according to some embodiments of the present invention.
[0047] This embodiment provides a calibration method for calibrating a system, wherein the calibration system includes a plane working robot and a camera device disposed on the plane working robot, such as Figure 1 As shown, the calibration method includes:
[0048] Step S102: obtaining a target calibration pattern;
[0049] Step S104: determining characteristic parameters of the target calibration pattern; the characteristic parameters include center coordinate parameters and size parameters of characteristic points;
[0050] Step S106: projecting the target calibration plate according to the characteristic parameters to generate an actual calibration pattern on the target calibration plate;
[0051] Step S108: adjusting the position of the camera device according to the external parameters of the plane working robot;
[0052] Step S110: calibrate the plane coordinates of the camera device.
[0053] The calibration method provided in this embodiment is mainly used for a calibration system composed of a plane working robot and a camera device. The camera device can use a low-cost monocular camera. The main goal of this solution is to generate a calibration pattern for monocular camera plane coordinate estimation to improve the accuracy of feature point coordinate extraction under the condition of distortion and distance. Specifically, the target calibration pattern is first obtained, which is usually a specially designed calibration plate with known features. Then the characteristic parameters of the target calibration pattern are determined, and the characteristics of the target calibration pattern are extracted, which usually include the center coordinate parameters and size parameters of the feature points. These parameters provide key information about the calibration pattern. The target calibration pattern is then projected onto the target calibration plate using the characteristic parameters to generate an actual calibration pattern. This process can consider the intrinsic parameters and distortion parameters of the camera to obtain an accurate projection. Finally, the position of the camera device is adjusted according to the external parameter parameters of the plane working robot, wherein the external parameter parameters include the robot's posture information, that is, the position and direction of the robot. In this step, the position of the camera device is adjusted according to these external parameter parameters to ensure that the camera device is correctly aligned with the calibration plate. Finally, calibration is performed, that is, the plane coordinate calibration of the camera device is performed. This can include determining the intrinsic parameters of the camera and calibrating the camera coordinate system with the working coordinate system of the planar working robot so that subsequent measurement and control can be carried out accurately.
[0054] It can be understood that in this solution, according to the working characteristics of the robot, the basic graphics of the calibration pattern, such as circles, grids, etc., are determined to design an ideal calibration plate. Then the camera is regarded as a projector, and the approximate internal and external parameters of the camera are used to perform projection transformation on the ideal calibration plate to generate a calibration plate with a projection effect. Since distortion is taken into account in the projection process, the actual imaging captured by the camera is closer to the ideal calibration plate. This method avoids the problem of calibration pattern deformation caused by distance and distortion, thereby improving the accuracy of subsequent feature point coordinate extraction.
[0055] In a specific embodiment, the plane working robot is a sweeping robot, and the calibration process is as follows:
[0056] Estimate the internal and external parameters of the sweeping robot used in the experiment (cameras of the same model and the same batch are roughly similar, select the internal and external parameters of any camera as the typical estimated parameters, or use the specification parameters in the camera manual as the approximate internal parameters, and use the robot-related structural parameters as the approximate external parameters).
[0057] Based on the camera posture and actual environment, the image area corresponding to the calibration range is determined, and the feature point pattern and arrangement are set. Coarse adjustments can be made through the projection formula and fine adjustments can be made based on the actual imaging effect.
[0058] Perform projection and other calculations based on the calibration plate designed in the previous step to generate the actual calibration plate.
[0059] The camera equipment is placed according to the requirements of the external parameters used, and the actual calibration plate is projected or printed and laid in front of the camera. The camera is controlled to take pictures to check the image effect. If the deviation is small, the next calibration calculation can be carried out. If the deviation is large, it means that the internal and external parameters selected in the first step are incorrect and the calibration plate should be readjusted, or there is a large deviation in the internal and external parameters of the camera, which means that the camera equipment to be calibrated is unqualified and the camera should be reinstalled or replaced.
[0060] The pixel coordinates of the center of the feature point are extracted, and the world coordinates of the center of the feature point are generated. After being associated by the relevant matching algorithm, the plane coordinates of the monocular camera can be calibrated.
[0061] Alternatively, if Figure 2 As shown, the target calibration plate is projected according to the characteristic parameters to generate an actual calibration pattern on the target calibration plate, specifically including:
[0062] Step S1062: Obtain the distance parameter between the camera device and the calibration plane;
[0063] Step S1064: determining a first dimension perpendicular to the optical axis and a second dimension parallel to the optical axis in the calibration area of the target calibration plate;
[0064] Step S1066: determining the imaging pixel coordinates on the target calibration plate according to the distance parameter, the first size, and the second size;
[0065] Step S1068: Projection is performed according to the imaging pixel coordinates, and the foreground points generated after the projection are connected and inverted according to the flood filling method.
[0066] In this embodiment, in the process of projecting the target calibration plate according to the characteristic parameters, first, it is necessary to determine the distance parameter between the camera device and the calibration plane. This parameter is used to correct the projection because the distance affects the relationship between the pixel coordinates and the actual coordinates. On the target calibration plate, it is necessary to determine the calibration area for projection. The size of this area includes a first size perpendicular to the optical axis direction and a second size parallel to the optical axis direction. These sizes will help determine the size and shape of the actual calibration pattern to be generated on the calibration plate. Using the distance parameter, the first size and the second size, the imaging pixel coordinates of the actual calibration pattern generated on the calibration plate can be calculated. These coordinates will determine the position and shape on the image. Specifically, the horizontal coordinate in the imaging pixel coordinate can be determined according to the distance parameter and the first size, and the vertical coordinate in the imaging pixel coordinate can be determined according to the distance parameter and the second size. Using the calculated imaging pixel coordinates, the actual calibration pattern is projected onto the calibration plate. It can be understood that this projection takes into account the distance parameter to ensure that the generated pattern is consistent with the shape of the actual calibration plate. Finally, in order to ensure that the generated calibration pattern is continuous and can be accurately measured, the foreground points can be connected using the flood filling method. Additionally, it may be necessary to invert the generated pattern so that subsequent measurements can be made accurately.
[0067] In general, the purpose of this solution is to generate an accurate calibration pattern for use in subsequent calibration processes. This calibration pattern will help the robot system accurately identify and measure the target calibration plate, thereby improving the positioning and measurement accuracy of the system.
[0068] In a specific embodiment, the center coordinates and radius length of the feature points at a certain resolution are obtained, and S3 uses this as input to project the ideal calibration plate. First, determine the points to be projected, including the center point and edge points of the feature points, and obtain a higher resolution edge point set by sampling the angles combined with the center point coordinates and radius. Then, dedistort the points in the point set using approximate distortion parameters. Then comes the projection, which is achieved through the camera coordinate transformation matrix, such as formulas #2 and #3. For the case where the calibration plane is parallel to the camera optical axis, the formula is as follows:
[0069] (u d , v d ) = undistort(y0, v0) #(1)
[0070]
[0071]
[0072] u0, v0 represent the pixel coordinates on the template image, u d , v d Represented as the pixel coordinate after dedistortion, yc represents the height of the camera from the calibration plane, w and h represent the width (perpendicular to the optical axis) and height (parallel to the optical axis) of the calibration area, respectively, and u and v represent the corresponding pixel coordinates on the calibration pattern. Due to limited sampling, the foreground points after projection need to be connected and inverted by flood fill.
[0073] It can be understood that the undistort function is the dedistortion and stereo correction operation in camera calibration.
[0074] Alternatively, if Figure 3 As shown, before projecting the target calibration plate according to the characteristic parameters, it also includes:
[0075] Step S1052: Obtain distortion parameters of the camera device;
[0076] Step S1054: determining an edge point set of the target calibration pattern according to the feature parameters;
[0077] Step S1056: performing a dedistortion operation on the points in the edge point set according to the distortion parameters.
[0078] In this embodiment, before the calibration plate is projected, it is also involved to obtain the distortion parameters of the camera device and perform a de-distortion operation. Specifically, the distortion parameters are first obtained from the camera device. These parameters generally include radial distortion and tangential distortion. Radial distortion is caused by the non-ideal lens shape, while tangential distortion is caused by the imperfect installation of the camera lens. The characteristic parameters (i.e., the center coordinate parameters and size parameters mentioned above) are then used to determine the edge point set of the target calibration pattern. These edge points are the outer contour of the calibration pattern. Finally, the obtained distortion parameters are used to perform a de-distortion operation on each point in the edge point set. This operation will correct the distortion and make the positions of the edge points closer to their positions in the absence of distortion.
[0079] This solution enables the feature points on the target calibration plate to be more accurately mapped into the image so that subsequent projection and measurement are accurate. Distortion correction can significantly improve the quality of the image, thereby improving the accuracy of calibration.
[0080] Alternatively, if Figure 4 As shown, the camera equipment is placed according to the external parameters of the plane working robot, including:
[0081] Step S1082: setting the target calibration plate in the framing direction of the camera device;
[0082] Step S1084: Acquire image information of the camera device;
[0083] Step S1086: determining a calibration deviation according to the image information;
[0084] Step S1088: Adjust the target calibration plate according to the size of the calibration deviation.
[0085] In this embodiment, in the process of adjusting the position of the camera device by the external parameter parameters of the plane working robot, the target calibration plate is set in the viewfinder direction of the camera device, that is, the target calibration plate is placed in a position visible to the camera device to ensure that the camera can see the calibration plate. The camera device is started to acquire an image of the calibration plate. This usually requires setting appropriate exposure time, focal length and other parameters on the camera to ensure image clarity and visibility. After acquiring the image, computer vision technology is used to analyze the image to determine the position and orientation of the calibration plate relative to the camera device. This can include detecting feature points on the calibration plate, such as corner points, and then calculating their relative positions. These feature points are usually known, so they can be used to determine the calibration deviation. Finally, based on the analysis results of the calibration deviation, it can be determined whether the position and orientation of the calibration plate are correct. If there is a deviation, the position of the calibration plate needs to be adjusted to ensure that its position within the camera field of view is accurate. This usually involves translating and rotating the calibration plate to correct the deviation.
[0086] Through these steps, the position of the camera device can be calibrated according to the external parameters to ensure that the camera can accurately see the target calibration plate. This is a critical step in the camera calibration process because the correct camera position is very important for subsequent calibration and measurement.
[0087] Based on any of the above embodiments, the feature points are arranged in multiple rows and columns to form a grid structure. Each row contains a set of circular feature points. Within each row, the distances between the circular feature points are equal, which means that they are evenly distributed in the same interval. In addition, the radius of these circular feature points is also the same, which helps to maintain their uniformity. Although the feature points in each row have the same point spacing and radius, the feature points between different rows can have different spacing and radius. This means that the calibration pattern has vertical changes, and the point spacing and radius may gradually increase or decrease.
[0088] This regular arrangement and variation helps the calibration algorithm detect and identify these feature points in the image. When the calibration plate is imaged with the camera, the regularity and variation of these feature points can be used to determine the internal and external parameters of the camera for calibration. Such calibration patterns are often used for camera calibration, calibration of machine vision systems, or for ranging and navigation in robots and autonomous navigation systems.
[0089] In a specific embodiment, a calibration pattern design method for monocular camera plane coordinate estimation is proposed, which can be applied to the camera calibration of a visual robot before leaving the factory, so as to estimate the coordinates of a given plane pixel point within the field of view of the robot when it is working.
[0090] First, according to the working characteristics of the robot, the basic graphics of the calibration pattern, such as circles, grids, etc., are determined to design an ideal calibration plate. Then the camera is regarded as a projector, and the approximate internal and external parameters of the camera are used to perform projection transformation on the ideal calibration plate to generate a calibration plate with projection effect. Since distortion is considered in the projection process, the actual image captured by the camera is closer to the ideal calibration plate. This method avoids the problem of calibration pattern deformation caused by distance and distortion, thereby improving the accuracy of subsequent feature point coordinate extraction. Design methods such as Fig.10 As shown, specifically including:
[0091] S1: Design of ideal calibration pattern and extraction of feature point center and radius. Taking the circular calibration pattern as an example, when the circular pattern is stretched or has noise at the edge, the center point changes less. Fig. 9 As shown, it is not difficult to imagine that if the actual image of the calibration plate taken by the robot is consistent with the calibration plate (all regular circles), the subsequent center extraction will be accurate and robust. Circular feature points are not easily lost during detection, and the center can be accurately located even at an inclined viewing angle. In the calibration plate, circular feature points appear in rows, and the spacing between feature points in each row is the same, and the radius of the feature points is the same. However, the spacing and radius of feature points in different rows are different, and the starting and ending points are different. These arrangement characteristics depend on some prior knowledge: the pixel coordinates of the top row of points are about 1.4m in front of the experimental robot in the present invention, that is, the maximum calibration distance. If feature points are set up further, the area of the point will be very large during actual projection; the two ends of the top row of points are not arranged to the edge of the image because they will be projected outside the calibration area; when the camera is shot at a horizontal viewing angle, the upper and lower images correspond to the distance respectively. Since the camera imaging is "near large and far small", that is, the small circle at the top will become a very large stretched figure after projection; the dots near the edge at the bottom are severely deformed after projection due to viewing angle and distortion, so they are eliminated.
[0092] S2: Extraction of feature point center and radius. The input here is the calibration plate (image format) designed in S1, and the output is the center pixel coordinates and radius pixel length of each dot. The main methods used are: 1. Use cv2.findContours() method to obtain the edge pixel position of the feature point; 2. Use cv2.moments() to calculate the image distance of the feature image, and infer the center point of the feature point from the image distance; 3. Use cv2.contourArea() to calculate the area of the feature point, and calculate the radius of the feature point from the area.
[0093] Among them, cv2.findContours(), cv2.moments(), and cv2.contourArea() are all functions in opencv.
[0094] S3: Actual calibration pattern generation (ideal calibration plate projection transformation and post-processing). Based on S2, the center coordinates and radius length of the feature points at a certain resolution have been obtained. S3 uses this as input to project the ideal calibration plate. First, determine the points to be projected, including the center point and edge points of the feature points, and obtain a higher resolution edge point set by sampling the angles combined with the center point coordinates and radius. Then, the points in the point set are dedistorted using approximate distortion parameters. Then comes the projection, which is achieved through the camera coordinate transformation matrix, such as formulas (#2) and (#3). For the case where the calibration plane is parallel to the camera optical axis, the formula is as follows:
[0095] (u d , v d ) = undistort(u0, v0) #(1)
[0096]
[0097]
[0098] u0, v0 represent the pixel coordinates on the template image, u d , v d Represented as the pixel coordinate after dedistortion, y c represents the height of the camera from the calibration plane, w and h represent the width (perpendicular to the optical axis) and height (parallel to the optical axis) of the calibration area, respectively, and u and v represent the corresponding pixel coordinates on the calibration pattern. Due to limited sampling, the foreground points after projection need to be connected and inverted by flood fill. The result is as follows: Figure 8 As shown, the ideal calibration plate is Fig. 9 As shown in the figure, the overall algorithm flow is as follows Fig.11 shown.
[0099] It can be understood that the undistort function is the dedistortion and stereo correction operation in camera calibration.
[0100] In a specific embodiment, a calibration process of a sweeping robot based on the above-mentioned calibration pattern design and matching calibration method is as follows:
[0101] Estimate the internal and external parameters of the sweeping robot used in the experiment (cameras of the same model and the same batch are roughly similar, select the internal and external parameters of any camera as the typical estimated parameters, or use the specification parameters in the camera manual as the approximate internal parameters, and use the robot-related structural parameters as the approximate external parameters).
[0102] Based on the camera posture and actual environment, the image area corresponding to the calibration range is determined, and the feature point pattern and arrangement are set. Coarse adjustments can be made through the projection formula and fine adjustments can be made based on the actual imaging effect.
[0103] Perform projection and other calculations based on the designed calibration plate to generate the actual calibration plate.
[0104] The camera equipment is placed according to the requirements of the external parameters used, and the actual calibration plate is projected or printed and laid in front of the camera. The camera is controlled to take pictures to check the image effect. If the deviation is small, the next calibration calculation can be carried out. If the deviation is large, it means that the selected internal and external parameters are incorrect and the calibration plate should be readjusted, or there is a large deviation in the internal and external parameters of the camera, which means that the camera equipment to be calibrated is unqualified and the camera should be reinstalled or replaced.
[0105] The pixel coordinates of the center of the feature point are extracted, and the world coordinates of the center of the feature point are generated. After being associated by the relevant matching algorithm, the plane coordinates of the monocular camera can be calibrated.
[0106] like Figure 5 As shown, the embodiment of the present application provides a calibration device 200. The calibration device 200 includes an acquisition module 202, a parameter determination module 204, a projection module 206 and a calibration module 208.
[0107] Among them, the acquisition module 202 is used to acquire the target calibration pattern; the parameter determination module 204 is used to determine the characteristic parameters of the target calibration pattern, and the characteristic parameters include the center coordinate parameters and size parameters of the characteristic points; the projection module 206 is used to project the target calibration plate according to the characteristic parameters to generate the actual calibration pattern on the target calibration plate; the calibration module 208 is used to adjust the position of the camera device according to the external parameter parameters of the planar working robot and perform plane coordinate calibration on the camera device.
[0108] This embodiment is mainly used for a calibration system composed of a plane working robot and a camera device. The camera device can use a low-cost monocular camera. The main goal of this solution is to generate a calibration pattern for monocular camera plane coordinate estimation to improve the accuracy of feature point coordinate extraction while considering distortion and distance. Specifically, first obtain a target calibration pattern, which is usually a specially designed calibration plate with known features. Then determine the characteristic parameters of the target calibration pattern, extract the characteristics of the target calibration pattern, and usually include the center coordinate parameters and size parameters of the feature points. These parameters provide key information about the calibration pattern. Then use the characteristic parameters to project the target calibration pattern onto the target calibration plate to generate an actual calibration pattern. This process can consider the intrinsic parameters and distortion parameters of the camera to obtain an accurate projection. Finally, according to the external parameter parameters of the plane working robot, the position of the camera device is adjusted, wherein the external parameter parameters include the robot's posture information, that is, the position and direction of the robot. In this step, according to these external parameter parameters, adjust the position of the camera device to ensure that the camera device is correctly aligned with the calibration plate. Finally, calibration is performed, that is, the plane coordinate calibration of the camera device is performed. This can include determining the intrinsic parameters of the camera and calibrating the camera coordinate system with the working coordinate system of the planar working robot so that subsequent measurement and control can be carried out accurately.
[0109] The calibration device 200 in the embodiment of the present application can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage 212 (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0110] The calibration device 200 in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0111] The calibration device 200 provided in the embodiment of the present application can achieve Figures 1 to 4To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0112] Alternatively, if Figure 7 As shown, an embodiment of the present application further provides a calibration system 100, comprising: any calibration device 200 as described above; a planar working robot 102, comprising a mobile device 1022 and a camera device 1024 disposed on the mobile device.
[0113] Alternatively, if Figure 6 As shown, the embodiment of the present application also provides a calibration system 100, including a processor 214, a memory 212, and a program or instruction stored in the memory 212 and executable on the processor 214. When the program or instruction is executed by the processor 214, each process of the embodiment of the above-mentioned calibration method is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0114] It should be noted that the calibration system in the embodiment of the present application includes electronic equipment and non-electronic equipment.
[0115] Among them, the processor 214 is used to obtain the target calibration pattern; determine the characteristic parameters of the target calibration pattern, the characteristic parameters include the center coordinate parameters and size parameters of the characteristic points; project the target calibration plate according to the characteristic parameters to generate an actual calibration pattern on the target calibration plate; adjust the position of the camera device according to the external parameter parameters of the planar working robot; and perform plane coordinate calibrating on the camera device.
[0116] Through the above scheme, a calibration pattern for monocular camera plane coordinate estimation can be generated to improve the accuracy of feature point coordinate extraction while taking distortion and distance into consideration.
[0117] Optionally, the processor 214 is also used to implement the following steps: obtaining a distance parameter between the camera device and the calibration plane; determining a first dimension perpendicular to the optical axis direction and a second dimension parallel to the optical axis direction in the calibration area of the target calibration plate; determining the imaging pixel coordinates on the target calibration plate according to the distance parameter, the first dimension and the second dimension; performing projection according to the imaging pixel coordinates, and connecting the foreground points generated after the projection according to the flood filling method and performing inversion processing.
[0118] Optionally, the processor 214 is further configured to obtain distortion parameters of the camera device; determine an edge point set of the target calibration pattern according to the characteristic parameters; and perform a dedistortion operation on points in the edge point set according to the distortion parameters.
[0119] Optionally, the processor 214 is further configured to set a target calibration plate in a framing direction of the camera device; obtain image information of the camera device; determine a calibration deviation according to the image information; and adjust the target calibration plate according to a size of the calibration deviation.
[0120] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned calibration method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0121] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0122] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned calibration method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0123] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0124] According to the embodiments of the calibration method, calibration device, calibration system, readable storage medium and chip of the present invention, the automatic calibration of the robot tool coordinate system is realized without setting a reference target, avoiding the influence of visual deviation on the precision, and improving the calibration accuracy. At the same time, this solution can be integrated with the robot program to realize full-automatic calibration, which improves the work efficiency and reduces the manual operation intensity for applications such as welding and gluing that require frequent tool changes or calibration tools.
[0125] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0126] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0127] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A calibration method, characterized in that: Used for calibrating a system, the calibration system comprising a plane working robot and a camera device arranged on the plane working robot, the calibration method comprising: Obtaining a target calibration pattern; Determining characteristic parameters of the target calibration pattern, wherein the characteristic parameters include center coordinate parameters and size parameters of characteristic points; Projecting the target calibration plate according to the characteristic parameters to generate an actual calibration pattern on the target calibration plate; Adjust the position of the camera device according to the external parameter parameters of the planar working robot; The camera device is calibrated in plane coordinates.
2. The calibration method according to claim 1, characterized in that: The projecting the target calibration plate according to the characteristic parameters to generate an actual calibration pattern on the target calibration plate specifically includes: Obtaining a distance parameter between the camera device and a calibration plane; Determine a first dimension perpendicular to the optical axis direction and a second dimension parallel to the optical axis direction in the calibration area of the target calibration plate; Determine the imaging pixel coordinates on the target calibration plate according to the distance parameter, the first size and the second size; Projection is performed according to the imaging pixel coordinates, and foreground points generated after the projection are connected and inverted according to a flood filling method.
3. The calibration method according to claim 2, characterized in that: Before projecting the target calibration plate according to the characteristic parameters, the method further includes: Obtaining distortion parameters of the camera device; Determine an edge point set of the target calibration pattern according to the characteristic parameters; A dedistortion operation is performed on the points in the edge point set according to the distortion parameters.
4. The calibration method according to claim 1, characterized in that: The step of adjusting the position of the camera device according to the external parameters of the planar working robot specifically includes: Setting the target calibration plate in the framing direction of the camera device; Acquire image information of the camera device; determining a calibration deviation based on the image information; The target calibration plate is adjusted according to the size of the calibration deviation.
5. The calibration method according to claim 1, characterized in that: The target calibration pattern includes a plurality of circular feature points arranged in an array, the circular feature points in each row have the same point spacing and the same radius, and the circular feature points in different rows have different point spacing and different radius.
6. A calibration device, characterized in that: Used for calibrating a system, the calibration system includes a plane working robot and a camera device arranged on the plane working robot, and the calibration device includes: An acquisition module, used for acquiring a target calibration pattern; A parameter determination module, used to determine characteristic parameters of the target calibration pattern, wherein the characteristic parameters include center coordinate parameters and size parameters of characteristic points; A projection module, used for projecting the target calibration plate according to the characteristic parameters to generate an actual calibration pattern on the target calibration plate; A calibration module is used to adjust the position of the camera device according to the external parameters of the planar working robot and calibrate the plane coordinates of the camera device.
7. A calibration system, characterized in that: include: The calibration device according to claim 6; A plane working robot comprises a moving device and a camera device arranged on the moving device.
8. A calibration system, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the calibration method as claimed in any one of claims 1 to 5.
9. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the calibration method according to any one of claims 1 to 5 are implemented.
10. A chip, characterized in that: The chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the calibration method according to any one of claims 1 to 5.
Citation Information
Patent Citations
Image-assisting system using calibration method
CN102163331A
Projector lens distortion correcting method
CN105528770A
Camera adjustable calibration system
CN109360249A
Equipment calibration method and device, electronic equipment and storage medium
CN113192145A
Cited By
Method, device and equipment for calibrating external parameters of contourgraph
CN120576686A
