Visual calibration method and system of mobile robot and operation method and system

Through the visual calibration method of mobile robots, the transformation matrix between the camera and the robot is obtained, and the problem of insufficient accuracy and real-time accuracy of 3D visual guidance of mobile robots in the prior art is solved, and accurate cutting and grasping of large-sized material boxes and materials is achieved. Improves the accuracy and reliability of the work.

CN118952222APending Publication Date: 2024-11-15SPEEDBOT ROBOTICS CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411370704.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-29
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing 3D visual guidance operation scheme of mobile robots has shortcomings in accuracy and real-time performance, and it is difficult to meet the needs of variable environments and different operating tasks, especially in scenarios such as material box cutting and grabbing.

Method used

A mobile robot visual calibration method is adopted to obtain the transformation matrix from the camera coordinate system to the calibration plate coordinate system when the robot is in the working position, and combine the robot's photography posture matrix to calculate the transformation matrix from the camera coordinate system to the robot coordinate system to achieve high accuracy. visual calibration and re-execute the hand-eye calibration process after each movement to ensure the recognition and positioning accuracy.

Benefits of technology

Effectively expand the camera shooting range, improve work accuracy and reliability, ensure the identification and positioning accuracy of plus or minus 1mm, overcome the problem of high accuracy requirements for mobile robots, and realize accurate cutting of large-sized material boxes and internal materials. and crawl.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118952222A_ABST
    Figure CN118952222A_ABST
Patent Text Reader

Abstract

The invention discloses a mobile robot vision calibration method and system and an operation method and system.The mobile robot vision calibration method comprises the following steps that when a robot is located at an operation position, a transformation matrix Tb1 from a camera coordinate system to a first calibration plate coordinate system and a current photographing posture matrix P1 of the robot are obtained; wherein the first calibration plate is located on a mechanical arm of the robot; s2, on the basis of Tx, Tb1 and P1, a transformation matrix T1 from a camera coordinate system to a robot coordinate system is obtained, and calibration is completed; wherein Tx is a transformation matrix from the first calibration plate coordinate system to the robot coordinate system. The method is suitable for visual guidance operation of the mobile robot with the eyes outside the hands, the shooting range of the camera can be expanded, the visual calibration precision is high, the real-time performance is good, even if the mobile robot cannot accurately reach the fixed position every time, the recognition and positioning precision can be guaranteed, and the operation precision and reliability are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of mobile robot vision-guided operation, and in particular to a mobile robot vision calibration method and system, and an operation method and system. Background Art

[0002] In the application field of mobile robots, 3D vision-guided operation technology is gradually emerging. In common solutions, robots and 3D cameras are mostly installed in the way of eyes on the hands. Limited by hardware conditions such as the robot's arm span, this method has a relatively narrow field of view, and the size range of objects that can guide operations and operations is also relatively limited. For the installation method of eyes outside the hand, although it has advantages such as a large field of view and a wide range of operable object sizes, most of them are used in fixed robot operation scenarios. If it is used for mobile robot operations, the robot is required to be in place accurately. In the prior art, since most robots are fixed in position or have a small range of movement, hand-eye calibration often only needs to be performed once.

[0003] If the existing 3D vision-guided operation scheme of mobile robots adopts the calibration method of eyes outside hands, the positioning accuracy of the mobile robots will be high, otherwise it will not be able to perform high-precision guidance operations. The existing calibration methods are insufficient in terms of accuracy and real-time performance, and cannot effectively cope with the challenges of mobile robots in changing environments and different operation tasks, and it is difficult to meet the operation requirements of mobile robots in different scenarios. For example, in specific operation scenarios such as cutting and grasping of material boxes, inaccurate cutting or grasping may occur. Summary of the invention

[0004] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and provide a mobile robot vision calibration method and system, an operation method and system, which are suitable for vision-guided operations of mobile robots with eyes outside hands, can expand the camera shooting range, have high visual calibration accuracy and good real-time performance, and improve operation accuracy and reliability.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A mobile robot visual calibration method, which is characterized by including:

[0007] When the robot is in working position, perform the following steps:

[0008] Step S1, obtaining the transformation matrix Tb1 from the camera coordinate system to the first calibration plate coordinate system and the robot's current photographing posture matrix P1; wherein the first calibration plate is located on the robot's mechanical arm;

[0009] Step S2, based on Tx, Tb1, and P1, obtain the transformation matrix T1 from the camera coordinate system to the robot coordinate system to complete the calibration; wherein Tx is the transformation matrix from the first calibration plate coordinate system to the robot coordinate system.

[0010] As a preferred method, the Tx acquisition process includes:

[0011] When the robot is in the initial position, perform the following steps:

[0012] Step S01, obtaining the transformation matrix T0 from the camera coordinate system to the robot coordinate system, the transformation matrix Tb0 from the camera coordinate system to the first calibration board coordinate system, and the robot shooting posture matrix P0;

[0013] Step S02: Based on T0, Tb0, and P0, obtain the transformation matrix Tx from the first calibration plate coordinate system to the robot coordinate system.

[0014] As a preferred embodiment, in step S01, obtaining the transformation matrix T0 from the camera coordinate system to the robot coordinate system includes:

[0015] Step S011, obtaining a coordinate value V1 of the second calibration plate in the camera coordinate system and a coordinate value V2 of the second calibration plate in the robot coordinate system; wherein the second calibration plate is located on the robot base;

[0016] Step S012: Based on V1 and V2, obtain the transformation matrix T0 from the camera coordinate system to the robot coordinate system.

[0017] As a preferred manner, in step S011, the second calibration plate has feature points, an image of the second calibration plate is captured by a camera, and coordinate values ​​of the feature points on the second calibration plate in the camera coordinate system are identified through the image.

[0018] As a preferred method, in step S011, the second calibration plate has a feature point, the calibration needle on the robot base is pointed to the feature point, and the coordinate value of the feature point on the second calibration plate in the robot coordinate system is obtained.

[0019] Based on the same inventive concept, the present invention also provides a mobile robot vision calibration system, and the mobile robot vision calibration system is used to execute the mobile robot vision calibration method.

[0020] Based on the same inventive concept, the present invention also provides a mobile robot operation method, and the mobile robot operation method performs visual calibration on the robot based on the mobile robot visual calibration method.

[0021] As a preferred embodiment, the operation includes material box cutting, and the material box cutting operation method includes:

[0022] The camera is used to collect the image of the material box and the corner point information of the material box in the image of the material box is identified;

[0023] Based on the material box corner point information and transformation matrix T1, the cutting point and cutting angle information of the material box corner point in the robot coordinate system are calculated;

[0024] The mobile robot completes the material box cutting operation based on the cutting point and cutting angle information.

[0025] As another preferred embodiment, the operation includes grabbing materials inside the material box, and the material grabbing operation method includes:

[0026] Use the camera to collect images inside the material box and identify the image information of each material in the image inside the material box;

[0027] Based on the image information of each material, obtain the size information and corner point information of each material;

[0028] Based on the size information of each material, the corner point information of each material, and the transformation matrix T1, the grasping point and grasping angle information of each material in the robot coordinate system are calculated;

[0029] The mobile robot completes the material grabbing operation based on the grabbing point and grabbing angle information.

[0030] Based on the same inventive concept, the present invention also provides a mobile robot operating system, which is used to execute the mobile robot operating method.

[0031] Compared with the prior art, the present invention is suitable for vision-guided operations of mobile robots with eyes outside hands, can expand the camera shooting range, has high visual calibration accuracy and good real-time performance, and can ensure that the recognition and positioning accuracy is within plus or minus 1 mm even if the mobile robot cannot accurately reach a fixed position every time, effectively overcoming the problem of excessively high requirements for the positioning accuracy of mobile robots in the prior art, and improving the operation accuracy and reliability; in addition, the present invention can accurately segment and smooth the edges of materials tightly stacked inside the material box, thereby achieving accurate material grabbing. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of an implementation method of a mobile robot operation according to the present invention.

[0033] Figure 2 This is the plane segmentation map of the material box point cloud.

[0034] Figure 3 This is the material box corner point identification diagram.

[0035] Figure 4 This is a material instance segmentation diagram.

[0036] Figure 5 It is a material identification and grabbing information identification diagram. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described below in conjunction with the drawings in the embodiment. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] The present invention relates to a 3D vision real-time calibration and guidance operation method for a mobile robot and a corresponding system, which is preferably but not limited to being applied to the cutting of large-sized material boxes and the grabbing of internal materials. In a specific operation scenario, the camera is fixed on a bracket, and the height and position of the bracket can be flexibly designed and adjusted according to the actual conditions on site. The height and position of the camera bracket can be flexibly adjusted according to various actual factors such as the on-site environment, the size of the material box, the placement of materials, etc., to ensure that the camera can obtain the best shooting angle and range, thereby improving the applicability and versatility of the present invention.

[0039] The present invention is suitable for an installation mode where the eye is outside the hand, which greatly expands the field of view and can cope with the needs of handling larger material boxes and other operating scenarios.

[0040] The present invention mainly includes two parts: visual calibration of the mobile robot, and visually guided mobile robot operation (preferably but not limited to material box cutting and material grabbing).

[0041] In view of the fact that it is difficult to ensure that a mobile robot arrives at a fixed position accurately every time, a hand-eye calibration process needs to be performed for each operation. In order to obtain accurate external parameters between the camera and the robot after each movement, the present invention adopts the following method: a first calibration plate is installed on the robot's mechanical arm, and a second calibration plate is installed on the robot's base (such as the end flange), and the transformation matrix between the camera coordinate system and the robot coordinate system is calculated through the transformation matrix between the camera and the two calibration plates.

[0042] like Figure 1 As shown, the mobile robot vision calibration method according to an embodiment of the present invention includes:

[0043] When the robot is in working position, perform the following steps:

[0044] Step S1, obtaining the transformation matrix Tb1 from the camera coordinate system to the first calibration plate coordinate system and the robot's current photo posture matrix P1; wherein the first calibration plate is fixedly set on the robot's mechanical arm. The Tb1 acquisition process preferably but not limited to includes: taking an image (such as a texture map, a depth map) of the first calibration plate by a camera, thereby obtaining the position and posture of the first calibration plate, and then calculating the transformation matrix Tb1 from the camera coordinate system to the first calibration plate coordinate system.

[0045] Step S2, based on Tx, Tb1, and P1, obtain the transformation matrix T1 from the camera coordinate system to the robot coordinate system to complete the calibration; wherein Tx is the transformation matrix from the first calibration plate coordinate system to the robot coordinate system.

[0046] The calculation formula of T1 is:

[0047] T1=Tb1·Tx·P1

[0048] The Tx acquisition process preferably includes but is not limited to:

[0049] When the robot is in the initial position, perform the following steps:

[0050] Step S01, obtain the transformation matrix T0 from the camera coordinate system to the robot coordinate system, the transformation matrix Tb0 from the camera coordinate system to the first calibration board coordinate system, and the robot shooting posture matrix P0.

[0051] Step S02: Based on T0, Tb0, and P0, obtain the transformation matrix Tx from the first calibration plate coordinate system to the robot coordinate system.

[0052] Among them, Tx is calculated and determined according to the following formula:

[0053] T0=Tb0·Tx·P0

[0054] In other preferred embodiments, for the hand-eye calibration process, in addition to using the above-mentioned method of calculating the hand-eye calibration matrix based on the robot's photo posture and camera photo data, a calibration algorithm based on machine learning can also be used to improve the accuracy and speed of calibration by learning from a large amount of sample data.

[0055] Step S011, obtaining a coordinate value V1 of the second calibration plate in the camera coordinate system and a coordinate value V2 of the second calibration plate in the robot coordinate system; wherein the second calibration plate is located on the robot base.

[0056] In step S011, obtaining the coordinate value V1 of the second calibration plate in the camera coordinate system preferably includes but is not limited to:

[0057] Step S012: Based on V1 and V2, obtain the transformation matrix T0 from the camera coordinate system to the robot coordinate system.

[0058] In some preferred schemes, in step S011, the second calibration plate has feature points, and the image of the second calibration plate is captured by a camera, and the coordinate values ​​of the feature points on the second calibration plate in the camera coordinate system are identified by the image. The specific process preferably includes but is not limited to: adjusting the position of the second calibration plate to be located under the camera field of view, ensuring that the feature points on the second calibration plate can be clearly photographed by the camera, and the depth of the second calibration plate is not missing, photographing the image of the second calibration plate (such as a texture map, a depth map) by the camera, and obtaining the pixel coordinates of the feature points of the second calibration plate on the image by feature detection, and calculating the coordinate values ​​of the feature points on the second calibration plate in the camera coordinate system according to the camera intrinsic parameters and the depth values ​​of the feature points.

[0059] In some preferred solutions, in step S011, the second calibration plate has characteristic points, and the calibration needle on the robot base is pointed to the characteristic points to obtain the coordinate values ​​of the characteristic points on the second calibration plate in the robot coordinate system. The specific process preferably includes but is not limited to: using the calibration needle installed at the end of the robot base flange to point to the characteristic points on the second calibration plate point by point to obtain the coordinate values ​​of these characteristic points in the robot coordinate system.

[0060] In summary, in the present invention, when the robot is in the initial position, the first calibration (preferably but not limited to hand-eye calibration) is performed to obtain the transformation matrix Tx from the first calibration plate coordinate system to the robot coordinate system. Each subsequent movement of the robot is followed by accurately calculating the transformation matrix T1 from the camera coordinate system to the robot coordinate system through the formula T1=Tb1·Tx·P1, and performing visual calibration to ensure that the mobile robot can accurately operate in the new position.

[0061] The calibration method of the present invention can effectively reduce the complexity of repeated calibration in practical applications, and ensure that an accurate transformation matrix can be obtained after each position update, thereby effectively ensuring that the operation process can be carried out accurately, and the identification and positioning accuracy can reach plus or minus 1 mm.

[0062] A second aspect of an embodiment of the present invention provides a mobile robot vision calibration system, wherein the mobile robot vision calibration system is used to execute the mobile robot vision calibration method.

[0063] like Figure 1 As shown, a third aspect of an embodiment of the present invention provides a mobile robot operation method, and the mobile robot operation method performs visual calibration on the robot based on the mobile robot visual calibration method.

[0064] The visually guided mobile robot operation process described in the present invention is preferably, but not limited to, the material box cutting and material grabbing process. The present invention can successfully overcome the problem of uncertainty in the position of the mobile robot based on real-time hand-eye calibration, precise image processing algorithms, and deep learning models, and achieve high-precision operation guidance for large-sized material boxes and the materials inside them.

[0065] As a preferred embodiment, the operation includes material box cutting, and the material box cutting operation method includes:

[0066] First, the 3D camera is mounted on a bracket to prepare for the cutting of the material box. The 3D camera is used to collect images of the material box through its field of view, including texture maps and depth maps. The texture map provides surface visual information of the material box, and the depth map provides its spatial depth information.

[0067] Then, based on the camera internal parameters, the image processing algorithm is used to generate point cloud data in the bin field of view. The point cloud plane fitting method is used to extract all plane data (such as Figure 2 ). By performing plane detection and fitting on the point cloud data, the system can identify the placement area of ​​the outfeed box and its height range.

[0068] Then, based on the extracted plane data, the corner point information of the box surface (such as Figure 3 ). The effective depth data of the corner points can help determine the cutting coordinates of the material box.

[0069] Through the effective depth data of the corner points of the material box and the calibrated camera to robot external parameter matrix T1, the cutting coordinates of the corner points of the material box are calculated to obtain the cutting point and cutting angle information.

[0070] Finally, the mobile robot completes the material box cutting operation based on the cutting point and cutting angle information.

[0071] As another preferred embodiment, the operation includes grabbing materials inside the material box, and the material grabbing operation method includes:

[0072] After the cutting of the material box is completed, the process proceeds to the step of grabbing the material inside the material box.

[0073] First, a 3D camera is used to capture images of the interior of the material box, including texture images and depth images, and then the image information of each material in the image of the interior of the material box is identified.

[0074] Then, the pre-trained instance segmentation model is used to segment the material instances in the texture map (such as Figure 4), and calculate the central pixel coordinates of each material instance: Based on the point cloud data generated by the depth map, the plane of the material is extracted to obtain plane parameter information, such as the normal vector and the plane equation coefficient. This information helps to filter out noise points and depth missing parts in the material contour and optimize the segmentation effect of the material edge. According to the plane parameter information and camera external parameters of each material instance, it is converted to the robot coordinate system to obtain the size information and corner point information of each material, and then the grasping center of each material in the robot coordinate system is calculated based on the transformation matrix T1 (such as Figure 5 ) to obtain the grasping point and grasping angle information to ensure the accuracy and reliability of material grasping.

[0075] Finally, the mobile robot completes the material grabbing operation based on the grabbing point and grabbing angle information.

[0076] In some preferred embodiments, the material grabbing process also includes a job completion judgment, that is, after the material inside the material box is grabbed, the material box job completion judgment is performed.

[0077] The operation completion judgment process preferably includes, but is not limited to: by identifying the size information of the material box, during each grabbing process, checking whether the depth of all planes fitted inside the material box is close to the bottom of the material box. If the detection result shows that it is close to the bottom, it is determined that the cutting and grabbing operation of the material box has been completed.

[0078] In the process of material grabbing, the present invention can use different types of grabbing tools, such as a clamping claw grabber, etc., in combination with corresponding control algorithms and visual recognition strategies to achieve accurate grabbing of materials.

[0079] For image processing algorithms, in addition to the point cloud plane extraction algorithm and polygon fitting algorithm, other similar image segmentation and feature extraction algorithms, such as semantic segmentation algorithms based on deep learning, can also be used to achieve similar effects.

[0080] The present invention can also use different types of mobile robot platforms, combined with corresponding mechanical structures and control methods, to achieve the same purpose of material box cutting and material grabbing.

[0081] The fourth aspect of the embodiments of the present invention provides a mobile robot operation system, which is used to perform the mobile robot operation method. In some preferred embodiments, a quick-change device is provided at the end of the robot, so that different operation tools can be quickly switched to improve the operation efficiency. In different operation links such as material box cutting and material grabbing, tools such as cutting knives or grabbing suction cups can be quickly replaced by quick-change devices, which reduces the operation preparation time and greatly improves the operation efficiency of the mobile robot, so that it can complete a variety of different tasks in a short time.

[0082] Compared with the prior art, the present invention has at least the following beneficial effects:

[0083] First, the eye-outside-the-hand camera installation method greatly expands the field of view. This significantly increases the size of the bins that can be processed and provides a wider range of camera options. This technical approach effectively solves the problem of the limited camera shooting range caused by the eye-outside-the-hand installation method due to the robot's arm span and other hardware. The eye-outside-the-hand installation method allows the camera to have a larger field of view, so that it can handle larger bins and more complex operating scenarios.

[0084] Secondly, real-time hand-eye calibration technology is used. Since it is difficult for mobile robots to accurately reach a fixed position every time in actual operations, this technology can quickly and accurately calculate the hand-eye calibration matrix based on the robot's photo posture and camera data each time. Hand-eye calibration technology is the key to the accurate operation of mobile robots. Even if the mobile robot cannot accurately reach a fixed position every time, it can ensure that the recognition and positioning accuracy is plus or minus 1mm, thereby ensuring the accuracy of each operation (cutting and grasping operations, etc.), thereby effectively overcoming the problem of excessively high positioning accuracy requirements for mobile robots in the existing technology, greatly improving the accuracy and reliability of the operation, and making each incoming material cutting and grasping operation can be carried out accurately, providing a solid technical foundation for realizing efficient and high-precision mobile robot operations.

[0085] Furthermore, the deep learning instance segmentation model and 3D point cloud data processing technology are used to accurately segment and smooth the edges of the densely stacked materials inside the material box, achieving accurate identification and grasping. The deep learning instance segmentation model can accurately segment densely stacked materials, while 3D point cloud data processing further improves the recognition accuracy and smoothing effect of the material edges, thereby accurately calculating the position, size and other key information of the material, providing a strong guarantee for the robot's accurate grasping.

[0086] The various technical means of the present invention work together to significantly improve the performance and application scope of the mobile robot in 3D vision real-time calibration and guidance operations.

[0087] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0088] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0089] In the embodiments provided by the present invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the above-described method and system embodiments are only illustrative, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0090] The units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0091] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0092] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0093] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A mobile robot visual calibration method, characterized in that: include: When the robot is in working position, perform the following steps: Step S1, obtaining the transformation matrix Tb1 from the camera coordinate system to the first calibration plate coordinate system and the robot's current photographing posture matrix P1; wherein the first calibration plate is located on the robot's mechanical arm; Step S2, based on Tx, Tb1, and P1, obtain the transformation matrix T1 from the camera coordinate system to the robot coordinate system to complete the calibration; wherein Tx is the transformation matrix from the first calibration plate coordinate system to the robot coordinate system.

2. The mobile robot visual calibration method according to claim 1, characterized in that: The Tx acquisition process includes: When the robot is in the initial position, perform the following steps: Step S01, obtaining the transformation matrix T0 from the camera coordinate system to the robot coordinate system, the transformation matrix Tb0 from the camera coordinate system to the first calibration board coordinate system, and the robot shooting posture matrix P0; Step S02: Based on T0, Tb0, and P0, obtain the transformation matrix Tx from the first calibration plate coordinate system to the robot coordinate system.

3. The mobile robot visual calibration method according to claim 2, characterized in that: In the step S01, obtaining the transformation matrix T0 from the camera coordinate system to the robot coordinate system includes: Step S011, obtaining a coordinate value V1 of the second calibration plate in the camera coordinate system and a coordinate value V2 of the second calibration plate in the robot coordinate system; wherein the second calibration plate is located on the robot base; Step S012: Based on V1 and V2, obtain the transformation matrix T0 from the camera coordinate system to the robot coordinate system.

4. The mobile robot visual calibration method according to claim 3, characterized in that: In step S011, the second calibration plate has feature points, and an image of the second calibration plate is captured by a camera, and coordinate values ​​of the feature points on the second calibration plate in the camera coordinate system are identified through the image.

5. The mobile robot visual calibration method according to claim 3 or 4, characterized in that: In step S011, the second calibration plate has a feature point, and the calibration needle on the robot base is pointed to the feature point to obtain the coordinate value of the feature point on the second calibration plate in the robot coordinate system.

6. A mobile robot visual calibration system, characterized in that: The mobile robot vision calibration system is used to execute the mobile robot vision calibration method as described in any one of claims 1 to 5.

7. A mobile robot operation method, characterized in that: The mobile robot operation method performs visual calibration on the robot based on the mobile robot visual calibration method according to any one of claims 1 to 5.

8. The mobile robot operation method according to claim 7, characterized in that: The operation includes material box cutting, and the material box cutting operation method includes: The camera is used to collect the image of the material box and the corner point information of the material box in the image of the material box is identified; Based on the material box corner point information and transformation matrix T1, the cutting point and cutting angle information of the material box corner point in the robot coordinate system are calculated; The mobile robot completes the material box cutting operation based on the cutting point and cutting angle information.

9. The mobile robot operation method according to claim 7 or 8, characterized in that: The operation includes grabbing materials inside the material box, and the material grabbing operation method includes: Use the camera to collect images inside the material box and identify the image information of each material in the image inside the material box; Based on the image information of each material, obtain the size information and corner point information of each material; Based on the size information of each material, the corner point information of each material, and the transformation matrix T1, the grasping point and grasping angle information of each material in the robot coordinate system are calculated; The mobile robot completes the material grabbing operation based on the grabbing point and grabbing angle information.

10. A mobile robot operation system, characterized in that: The mobile robot operation system is used to execute the mobile robot operation method as described in any one of claims 7 to 9.

Citation Information

Cited By

  • Visual calibration method and system for mobile robot

    CN119489446A

  • A method and system for visual calibration of mobile robots

    CN119489446B