Manipulator operation positioning method

Abstract

The application discloses a mechanical hand operation positioning method, which comprises an initialization step. In the initialization scene, the mechanical hand obtains the relationship between the actual distance B between positioning elements on a positioning mark in X and Y directions and the pixel distance A under different operation distances Z, that is, obtains the ratio C of B and A, or obtains the function relationship f(Z)=C of C and Z. When a task is performed, the distance between the mechanical hand and a target object is the operation distance Z, which is directly measured by a distance measuring device. The displacement between the mechanical hand and the target object in the XY direction is calculated through the relationship between C and Z. The mechanical hand and the target object are repeatedly translated in the XY direction to meet the accuracy requirement range of the mechanical hand performing the task. The positioning method with multiple translations realizes accurate and efficient positioning, can meet the application scene of the mechanical hand with high accuracy requirement, and ensures that the mechanical hand accurately and reliably performs related tasks.

Description

[Technical Field]

[0001] This invention relates to the field of robotics, and more particularly to a method for positioning and manipulating a robotic arm. [Background Technology]

[0002] There are two main types of robots: one type is installed in a fixed position and operates on a fixed target according to a fixed process, with the operation steps determined by a program, such as ordinary industrial robots; the other type is manually controlled robots, such as drones and robot dogs, which can only operate on the corresponding target object through real-time human control and intervention.

[0003] In robot manipulation, target object localization is crucial. Accurate and efficient target object localization is a prerequisite for robot task execution. Current technologies typically employ binocular vision or LiDAR for localization. Both methods rely on angular positioning calculations, which suffer from amplified and uncorrectable errors, with accuracy exceeding 5cm. Therefore, robots using these localization methods are unsuitable for applications requiring high precision.

[0004] This invention was developed to address the shortcomings of existing technologies. [Summary of the Invention]

[0005] This invention overcomes the shortcomings of existing technologies and provides a robotic arm operation and positioning method, including an initialization step. During initialization, the robotic arm, in an initialization scene, obtains the relationship between the actual distance B and the pixel distance A of the positioning elements on the positioning marker in the X and Y directions at a certain shooting focal length and different operation distances Z. This relationship is obtained as a set of ratios C of B to A, or a functional relationship f(Z) = C between C and Z. When performing a specific task, the distance between the robotic arm and the target object is the operation distance Z, which is directly measured by a ranging device. The displacement between the robotic arm and the target object in the X and Y directions is calculated based on the relationship between the C value obtained during initialization and the operation distance Z, as well as the principle of visual calibration. Finally, the robotic arm and the target object are repeatedly translated and corrected in the X and Y directions to meet the accuracy requirements of the robotic arm in performing the task. By adopting a positioning method of multiple translations, accurate and efficient positioning is achieved, which can meet the application scenarios of robotic arms with high precision requirements and ensure that the robotic arm performs related tasks accurately and reliably.

[0006] To achieve the above objectives, the present invention provides a robotic arm operation and positioning method, wherein the robotic arm is equipped with a visual imaging device and a ranging device, comprising the following steps:

[0007] S1. Set up an initial scene containing a positioning identifier. The positioning identifier has several positioning elements, and the actual distance between two adjacent positioning elements in the X and Y directions is a constant B. The robot arm is configured to take pictures of the initial scene at different operating distances Z.

[0008] S2. The visual imaging device takes pictures of the acquisition area containing the positioning marker to obtain an image containing the positioning marker;

[0009] S3. Identify the positioning markers in the image to obtain the pixel distance A between the positioning elements in the X and Y directions in the image;

[0010] S4. Under the operating distance Z, determine the ratio of the actual distance B to the pixel distance A as C;

[0011] S5. Repeat S1-S4 to obtain the set of ratios C of actual distance B to pixel distance A under different operating distances Z, or obtain the functional relationship f(Z) = C between C and Z under different operating distances Z.

[0012] As described above, the robotic arm positioning method further includes the following step after step S4 or S5:

[0013] S6. The robotic arm accepts the work task;

[0014] S7. The robotic arm moves to the working position and determines the data collection range;

[0015] S8. Send a detection signal to the acquisition range, and obtain the operating distance Z between the robot and the acquisition range after feedback;

[0016] S9. The visual imaging device takes pictures of the acquisition area containing the positioning marker to obtain an image containing the positioning marker and the target object, and identifies the positioning marker in the image to obtain the pixel coordinates (X) of its positioning element. 0 Y 0 ), and to identify the target object in the image and obtain its pixel coordinates (X). 1 Y 1 Based on the pixel coordinates (X) of the positioning elements of the positioning markers in the image, 0 Y 0 ) and the pixel coordinates (X) of the target object 1 Y 1 The relative pixel coordinates (X, X) of the target object between the positioning element of the positioning marker in the image and the target object are calculated. 2 Y 2 Then, based on the ratio C corresponding to the corresponding operation distance Z and the relative target pixel coordinates (X... 2 Y 2 The physical displacement S of the robot relative to the target object in the XY direction is calculated.

[0017] Alternatively, a visual imaging device can take a picture of the area containing the target object to obtain an image containing the target object and the center coordinates (X) of the image. 3 Y 3 ), and to identify the target object in the image and obtain its pixel coordinates (X). 1 Y 1 According to the center coordinates (X) of the image 3 Y 3 ) and the pixel coordinates (X) of the target object 1 Y 1 The relative image center pixel coordinates (X) between the image center and the target object are calculated. 4 Y 4 Then, based on the ratio C corresponding to the corresponding operation distance Z and the relative image center pixel coordinates (X... 4 Y 4 The physical displacement S of the robot relative to the target object in the XY direction is calculated.

[0018] S10. Based on the physical displacement S, the robot arm moves towards the target object accordingly;

[0019] S11. Repeat steps S8 to S10 until the relative displacement between the robot and the target is within the preset accuracy range of the robot's operation.

[0020] S12. Complete the robot arm's positioning and execute the corresponding work tasks.

[0021] In the robotic arm positioning method described above, the visual imaging device is a monocular camera, and the ranging device is a laser rangefinder or a radar ranging device.

[0022] As described above, the robot operation positioning method includes steps S1-S5. In step S5, the set of ratios C is stored in the robot system in the form of a data table for subsequent operation calls, or the corresponding C values ​​under different Z values ​​are obtained according to the functional relationship between C and Z f(Z)=C.

[0023] As described above, in a robotic arm operation and positioning method, the robotic arm is equipped with a projection device for projecting positioning marks onto an initialization scene. In step S1, the projection device projects positioning marks onto the initialization scene at different operation distances Z.

[0024] As described above, in the robotic arm operation positioning method, in step S1, a positioning marker is manually set in the initialization scene.

[0025] In the robotic arm operation and positioning method described above, in steps S1-S4, the robotic arm moves relative to the positioning marker and automatically obtains a set of ratios C of the actual distance B to the pixel distance A under different operation distances Z.

[0026] In the robotic arm positioning method described above, in step 8, if a detection signal is sent to the acquisition range and no feedback is received, the operating distance Z between the robotic arm and the acquisition range is obtained; then the operating distance is assigned as Z. n According to set C at operational distance Z n Below, we obtain the corresponding C. n Then, the physical displacement S of the robot relative to the target object is calculated. The robot moves towards the target object according to the physical displacement S, and then sends a detection signal to the acquisition range. After feedback, the operating distance Z between the robot and the acquisition range is obtained.

[0027] As described above, in the robotic arm operation and positioning method, if the robotic arm moves beyond the target object in the XY direction in step S10, the movement value in the XY direction is calculated based on the C value corresponding to the current Z and the robotic arm moves in the opposite direction. If the robotic arm moves beyond the target object in the XY direction in the opposite direction, the movement value in the XY direction is calculated based on the C value corresponding to the current Z and the robotic arm moves in the forward direction.

[0028] Alternatively, in step S10, if the robot moves beyond the target object in the XY direction, it moves in the opposite direction with a value less than the previous XY direction movement value. If the robot moves in the opposite direction in the XY direction beyond the target object, it continues to move in the forward direction with a value less than the previous XY direction movement value.

[0029] Compared with existing technologies, the robotic arm operation and positioning method of the present invention has the following advantages:

[0030] 1. The distance between the robot and the target in the Z direction is directly measured by a ranging device, while the displacement between the robot and the target in the XY direction is calculated based on the relationship between the C value obtained from initialization and the operating distance Z, as well as the principle of visual calibration. Furthermore, the robot undergoes multiple translation corrections in the XY direction to meet the accuracy requirements of the robot's task execution, thereby achieving accurate and high-precision positioning of the target.

[0031] 2. This invention can obtain the set of ratios C of actual distance B to pixel distance A under different operating distances Z through the initialization step, or obtain the functional relationship f(Z)=C between C and Z under different operating distances Z, which lays the foundation for the robot to achieve accurate positioning through multiple translation calculations.

[0032] 3. The method of the present invention calculates the physical displacement S of the robot relative to the target object, and then translates the robot relative to the target object in the XY direction to achieve accurate positioning. This method avoids the defects of angle positioning, such as the inability to correct and amplify errors. Therefore, the method of the present invention can be applied to positioning the target object by the robot in scenarios with high precision requirements.

[0033] 4. By using visual calibration, the physical displacement S of the robot relative to the target object is calculated by using the positioning mark as a reference at different operating distances Z. Therefore, the robot can obtain multiple translation corrections in the XY direction, ensuring accurate positioning of the target object. [Image Description]

[0034] Figure 1 This is a schematic diagram of a positioning identifier containing positioning elements according to the present invention.

[0035] Figure 2 This is a schematic diagram of another positioning mark containing positioning elements in this invention.

[0036] Figure 3 The logic diagram for initializing this invention.

[0037] Figure 4 The working logic diagram for positioning in this invention. [Detailed Implementation]

[0038] The technical solutions in the embodiments of the present invention will now be clearly and completely described in conjunction with the accompanying drawings.

[0039] This invention discloses a robotic arm operation and positioning method. The robotic arm is configured to move within a working environment. The robotic arm is equipped with a visual imaging device and a ranging device. The visual imaging device is a monocular camera, such as the Hikvision monocular camera, model DS-2CD6B45DWD-U400w. The ranging device is a laser rangefinder or radar ranging device, such as the Tianjin Yike laser range sensor, model OSM40-KL800CB6 / 485. During initialization, as... Figure 3 As shown, the steps include:

[0040] S1. Set up an initialization scene containing positioning markers. Each positioning marker has several positioning elements, and the actual distance between adjacent positioning elements in the X and Y directions is a constant B. The robotic arm is configured to capture images of the initialization scene at different operating distances Z. In this step, a projection device, such as the projection device disclosed in Chinese Patent 202211204537.0, is set on the robotic arm. The projection device automatically projects the positioning markers onto the initialization scene at different operating distances Z, and its parameters remain unchanged during this process. Alternatively, in this step, the positioning markers are manually set in the initialization scene. For the graphic of the positioning markers, such as... Figure 1 or Figure 2 As shown.

[0041] S2. Adjust the shooting focal length of the visual shooting device and keep it unchanged. The visual shooting device takes pictures of the collection range containing the positioning marks that are automatically placed or manually set, and obtains an image containing the positioning marks.

[0042] S3. Identify the positioning markers in the image to obtain the pixel distance A between the positioning elements in the X and Y directions.

[0043] S4. Under the operating distance Z, determine the ratio of the actual distance B to the pixel distance A as C.

[0044] S5. Repeat S1-S4 to obtain a set of ratios C of the actual distance B to the pixel distance A under different operating distances Z, or obtain the functional relationship f(Z) = C between C and Z under different operating distances Z. In this step, the set of ratios C is stored in the robot system in the form of a data table for subsequent operation, or the corresponding C value under different Z values ​​is obtained according to the functional relationship f(Z) = C between C and Z. Based on this, the robot initialization is completed.

[0045] In steps S1-S4 above, the robotic arm moves relative to the positioning marker and automatically obtains the set of ratios C between the actual distance B and the pixel distance A at different operating distances Z.

[0046] like Figure 4 As shown, the robotic arm operation positioning method of the present invention includes the following steps when the robotic arm performs operation positioning after completing the above initialization steps:

[0047] S6. The robotic arm accepts the work task.

[0048] S7. The robotic arm moves to the working position and determines the collection range.

[0049] S8. Send a detection signal to the acquisition range, and obtain the operating distance Z between the robot arm and the acquisition range after feedback; in this step, if a detection signal is sent to the acquisition range but no operating distance Z is obtained after feedback, then assign the operating distance as Z. n According to set C at operational distance Z n Below, we obtain the corresponding C. n Based on visual calibration, the physical displacement S of the robot relative to the target object in the XY direction is calculated. The robot moves towards the target object accordingly based on the physical displacement S, and then sends a detection signal to the acquisition range. After feedback, the operating distance Z between the robot and the acquisition range is obtained.

[0050] S9. The vision imaging device takes pictures of the acquisition range containing the positioning marker to obtain an image containing the positioning marker and the target object. The positioning marker is relatively fixed on the robotic arm. When the vision imaging device captures the target object within the acquisition range, it can also capture the positioning marker at the same time. The device identifies the positioning marker in the image to obtain the pixel coordinates (X) of its positioning element. 0 Y 0 ), and to identify the target object in the image and obtain its pixel coordinates (X). 1 Y 1 Based on the pixel coordinates (X) of the positioning elements of the positioning markers in the image, 0 Y 0 ) and the pixel coordinates (X) of the target object 1 Y 1 The relative pixel coordinates (X, X) of the target object between the positioning element of the positioning marker in the image and the target object are calculated. 2 Y 2 Then, based on the ratio C corresponding to the corresponding operation distance Z and the relative target pixel coordinates (X... 2 Y 2 The physical displacement S of the robotic arm relative to the target object in the XY direction is calculated; or the visual imaging device takes pictures of the acquisition range containing the target object to obtain an image containing the target object and the center coordinates (X, Y, X) of the image. 3 Y 3 ), and to identify the target object in the image and obtain its pixel coordinates (X). 1 Y 1 According to the center coordinates (X) of the image 3 Y 3 ) and the pixel coordinates (X) of the target object 1 Y 1 The relative image center pixel coordinates (X) between the image center and the target object are calculated. 4 Y 4 Then, based on the ratio C corresponding to the corresponding operation distance Z and the relative image center pixel coordinates (X... 4Y 4 The physical displacement S of the robot relative to the target object in the XY direction is calculated.

[0051] S10. Based on the physical displacement S, the robot arm translates towards the target object along the XY direction. In this step, if the robot arm moves beyond the target object in the XY direction, the movement value in the XY direction is calculated based on the current C value corresponding to Z, and it moves in the opposite direction. If the robot arm moves beyond the target object in the opposite direction in the XY direction, the movement value in the XY direction is calculated based on the current C value corresponding to Z, and it moves in the forward direction. Alternatively, in step S10, if the robot arm moves beyond the target object in the XY direction, a movement value less than the previous movement value in the XY direction is taken, and it moves in the opposite direction. If the robot arm moves beyond the target object in the opposite direction in the XY direction, a movement value less than the previous movement value in the XY direction is taken, and it moves in the forward direction.

[0052] S11. Repeat steps S8 to S10, with the robot arm translating relative to the target object multiple times in the XY direction until the relative displacement between the robot arm and the target object is within the preset accuracy range of the robot arm operation.

[0053] S12. Complete the robot arm's positioning and execute the corresponding work tasks.

[0054] Therefore, in the method of the present invention, when the robot arm positions the target object, the operating distance Z is first directly measured. Using the relationship between the operating distance Z and C, the physical displacement S of the robot arm relative to the target object in the XY direction is calculated according to visual calibration. The robot arm then translates relative to the target object in the XY direction to achieve accurate positioning. This method avoids the defects of angle positioning, such as the inability to correct and amplify errors. Therefore, the method of the present invention is applicable to positioning the target object by the robot arm in scenarios with high precision requirements.

[0055] It is important to note that, for the method of this invention, to ensure normal and reliable operation, the focal length of the visual imaging device on the robotic arm is set after initialization. During subsequent task capture, the same focal length as during initialization is used. If the focal length of the visual imaging device is adjusted, the robotic arm needs to be re-initialized. Furthermore, during the robotic arm's task execution, if its projection device automatically projects positioning markers onto the acquisition range at different operating distances Z, the projection device parameters are also kept consistent with the parameters adjusted and set during initialization.

Claims

1. A robot operation positioning method characterized by The mechanical arm is provided with a visual shooting device and a ranging device, and comprises the following steps: S1, setting an initialization scene containing a positioning mark, the positioning mark having a plurality of positioning elements, and the actual distance between two adjacent positioning elements in the X and Y directions being a constant B, the mechanical arm being configured to shoot the initialization scene at different operating distances Z; S2, the visual shooting device shooting the collection range containing the positioning mark to obtain an image containing the positioning mark; S3, identifying the positioning mark in the image to obtain the pixel distance A between the positioning elements in the X and Y directions in the image; S4, determining the ratio C of the actual distance B and the pixel distance A at the operating distance Z; S5, repeating S1-S4 to obtain a set of ratios C of the actual distance B and the pixel distance A at different operating distances Z, or to obtain a function relationship f(Z) = C between C and Z at different operating distances Z; After step S4 or S5, the following steps are further included: S6, the mechanical arm receiving a work task; S7, the mechanical arm going to a work position and determining a collection range; S8, sending a detection signal to the collection range, and obtaining the operating distance Z between the mechanical arm and the collection range after feedback; S9. The visual imaging device takes pictures of the acquisition area containing the positioning marker to obtain an image containing the positioning marker and the target object, and identifies the positioning marker in the image to obtain the pixel coordinates (X) of its positioning element. 0 ,Y 0 ), and to identify the target object in the image and obtain its pixel coordinates (X). 1 ,Y 1 Based on the pixel coordinates (X) of the positioning elements of the positioning markers in the image, 0 ,Y 0 ) and the pixel coordinates (X) of the target object 1 ,Y 1 The relative pixel coordinates (X, X) of the target object between the positioning element of the positioning marker in the image and the target object are calculated. 2 ,Y 2 Then, based on the ratio C corresponding to the corresponding operation distance Z and the relative target pixel coordinates (X... 2 ,Y 2 The physical displacement S of the robot relative to the target object in the XY direction is calculated. Or the visual camera takes a picture of the collection range containing the target object to obtain an image containing the target object and the center coordinates (X 3 ,Y 3 ) of the image, and identifies the target object in the image to obtain the pixel coordinates (X 1 ,Y 1 ) of the target object, according to the center coordinates (X 3 ,Y 3 ) of the image and the pixel coordinates (X 1 ,Y 1 ) of the target object, the relative image center pixel coordinates (X 4 ,Y 4 ) between the center of the image and the target object are calculated, and according to the corresponding ratio C of the operation distance Z and the relative image center pixel coordinates (X 4 ,Y 4 ), the physical displacement S of the manipulator relative to the target object in the XY direction is calculated; S10, the mechanical arm moving to a target object according to the physical displacement S; S11, repeating step S8 to step S10 until the relative displacement between the mechanical arm and the target object is within a preset accuracy range of the mechanical arm operation; S12, completing the mechanical arm positioning and performing a corresponding work task.

2. The method of claim 1, wherein The visual shooting device is a monocular camera, and the ranging device is a laser range finder or a radar ranging device.

3. The method of claim 1, wherein The initialization setting of the mechanical arm comprises steps S1-S5, and the set of ratios C in step S5 is stored in the mechanical arm system in the form of a data table for subsequent operation calling, or the corresponding C values at different Z values are obtained according to the function relationship f(Z) = C between C and Z.

4. The method of claim 1, wherein The mechanical arm is provided with a projection device for projecting the positioning mark to the initialization scene, and the projection device projects the positioning mark to the initialization scene at different operating distances Z in step S1.

5. The method of claim 1, wherein In step S1, the positioning mark is manually set in the initialization scene.

6. The method of claim 1, wherein In steps S1-S4, the mechanical arm moves relative to the positioning mark to automatically obtain the set of ratios C of the actual distance B and the pixel distance A at different operating distances Z.

7. The method of claim 1, wherein In step 8, if a detection signal is sent to the collection range and no feedback is received, the operation distance Z between the manipulator and the collection range is obtained, and the operation distance is assigned as Z n , according to the set C at the operation distance Z n , the corresponding C n is obtained, and the physical displacement S of the manipulator relative to the target object is calculated, and the manipulator moves to the target object according to the physical displacement S, and a detection signal is sent to the collection range, and the operation distance Z between the manipulator and the collection range is obtained after feedback.

8. The method of claim 1, wherein In step S10, if the mechanical arm moves beyond the target object in the XY direction, the mechanical arm is moved reversely by taking a smaller value than the last XY direction movement value, and if the mechanical arm moves beyond the target object in the XY direction reversely, the mechanical arm is moved forwardly by continuing to take a smaller value than the last XY direction movement value. Or in step S10, if the mechanical arm moves beyond the target object in the XY direction, the mechanical arm is moved reversely by taking a smaller value than the last XY direction movement value, and if the mechanical arm moves beyond the target object in the XY direction reversely, the mechanical arm is moved forwardly by continuing to take a smaller value than the last XY direction movement value.

Images