A method for controlling the movement of a robotic arm along a set path and a robotic arm control system

Abstract

This invention relates to a method for controlling the movement of a robotic arm along a set path, comprising the following steps: Step 1: Visually recognizing the positions of the actuator and the target object to obtain the position of the actuator's end effector and the target layer position of the target object; Step 2: Determining the entry position and controlling the actuator's end effector to reach the entry position; Step 3: Instructing the robotic arm to move the actuator along the set path within the target object, calculating the compensation depth of the actuator by detecting the force acting on the actuator, and instructing the actuator to perform the compensation depth movement; Step 4: After the actuator's end effector completes its movement, it exits along the original path. This invention addresses the problem of the actuator's inaccurate movement along the set path due to the contact force between the actuator and the target object. Position compensation is performed based on the resistance of the target object, allowing the actuator to move along the set path before performing the compensation position movement, thus enabling the actuator's end effector to more accurately reach the position point on the set path in each step.

Description

Technical Field

[0001] This invention relates to the field of robotic arm control, and more specifically, to a method for controlling the movement of a robotic arm along a set path. Background Technology

[0002] Surgical procedures are increasingly involving the use of surgical robotic systems, especially in ophthalmic surgery. The process by which ophthalmic surgeons manipulate the scalpel using a combination of visual and tactile sensations is similar to the Remote Center of Motion (RCM) mechanism in robotic kinematics. This mechanism minimizes changes in the center of rotation during movement, thus avoiding traction on surrounding tissues. Therefore, RCM is a suitable technique for ophthalmic surgery. The RCM technology in this invention is a software-based RCM technology that ensures the RCM point remains stationary during RCM movement, thereby reducing tearing of surrounding tissues.

[0003] For example, the patent document with announcement number "CN110215339B" discloses an existing method for realizing automated mechanical manipulator path planning. The method uses the visual image obtained by the up-and-down moving microscope system to calculate the three-dimensional coordinates of the scalpel in the eye and the position of the iris, and sets the incision path so that the robotic manipulator carries the scalpel and automatically completes the corneal main incision surgery along the incision path.

[0004] However, the methods described above rely solely on visual recognition as input, resulting in inadequate monitoring of the forces acting on the eye during surgery and an inability to accurately assess the degree of tearing the incision by the scalpel. Furthermore, the contact force between the eye and the actuators is ignored, lacking a corresponding compensation mechanism. Both of these factors lead to a decrease in the accuracy of the robotic arm's movement along the set path, resulting in a deviation between the actual movement path and the pre-set path. Summary of the Invention

[0005] To overcome the problem of needing to improve the accuracy of robotic arm movement along a set path in the prior art, this invention provides a robotic arm movement control method and a robotic arm control system that combine force feedback and visual recognition to improve the accuracy of robotic arm movement.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: to provide a method for controlling the movement of a robotic arm along a set path, comprising a robotic arm, an actuator mounted on the robotic arm, and a force sensor for monitoring the force on the actuator, characterized by comprising the following steps:

[0007] Step 1: Visually identify the positions of the actuator and the target object to obtain the position of the actuator's end and the target layer position of the target object;

[0008] Step 2: Determine the entry positions of the actuator and the target object based on the position of the actuator tip and the target layer position, and control the actuator tip to reach the entry position;

[0009] Step 3: The robotic arm drives the actuator to move within the target object along the set path. During each step of the robotic arm's movement, the compensation depth of the actuator is calculated by detecting the force on the actuator, and the actuator is then made to perform the compensation depth.

[0010] Step 4: After the end of the actuator completes its movement along the set path, it exits the target object along the original path.

[0011] In the above technical solution, the actuator is the entire component body, and the end of the actuator is the endpoint of the actuator. The entry position is the starting point of the set path.

[0012] When the robotic arm moves within the target object, the contact force exerted by the object's material on the actuator is considered. This force causes relative movement between the actuator and the end effector as it moves along the set path, resulting in a positional offset of the actuator. This is usually due to insufficient movement depth. Therefore, a force sensor detects the specific force acting on the actuator to obtain a force feedback value. Based on this force feedback value, the compensation depth of the actuator's end effector is calculated. This compensation depth is used to compensate for the positional offset, ensuring that the actuator's end effector can more accurately reach the set path position point in each step.

[0013] Preferably, in step three, the calculation of the compensation depth is specifically as follows:

[0014]

[0015] In the formula, F is the contact force between the actuator and the target; P is the resistance of the target; Δl y R is the compensation depth, and R is the radius of the target object. When the control F remains constant, the entry depth Δl of the actuator can be compensated based on the change in P. y .

[0016] Preferably, the specific process of step one is as follows:

[0017] S1.1: Instruct the robotic arm to move three times and record the movement amount to obtain the spatial markers (x0,y0,z), (x1,y0,z), and (x0,y2,z) at the end of the actuator, and record the images after each of the three movements.

[0018] S1.2: Process the images to obtain the coordinates of the pixel positions at the ends of the actuators in the three images as (u0, v0), (u1, v2), and (u1, v2), respectively.

[0019] S1.3: After obtaining the pixel position at the end of the actuator, calculate the coordinates (x, y) of the actuator's end position. d ,y d ,z d Specifically:

[0020]

[0021]

[0022] z d =z.

[0023] Since image recognition cannot recognize changes in height in the vertical direction, that is, changes in the Z-axis direction, because the coordinate point of the actuator end position on the Z-axis will not change, and in order to calculate the position of the actuator end accurately, it is best if the actuator end does not move in the Z-axis direction after three movements.

[0024] Preferably, the specific process of S1.2 is as follows:

[0025] The obtained image is preprocessed; specifically, the image is converted into a grayscale image and then binarized according to a threshold.

[0026] Feature recognition is performed on the processed image. A gradient algorithm is used to identify the target layer contour and its center point, and then the contour is filled. The original image is then subtracted from the identified target layer to obtain the remaining image. The edges of the remaining image are extracted, and weights are calculated for the edge pixels. The weight is the number of pixels within a circle with radius r centered on the edge pixel, using the following formula:

[0027] Weight = Number of pixels with intensity of 0 in the vicinity / Total number of pixels in the vicinity

[0028] Sort the images by weight from largest to smallest, and the one with the largest weight is considered the end of the actuator. Based on the recognition results, obtain the target layer position and the pixel position of the actuator end from the three images respectively.

[0029] Preferably, the three movements of the robotic arm are: the robotic arm returning to its initial position, the robotic arm moving in the X-axis direction, and the robotic arm moving in the Y-axis direction.

[0030] Preferably, in step two, the entry position of the target object is determined as follows: based on the direction of the end of the actuator, the direction of the tangent perpendicular to the end of the actuator is determined, and the intersection of the edge contour of the target object and the tangent is determined as the entry position.

[0031] Preferably, in step two, based on the pixel coordinates (u) of the entry position... α ,v α ), calculate the x-direction and y-direction components of its spatial coordinates (x d,y d Based on the current position (x0, y0, z0) of the actuator end effector, the required position difference for the robotic arm to move is calculated. The robotic arm is then controlled to move from its current position to above the target position (x0, y0, z0) while maintaining its z-axis coordinate unchanged. d ,y d ,z0) location.

[0032] Preferably, in step two, when the end of the actuator reaches above the entry position, the position of the end of the actuator is adjusted, specifically as follows:

[0033] The actuator tip moves along the z-axis at a set speed and contacts the target object. When the force on the actuator reaches a preset stop threshold, the actuator stops moving and its tip retracts a set distance along its original path. Since image recognition alone is insufficient to pinpoint the positional difference between the target object and the actuator tip along the z-axis, when the actuator is positioned above the entry point, moving it at the set speed along the z-axis to contact the target object, and the force reaching the preset threshold, indicates that the actuator tip has penetrated a certain depth into the target object. Retracting the actuator tip a set distance at this point ensures that the actuator tip contacts the outer contour of the target object. The relationship between the stop threshold and the set retraction distance can be determined through multiple measurements, ensuring that after the force on the actuator reaches the stop threshold, its retraction distance results in contact with the outer contour of the target object.

[0034] A robotic arm control system includes a robotic arm, a microscopic imaging system, and an electronic device. The electronic device is electrically connected to the microscopic imaging system and the robotic arm. The microscopic imaging system is used to acquire images of the manipulated object in real time. The electronic device includes a memory and a processor. The memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-mentioned robotic arm control method for executing motion trajectories with turning points.

[0035] Compared with existing technologies, the beneficial effects are as follows: This invention addresses the problem of the actuator not moving accurately along the set path due to the contact force between the actuator and the target object. It first accurately locates the relative position between the actuator and the target object through visual recognition, thus pinpointing the entry position, which is the starting point of the set path. Movement begins at this highly accurate starting point. Then, position compensation is performed based on the resistance of the target object, allowing the actuator to move along the set path before performing the compensation movement. This ensures that the actuator's end effector can more accurately reach the position point on the set path in each step. Because the actuator's end effector starts moving from a highly accurate starting point, and each step accurately reaches the position point on the set path, the robotic arm ultimately achieves higher accuracy in driving the actuator along the set path. Attached Figure Description

[0036] Figure 1 This is a flowchart of a method for controlling the movement of a robotic arm along a set path according to the present invention;

[0037] Figure 2 This is a schematic diagram illustrating the entry location confirmation method of the present invention;

[0038] Figure 3 This is a schematic diagram of the structure of a robotic arm control system according to the present invention;

[0039] Figure 4 This is a schematic diagram of the structure of the robotic arm of the present invention. Detailed Implementation

[0040] The accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. To better illustrate this embodiment, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings. The positional relationships described in the drawings are for illustrative purposes only and should not be construed as limiting this patent.

[0041] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "long," and "short" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present patent. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0042] The technical solution of the present invention will be further described in detail below through specific embodiments and in conjunction with the accompanying drawings:

[0043] Example 1

[0044] like Figure 1-2 The illustration shows an embodiment of a method for controlling the movement of a robotic arm along a predetermined path, comprising a robotic arm, an actuator mounted on the robotic arm, and a force sensor for monitoring the force applied to the actuator. The method is characterized by the following steps:

[0045] Step 1: Visually identify the positions of the actuator and the target object to obtain the position of the actuator's end and the target layer position of the target object;

[0046] S1.1: Instruct the robotic arm to make three movements and record the movement amounts to obtain the spatial markers (x0, y0, z), (x1, y0, z), and (x0, y2, z) at the end of the actuator, respectively. The three movements are the robotic arm returning to the initial position, the robotic arm moving in the X-axis direction, and the robotic arm moving in the Y-axis direction, respectively. At the same time, record the images after the three movements.

[0047] S1.2: Process the image and preprocess the obtained image; specifically, convert the image to a grayscale image and then binarize it according to a threshold.

[0048] Feature recognition is performed on the processed image. A gradient algorithm is used to identify the target layer contour and its center point, and then the contour is filled. The original image is then subtracted from the identified target layer to obtain the remaining image. The edges of the remaining image are extracted, and weights are calculated for the edge pixels. The weight is the number of pixels within a circle with radius r centered on the edge pixel, using the following formula:

[0049] Weight = Number of pixels with intensity of 0 in the vicinity / Total number of pixels in the vicinity

[0050] Sort the images by weight from largest to smallest, and the one with the largest weight is considered the end of the execution element. Based on the recognition results, the target layer position and the pixel position of the end of the execution element are obtained from the three images respectively. The coordinates of the pixel positions of the end of the execution element in the three images are (u0,v0), (u1,v2), and (u1,v2).

[0051] S1.3: After obtaining the pixel position at the end of the actuator, calculate the coordinates (x, y) of the actuator's end position. d ,y d ,z d Specifically:

[0052]

[0053]

[0054] z d =z.

[0055] Since image recognition cannot recognize changes in height in the vertical direction, that is, changes in the Z-axis direction, because the coordinate point of the actuator end position on the Z-axis will not change, and in order to calculate the position of the actuator end accurately, it is best if the actuator end does not move in the Z-axis direction after three movements.

[0056] Step 2: Determine the entry positions of the actuator and the target object based on the position of the actuator tip and the target layer position, and control the actuator tip to reach the entry position;

[0057] The method for determining the entry location of the target object is as follows: Figure 2As shown, the direction of the tangent perpendicular to the end of the actuator is determined based on the direction of the end of the actuator, and the intersection of the target object's edge contour and the tangent is determined as the entry position.

[0058] Based on the pixel coordinates of the entry position (u) α ,v α ), calculate the x-direction and y-direction components of its spatial coordinates (x d ,y d Based on the current position (x0, y0, z0) of the actuator end effector, the required position difference for the robotic arm to move is calculated. The robotic arm is then controlled to move the actuator from its current position to above the target position, while keeping the z-axis coordinate unchanged. This means the actuator end effector will be positioned at (x0, y0, z0) after the movement. d ,y d ,z0).

[0059] Because image recognition alone is insufficient to pinpoint the positional difference between the target object and the actuator tip along the Z-axis, the actuator tip can only reach above the entry position, failing to accurately reach it. Therefore, adjustment of the tip's position is necessary. In this implementation, the specific adjustment method is as follows: the actuator tip moves along the Z-axis at a set speed and contacts the target object. When the force on the actuator reaches a preset stop threshold, the actuator stops moving and the actuator tip retracts a set distance along its original path. When the force on the actuator reaches the preset threshold, it indicates that the actuator tip has entered the target object to a certain depth. Retracting the actuator tip a set distance along its original path at this point will allow the actuator tip to contact the outer contour of the target object. The relationship between the stop threshold and the set retraction distance can be obtained through multiple measurements, ensuring that after the force on the actuator reaches the stop threshold, its retraction distance just brings it into contact with the outer contour of the target object.

[0060] Step 3: The robotic arm drives the actuator to move within the target object along a set path. During each step of the robotic arm's movement, the compensation depth of the actuator is calculated by detecting the force applied to it, and the actuator is then executed to achieve the compensation depth. The calculation of the compensation depth is specifically as follows:

[0061]

[0062] In the formula, F is the contact force between the actuator and the target; P is the resistance of the target; Δl y R is the compensation depth, and R is the radius of the target object. When the control F remains constant, the entry depth Δl of the actuator can be compensated based on the change in P. y .

[0063] Step 4: After the end of the actuator completes its movement along the set path, it exits the target object along the original path.

[0064] The working principle of this embodiment is as follows: When the robotic arm moves within the target object, the material of the target object itself will generate a certain resistance to the actuator. This resistance will cause the end of the actuator to deviate in position when moving along the set path. Generally, the depth of movement is insufficient. Therefore, the force sensor detects the specific force on the actuator and obtains a force feedback value. The compensation depth of the actuator end is calculated based on the force feedback value. The compensation depth is used to make up for the position deviation, so that the end of the actuator can reach the position point of the set path more accurately in each step.

[0065] The beneficial effects of this embodiment are as follows: This invention addresses the problem of the actuator not moving accurately along the set path due to the contact force between the actuator and the target object. First, it accurately locates the relative position between the actuator and the target object through visual recognition, thus pinpointing the entry position, which is the starting point of the set path. Movement begins at this highly accurate starting point. Then, position compensation is performed based on the resistance of the target object, allowing the actuator to move along the set path before performing the compensation movement. This ensures that the actuator's end effector can more accurately reach the position point on the set path in each step. Because the actuator's end effector starts moving from a highly accurate starting point, and each step accurately reaches the position point on the set path, the robotic arm ultimately achieves higher accuracy in driving the actuator along the set path.

[0066] Example 2

[0067] like Figure 1-2 This illustration shows another embodiment of a robotic arm motion control method along a set path, applied to cataract surgery. The target object is the eyeball, and the target layer should be the cornea. However, due to the transparency of the cornea, the target layer is set as the iris, and the cornea is identified by recognizing the outline of the iris. F is the contact force between the actuator and the cornea; P is the intraocular pressure; R is the radius of the eyeball; the stop threshold is 60 mN, and the set backward distance is 1.5 mm. Because the eyeball cannot be completely fixed, it will shift slightly along the direction of the contact force. Therefore, if the actuator merely moves along the set path, the movement will be insufficient and inaccurate. A compensation depth is added to compensate for the eyeball's shift, allowing the actuator to accurately reach the nodes of the set path.

[0068] The path is divided into 5 segments, as follows:

[0069] Section A: The scalpel advances 2mm in the y-axis direction.

[0070] R1 segment: The scalpel moves in an RCM motion around the tip, rotating upwards by 22.5 degrees.

[0071] Section B: The scalpel advances 1.75mm towards the tip.

[0072] R2 segment: The scalpel moves in an RCM motion around the tip, rotating downwards by 22.5 degrees.

[0073] Segment C: The scalpel advances 4mm in the y-axis direction.

[0074] Example 3

[0075] A robotic arm control system, such as Figure 3 As shown, the device includes a robotic arm 1, a microscopic imaging system 2, and an electronic device. The electronic device is electrically connected to the microscopic imaging system 2 and the robotic arm 1. The microscopic imaging system 2 is used to acquire images of the manipulated object in real time. The electronic device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute computer programs to implement a robotic arm motion control method along a set path according to embodiment 1.

[0076] In this embodiment, robotic arm 1, as Figure 4 As shown, the joint includes a first joint, a second joint, and an actuating joint. The first joint includes a first linear motor 3, a second linear motor 4, and a first mounting plate 5 rotatably connected to the output ends of the first linear motor 3 and the second linear motor 4, respectively. The second joint includes a third linear motor 6 and a fourth linear motor 7, both mounted on the first mounting plate 5, and a second mounting plate 8 rotatably connected to the output ends of the third linear motor 6 and the fourth linear motor 7, respectively. The third joint includes a fifth linear motor 9 mounted on the second mounting plate 8, an actuating element 10 mounted on the output of the fifth linear motor 9, and a force sensor 11 mounted on the actuating element 10.

[0077] Example 4

[0078] A computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the above-described method for controlling the movement of a robotic arm along a set path.

[0079] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for controlling the movement of a robotic arm along a predetermined path, comprising a robotic arm, an actuator mounted on the robotic arm, and a force sensor for monitoring the force applied to the actuator, characterized in that, Includes the following steps: Step 1: Visually identify the positions of the actuator and the target object to obtain the position of the actuator's end and the target layer position of the target object; Step 2: Determine the entry positions of the actuator and the target object based on the position of the actuator tip and the target layer position, and control the actuator tip to reach the entry position; Step 3: The robotic arm drives the actuator to move within the target object along a set path. During each step of the robotic arm's movement, the compensation depth of the actuator is calculated by detecting the force applied to it, and the actuator is then executed to achieve the compensation depth. The calculation of the compensation depth is as follows: In the formula, It is the contact force between the actuating element and the target object; It is the resistance of the target object; It is the depth of compensation. It is the radius of the target object; when controlling When unchanged, according to The change can compensate for the entry depth of the actuator. ; Step 4: After the end of the actuator completes its movement along the set path, it exits the target object along the original path.

2. The method for controlling the movement of a robotic arm along a set path according to claim 1, characterized in that, The specific process for step one is as follows: S1.1: Instruct the robotic arm to perform three movements and record the movement amounts to obtain the spatial markings at the end of the actuator. , , Simultaneously, the images after each of the three movements were recorded; S1.2: Process the images to obtain the coordinates of the pixel positions at the ends of the actuators in the three images. , , S1.3: After obtaining the pixel position at the end of the actuator, calculate the coordinates of the actuator's end position. Specifically: 。 3. The method for controlling the movement of a robotic arm along a set path according to claim 2, characterized in that, The specific process of S1.2 is as follows: The obtained image is preprocessed; specifically, the image is converted into a grayscale image and then binarized according to a threshold. Feature recognition is performed on the processed image. A gradient algorithm is used to identify the target layer contour and its center point, and then the contour is filled. The original image is then subtracted from the identified target layer to obtain the remaining image. The edges of the remaining image are extracted, and weights are calculated for the edge pixels. The weight is the number of pixels within a circle with radius r centered on the edge pixel, using the following formula: Weight = Number of pixels with intensity of 0 in the vicinity / Total number of pixels in the vicinity Sort the images by weight from largest to smallest, and the one with the largest weight is considered the end of the actuator. Based on the recognition results, obtain the target layer position and the pixel position of the actuator end from the three images respectively.

4. The method for controlling the movement of a robotic arm along a set path according to claim 3, characterized in that, The three movements of the robotic arm are: returning to the initial position, moving the robotic arm in the X-axis direction, and moving the robotic arm in the Y-axis direction.

5. The method for controlling the movement of a robotic arm along a set path according to claim 3, characterized in that, In step two, the entry position of the target object is determined as follows: based on the direction of the end of the actuator, the direction of the tangent perpendicular to the end of the actuator is determined, and the intersection of the target layer contour and the tangent is determined as the entry position.

6. The method for controlling the movement of a robotic arm along a set path according to claim 5, characterized in that, In step two, based on the pixel coordinates of the entry position Calculate the x- and y-direction components of its spatial coordinates. Based on the current position of the end of the actuator. The required positional difference for the robotic arm to move is calculated, and the robotic arm is controlled to move from its current position to above the target position, i.e., the coordinate point, while keeping its z-axis coordinate unchanged. Place.

7. The method for controlling the movement of a robotic arm along a set path according to claim 6, characterized in that, In step two, the end of the actuator reaches above the entry position, and the position of the end of the actuator is adjusted, specifically as follows: The end of the actuator moves in the z-axis direction at a set speed and contacts the target object. When the force on the actuator reaches a preset stop threshold, the actuator stops moving and the end of the actuator moves back a set distance along the original path.

8. A robotic arm control system, characterized in that, The device includes a robotic arm, a microscopic imaging system, and an electronic device, wherein the electronic device is electrically connected to the microscopic imaging system and the robotic arm; the microscopic imaging system is used to acquire images of the manipulated object in real time; the electronic device includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the robotic arm motion control method along a set path as described in any one of claims 1-7.

9. A computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the method for controlling the movement of a robotic arm along a set path as described in any one of claims 1-7.

Images