A robot arm trajectory planning method, device, equipment and medium

Abstract

The present application relates to the field of robotic arm control technology, and provides a robotic arm trajectory planning method, device, equipment, and medium, including: fitting the position and posture of the obstacle bounding box in the robot coordinate system; fitting the robot bounding box; using the center point of the tool at the end of the manipulator arm as the starting point and the center point of the tool at the end of the conveying arm as the ending point, and combining the length of the trajectory to be fitted and the obstacle avoidance space to generate a motion trajectory curve of the manipulator arm; extracting trajectory points from the motion trajectory curve, planning the posture of the manipulator arm at each trajectory point, verifying whether the trajectory points of the manipulator arm are reachable, and if not reachable, adjusting the posture of the manipulator arm at the corresponding trajectory point; if reachable, the trajectory points of the manipulator arm are matched by the current endoscope conveying length, and the trajectory points of the manipulator arm are described by the position and posture of the manipulator arm. The present application achieves precise obstacle avoidance in a complex and changing surgical environment through environmental perception, kinematic modeling, and trajectory generation, thereby improving safety and reliability.

Description

Technical Field

[0001] The present application relates to the field of robotic arm control technology, and in particular to a robotic arm trajectory planning method, device, equipment and medium. Background Art

[0002] Natural cavities such as the digestive tract and respiratory tract are common sites of human disease. Because lesions are located within the body's natural cavities, flexible endoscopy is necessary for examination.

[0003] Traditional flexible endoscopic examinations or surgeries require two hands to coordinate movements, such as holding the scope, operating knobs, and manually maneuvering the scope. Some examinations or surgeries require image guidance from radiographs, requiring medical personnel to wear heavy lead-based protective clothing and manually operate the flexible endoscope for extended periods. This significantly impacts the quality of diagnosis and treatment, their physical strength, and their health. This is particularly restrictive for experienced, older, or female medical personnel, who may even be forced to abandon endoscopic work.

[0004] With the development of robot-assisted technology, doctors can adjust the delivery length and posture of flexible endoscopes by adjusting the handle switch and button, which greatly reduces the doctor's physical strength and manual operation fatigue, reduces the operation requirements of the operation, reduces the radiation to medical staff, and at the same time improves the interaction between medical staff and images.

[0005] However, the related flexible endoscope control robot system still has significant flaws. During the endoscope delivery process, the robot needs to plan the trajectory of the manipulator arm in advance based on the patient's natural cavity position to prevent damage to the endoscope or actuator due to improper operation. In actual surgery, the environment is complex and changing. The established trajectory planning method cannot ensure the robot's accurate obstacle avoidance and is clearly insufficient in terms of reliability in ensuring safe movement. Summary of the Invention

[0006] The present application provides a robot arm trajectory planning method, device, equipment and medium to solve the problem that the established trajectory planning method is difficult to ensure the robot's accurate obstacle avoidance and is insufficiently reliable in ensuring safe movement.

[0007] To solve the above technical problems, the technical solutions provided by this application are as follows:

[0008] In a first aspect, the present application provides a robot arm trajectory planning method, which is applied to a flexible endoscope manipulation robot system, wherein the flexible endoscope manipulation robot system includes a delivery arm and a manipulation arm, and the method includes:

[0009] Obtain point cloud data of obstacles in the surgical environment and fit the position and posture of the obstacle bounding box in the robot coordinate system;

[0010] Obtain the position of the tool center point at the end of the manipulator arm and the tool center point at the end of the conveyor arm in the robot coordinate system, and fit the robot bounding box based on the robot structural parameters;

[0011] The length of the trajectory to be fitted is obtained based on the current endoscope delivery length, the obstacle avoidance space is obtained based on the position of the obstacle bounding box in the robot coordinate system, and the motion trajectory curve of the manipulator arm is generated by combining the length of the trajectory to be fitted and the obstacle avoidance space with the center point of the tool at the end of the manipulator arm as the starting point and the center point of the tool at the end of the delivery arm as the ending point;

[0012] Extracting trajectory points from the motion trajectory curve and planning the posture of the manipulator at each of the trajectory points;

[0013] Verify whether the trajectory point of the operating arm is reachable. If the trajectory point of the operating arm is not reachable, adjust the posture of the operating arm corresponding to the trajectory point; if the trajectory point of the operating arm is reachable, the trajectory point of the operating arm is matched by the current endoscope delivery length, and the trajectory point of the operating arm is described by the posture of the operating arm.

[0014] In some possible implementations, acquiring point cloud data of obstacles in the surgical environment and fitting the position and posture of the obstacle bounding box in the robot coordinate system includes:

[0015] Collecting image information of obstacles in the surgical environment, and obtaining point cloud data of the obstacles in the surgical environment through image matching and three-dimensional reconstruction technology;

[0016] Based on the acquired point cloud data of the obstacle in the surgical environment, a bounding box algorithm is used to fit the position of the obstacle bounding box in the robot coordinate system. The bounding box algorithm includes at least one of an axis-aligned bounding box and an oriented bounding box.

[0017] In some possible implementations, the flexible endoscope manipulation robot system further includes a delivery length sensor mounted on the delivery arm, and obtaining the length of the trajectory to be fitted based on the current endoscope delivery length includes:

[0018] Acquiring the current endoscope delivery length through the delivery length sensor;

[0019] The length of the trajectory to be fitted is obtained by combining the current endoscope delivery length and the endoscope length parameter.

[0020] In some possible implementations, the motion trajectory curve of the manipulator arm is generated by taking the center point of the tool at the end of the manipulator arm as the starting point and the center point of the tool at the end of the conveyor arm as the ending point, combining the length of the trajectory to be fitted and the obstacle avoidance space, including:

[0021] The motion trajectory curve is described by a B-spline curve, which is solved by an optimization model. The objective function of the optimization model is:

[0022]

[0023] in:

[0024] —Control points that affect the current curve value;

[0025] — node vector;

[0026] —the square integral of the second derivative of the curve;

[0027] —B-spline curve;

[0028] —weight coefficient;

[0029] -node;

[0030] —Total length of the endoscope;

[0031] —Current endoscope delivery length;

[0032] The constraints of the optimization model include:

[0033]

[0034]

[0035]

[0036] in:

[0037] —k-order B-spline basis function;

[0038] — starting point;

[0039] — termination point;

[0040] —Obstacle bounding box;

[0041] —Obstacle avoidance safety distance.

[0042] In some possible implementations, extracting trajectory points from the motion trajectory curve includes:

[0043] The motion trajectory curve is divided into equal parts according to unit length, and the equal division points are the trajectory points.

[0044] In some possible implementations, verifying whether the trajectory point of the operating arm is reachable includes:

[0045] Verifying whether a first constraint condition and a second constraint condition are simultaneously satisfied, whereby if both the first constraint condition and the second constraint condition are simultaneously satisfied, the trajectory point of the operating arm is reachable; and if both the first constraint condition and the second constraint condition are not simultaneously satisfied, the trajectory point of the operating arm is unreachable;

[0046] The first constraint condition is that when moving from the adjacent previous trajectory point to the trajectory point, the rotation angle of each joint of the operating arm does not exceed the rotation angle range;

[0047] The second constraint condition is that at the trajectory point, the robot bounding box and the obstacle bounding box do not interfere with each other.

[0048] In some possible implementations, the method further includes:

[0049] When the obstacle bounding box enters the working space of the operating arm and the position of the obstacle bounding box changes, and the obstacle bounding box is on the motion trajectory curve, the motion trajectory curve is regenerated and the operating arm switches to the new motion trajectory curve action.

[0050] In a second aspect, the present application provides a robot arm trajectory planning device, which is applied to a flexible endoscope manipulation robot system, wherein the flexible endoscope manipulation robot system includes a conveying arm and a manipulator arm; the device includes:

[0051] The first fitting unit is used to obtain point cloud data of obstacles in the surgical environment and fit the position and posture of the obstacle bounding box in the robot coordinate system;

[0052] The second fitting unit is used to obtain the position and posture of the tool center point at the end of the manipulator arm and the tool center point at the end of the conveyor arm in the robot coordinate system, and fit the robot bounding box based on the robot structural parameters;

[0053] a generating unit, configured to obtain a length of a trajectory to be fitted based on a current endoscope delivery length, obtain an obstacle avoidance space based on a position of the obstacle bounding box in the robot coordinate system, and generate a motion trajectory curve of the manipulator arm by combining the length of the trajectory to be fitted and the obstacle avoidance space with a center point of the tool at the end of the manipulator arm as a starting point and a center point of the tool at the end of the delivery arm as an end point;

[0054] a planning unit, configured to extract trajectory points from the motion trajectory curve and plan the posture of the manipulator at each of the trajectory points;

[0055] A verification unit is used to verify whether the trajectory point of the operating arm is reachable. If the trajectory point of the operating arm is not reachable, the posture of the operating arm corresponding to the trajectory point is adjusted; if the trajectory point of the operating arm is reachable, the trajectory point of the operating arm is matched with the current endoscope conveying length, and the trajectory point of the operating arm is described by the posture of the operating arm.

[0056] In some possible implementations, the first fitting unit includes:

[0057] A first acquisition subunit is used to collect image information of obstacles in the surgical environment, and obtain point cloud data of the obstacles in the surgical environment through image matching and three-dimensional reconstruction technology;

[0058] The first fitting subunit is used to fit the position of the obstacle bounding box in the robot coordinate system using a bounding box algorithm based on the acquired point cloud data of the obstacle in the surgical environment, wherein the bounding box algorithm includes at least one of an axis-aligned bounding box and an oriented bounding box.

[0059] In a third aspect, the present application provides a computing device comprising a processor and a memory, wherein the memory stores instructions, and the processor executes the instructions so that the computing device executes the robot arm trajectory planning method as described in the first aspect of the present application or any implementation method of the first aspect.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed on a computing device, the computing device executes the robot arm trajectory planning method described in the first aspect or any one of the implementations of the first aspect.

[0061] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods.

[0062] Based on the above description, it can be seen that the technical solution of this application has the following beneficial effects:

[0063] The embodiment of the present application first obtains point cloud data of obstacles in the surgical environment and fits the position of the obstacle bounding box in the robot coordinate system to accurately identify and locate obstacles in the surgical environment. This step provides accurate environmental information for subsequent trajectory planning, so that the robot can effectively avoid obstacles when planning the motion trajectory and avoid collisions between the robot and obstacles; secondly, the position of the tool center point of the end tool of the operating arm and the tool center point of the end tool of the conveying arm in the robot coordinate system is determined by the robot's positive kinematics, and the robot bounding box is fitted in combination with the robot's structural parameters to accurately describe the robot's motion range and posture, providing the robot's own kinematic information for trajectory planning, so that the robot can better adapt to different surgical environments and operation requirements, and improve the flexibility and adaptability of the system; on this basis, the embodiment of the present application obtains the length of the trajectory to be fitted based on the current endoscope delivery length, obtains the obstacle avoidance space based on the position of the obstacle bounding box in the robot coordinate system, takes the tool center point of the end tool of the operating arm as the starting point, and takes the tool center point of the end tool of the conveying arm as the end point, and concludes. The length of the trajectory to be fitted and the obstacle avoidance space are combined to generate a motion trajectory curve of the operating arm. This step comprehensively considers the endoscope delivery length and the obstacle avoidance space, and generates a motion trajectory that meets the operation requirements and avoids obstacles while ensuring safety, thereby significantly improving the reliability and safety of trajectory planning. Furthermore, the embodiment of the present application also extracts trajectory points from the motion trajectory curve, plans the posture of the operating arm at each trajectory point, and verifies whether the trajectory point of the operating arm is reachable. When the trajectory point of the operating arm is not reachable, the posture of the operating arm at the corresponding trajectory point is adjusted to ensure that the operating arm can smoothly reach the predetermined trajectory point. This step further improves the feasibility and reliability of trajectory planning, ensuring that the robot can safely and accurately complete the endoscope delivery task according to the predetermined trajectory in actual operation. When the trajectory point of the operating arm is reachable, the trajectory point of the operating arm is matched by the current endoscope delivery length, ensuring that the positioning of the operating arm is accurate. The trajectory point of the operating arm is described by the posture of the operating arm, so that the posture of the operating arm at each trajectory point is determined.

[0064] In summary, the robotic arm trajectory planning method provided in the embodiment of the present application can achieve precise obstacle avoidance in complex and changeable surgical environments through precise environmental perception, kinematic modeling and trajectory generation, and significantly improve the operational safety and reliability of the flexible endoscope control robot system. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and other features, advantages, and aspects of the various embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0066] Figure 1a A schematic diagram of a flexible endoscope provided in an embodiment of the present application;

[0067] Figure 1b A schematic structural diagram of a flexible endoscope operation robot system provided in an embodiment of the present application;

[0068] Figure 2 This is a flowchart of a specific embodiment of the robot arm trajectory planning method provided in this application;

[0069] Figure 3 This is a simplified structural diagram of a specific embodiment of the robot arm trajectory planning device provided in this application.

[0070] in, Figure 1a-Figure 3 The reference numerals are as follows:

[0071] 1-Flexible endoscope; 11-Operating unit; 110-Endoscope workstation interface; 111-Large impeller; 112-Small impeller; 113-Function button; 114-Suction valve button; 115-Water vapor valve button; 116-Instrument channel; 12-Insertion unit; 13-Tip end;

[0072] 100-Robotic trolley; 101-Transport arm; 102-Operating arm; 103-Transport device; 104-Operating device; 200-Endoscopic workstation; 300-Instrument operating table; 400-Operating table; 500-Binocular camera; 600-Robotic arm trajectory planning device; 601-First fitting unit; 602-Second fitting unit; 603-Generation unit; 604-Planning unit; 605-Verification unit. DETAILED DESCRIPTION

[0073] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are for illustrative purposes only and are not intended to limit the scope of protection of the present application.

[0074] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0075] It should be noted that the concepts of "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0076] It should be noted that the modifications of "one" and "multiple" mentioned in this application are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0077] Natural cavities such as the digestive tract and respiratory tract are common sites of human disease. Because lesions are located within the body's natural cavities, flexible endoscopy is necessary for examination.

[0078] Traditional flexible endoscopic examinations or surgeries require two hands to coordinate movements, such as holding the scope, operating knobs, and manually maneuvering the scope. Some examinations or surgeries require image guidance from radiographs, requiring medical personnel to wear heavy lead-based protective clothing and manually operate the flexible endoscope for extended periods. This significantly impacts the quality of diagnosis and treatment, their physical strength, and their health. This is particularly restrictive for experienced, older, or female medical personnel, who may even be forced to abandon endoscopic work.

[0079] With the development of robot-assisted technology, doctors can adjust the delivery length and posture of flexible endoscopes by adjusting the handle switch and button, which greatly reduces the doctor's physical strength and manual operation fatigue, reduces the operation requirements of the operation, reduces the radiation to medical staff, and at the same time improves the interaction between medical staff and images.

[0080] However, existing flexible endoscope control robotic systems still have significant flaws. During the endoscope delivery process, the robot must pre-plan the trajectory of the manipulator arm based on the patient's natural orifice position to prevent damage to the endoscope or actuators due to improper operation. During actual surgery, the environment is complex and changing, and established trajectory planning methods cannot ensure accurate obstacle avoidance, and their reliability in ensuring safe movement is clearly insufficient.

[0081] Based on this, an embodiment of the present application provides a robot arm trajectory planning method, which is applied to a soft endoscope manipulation robot system, aiming to solve the reliability and safety problems of the trajectory planning of the manipulator arm in the mirror delivery link of the soft endoscope manipulation robot system in the related technology. Specifically, the embodiment of the present application first obtains point cloud data of obstacles in the surgical environment and fits the position of the obstacle bounding box in the robot coordinate system to accurately identify and locate obstacles in the surgical environment. This step provides accurate environmental information for subsequent trajectory planning, so that the robot can effectively avoid obstacles when planning the motion trajectory, and avoid damage to the soft endoscope due to improper operation; secondly, the position of the tool center point of the end tool of the manipulator arm and the tool center point of the end tool of the delivery arm in the robot coordinate system is determined by the robot's positive kinematics, and the robot bounding box is fitted in combination with the robot's structural parameters to accurately describe the robot's motion range and posture, providing the robot's own kinematic information for trajectory planning, so that the robot can better adapt to different surgical environments and operation requirements, and improve the flexibility and adaptability of the system; on this basis, the embodiment of the present application obtains the length of the trajectory to be fitted based on the current endoscope delivery length, and obtains the length of the trajectory to be fitted based on the obstacle bounding box in the robot coordinate system. The obstacle avoidance space is obtained by taking the center point of the tool at the end of the operating arm as the starting point and the center point of the tool at the end of the conveying arm as the ending point. The motion trajectory curve of the operating arm is generated by combining the length of the trajectory to be fitted and the obstacle avoidance space. This step comprehensively considers the endoscope conveying length and the obstacle avoidance space, and generates a motion trajectory that meets the operation requirements and avoids obstacles while ensuring safety, thereby significantly improving the reliability and safety of trajectory planning. Furthermore, the embodiment of the present application also extracts trajectory points from the motion trajectory curve, plans the posture of the operating arm at each trajectory point, and verifies whether the trajectory point of the operating arm is reachable. When the trajectory points of the manipulator arm are unreachable, the posture of the manipulator arm at the corresponding trajectory points is adjusted to ensure that the manipulator arm can smoothly reach the predetermined trajectory points. This step further improves the feasibility and reliability of trajectory planning, ensuring that the robot can complete the operation task safely and accurately according to the predetermined trajectory in actual operation. When the trajectory points of the manipulator arm are reachable, the trajectory points of the manipulator arm are matched by the current endoscope delivery length to ensure the accurate positioning of the manipulator arm. The trajectory points of the manipulator arm are described by the posture of the manipulator arm, so that the posture of the manipulator arm at each trajectory point is determined.

[0082] In summary, the robotic arm trajectory planning method provided in the embodiment of the present application can achieve precise obstacle avoidance in complex and changeable surgical environments through precise environmental perception, kinematic modeling and trajectory generation, and significantly improve the operational safety and reliability of the flexible endoscope control robot system.

[0083] It is understandable that the defects in the above solutions are the results obtained by the applicant after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed in the embodiments of this application should be regarded as the applicant's contribution to the embodiments of this application during the application process.

[0084] In order to facilitate understanding of the robot arm trajectory planning method provided in the embodiment of the present application, the following Figure 1a and Figure 1b Provide explanation. Figure 1a This is a schematic structural diagram of a traditional flexible endoscope provided in an embodiment of the present application. Figure 1b A schematic structural diagram of a flexible endoscope operation robot system provided in an embodiment of the present application.

[0085] In the embodiment of the present application, the flexible endoscope is taken as an example of a digestive flexible endoscope. Figure 1a As shown, the flexible endoscope 1 includes an operating unit 11, an insertion unit 12, and a tip 13. The operating unit 11 includes an endoscope workstation interface 110, a large impeller 111, a small impeller 112, a function button 113, a suction valve button 114, a water vapor valve button 115, and an instrument channel 116. When the flexible endoscope 1 is transported, the tip 13 is first inserted into the patient's mouth, and then the insertion unit 12 is inserted into the patient's mouth.

[0086] Combine Figure 1a Taking a flexible digestive endoscope as an example, the traditional manual operation method of a flexible digestive endoscope is as follows: the medical staff holds the front end of the flexible endoscope with their right hand, holds the flexible endoscope operating part 11 with their left hand, and places it on their chest. They use their thumb, middle finger, and ring finger to adjust the size knob. The middle finger controls the air and water injection, and the index finger controls the suction valve button 114. When controlling the advancement and retraction of the flexible endoscope, it is necessary to rotate the scope body to always maintain a good field of view. To reach the lesion or the area requiring biopsy, the corresponding instrument is inserted from the instrument channel under the instrument channel, through the instrument channel inside the insertion part 12, to the tip end 13. The corresponding operation is completed with the cooperation of the visual field under the microscope.

[0087] Combine Figure 1bThe flexible endoscope manipulation robot system includes a robot, an endoscope workstation 200, an instrument operating table 300, and an operating table 400. The robot comprises a robot trolley 100, a transport arm (also called a master arm) 101, a manipulator arm (also called a slave arm) 102, a transport device 103, and a manipulator 104. The transport arm 101 and the manipulator arm 102 are collectively referred to as a robotic arm. Both the transport arm 101 and the manipulator arm 102 are multi-degree-of-freedom robotic arms. In a flexible endoscope manipulation robot system, a tool is typically secured to the end of the robotic arm. For example, a manipulator 104 is secured to the end of the manipulator arm 102. The manipulator 104 is secured to the manipulator 104, and the manipulator 11 of the flexible endoscope 1 is secured to the end of the transport arm 101. The manipulator 104 and the manipulator 11 of the flexible endoscope 1 can be considered tools secured to the end of the manipulator arm 102, while the transport device 103 can be considered tools secured to the end of the transport arm 101. The tools can also be referred to as end effectors.

[0088] In master-slave control mode, the doctor's commands are converted into movements of the delivery arm 101, the manipulator arm 102, the delivery device 103, and the manipulator 104. The manipulator 104 controls the curvature of the flexible endoscope 1, while the delivery device 103 controls the length of time the flexible endoscope 1 enters the human body. Both control the rotation of the flexible endoscope 1 simultaneously, or one can control the other independently. Furthermore, the manipulator 104 can drive the flexible endoscope's associated instruments.

[0089] To facilitate understanding of the present application, a robot arm trajectory planning method provided in an embodiment of the present application is described below with reference to the accompanying drawings.

[0090] See also Figure 2 , Figure 2 This is a flowchart of a specific embodiment of the robot arm trajectory planning method provided in this application.

[0091] The robot arm trajectory planning method of the embodiment of the present application is applied to a flexible endoscope manipulation robot system, which includes a conveying arm and a manipulation arm. The method may include S201-S205:

[0092] S201: Obtain point cloud data of obstacles in the surgical environment and fit the position and posture of the obstacle bounding box in the robot coordinate system.

[0093] In such Figure 1bIn the illustrated embodiment, a binocular camera 500 is used to capture images of obstacles within the surgical environment, such as the endoscopic workstation 200 and the instrument console 300. The binocular camera 500 captures image information of the obstacles. Based on this image information, image matching and 3D reconstruction techniques are used to obtain point cloud data of the obstacles. This point cloud data contains the 3D coordinate information of numerous discrete points on the obstacle surface. A bounding box algorithm, such as an axis-aligned bounding box algorithm or an oriented bounding box algorithm, is then used to calculate the minimum bounding box that can completely enclose the obstacle based on the distribution range of the point cloud data. This bounding box has clear position and posture information in the robot coordinate system, providing basic data for subsequent obstacle avoidance planning.

[0094] For example, for the instrument operating table 300, the minimum boundary coordinate of its bounding box is obtained by the algorithm, and the minimum position of the X axis is obtained. , X-axis maximum position , minimum position of Y axis , Y-axis maximum position Due to its regular spatial shape, it is fitted into a cuboid with the same Z axis as the robot coordinate system and known side lengths. ,in:

[0095] ——The bounding box of the instrument operating table is rotated based on the Z axis The rotation matrix of the angle;

[0096] ——The initial position point set of the instrument operation table bounding box based on the robot coordinate system;

[0097] ——The zero offset of the instrument operating table bounding box determines the spatial position of the instrument operating table bounding box in the robot coordinate system.

[0098] Among them, the binocular camera 500, also known as a stereo camera, is a device that simulates the depth perception ability of the human eyes. It consists of two cameras arranged side by side and can capture two slightly different perspective images of the same scene. Before actually controlling the soft endoscope manipulation robot system, the binocular camera 500 needs to be calibrated by hand and eye. Hand-eye calibration can be understood as a process for determining the relationship between the coordinate system of the binocular camera 500 and the coordinate system of the robot. In other words, through hand-eye calibration, the conversion relationship between the coordinate system of the binocular camera 500 and the coordinate system of the robot can be obtained. In this way, for the image information collected during the actual operation of the binocular camera 500, the coordinate values ​​of the points in the image information in the robot coordinate system can be obtained.

[0099] In some other embodiments of the present application, point cloud data of obstacles in the surgical environment can also be obtained through a lidar scanning system, a depth camera, etc.

[0100] Based on this, in the robot arm trajectory planning method of the embodiment of the present application, point cloud data of obstacles in the surgical environment is obtained, and the position and posture of the obstacle bounding box in the robot coordinate system are fitted, including:

[0101] Collect image information of obstacles in the surgical environment, and obtain point cloud data of obstacles in the surgical environment through image matching and 3D reconstruction technology;

[0102] Based on the acquired point cloud data of obstacles in the surgical environment, a bounding box algorithm is used to fit the position and posture of the obstacle bounding box in the robot coordinate system. The bounding box algorithm includes at least one of an axis-aligned bounding box and an oriented bounding box.

[0103] As set up above, by acquiring point cloud data of obstacles in the surgical environment and fitting the position of the obstacle bounding box in the robot coordinate system, obstacles in the surgical environment can be accurately identified and located. This step provides accurate environmental information for subsequent trajectory planning, allowing the robot to effectively avoid obstacles when planning the motion trajectory and prevent damage to the flexible endoscope due to improper operation.

[0104] S202: Obtain the positions of the tool center point at the end of the manipulator arm and the tool center point at the end of the conveyor arm in the robot coordinate system, and fit the robot bounding box based on the robot structural parameters.

[0105] Specifically, the robot's forward kinematics model can be used to calculate the position of the tool center point at the end of the operating arm and the tool center point at the end of the conveying arm in the robot coordinate system according to the angle values ​​of each joint of the robot and the kinematic transformation formula to determine their spatial positions.

[0106] Given the structural parameters of the robot's components, such as size and shape, as well as the spatial positions of the manipulator and conveyor arm end-of-tool centers obtained through the robot's forward kinematics model, the positions and postures of the robot body, conveyor device 103, and manipulator 104 in the robot coordinate system are determined using the manipulator and conveyor arm end-of-tool centers as references and combining the relative positional relationships of each component with the manipulator and conveyor arm end-of-tool centers. Then, using an appropriate bounding box algorithm, such as the axis-aligned bounding box algorithm, the minimum bounding box that encloses the robot body, conveyor device 103, and manipulator 104 is calculated. This is used to assess the robot's spatial occupancy during motion and prevent collisions with obstacles.

[0107] The robot bounding box includes the robot body bounding box ,in:

[0108] ——Robot body bounding box rotates based on the Z axis The rotation matrix of the angle;

[0109] ——The robot body bounding box is based on the initial position point set of the robot coordinate system;

[0110] ——The zero offset of the robot body bounding box determines the spatial position of the robot body bounding box in the robot coordinate system.

[0111] Conveyor device bounding box ,in:

[0112] ——The conveyor bounding box is rotated based on the Z axis The rotation matrix of the angle;

[0113] ——The initial position point set of the conveyor device bounding box based on the robot coordinate system;

[0114] ——The zero offset of the conveyor bounding box determines the spatial position of the conveyor bounding box in the robot coordinate system.

[0115] and the operating device bounding box ,in:

[0116] ——The operating device bounding box is rotated based on the Z axis The rotation matrix of the angle;

[0117] ——The initial position point set of the operating device bounding box based on the robot coordinate system;

[0118] ——The zero offset of the manipulator bounding box determines the spatial position of the manipulator bounding box in the robot coordinate system.

[0119] As set up above, the position of the tool center point at the end of the manipulator arm and the tool center point at the end of the conveyor arm in the robot coordinate system is determined through the robot's forward kinematics, and the robot bounding box is fitted in combination with the robot's structural parameters to accurately describe the robot's range of motion and posture, providing the robot's own kinematic information for trajectory planning, so that the robot can better adapt to different surgical environments and operational requirements, and improve the flexibility and adaptability of the system.

[0120] S203: Based on the current endoscope delivery length, the length of the trajectory to be fitted is obtained, and based on the position of the obstacle bounding box in the robot coordinate system, the obstacle avoidance space is obtained. The center point of the tool at the end of the operating arm is used as the starting point, and the center point of the tool at the end of the delivery arm is used as the end point. In combination with the length of the trajectory to be fitted and the obstacle avoidance space, the motion trajectory curve of the operating arm is generated.

[0121] In the flexible endoscope operating robot system, the conveying device 103 conveys the flexible endoscope 1 through the natural cavity into the human body. During the delivery process, the conveying arm 101 will keep the delivery position stopped, and the conveying device 103 at the end of the conveying arm 101 conveys the flexible endoscope 1. The end of the operating arm 102 supports and fixes the flexible endoscope 1, operates the curvature of the flexible endoscope 1, and the operating arm 102 moves in the direction close to the conveying arm 101 following the flexible endoscope 1. In the embodiment of the present application, the trajectory planning of the robot arm is the trajectory planning of the operating arm. Therefore, the motion trajectory curve of the operating arm uses the center point of the tool at the end of the operating arm as the starting point of the trajectory planning. , with the center point of the tool at the end of the conveyor arm as the end point .

[0122] Among them, the obstacle avoidance space is obtained based on the position of the obstacle bounding box in the robot coordinate system. Specifically, in the mathematical model, by setting appropriate constraints, such as setting a certain safety distance around the obstacle bounding box to form an obstacle avoidance space, it is ensured that the planned operating arm trajectory will not enter the dangerous area, and the collision between the operating arm and the obstacle is avoided as much as possible.

[0123] The length of the trajectory to be fitted is obtained based on the current endoscope delivery length. The flexible endoscope manipulation robot system of the embodiment of the present application further includes a delivery length sensor installed on the delivery arm 101. In the robot arm trajectory planning method of the embodiment of the present application, the length of the trajectory to be fitted is obtained based on the current endoscope delivery length, including:

[0124] The current endoscope delivery length is obtained through the delivery length sensor;

[0125] The length of the trajectory to be fitted is obtained by combining the current endoscope delivery length and the endoscope length parameter.

[0126] The current endoscope delivery length is the length of the flexible endoscope 1 extending from the delivery arm 101 . The difference between the endoscope length parameter and the current endoscope delivery length is the length of the trajectory to be fitted.

[0127] Based on the determined starting point and ending point of trajectory planning, the length of the trajectory to be fitted, and the obstacle avoidance space, a suitable mathematical model, such as a spline curve model, is adopted. By adjusting the control points and parameters of the curve, the curve can avoid the obstacle avoidance space and maintain smoothness while meeting the starting point and ending point conditions.

[0128] Furthermore, in the robot arm trajectory planning method of the embodiment of the present application, the center point of the tool at the end of the manipulator arm is used as the starting point, the center point of the tool at the end of the conveyor arm is used as the ending point, and the length of the trajectory to be fitted and the obstacle avoidance space are combined to generate the motion trajectory curve of the manipulator arm, including:

[0129] The motion trajectory curve is described by the B-spline curve, which is solved by the optimization model. The objective function of the optimization model is:

[0130]

[0131] in:

[0132] —Control points that affect the current curve value;

[0133] — node vector;

[0134] —the square integral of the second derivative of the curve;

[0135] —weight coefficient;

[0136] —B-spline curve;

[0137] -node;

[0138] —Total length of the endoscope;

[0139] —Current endoscope delivery length;

[0140] The constraints of the optimization model include:

[0141]

[0142]

[0143]

[0144] in:

[0145] —k-order B-spline basis function;

[0146] — starting point;

[0147] — termination point;

[0148] —Obstacle bounding box;

[0149] —Obstacle avoidance safety distance.

[0150] Specifically, the expression of the B-spline curve is as follows:

[0151]

[0152] is the control point that affects the current curve value, is the k-order B-spline basis function, which can be calculated by the De Boor-Cox recursion formula:

[0153]

[0154] Node Vector , and satisfies .

[0155] in:

[0156] First, the starting point of trajectory planning needs to meet , that is, the starting point constraint is expressed as: .

[0157] Second, the end point of trajectory planning needs to meet , that is, the termination point constraint is expressed as: .

[0158] Third, B-spline curve The arc length L is calculated as follows:

[0159]

[0160] is the first-order derivative of the curve, , and are the starting and ending parameters within the curve definition domain, , , the current endoscope delivery length is known The remaining arc length is based on the total length of the endoscope Calculate the arc length of the B-spline, and the arc length constraint is expressed as: .

[0161] Fourth, the curve needs to avoid the obstacle space location , so the obstacle avoidance space is expressed as: , To avoid obstacles at a safe distance.

[0162] Fifth, in order to ensure the smoothness of the curve, the second-order derivative of the curve can be constrained. The second-order derivative of the curve is defined as , smoothness can be ensured by minimizing the square integral of the second-order derivative of the curve, that is, part of the objective function can be set as: .

[0163] Based on the above constraints, the above optimization model is constructed. is a weight coefficient used to balance the relative importance of the curve length constraint and the smoothness constraint. The motion trajectory curve can be obtained by calculating the optimal control points and node vectors through the optimization model, which significantly improves the reliability and safety of trajectory planning.

[0164] S204: Extracting trajectory points from the motion trajectory curve and planning the posture of the manipulator at each trajectory point.

[0165] Based on the fitted bounding box of the manipulator, the motion trajectory curve is divided into equal parts according to unit length (1 mm). The equal division points are the trajectory points. The influence of the posture of the manipulator 102 at the trajectory points on the spatial position of the bounding box is analyzed.

[0166] By changing the angles of the joints of the manipulator 102, the posture of the manipulator 102 at each trajectory point is adjusted, thereby causing the spatial position of the bounding box to change accordingly to meet the obstacle avoidance and movement requirements of the manipulator 102 at that trajectory point. For example, when the manipulator 102 approaches an obstacle, the joint angles are adjusted to tilt the manipulator 102 away from the obstacle, thereby changing the direction and position of the bounding box to avoid collision with the obstacle.

[0167] Based on this, in the robot arm trajectory planning method of the embodiment of the present application, the trajectory points are extracted from the motion trajectory curve, including:

[0168] The motion trajectory curve is divided into equal parts according to unit length, and the dividing points are the trajectory points.

[0169] S205: Verify whether the trajectory point of the manipulator arm is reachable. If the trajectory point of the manipulator arm is not reachable, adjust the posture of the manipulator arm at the corresponding trajectory point. If the trajectory point of the manipulator arm is reachable, the trajectory point of the manipulator arm is matched with the current endoscope delivery length, and the trajectory point of the manipulator arm is described by the posture of the manipulator arm.

[0170] In the robot arm trajectory planning method of the embodiment of the present application, verifying whether the trajectory points of the manipulator are reachable includes:

[0171] Verify whether the first constraint and the second constraint are satisfied at the same time. If the first constraint and the second constraint are satisfied at the same time, the trajectory point of the manipulator is reachable. If the first constraint and the second constraint are not satisfied at the same time, the trajectory point of the manipulator is unreachable.

[0172] The first constraint condition is that when moving from the adjacent previous trajectory point to the corresponding trajectory point, the rotation angle of each joint of the manipulator does not exceed the rotation angle range;

[0173] The second constraint is that at the corresponding trajectory points, the robot bounding box and the obstacle bounding box do not interfere with each other.

[0174] The rotation angle limits for each joint in the manipulator are based on the physical limitations and kinematic constraints of the manipulator's joints. Each joint has a rotation angle range. During the verification process, the rotation angle change of each joint in the manipulator is calculated when moving from the adjacent previous trajectory point to the corresponding trajectory point. If the rotation angle of each joint in the manipulator does not exceed the rotation angle range, the first constraint condition is determined to be met. If the rotation angle of each joint in the manipulator does not exceed the rotation angle range, the first constraint condition is determined to be not met, and the trajectory point is unreachable.

[0175] For the spatial reachability of the manipulator, check whether the robot bounding box and the obstacle bounding box interfere with each other at the current trajectory point in combination with the fitted robot bounding box. If the robot bounding box and the obstacle bounding box do not interfere with each other, it is determined that the second constraint condition is met and the trajectory point is reachable. If the robot bounding box and the obstacle bounding box interfere with each other, it is determined that the second constraint condition is not met and the trajectory point is unreachable. It is necessary to adjust the rotation posture of the manipulator, adjust the rotation matrix of the manipulator's spatial bounding box, change the spatial position of the manipulator's bounding box, and adjust the manipulator's obstacle avoidance posture.

[0176] In the flexible endoscope manipulation robot system, the trajectory points of the manipulator arm are matched to the current endoscope delivery length. Specifically, during the surgical procedure, the delivery length sensor provides real-time feedback on the current endoscope delivery length. Based on this, the robot searches for the corresponding trajectory point within a validated motion trajectory curve. Then, by controlling the motion of each robot joint, the manipulator 104 is precisely moved to this trajectory point, ensuring that the flexible endoscope is safely and accurately delivered along the planned trajectory. During this process, the current endoscope delivery length and the position of the manipulator 104 are continuously monitored, and the motion parameters of each joint are adjusted in real time to ensure accuracy and stability.

[0177] The trajectory points of the manipulator are described by the posture of the manipulator, that is, the posture of each trajectory point is determined, and the posture includes spatial position and posture.

[0178] In some possible implementations, the robot arm trajectory planning method of the embodiment of the present application further includes:

[0179] When the obstacle bounding box enters the working space of the manipulator, the position of the obstacle bounding box changes, and the obstacle bounding box is on the motion trajectory curve, the motion trajectory curve is regenerated and the manipulator switches to the new motion trajectory curve action.

[0180] In this way, the robot arm trajectory planning method of the embodiment of the present application can respond to environmental changes in real time. When the obstacle bounding box enters the working space of the operating arm and the position of the obstacle bounding box changes, and the obstacle bounding box is on the motion trajectory curve, the motion trajectory curve is regenerated, and the operating arm switches to the new motion trajectory curve action, ensuring that the robot accurately avoids obstacles, better adapts to the complex and changeable surgical environment, and ensures the safety and reliability of the robot's movement.

[0181] Based on the robotic arm trajectory planning method provided in the above method embodiment, the present application also provides a robotic arm trajectory planning device. The robotic arm trajectory planning device will be described below with reference to the accompanying drawings. Since the principle of the problem solved by the device in the embodiment of the present disclosure is similar to that of the robotic arm trajectory planning method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0182] Please refer to Figure 3 , Figure 3 This is a simplified structural diagram of a specific embodiment of the robot arm trajectory planning device provided in this application.

[0183] The present application also provides a robot arm trajectory planning device 600, which is applied to a flexible endoscope manipulation robot system. The flexible endoscope manipulation robot system includes a conveying arm 101 and a manipulation arm 102. The robot arm trajectory planning device 600 includes:

[0184] The first fitting unit 601 is used to obtain point cloud data of obstacles in the surgical environment and fit the position and posture of the obstacle bounding box in the robot coordinate system;

[0185] The second fitting unit 062 is used to determine the position of the tool center point of the end tool of the manipulator arm and the tool center point of the conveyor arm in the robot coordinate system through the robot forward kinematics, and fit the robot bounding box in combination with the robot structural parameters;

[0186] A generating unit 603 is configured to obtain a length of a trajectory to be fitted based on the current endoscope delivery length, obtain an obstacle avoidance space based on the position of the obstacle bounding box in the robot coordinate system, and generate a motion trajectory curve of the manipulator arm by combining the length of the trajectory to be fitted and the obstacle avoidance space with the center point of the tool at the end of the manipulator arm as the starting point and the center point of the tool at the end of the delivery arm as the ending point;

[0187] The planning unit 604 is used to extract trajectory points from the motion trajectory curve and plan the posture of the manipulator at each trajectory point;

[0188] The verification unit 605 is used to verify whether the trajectory point of the manipulator arm is reachable. If the trajectory point of the manipulator arm is not reachable, the posture of the manipulator arm at the corresponding trajectory point is adjusted; if the trajectory point of the manipulator arm is reachable, the trajectory point of the manipulator arm is matched by the current endoscope delivery length, and the trajectory point of the manipulator arm is described by the posture of the manipulator arm.

[0189] In some possible implementations, the first fitting unit 601 includes:

[0190] A first acquisition subunit is used to collect image information of obstacles in the surgical environment, and obtain point cloud data of the obstacles in the surgical environment through image matching and three-dimensional reconstruction technology;

[0191] The first fitting subunit is used to fit the position of the obstacle bounding box in the robot coordinate system using a bounding box algorithm based on the acquired point cloud data of the obstacle in the surgical environment, wherein the bounding box algorithm includes at least one of an axis-aligned bounding box and an oriented bounding box.

[0192] It should be noted that the specific implementation of each unit in this embodiment can refer to the relevant description in the above method embodiment. The division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The functional units in the embodiment of the present application can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0193] An embodiment of the present application further provides a computing device, which may be a cloud computing device or an edge computing device, including: a processor, a memory, and a system bus;

[0194] The processor and memory are connected via a system bus;

[0195] The memory is used to store one or more programs, wherein the one or more programs include instructions, which, when executed by the processor, enable the processor to execute the above-mentioned robot arm trajectory planning method.

[0196] The present application also provides a computer-readable storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed on a computing device, the computing device executes the above-mentioned robot arm trajectory planning method.

[0197] In the context of this application, a computer-readable storage medium, also known as a machine-readable medium, may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0198] It should be noted that the computer-readable medium referred to in this application may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. This propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wire, optical cable, RF (radio frequency), or any suitable combination thereof.

[0199] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

[0200] Although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this application. Certain features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0201] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the disclosure herein is not limited to technical solutions formed by specific combinations of the aforementioned technical features. It also encompasses other technical solutions formed by any combination of the aforementioned technical features or their equivalents, without departing from the aforementioned disclosure. For example, a technical solution formed by replacing the aforementioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A robot arm trajectory planning method, characterized in that: Applied to a flexible endoscope manipulation robot system, the flexible endoscope manipulation robot system includes a delivery arm and a manipulation arm, and the method includes: Obtain point cloud data of obstacles in the surgical environment and fit the position and posture of the obstacle bounding box in the robot coordinate system; Obtain the position of the tool center point at the end of the manipulator arm and the tool center point at the end of the conveyor arm in the robot coordinate system, and fit the robot bounding box based on the robot structural parameters; The length of the trajectory to be fitted is obtained based on the current endoscope delivery length, the obstacle avoidance space is obtained based on the position of the obstacle bounding box in the robot coordinate system, and the motion trajectory curve of the manipulator arm is generated by combining the length of the trajectory to be fitted and the obstacle avoidance space with the center point of the tool at the end of the manipulator arm as the starting point and the center point of the tool at the end of the delivery arm as the ending point; Extracting trajectory points from the motion trajectory curve and planning the posture of the manipulator at each of the trajectory points; Verifying whether the trajectory point of the manipulator arm is reachable, and if the trajectory point of the manipulator arm is not reachable, adjusting the posture of the manipulator arm corresponding to the trajectory point; if the trajectory point of the manipulator arm is reachable, matching the trajectory point of the manipulator arm with the current endoscope delivery length, and the trajectory point of the manipulator arm is described by the posture of the manipulator arm; The motion trajectory curve of the manipulator arm is generated by taking the center point of the tool at the end of the manipulator arm as the starting point and the center point of the tool at the end of the conveyor arm as the ending point, combining the length of the trajectory to be fitted and the obstacle avoidance space, including: The motion trajectory curve is described by a B-spline curve, which is solved by an optimization model. The objective function of the optimization model is: ； in: —Control points that affect the current curve value; — node vector; —the square integral of the second derivative of the curve; —weight coefficient; —B-spline curve; -node; —Total length of the endoscope; —Current endoscope delivery length; The constraints of the optimization model include: ； in: —k-order B-spline basis function; — starting point; — termination point; —Obstacle bounding box; —Obstacle avoidance safety distance.

2. The method according to claim 1, characterized in that The step of obtaining point cloud data of obstacles in the surgical environment and fitting the position and posture of the obstacle bounding box in the robot coordinate system includes: Collecting image information of obstacles in the surgical environment, and obtaining point cloud data of the obstacles in the surgical environment through image matching and three-dimensional reconstruction technology; Based on the acquired point cloud data of the obstacle in the surgical environment, a bounding box algorithm is used to fit the position of the obstacle bounding box in the robot coordinate system. The bounding box algorithm includes at least one of an axis-aligned bounding box and an oriented bounding box.

3. The method according to claim 1, characterized in that The flexible endoscope manipulation robot system further includes a delivery length sensor installed on the delivery arm, and obtaining the length of the trajectory to be fitted based on the current endoscope delivery length includes: Acquiring the current endoscope delivery length through the delivery length sensor; The length of the trajectory to be fitted is obtained by combining the current endoscope delivery length and the endoscope length parameter.

4. The method according to claim 1, wherein The step of extracting trajectory points from the motion trajectory curve includes: The motion trajectory curve is divided into equal parts according to unit length, and the equal division points are the trajectory points.

5. The method according to claim 1, characterized in that The verifying whether the trajectory point of the operating arm is reachable includes: Verifying whether a first constraint condition and a second constraint condition are simultaneously satisfied, whereby if both the first constraint condition and the second constraint condition are simultaneously satisfied, the trajectory point of the operating arm is reachable; and if both the first constraint condition and the second constraint condition are not simultaneously satisfied, the trajectory point of the operating arm is unreachable; The first constraint condition is that when moving from the adjacent previous trajectory point to the trajectory point, the rotation angle of each joint of the operating arm does not exceed the rotation angle range; The second constraint condition is that at the trajectory point, the robot bounding box and the obstacle bounding box do not interfere with each other.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: When the obstacle bounding box enters the working space of the operating arm and the position of the obstacle bounding box changes, and the obstacle bounding box is on the motion trajectory curve, the motion trajectory curve is regenerated and the operating arm switches to the new motion trajectory curve action.

7. A robot arm trajectory planning device, characterized in that: Applicable to a flexible endoscope manipulation robot system, the flexible endoscope manipulation robot system includes a delivery arm and a manipulation arm; the device includes: The first fitting unit is used to obtain point cloud data of obstacles in the surgical environment and fit the position and posture of the obstacle bounding box in the robot coordinate system; The second fitting unit is used to obtain the position and posture of the tool center point at the end of the manipulator arm and the tool center point at the end of the conveyor arm in the robot coordinate system, and fit the robot bounding box based on the robot structural parameters; a generating unit, configured to obtain a length of a trajectory to be fitted based on a current endoscope delivery length, obtain an obstacle avoidance space based on a position of the obstacle bounding box in the robot coordinate system, and generate a motion trajectory curve of the manipulator arm by combining the length of the trajectory to be fitted and the obstacle avoidance space with a center point of the tool at the end of the manipulator arm as a starting point and a center point of the tool at the end of the delivery arm as an end point; a planning unit, configured to extract trajectory points from the motion trajectory curve and plan the posture of the manipulator at each of the trajectory points; a verification unit, configured to verify whether the trajectory point of the operating arm is reachable, and if the trajectory point of the operating arm is not reachable, adjust the posture of the operating arm corresponding to the trajectory point; if the trajectory point of the operating arm is reachable, the trajectory point of the operating arm is matched with the current endoscope transport length, and the trajectory point of the operating arm is described by the posture of the operating arm; The motion trajectory curve of the manipulator arm is generated by taking the center point of the tool at the end of the manipulator arm as the starting point and the center point of the tool at the end of the conveyor arm as the ending point, combining the length of the trajectory to be fitted and the obstacle avoidance space, including: The motion trajectory curve is described by a B-spline curve, which is solved by an optimization model. The objective function of the optimization model is: ； in: —Control points that affect the current curve value; — node vector; —the square integral of the second derivative of the curve; —weight coefficient; —B-spline curve; -node; —Total length of the endoscope; —Current endoscope delivery length; The constraints of the optimization model include: ； in: —k-order B-spline basis function; — starting point; — termination point; —Obstacle bounding box; —Obstacle avoidance safety distance.

8. The device according to claim 7, characterized in that The first fitting unit includes: A first acquisition subunit is used to collect image information of obstacles in the surgical environment, and obtain point cloud data of the obstacles in the surgical environment through image matching and three-dimensional reconstruction technology; The first fitting subunit is used to fit the position of the obstacle bounding box in the robot coordinate system using a bounding box algorithm based on the acquired point cloud data of the obstacle in the surgical environment, wherein the bounding box algorithm includes at least one of an axis-aligned bounding box and an oriented bounding box.

9. A computing device, characterized in that The computing device includes a processor and a memory, wherein instructions are stored in the memory, and the processor executes the instructions so that the processor executes the robot arm trajectory planning method according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed on a computing device, the computing device executes the robot arm trajectory planning method according to any one of claims 1 to 6.

Images