Micromanipulation system and microscopic control method based on rigid-flexible combined mechanical arm

Abstract

The invention discloses a microscopic operation system and a microscopic control method based on a rigid-flexible combined mechanical arm, belongs to the field of organoid culture, and aims to solve the problem of control precision of a flexible structure. The mechanical arm comprises a rigid-flexible combined mechanical arm body used for conducting decoupling operation on the space pose of a tail end tool installed on the rigid-flexible combined mechanical arm body; the multi-sensor measuring system is used for collecting visual data and strain data of the end tool in real time; and the self-adaptive control module is used for carrying out online estimation and disturbance compensation on time-varying parameters of the rigid-flexible combined mechanical arm according to the data acquired by the multi-sensor measurement system, and controlling the space pose of the tail-end tool. Further, a concentric push-pull flexible body is driven to adjust the space posture of a tail end tool of the flexible mechanical arm; according to the flexible mechanical arm, the single rotation freedom degree of a concentric push-pull flexible body is achieved through two-stage stretching design, the concentric push-pull flexible body is of a three-bending-section concentric pipe structure, and a double-row staggered strain gauge array is attached to the back of an outer pipe.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of organoid culture, and particularly relates to a micro-operation system based on a rigid-flexible combined manipulator and a micro-control method. BACKGROUND

[0002] Organoid culture technology is a new culture technology emerging in recent years. By constructing a three-dimensional culture environment in vitro and inducing differentiation, cells self-organize into a cell cluster with complex structure, which can reflect the true characteristics of in vivo tissues and organs, and provides an excellent research platform for the fields of tissue engineering, regenerative medicine, precision medicine and drug development. Due to the involvement of a large number of complex three-dimensional operations and processing, the stability of organoids cultured by manual operation is difficult to guarantee, and the development of corresponding automatic operation equipment becomes the top priority of organoid technology research.

[0003] Unlike traditional micro-operation systems that only need to move in a two-dimensional plane, since the organoid cell cluster has a complex three-dimensional structure, the matching operation equipment needs to have three-dimensional pose control capability. However, due to the narrow operation space and small linear actuator stroke, it is difficult to add the pose adjustment degrees of freedom of the traditional structure.

[0004] To this end, the existing scheme realizes the pose adjustment of the operation object through a flexible structure, and at the same time, in order to ensure the accurate measurement of the flexible structure, a matching measurement device needs to be introduced. However, the current measurement device is difficult to handle nonlinear errors, affecting the control accuracy of the flexible structure. SUMMARY

[0005] In order to solve the problem of control accuracy of the flexible structure, the present application provides a micro-operation system based on a rigid-flexible combined manipulator and a micro-control method.

[0006] The present application provides a micro-operation system based on a rigid-flexible combined manipulator, comprising:

[0007] The rigid-flexible combined manipulator is used for decoupling operation on the spatial pose of the end tool 8 installed on the rigid-flexible combined manipulator;

[0008] The multi-sensor measurement system is used for real-time acquisition of visual data and strain data of the end tool 8;

[0009] The adaptive control module is used for online estimation and disturbance compensation of time-varying parameters of the rigid-flexible combined manipulator according to the data obtained by the multi-sensor measurement system, and control of the spatial pose of the end tool 8.

[0010] In one possible design, the spatial pose of the end tool 8 of the present application includes spatial position and spatial pose, and the rigid-flexible combined manipulator comprises:

[0011] Cartesian robot 6, including XYZ three-axis orthogonal arrangement of lead screw sliding table, used for compensating adjustment of spatial position of end tool 8;

[0012] Flexible robot 5, including concentric push-pull flexible body 5-9, flexible robot 5 is connected with Z-axis sliding table of Cartesian robot 6, and the spatial posture of end tool 8 is adjusted by driving concentric push-pull flexible body 5-9; wherein, end tool 8 is specifically installed on flexible robot 5.

[0013] In a possible design, the flexible robot 5 of the application realizes single rotation degree of freedom of the concentric push-pull flexible body 5-9 through a two-stage stretching design, and the concentric push-pull flexible body 5-9 is attached with a double-row staggered strain gauge array, and the strain gauge array is used for measuring the curvature of the concentric push-pull flexible body 5-9.

[0014] In a possible design, the multi-sensor measurement system of the application includes:

[0015] Microscopic camera 2, through sub-micron level image acquisition, extracting visual information of biological objects and end tool 8 in XY plane;

[0016] Lateral camera 7, through sub-millimeter level image acquisition, extracting Z-axis displacement information and the shape of the concentric push-pull flexible body 5-9.

[0017] In a possible design, the adaptive control module of the application includes:

[0018] Lower machine main control board, running asynchronous Kalman filtering algorithm, fusing visual data and strain data to obtain spatial posture of end tool 8;

[0019] Adaptive robust quasi-static controller, according to the data obtained by the multi-sensor measurement system, online estimating the time-varying parameters of the rigid-flexible combined robot and compensating the disturbance, and controlling the spatial posture of the end tool 8.

[0020] In a possible design, the concentric push-pull flexible body 5-9 of the application includes an outer concentric tube 5-9-1 and an inner concentric tube 5-9-3 arranged in the outer concentric tube 5-9-1, and a clamping section 5-9-4 is arranged at the leading end of the inner concentric tube 5-9-3, the clamping section 5-9-4 is inserted into the inner tube clamp 5-8 for self-centering fixation; the leading end of the outer concentric tube 5-9-1 is provided with a fixed terminal 5-9-2, and the fixed terminal 5-9-2 is locked to the outer concentric tube base 5-10 through a screw; the back of the outer concentric tube 5-9-1 is attached with a double-row staggered strain gauge array.

[0021] In another possible design, the flexible robot 5 of the application further includes a fixed platform 5-1, a T1 moving table, a T2 moving table, an R-axis rotating motor 5-6, a rotating frame 5-7 and an inner tube clamp 5-8;

[0022] The fixed platform 5-1 is fixed to the Z-axis sliding table of the Cartesian robot arm 6;

[0023] The T1 moving table includes a T1 sliding table 5-2 and a T1 motor 5-3. The base of the T1 sliding table 5-2 is fixed to the fixed platform 5-1. The sliding block of the T1 sliding table 5-2 is driven by the T1 motor 5-3 to move linearly along the T1 axis, realizing primary driving.

[0024] The T2 moving table includes a T2 sliding table 5-4 and a T2 motor 5-5. The base of the T2 sliding table 5-4 is rigidly connected to the T1 sliding block by bolts. The sliding block of the T2 sliding table 5-4 is driven by the T2 motor 5-5 to move linearly along the T2 axis, realizing secondary driving.

[0025] The base of the R-axis rotary motor 5-6 is fixed to the sliding block of the T2 sliding table 5-4. The leading end of the rotary frame 5-7 is coaxially fixed to the rotor of the R-axis rotary motor 5-6. The rotary frame 5-7 is provided with a clamping groove type outer concentric tube base 5-10 at the trailing end, and is provided with a self-centering inner tube chuck 5-8 at the inner side. The rotary frame 5-7 rotates around the R-axis by the R-axis rotary motor, adjusting the pointing angle of the end tool 8.

[0026] The T1 axis, the T2 axis and the R-axis are parallel. The R-axis drives the end tool 8 to rotate around the axis, and the T1 axis and / or the T2 axis controls the bending angle of the concentric push-pull flexible body 5-9 by pushing and pulling.

[0027] In a possible design, the main body part of the outer concentric tube 5-9-1 and the inner concentric tube 5-9-3 of the concentric push-pull flexible body 5-9 is a three-section design, which is composed of three continuous bending sections, and is divided into a first section, a second section and a third section from the leading end to the trailing end. The notches of each section are asymmetrically cut into the opposite directions of the tube wall.

[0028] The T1 and / or T2 axis secondary driving pushes and pulls the outer concentric tube 5-9-1 to control the first section and the second section to bend forward and the third section to bend reversely. The end tool 8 is adjusted to the target angle by the bending of the T1 / T2 axis push-pull concentric push-pull flexible body 5-9, and the direction of the end tool 8 is fine-tuned by the R-axis rotation.

[0029] In a possible design, the cross section of the outer concentric tube 5-9-1 and the inner concentric tube 5-9-3 is a polygon containing more than 4 sides.

[0030] The application also provides a micro-control method of a micro-operation system based on a rigid-flexible combined robot arm, which comprises:

[0031] The rigid-flexible combined robot arm is used to perform spatial pose decoupling operation on the end tool 8;

[0032] Real-time collection of visual data and strain data of the end tool 8 by a multi-sensor measurement system;

[0033] Online estimation of time-varying parameters of the rigid-flexible combined mechanical arm and compensation of disturbances according to the data obtained by the multi-sensor measurement system by using an adaptive control module, and control of the spatial pose of the end tool 8.

[0034] The beneficial effects of the present application: the present application realizes the spatial pose decoupling of the end tool through the rigid-flexible combined mechanical arm structure, combines the real-time collection of visual data and strain data by the multi-sensor measurement system, and then controls the spatial pose of the end tool by online estimation of time-varying parameters and compensation of detours through the adaptive control module, solves the problems of poor repeatability positioning accuracy of flexible structure, serious nonlinear disturbance, etc. Specifically, the rigid-flexible combined mechanical arm structure can be composed of a Cartesian mechanical arm + thermoplastic urethane (TPU) three-bending segment flexible body, the multi-sensor measurement system can be realized by dual-camera vision + strain gauge array, and the adaptive control module can be realized by recursive least squares (RLS) online estimation + periodic compensation through robust control, solving the problems of poor repeatability positioning accuracy of flexible body (x-axis displacement compression 97.32%), serious nonlinear disturbance, etc. It is suitable for precise operations such as microinjection and cell manipulation in organoid culture. Specifically, the following points are included:

[0035] 1. Use thermoplastic urethane elastomer material to make rectangular cross-section concentric push-pull structure (CPPR), solve the problems of shape limitation and short service life of existing process, and because the bending part is symmetrical, the deformation process has smaller uncertainty.

[0036] 2. Design a two-stage stretching structure for the flexible structure, obtain a single rotational degree of freedom at the end of the flexible segment through redundant driving. Combined with rotary motor and orthogonal linear mechanical arm, precise spatial pose control is realized.

[0037] 3. Design a double-row multi-piece strain gauge array, realize variable curvature measurement through multi-point sampling, and improve the accuracy of variable curvature structure measurement. Use sub-pixel processing technology for macro camera image to obtain millimeter level positioning accuracy. Use nonlinear data fusion technology to obtain flexible segment characteristics considering accuracy and cutoff frequency.

[0038] 4. Design an adaptive robust quasi-static controller, accurately online estimate the machining uncertainty and time-varying parameters of the flexible segment, and realize synchronous high-precision control combined with kinematic priori knowledge. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1is a schematic diagram of an overall structure of a micromanipulation system based on a rigid-flexible combined mechanical arm provided in the present application;

[0040] Figure 2 is a schematic diagram of a structure of a rigid-flexible combined mechanical arm related in the present application;

[0041] Figure 3 is a schematic diagram of a concentric push-pull flexible body with an inner tube and an outer tube assembled together related in the present application;

[0042] Figure 4 is a schematic diagram of a concentric push-pull flexible body with an inner tube and an outer tube not assembled together related in the present application;

[0043] Figure 5 is a schematic diagram of a strain gauge array coordinate of a concentric push-pull flexible body related in the present application;

[0044] Figure 6 is a schematic diagram of bending of a concentric push-pull flexible body when pushed and pulled related in the present application;

[0045] Figure 7 is an enlarged view of an inner tube and an outer tube of a concentric push-pull flexible body related in the present application;

[0046] Figure 8 is a bending principle diagram of a concentric push-pull flexible body related in the present application;

[0047] Figure 9 is a simulation diagram of a bending process of a concentric push-pull flexible body related in the present application;

[0048] Figure 10 is a system control block diagram related in the present application;

[0049] Figure 11 is a flowchart of a micromanipulation control method of a micromanipulation system based on a rigid-flexible combined mechanical arm provided in the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0051] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0052] The present application will be further described below with reference to the drawings and specific embodiments, but is not limited by the present application.

[0053] As a new type of flexible mechanical arm structure, concentric push-pull structure (CPPR) can flexibly adjust the end position by pulling two laterally open flexible tubes. The CPPR has simple structure, good motion stability, high lateral stiffness, and strong miniaturization potential. The posture of the operation object is adjusted by the CPPR, the position is adjusted by the linear orthogonal mechanical arm, and the rigid-flexible combined structure mechanical arm is composed to realize high-precision operation. The single-stage bending structure used in the existing concentric push-pull structure has an error displacement of nearly 20 mm in the x-axis, and the positioning accuracy is poor.

[0054] To ensure accurate measurement of the flexible structure, a matching measurement device needs to be introduced. Existing methods mainly include visual acquisition scheme and curvature sensor scheme. The former uses a macro industrial camera to take pictures, which can obtain diversified elements, but the sampling frequency is low, and it is difficult to achieve high-precision measurement due to the limitation of pixel width. The latter has higher measurement accuracy and cutoff frequency, but is limited by fixed curvature measurement and installation difficulty, and is difficult to handle nonlinear errors. How to fuse the data of the two is a major focus of current research.

[0055] In view of this, the embodiment of the present application provides a micro-operation system based on a rigid-flexible combined mechanical arm, comprising

[0056] The rigid-flexible combined mechanical arm is used for decoupling operation on the spatial pose of the end tool 8 installed on the rigid-flexible combined mechanical arm;

[0057] The multi-sensor measurement system is used for real-time acquisition of visual data and strain data of the end tool 8;

[0058] The adaptive control module is used for online estimation of time-varying parameters of the rigid-flexible combined mechanical arm and compensation of disturbances according to the data obtained by the multi-sensor measurement system, and control of the spatial pose of the end tool 8.

[0059] The embodiment realizes decoupling operation on the spatial pose of the end tool 8 through the rigid-flexible combined mechanical arm, improves the flexibility and accuracy of the system, the multi-sensor measurement system realizes real-time acquisition of visual data and strain data, enhances the feedback capability of the system, and the adaptive control module can online estimate time-varying parameters and compensate disturbances, which significantly improves the stability and anti-interference capability of the system.

[0060] In some embodiments, the spatial pose of the end tool 8 of the embodiment includes spatial position and spatial attitude, and the rigid-flexible combined mechanical arm comprises:

[0061] The Cartesian mechanical arm 6 comprises a three-axis orthogonal arrangement of a lead screw sliding table, and is used for compensating adjustment of the spatial position of the end tool 8;

[0062] The flexible mechanical arm 5 includes a concentric push-pull flexible body 5-9, and is connected with a Z-axis sliding table of the Cartesian mechanical arm 6, and the spatial posture of an end tool 8 is adjusted by driving the concentric push-pull flexible body 5-9; wherein the end tool 8 is specifically mounted on the flexible mechanical arm 5.

[0063] In the embodiment, the Cartesian mechanical arm 6 realizes accurate compensation adjustment of the spatial position of the end tool 8 through an XYZ three-axis orthogonal lead screw sliding table, and the flexible mechanical arm 5 realizes independent control of the position and the posture through the concentric push-pull flexible body 5-9, thereby simplifying the operation process and improving the positioning accuracy.

[0064] In some embodiments, the flexible mechanical arm 5 realizes single rotation degree of freedom of the concentric push-pull flexible body 5-9 through a two-stage stretching design, and a double-row staggered strain gauge array is attached to the concentric push-pull flexible body 5-9, and the strain gauge array is used for measuring the curvature of the concentric push-pull flexible body.

[0065] The two-stage stretching design in the embodiment enables the concentric push-pull flexible body 5-9 to complete posture adjustment only through single rotation degree of freedom, thereby reducing the control complexity; and the double-row staggered strain gauge array can accurately measure the curvature of the flexible body, thereby providing high-precision feedback data for adaptive control.

[0066] In some embodiments, the concentric push-pull flexible body 5-9 includes an outer concentric tube 5-9-1 and an inner concentric tube 5-9-3 which are sleeved, a clamping section 5-9-4 is arranged at the leading end of the inner concentric tube 5-9-3, the clamping section 5-9-4 is inserted into an inner tube chuck 5-8 for self-centering fixation, a fixed terminal 5-9-2 is arranged at the leading end of the outer concentric tube 5-9-1, and the fixed terminal 5-9-2 is locked to an outer concentric tube base 5-10 through a screw, and a double-row staggered strain gauge array is attached to the back of the outer concentric tube 5-9-1.

[0067] In the embodiment, the design of the inner and outer concentric tubes realizes self-centering fixation through the clamping section 5-9-4 and the fixed terminal 5-9-2, thereby improving the assembly accuracy and stability; and the double-row staggered strain gauge array is attached to the back of the outer concentric tube 5-9-1, thereby enhancing the sensitivity and reliability of the strain measurement.

[0068] In still some embodiments, the flexible mechanical arm 5 further includes a fixed platform 5-1, a T1 moving table, a T2 moving table, an R-axis rotating motor 5-6, a rotating frame 5-7 and the inner tube chuck 5-8.

[0069] The fixed platform 5-1 is fixed to the Z-axis sliding table of the Cartesian mechanical arm 6.

[0070] Fixed platform 5-1, yellow component, fixed on the Z-axis slide table of the Cartesian robot arm 6; the inclination angle can be manually adjusted. It is the base of the whole flexible robot arm 5, which is made of nylon material by 3D printing.

[0071] The T1 mobile station (blue component) includes a T1 slide table 5-2 and a T1 motor 5-3. The base of the T1 slide table 5-2 is fixed on the fixed platform 5-1, and the slider of the T1 slide table 5-2 is driven by the T1 motor 5-3 to move linearly along the T1 axis to realize primary driving. It is made of nylon material by 3D printing. A tension spring is installed between the T1 and T2 slide tables and the slider to eliminate backlash, and a limit switch is installed at the end of the stroke.

[0072] The T2 mobile station (red component) includes a T2 slide table 5-4 and a T2 motor 5-5. The base of the T2 slide table 5-4 is rigidly connected to the T1 slider by bolts, and the slider of the T2 slide table 5-4 is driven by the T2 motor 5-5 to move linearly along the T12 axis to realize secondary driving. It is made of nylon material by 3D printing.

[0073] A tension spring is installed between the T1 and T2 slide tables and the slider to eliminate backlash, and a limit switch is installed at the end of the stroke.

[0074] The base of the R-axis rotary motor 5-6 is fixed on the slider of the T2 slide table 5-4, and the leading end of the rotary frame 5-7 is coaxially fixed on the rotor of the R-axis rotary motor 5-6. The rotary frame 5-7 is provided with a clamping groove type outer concentric tube base 5-10 at the end, and is provided with a self-centering inner tube chuck 5-8 on the inner side. The rotary frame 5-7 rotates around the R-axis by the R-axis rotary motor to adjust the pointing angle of the end tool 8. The rotary frame 5-7 is made of aluminum alloy machining parts.

[0075] The axes of the T1 axis, the T2 axis and the R-axis are parallel. The R-axis drives the end tool 8 to rotate around the axis, and the T1 axis and / or the T2 axis controls the bending angle of the concentric push-pull flexible body 5-9 by pushing and pulling.

[0076] The end of the concentric push-pull flexible body 5-9 is provided with an end tool 8. Specifically, the end tool 8 adopts a membrane clamp, and is fixed by α-cyanoethyl acrylate.

[0077] In this embodiment, the T1 and T2 mobile stations realize the push-pull operation of the concentric push-pull flexible body 5-9 through secondary driving, and the pointing angle of the end tool 8 is adjusted by the R-axis rotary motor 5-6, realizing the cooperative control of multiple degrees of freedom and improving the flexibility and operation range of the system.

[0078] A specific embodiment is given in combination with the above embodiments, which is described below with reference to Figure 1The other components of the platform include a hollow stage 3 to hold biological objects and operating environment (such as a petri dish), a parallel light source 4 to provide adjustable color temperature and intensity light for the camera, and a precision guide rail 1 to hold and adjust the position of the components to adapt to different types of operating objects.

[0079] For the rear flexible body driving mechanism, the application has three independent actuators: T1 motor 5-3, T2 motor 5-5, and R-axis rotary motor 5-6. T1 and T2 are linear degrees of freedom, and the moving axis and the R-axis are parallel to each other. The three actuator operation axes of the Cartesian robot 6 are arranged orthogonally. The included angle between the R-axis and the Z-axis can be manually adjusted.

[0080] Degrees of freedom Stroke Sensor resolution Rated torque Maximum operating speed x / y / z ±10 mm 0.0265 um 0.6 Nm 1.5 mm / s T1 / T2 35 mm 0.24 um 0.45 Nm 2 mm / s R ±360° 0.011° 0.55 Nm 26.17 rad / s

[0081] In some embodiments, the main body of the outer concentric tube 5-9-1 and the inner concentric tube 5-9-3 of the concentric push-pull flexible body 5-9 is a three-segment design, composed of three consecutive curved segments, divided into a first segment, a second segment, and a third segment from the first end to the end, and the notches of each segment are asymmetrically cut into the opposite directions of the tube wall.

[0082] The T1 and / or T2 axis secondary drive push-pull outer concentric tube 5-9-1 controls the first segment and the second segment to bend forward, and the third segment to bend backward. By pushing and pulling the concentric push-pull flexible body 5-9 through the T1 / T2 axis, the end tool 8 is adjusted to the target angle, and then the R-axis rotary motor 5-6 is used to fine-tune the direction of the end tool 8.

[0083] The notches of the concentric tubes are asymmetrically cut into their sides in opposite directions. The tips of the two tubes are connected to each other, and the push-pull translation of the tube base relative to each other changes the curvature along the length of the combined tube. By changing the notch parameters along the tube, a custom variable curvature shape can be achieved.

[0084] The working space required for a general 3D printer to process a single-bend-segment concentric push-pull flexible body (outer diameter above 5mm) to achieve an effective and stable 90° bend is generally significantly larger than the stroke of mainstream Cartesian robot in the micro-operation field (±6-10mm), which greatly limits the practicality of concentric push-pull structures in the micro-operation field. To solve this problem, the application proposes a three-bend-segment concentric tube structure, which greatly reduces the displacement of the membrane clamp end in the x-axis. Using the stretching degree of freedom and an additional linear degree of freedom, reciprocating injection and fixed-point rotation of the end gripper tip are achieved. See Figures 3-7 The three-segment parameters of the concentric push-pull flexible body 5-9 are:

[0085] First section: groove number = 7, total length = 23 mm, outer tube outer diameter = 6 mm, inner tube outer diameter = 3.7 mm, outer tube inner diameter = 4 mm, inner tube inner diameter = 2.2 mm, groove width = 1.5 mm, tooth width = 2 mm;

[0086] Second section: groove number = 7, total length = 23 mm, outer tube outer diameter = 6 mm, inner tube outer diameter = 3.7 mm, outer tube inner diameter = 4 mm, inner tube inner diameter = 2.2 mm, groove width = 1.5 mm, tooth width = 2 mm;

[0087] Third section: groove number = 11, total length = 37 mm, outer tube outer diameter = 6 mm, inner tube outer diameter = 3.7 mm, outer tube inner diameter = 4 mm, inner tube inner diameter = 2.2 mm, groove width = 1.5 mm, tooth width = 2 mm.

[0088] Simulation is performed using matlab, when the tip is rotated from 0° to 90°, the end stretch is 3mm, the maximum displacement of x-axis is 0.428mm, and the stroke of z-axis is 17mm. The displacement of single-stage bending structure in x-axis is 19.953m, and the compression ratio of the structure to x-axis displacement is about 97.32%.

[0089] Since a strain gauge can only feedback a single curvature, in order to realize the variable curvature measurement of the flexible body, an array composed of multiple strain gauges needs to be deployed. In order to fully utilize the space and realize the maximum measurement resolution, two rows of strain gauges are deployed in a staggered manner, and cubic spline interpolation is used to estimate the overall curvature.

[0090] The strain gauge array is attached to the back of the outer tube along the length direction of the push-pull flexible body 5-9, and there are a total of two rows. The two rows of strain gauges are arranged in a staggered manner along the length y-axis direction of the flexible body with a spacing of 0.5b, and are distributed in an alternating manner in the width x-axis direction with an offset of 0.5a, where a is the width of the strain gauge and b is the length of the strain gauge.

[0091] The strain gauge array is used to collect multi-point strain data of the flexible body and calculate the continuous curvature of the flexible body through cubic spline interpolation, and then the coordinates of the end tool 8 are obtained. The specific process includes the following steps:

[0092] Step A1, divide the flexible body into n sections according to the distribution of the strain gauge array, n is the number of strain gauges minus 1, the number of strain gauges in the array is k and k+1 respectively, which satisfies the relationship: n = 2k, k = 3-8; the interpolation node coordinates are the position coordinates of n+1 strain gauges:

[0093]

[0094] When k = 4, the strain gauge array is shown in FIG. 4, wherein the upper row has 4 strain gauges and the lower row has 5 strain gauges. The coordinates of the upper left corner of each strain gauge are: Figure 7 ​

[0095]

[0096] Step A2, [y j ,y j+1 ] interval to construct a cubic spline interpolation function:

[0097] S j (y)=a j +b j (y-y j )+c j (y-y j ) 2 +d j (y-y j ) 3

[0098] j is the spline segment index, j=0,1,…,n, a j , b j , c j , d j are the coefficients to be solved:

[0099] The constraint conditions are:

[0100] The function value continuity constraint condition: S j (y j+1 )=S j+1 (y j+1 ), S j (y j+1 ) represents the end point of the jth segment of the spline curve, and S j+1 (y j+1 ) represents the starting point of the j+1th segment of the spline curve.

[0101] The first derivative continuity constraint condition: S j ′(y j+1 )=S j+1 ′(y j+1 ), S j ′(y j+1 ) represents the first derivative of the end point of the jth segment of the spline curve, and S j+1 ′(y k+1 ) represents the first derivative of the starting point of the j+1th segment of the spline curve.

[0102] The second derivative continuity constraint condition: S j ″(y j+1 )=S j+1 ″(y j+1 ), S j ″(y j+1 ) represents the second derivative of the end point of the jth segment of the spline curve, and S j+1 ″(y j+1) represents the second derivative of the start of the j+1 segment spline curve;

[0103] Natural boundary condition: S"(y0) = S"(y n

[0104] Step A3, solve the equation of step A2 to obtain the continuous curvature of the outer tube of the flexible body, and then obtain the curvature of the flexible body axis;

[0105] Step A4, according to the curvature of the flexible body axis, the spatial pose of the end tool 8 in the base coordinate system of the robot arm is calculated. According to the curvature of the flexible body axis, the complete spatial coordinates of the flexible body in the end coordinate system are obtained by using the fourth order Runge-Kutta integral. The homogeneous equation formed by the encoder results of the remaining degrees of freedom is multiplied to obtain the spatial coordinates of the flexible body in the base coordinate system of the robot arm.

[0106] The three-segment concentric push-pull flexible body 5-9 of the embodiment is segmented and bent by asymmetric notches, and the T1 / T2 axis push-pull and the R axis rotation work cooperatively to accurately adjust the angle and direction of the end tool 8, and meet the complex operation requirements.

[0107] In some embodiments, the cross section of the outer concentric tube 5-9-1 and the inner concentric tube 5-9-3 is a polygon containing more than 4 sides. The outer concentric tube 5-9-1 and the inner concentric tube 5-9-3 are made of thermoplastic polyurethane (TPU) material by horizontal 3D printing. The cross section of the outer concentric tube 5-9-1 and the inner concentric tube 5-9-3 is rectangular (quadrilateral).

[0108] The polygonal cross section of the inner and outer concentric tubes of the embodiment enhances the rigidity and torsional performance of the structure, avoids the sliding problem of the circular cross section tube during the push-pull process, and further improves the stability and control accuracy of the system.

[0109] In some embodiments, the multi-sensor measurement system of the embodiment includes:

[0110] The microscopic camera 2 extracts the visual information of the biological object and the end tool 8 in the XY plane by sub-micron level image acquisition;

[0111] The lateral camera 7 extracts the shape of the concentric push-pull flexible body 5-9 and the Z-axis displacement information by sub-millimeter level image acquisition.

[0112] In the embodiment, the microscopic camera provides sub-micron level image acquisition to ensure accurate positioning of the biological object and the end tool 8 in the XY plane; the lateral camera 2 provides sub-millimeter level image acquisition to monitor the shape of the flexible body and the Z-axis displacement in real time, realizing multi-dimensional high-precision measurement.

[0113] In some embodiments, the adaptive control module of the embodiment includes:

[0114] ​The lower computer main control board runs an asynchronous Kalman filter algorithm, fuses visual data and strain data to obtain the spatial pose of the end tool 8;

[0115] The adaptive robust quasi-static controller performs online estimation on the time-varying parameters of the rigid-flexible combined robot arm according to the data obtained by the multi-sensor measurement system, compensates disturbances, and controls the spatial pose of the end tool 8.

[0116] The process of running the asynchronous Kalman filter algorithm to fuse visual data and sensor data to obtain the spatial position of the end tool 8 includes the following steps:

[0117] Step B1, mechanical arm sensor coordinate unification:

[0118] The motor encoder data of the flexible robot arm 5 is converted into a homogeneous coordinate system to obtain the membrane clamp tip position in the robot base coordinate system; the motor encoder data refers to the T1 axis, T2 axis and R axis position;

[0119] According to the continuous curvature of the flexible body measured by the strain gauge array, the offset of the membrane clamp tip position is compensated, and the compensated data is used as the feedback of the robot arm;

[0120] Step B2, after processing the images collected by the microscope camera 2 and the lateral camera 7, the pixel coordinates are obtained, the microscope camera 2 image coordinates are multiplied by the visual Jacobian matrix to obtain the XY plane robot coordinates, the lateral camera 7 image coordinates are multiplied by the visual Jacobian matrix to obtain the Z axis coordinates, and then the membrane clamp tip position under visual fusion is obtained as the visual feedback; the information of the overlapping axis of the two image positioning is fused using weighting;

[0121] Step B3, a linear Kalman filter using a second-order integral model is deployed in the motion control thread, and only the feedback of the robot arm is used as the state variable for iteration in the control period without updating the visual data; when the visual data is updated, the visual feedback is fused to correct the state, ensuring continuous pose feedback. In this step, when the visual data is updated, the weight matrix of the filter is augmented to include the visual feedback, achieving the purpose of continuous feedback.

[0122] Due to the less external disturbance and single motion form in the micro-operation environment, many experimental operations can be regarded as periodic repetition of single action. The flexible robot arm has obvious nonlinear disturbances such as hysteresis, and can be regarded as having periodicity in the same stretching input motion process. In addition, due to the large processing error of TPU material, the change of material properties with time, and other problems, the parameters of flexible arms of different batches and time also have parameter differences.

[0123] To address this series of problems, this embodiment proposes a feedback controller with periodic compensation to perform online estimation of parameters and disturbances. Figure 10 ,By estimating the flexible body parameters and disturbances online, and combining the adaptive ,controller closed-loop controller with periodic iterative compensation, the process of achieving ,high-precision posture control includes the following steps:

[0124] Step C1: Construct the kinematic equation for the patch clamp tip:

[0125]

[0126] Where, p=[p x ,p y ,p z ]、 are the position coordinates and linear velocity of the patch clamp tip in the world coordinate system, respectively;

[0127] r and are the R-axis motor angle and speed respectively;

[0128] m=[m x ,m y ,m z ]、 are the position coordinates and moving speed of the Cartesian manipulator 6 in the world coordinate system respectively;

[0129] d=[d x ,d y ,d z ] is the unmodeled disturbance term, including periodic disturbance and non-periodic disturbance; this reference is to be estimated, and the estimation is completed by step C3;

[0130] θ i =[θ i,1 ,θ i,2 ] T , i=x, y, z are the parameters to be estimated; the estimation is completed by step C2;

[0131] Step C2: Parameter θ to be estimated i Use the saturation projection adaptive law to estimate online:

[0132]

[0133] Where sat function represents the limiting function, represents the projection operator, is the predefined upper limit of the parameter update rate, Γ is the adaptive gain matrix, ζ is the regression vector,

[0134] Step C3, using a nonlinear disturbance observer to estimate the unmodeled disturbance term;

[0135] The step C1 kinematics equation is converted into matrix standard form and a nonlinear disturbance observer z is established:

[0136]

[0137] Wherein, is the first derivative of z, L is a positive definite diagonal gain matrix, f θ (r) is a rotation coupling matrix,

[0138] Wherein, p(r, m) is a design function, p(r, m) = L -1 (r·f θ (r) + m); is the disturbance to be estimated, the disturbance estimated by the nonlinear disturbance observer is recorded into the storage queue;

[0139] Step C4, in the periodic motion, the disturbance estimated in the last period is superimposed on the user input p f to generate the controller desired pose p d ; p d is input into the subsequent kinematics solution and closed-loop feedback controller;

[0140] Step C5, based on the parameters θ i estimated in step C2, the inverse kinematics is solved in real time to generate the coarse adjustment instruction u a for the manipulator.

[0141] In this step, the inverse kinematics is completed based on the kinematics equation in step C1, and the inverse kinematics is to calculate the motor execution instruction from the known membrane patch probe tip coordinates. The θ i in the inverse kinematics in this step comes from the online estimation in step C2.

[0142] Step C6, a feedback controller is used to generate the fine adjustment instruction u b for the manipulator; the feedback controller adopts PD controller and sliding mode controller to jointly perform feedback control to compensate for the actual pose; the PD controller is adjusted to achieve the desired cutoff frequency and damping ratio; the sliding mode controller is used to eliminate high-frequency nonlinear disturbances and relative kinematics model deviation from step C4, and the output value of the feedback controller is u b .

[0143] Step C7, the instructions u = u a + u b are output to each motor in the rigid-flexible combined manipulator.

[0144] The asynchronous Kalman filtering algorithm is adopted to fuse multi-sensor data, so as to improve the accuracy and real-time performance of the pose estimation of the end tool 8; the adaptive robust quasi-static controller can effectively compensate the disturbance, and ensure the stability and robustness of the system in a complex environment.

[0145] Referring to Figure 11 The micro-control method for the above micro-operation system comprises the following steps:

[0146] The spatial pose of the end tool 8 is decoupled by using the rigid-flexible combined manipulator;

[0147] The visual data and strain data of the end tool 8 are collected in real time by using the multi-sensor measurement system;

[0148] The adaptive control module is used to perform online estimation and disturbance compensation on the time-varying parameters of the rigid-flexible combined manipulator according to the data obtained by the multi-sensor measurement system, and to control the spatial pose of the end tool 8.

[0149] The cooperative work of the rigid-flexible combined manipulator, the multi-sensor measurement system and the adaptive control module realizes the high-precision decoupling control of the spatial pose of the end tool 8, and is suitable for micro-operation and other scenes with extremely high precision requirements, so as to significantly improve the operation efficiency and success rate.

[0150] Although the present application is described herein with reference to particular embodiments, it should be understood that these examples are merely illustrative of the principles and applications of the present application. It should therefore be understood that numerous modifications can be made to the illustrative embodiments and that other arrangements can be devised without departing from the spirit and scope of the present application as defined by the appended claims. It should be understood that the features described in connection with one embodiment can be used in conjunction with other embodiments described herein. It should also be understood that features described in connection with separate embodiments can be used together.

Claims

1. A micromanipulation system based on a rigid-flexible combined robotic arm, characterized in that: include: A rigid-flexible combined robotic arm is used to decouple the spatial posture of an end tool (8) mounted on the rigid-flexible combined robotic arm; A multi-sensor measurement system for collecting visual data and strain data of the end tool (8) in real time; The adaptive control module is used to estimate the time-varying parameters of the rigid-flexible combined manipulator online and compensate for disturbances based on data obtained by the multi-sensor measurement system, and to control the spatial posture of the end tool (8).

2. The micromanipulation system based on a rigid-flexible combined robotic arm according to claim 1, characterized in that: The spatial posture of the end tool (8) includes a spatial position and a spatial posture, and the rigid-flexible combined mechanical arm includes: A Cartesian manipulator (6) includes a screw slide arranged orthogonally in three axes (X, Y, and Z) for compensating and adjusting the spatial position of an end tool (8); The flexible robotic arm (5) includes a concentric push-pull flexible body (5-9). The flexible robotic arm (5) is connected to the Z-axis slide of the Cartesian robotic arm (6) and adjusts the spatial posture of the end tool (8) by driving the concentric push-pull flexible body (5-9). The end tool (8) is specifically installed on the flexible robotic arm (5).

3. The micromanipulation system based on a rigid-flexible combined robotic arm according to claim 2, characterized in that: The flexible robotic arm (5) realizes a single rotational degree of freedom of a concentric push-pull flexible body (5-9) through a secondary stretching design. The concentric push-pull flexible body (5-9) is mounted with a double-row offset strain gauge array, and the strain gauge array is used to measure the curvature of the concentric push-pull flexible body (5-9).

4. The micromanipulation system based on a rigid-flexible combined robotic arm according to claim 2, characterized in that: The multi-sensor measurement system comprises: A microscopic camera (2) extracts visual information of the biological object and the end tool (8) in the XY plane through submicron-level image acquisition; The lateral camera (7) extracts the morphology and Z-axis displacement information of the concentric push-pull flexible body (5-9) through sub-millimeter image acquisition.

5. The micromanipulation system based on a rigid-flexible combined robotic arm according to claim 2, characterized in that: The adaptive control module includes: The lower computer main control board runs an asynchronous Kalman filter algorithm to fuse the visual data and strain data to obtain the spatial posture of the end tool (8); An adaptive robust quasi-static controller estimates the time-varying parameters of the rigid-flexible combined manipulator online and compensates for disturbances based on data obtained by a multi-sensor measurement system, thereby controlling the spatial posture of the end tool (8).

6. The micromanipulation system based on a rigid-flexible combined robotic arm according to claim 2, characterized in that: The concentric push-pull flexible body (5-9) comprises an outer concentric tube (5-9-1) and an inner concentric tube (5-9-3) arranged inside and outside. The head end of the inner concentric tube (5-9-3) is provided with a clamping section (5-9-4), and the clamping section (5-9-4) is inserted into the inner tube clamp (5-8) for self-centering fixation; the head end of the outer concentric tube (5-9-1) is provided with a fixed terminal (5-9-2), and the fixed terminal (5-9-2) is locked to the outer concentric tube base (5-10) by screws; and a double-row offset strain gauge array is mounted on the back of the outer concentric tube (5-9-1).

7. The micromanipulation system based on a rigid-flexible combined robotic arm according to claim 6, characterized in that: The flexible robotic arm (5) further comprises a fixed platform (5-1), a T1 moving platform, a T2 moving platform, an R-axis rotating motor (5-6), a rotating frame (5-7) and an inner tube chuck (5-8); A fixed platform (5-1) is fixed on the Z-axis slide of the Cartesian robot arm (6); The T1 moving platform includes a T1 slide (5-2) and a T1 motor (5-3). The base of the T1 slide (5-2) is fixed to the fixed platform (5-1). The slider of the T1 slide (5-2) is driven by the T1 motor (5-3) through a screw rod to move linearly along the T1 axis, thereby realizing a first-level drive. The T2 moving platform includes a T2 slide (5-4) and a T2 motor (5-5). The base of the T2 slide (5-4) is rigidly connected to the T1 slider by bolts. The slider of the T2 slide (5-4) is driven by the T2 motor (5-5) through a screw rod to perform linear motion along the T2 axis, thereby realizing a two-stage drive. The base of the R-axis rotary motor (5-6) is fixed to the slider of the T2 slide (5-4), and the head end of the rotary frame (5-7) is coaxially fixed to the rotor of the R-axis rotary motor (5-6); a slot-type outer concentric tube base (5-10) is provided at the end of the rotary frame (5-7), and a self-centering inner tube chuck (5-8) is provided on the inner side of the rotary frame (5-7); the rotary frame (5-7) is rotated around the R-axis by the R-axis rotary motor to adjust the pointing angle of the end tool (8); The axes of the T1 axis, the T2 axis and the R axis are parallel; the R axis drives the end tool (8) to rotate around the axis, and the T1 axis and / or the T2 axis control their bending angles by pushing and pulling the concentric push-pull flexible body (5-9).

8. The micromanipulation system based on a rigid-flexible combined robotic arm according to claim 7, characterized in that: The main body of the outer concentric tube (5-9-1) and the inner concentric tube (5-9-3) of the concentric push-pull flexible body (5-9) is a three-section design, consisting of three continuous curved sections, which are divided into the first section, the second section, and the third section from the beginning to the end, and the notches of each section are asymmetrically cut into the tube wall in opposite directions; The T1 and / or T2 axis secondary drive push-pull outer concentric tube (5-9-1) controls the first and second sections to bend in the forward direction and the third section to bend in the reverse direction. The end tool (8) is adjusted to the target angle by pushing and pulling the concentric push-pull flexible body (5-9) through the T1 / T2 axis, and the direction of the end tool (8) is fine-tuned by rotating the R axis.

9. The micromanipulation system based on a rigid-flexible combined robotic arm according to claim 6, characterized in that: The cross-sections of the outer concentric tube (5-9-1) and the inner concentric tube (5-9-3) are polygons having more than four sides.

10. A micromanipulation control method for a micromanipulation system based on a rigid-flexible combined robotic arm according to any one of claims 1 to 9, characterized in that: include: Using a rigid-flexible combined robotic arm to perform spatial posture decoupling operations on an end tool (8); Using a multi-sensor measurement system to collect visual data and strain data of the end tool (8) in real time; An adaptive control module is used to estimate the time-varying parameters of the rigid-flexible combined manipulator online and compensate for disturbances based on data obtained by a multi-sensor measurement system, thereby controlling the spatial posture of the end tool (8).

Images