Autonomous three-dimensional tissue ablation control system based on soft robot

Abstract

The invention provides an autonomous three-dimensional tissue ablation control system based on a soft-bodied robot, and relates to the field of soft-bodied robots. In order to realize precise ablation of lesion tissues of a laser ablation system in a narrow in-vivo environment, a laser tissue interaction model is pre-constructed to describe an interaction relationship between blue laser and tissues, so that a robust ablation depth control method is developed; in addition, a self-adaptive laser ablation path tracking method combining a soft robot main body and an OCT imaging system is innovatively provided; finally, a surgical robot system with autonomous and precise three-dimensional laser ablation capability is constructed, and the precision, safety and automation level of a laser ablation operation can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of soft robots, and in particular to an autonomous three-dimensional tissue ablation control system based on a soft robot. Background Art

[0002] Tissue ablation is an important treatment method in the field of tumor treatment. Traditional ablation methods mainly rely on manual operation by doctors and have many disadvantages.

[0003] In related technologies, soft robots are a new type of robot made of soft materials that can adapt to various unstructured environments and interact with humans more safely. Through the flexibility and adaptability of soft robots, a robot motion control method that can freely navigate in narrow and complex natural cavities is developed to ensure that the laser ablation device can accurately reach the designated treatment area. To this end, the researchers introduced the motion control technology of soft robots, and achieved highly flexible motion and precise operation of complex environments and non-uniform targets through nonlinear modeling of flexible structures, dynamic trajectory planning and adaptive control.

[0004] However, in the application of flexible actuators or soft robots, the inherent nonlinear properties of soft materials (including material, geometric and drive nonlinearities) and dynamic uncertainties (such as material hysteresis and environmental coupling) make it impossible to establish accurate models. In addition, the existing technology lacks efficient motion control algorithms, resulting in large path tracking errors and insufficient motion flexibility.

[0005] In addition, in existing laser ablation surgeries, the control of tissue ablation depth mainly relies on preset experimental data and fixed model parameters to determine the laser output power and irradiation time. This open-loop control method cannot dynamically adjust the laser parameters according to the actual response of the tissue during the operation, which can easily lead to insufficient or excessive ablation, affecting the surgical effect and safety. Summary of the invention

[0006] 1. Technical issues to be resolved In view of the deficiencies in the prior art, the present invention provides an autonomous three-dimensional tissue ablation control system based on a soft robot, which solves the technical problems of large path tracking errors and insufficient dynamic control of ablation depth.

[0007] (II) Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: An autonomous three-dimensional tissue ablation control system based on a soft robot, wherein the soft robot comprises a soft robot body, a laser ablation system, and an OCT imaging system; comprising: A construction module for pre-constructing a laser tissue interaction model to describe the interaction relationship between the blue laser and the tissue and determine the target ablation depth of the area to be ablated; A planning module, used for receiving a plurality of two-dimensional ablation points planned by a doctor on an endoscopic image and generating a laser ablation path; A tracking module, used to control the soft robot body to track the laser ablation path; comprising: Establishing a unit for establishing a linearized model of the soft robot body using the Koopman method; A control unit, used to optimize the robot control strategy based on the linearized model of the soft robot body and using the MPC method to ensure the tracking accuracy of the laser ablation path; The ablation module is used to adjust the energy output of the laser ablation system during the path tracking process, combining the interaction between the blue laser and the tissue and the real-time feedback of the OCT imaging system, so that the tissue ablation depth of the to-be-ablated area is consistent with the target ablation depth.

[0008] Preferably, during the laser irradiation of tissue, the state change of the tissue is divided into three stages: light absorption, heat diffusion and tissue damage. The pre-construction method of the laser tissue interaction model is correspondingly: For the light absorption stage, the laser-tissue interaction model is constructed using a dynamic Monte Carlo method; For the heat diffusion stage, the laser-tissue interaction model is constructed using the finite difference element method; For the tissue damage stage, the laser-tissue interaction model was constructed using Arrhenius integration.

[0009] Preferably, the receiving of multiple two-dimensional ablation points planned by the doctor on the endoscopic image and generating a laser ablation path includes: Capturing the endoscopic image including the area to be ablated by a camera and a light source; receiving a plurality of two-dimensional ablation points planned by the doctor on the endoscopic image, and traversing all points on the area to be ablated; The laser ablation path is generated by making the energy distribution of each of the two-dimensional ablation points uniform.

[0010] Preferably, the method of establishing a linear model of the soft robot body using the Koopman method comprises: The state dynamics of the soft robot body are defined using a nonlinear system: Among them, x(t) is the system state of the soft robot body at time t, corresponding to the position of the indicating laser point; is the state dynamics of the soft robot body at time t; u(t) is the control input, corresponding to the air pressure control signal; f(⋅) is the nonlinear dynamic function of the system, which is used to describe the working principle of the pneumatic system; For nonlinear systems , using the Koopman operator Map the system's observation function g(x) from time t to time t+Δt: The state dynamics of the soft robot body are approximated using a linear system: Among them, A is the state transfer matrix, which represents the evolution of the observation function; B is the control input matrix, which represents the influence of the air pressure control signal on the observation function; The least squares method is used to solve A and B to transform the evolution of the system into the following linear state space equation: Among them, g(x(t+1)) represents the observation function of the system at time t+1; The linear state space equation is used as the linearized model of the soft robot body.

[0011] Preferably, the method of optimizing the robot control strategy based on the linearized model of the soft robot body and using the MPC method to ensure the tracking accuracy of the laser ablation path includes: Based on the linearized model of the soft robot body, the future k-time states x(t+k) of the system state x(t) at time t are calculated by the following prediction equation: Among them, g(x(t+k+1)) and g(x(t+k)) are the observation functions of the system at time t+k+1 and t+k respectively; u(t+k) is the control input at time t+k; The deviation of the system state and the energy consumption of the control input are combined to minimize the first cost function J in the next N moments as the objective function of the MPC problem: Where p(t+k) is the predicted position of the laser point at time t+k; p d (t+k) is the expected position of the two-dimensional ablation point on the laser ablation path at time t+k; Q path , R path They are all symmetric positive definite weight matrices in the path tracking process, which control the path tracking error and the cost of the control input respectively. The subscript path represents path tracking; is the norm of the vector; And set the constraints of the MPC problem: Among them, u min 、u max are the minimum and maximum values ​​of the control input respectively; Based on the objective function and constraints of the MPC problem, it is converted into solving the following optimization problem: The objective function is: Where min is the minimization function; J(u) is the first cost function with respect to variable u; The constraints are: By solving the optimization problem, we can obtain the optimal control input sequence u * (t),u * (t+1),…,u * (t+N-1), and select the first optimal control input u * (t) Real-time adjustment of the movement of the soft robot body to ensure the tracking accuracy of the laser ablation path.

[0012] Preferably, in the path tracking process, the energy output of the laser ablation system is adjusted in combination with the interaction between the blue laser and the tissue and the real-time feedback of the OCT imaging system so that the tissue ablation depth of the to-be-ablated area is consistent with the target ablation depth, including: Define the functional mapping relationship between the laser energy output per unit time and the change in tissue depth: Among them, d t is the tissue ablation depth at time t, which is determined by the tissue depth change fed back by the OCT image in real time; the input variable of the system The tissue ablation depth d at time t t And the output energy per unit time u t Composition, Δd t =d t+1 -d t is the change in ablation depth at adjacent moments, is the output variable of the system; f is the mapping function between the laser energy output per unit time and the change in tissue depth; The data is obtained through offline sampling, and the dynamic relationship of tissue depth change is obtained by using Gaussian process regression GPR, and a nonlinear fitting is established: in, They are training data sets of input and output variables obtained from real environments, and it is assumed that the output variables of the system conform to the Gaussian distribution with a mean of 0. ; K(X,X) is the covariance matrix; I is the identity matrix; σ 2 is the variance of Gaussian noise; Given the training data set and the input variables of the system , construct the following joint probability distribution and predict the depth change of the system given any sequence of laser energy input : Among them, K(X,X)+Iσ 2 Add a noise term to the covariance function between the input data set X and itself; Is the covariance function between the input data set X and; For input variables The covariance function between the input data set X; For input variables The covariance function with itself; Based on the dynamic relationship of tissue depth change and the target ablation depth, a loss function is constructed to predict the tissue ablation depth in the future K moments through forward mapping: Among them, J ' is the second cost function; is the mean value of the random variable; d K is the tissue ablation depth in the next K moments; d d is the target ablation depth; Q depth , S depth are all symmetric positive definite weight matrices in ablation depth control, which control the ablation depth control error and the cost of control input respectively. The subscript depth represents ablation depth control; l(d t ) is the process loss function, h(d T ) is the terminal loss function; the superscript T indicates transposition; Minimize the second cost function J ' , determine the optimal tissue ablation depth at each moment in the future K moments , and then obtain a set of optimal energy laser sequences, combined with the depth variation , to set the laser power at the current moment and adjust the energy output of the laser ablation system in real time; The tissue depth change fed back in real time by the OCT image is used to update the dynamic relationship of the tissue depth change, and the above optimization process is repeated until the tissue ablation depth of the to-be-ablated area is consistent with the target ablation depth.

[0013] A storage medium stores a computer program for autonomous three-dimensional tissue ablation control based on a soft robot, wherein the computer program enables a computer to control the autonomous three-dimensional tissue ablation control system as described above to perform an autonomous three-dimensional tissue ablation method.

[0014] An electronic device, comprising: One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, the programs including a program for controlling the autonomous three-dimensional tissue ablation control system as described above to perform an autonomous three-dimensional tissue ablation method.

[0015] (III) Beneficial effects The present invention provides an autonomous three-dimensional tissue ablation control system based on a soft robot. Compared with the prior art, it has the following beneficial effects: In order to achieve precise ablation of diseased tissue in a narrow environment in the body by the laser ablation system, the present invention pre-constructs a laser-tissue interaction model to describe the interaction between the blue laser and the tissue, thereby developing a robust ablation depth control method; in addition, an innovative adaptive laser ablation path tracking method combining a soft robot body with an OCT imaging system is proposed; finally, a surgical robot system with autonomous and precise three-dimensional laser ablation capabilities is constructed, which can significantly improve the accuracy, safety and automation level of laser ablation surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A schematic diagram of the structure of a laser ablation system provided by an embodiment of the present invention; Figure 2 An integrated schematic diagram of an OCT imaging optical fiber, a laser ablation optical fiber, and an indicator laser optical fiber provided in an embodiment of the present invention; Figure 3 A schematic structural diagram of an autonomous three-dimensional tissue ablation control system based on a soft robot provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The embodiments of the present application provide an autonomous three-dimensional tissue ablation control system based on a soft robot, thereby solving the technical problems of large path tracking errors and insufficient dynamic control of ablation depth.

[0020] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows: First, add the terminology explanation: 1) Optical coherence tomography (OCT) is a three-dimensional tomography technology that obtains tomography capabilities in the depth direction based on the principle of low-coherence interference. It can reconstruct two-dimensional or three-dimensional images of the internal structure of biological tissues or materials through scanning. Its signal contrast comes from the spatial changes in the optical reflection (scattering) characteristics of biological tissues or materials.

[0021] 2) Model Predictive Control (MPC) is a model-based closed-loop optimization control strategy that is widely used in autonomous driving, path planning and other fields. MPC predicts the behavior of the system in the future and optimizes the control input to make the system as close to the target state as possible while satisfying various constraints.

[0022] 3) Gaussian Process Regression (GPR) is a non-parameteric model that uses Gaussian Process (GP) prior to perform regression analysis on data. Its model assumptions include noise (regression residual) and Gaussian process prior, and its solution is based on Bayesian inference.

[0023] In view of the shortcomings of existing laser ablation technologies, the embodiments of the present invention construct a laser ablation surgical system to achieve precise control of ablation depth. The core points involved include at least: (1) Robust laser ablation depth control algorithm based on real-time OCT feedback By combining the interaction mechanism between blue laser and tissue and OCT image feedback, the ablation depth is monitored and adjusted in real time. The interaction relationship between tissue and laser is learned online to update the system model in real time to predict the ablation depth, thereby dynamically adjusting the laser parameters to achieve robust ablation depth control.

[0024] (2) Laser ablation path tracking algorithm based on soft robot In order to solve the problem of insufficient integration of soft robot's flexible structure and precise control, the Koopman plus MPC method is adopted. The Koopman operator can transform complex nonlinear systems into high-dimensional linear system representations, automatically capture complex dynamic characteristics such as material hysteresis in a purely data-driven way, avoid the difficulties of traditional physical modeling, and support online model updates to adapt to system parameters and environmental changes; at the control level, the predictive optimization characteristics of MPC can plan the optimal trajectory in advance, explicitly consider system constraints such as actuator limitations, and balance tracking accuracy and control efficiency through multi-objective optimization.

[0025] It is particularly important to emphasize that the autonomous three-dimensional tissue ablation control system provided by the embodiment of the present invention relies on a soft robot. Specifically, the soft robot includes a soft robot body, a laser ablation system, and an OCT imaging system; wherein: The soft robot body precisely positions the light beam emitted by the laser fiber, performs flexible path tracking in space, and then moves the laser to perform laser ablation operations. Exemplarily, the soft robot body uses a pneumatic drive mechanism to make the robot's movement flexible and adaptable, and can bend and stretch freely in a small and complex tissue environment.

[0026] The laser ablation system provides a high-energy laser beam through a laser to ablate the diseased tissue, i.e., the area to be ablated. Theoretically, the laser ablation system can accurately control the power, frequency, and irradiation time of the laser to ensure accurate control of the ablation depth. Exemplarily, the tissue ablation depth is adjusted here by controlling the laser power.

[0027] The OCT imaging system includes an OCT imaging optical fiber, which uses optical coherence tomography (OCT) technology to provide real-time tissue imaging feedback. The system can obtain high-resolution three-dimensional images of diseased tissue and provide feedback information for real-time depth control.

[0028] Furthermore, Figure 1 A feasible laser ablation system was demonstrated. Figure 1 In the system, there are three parts, namely, a passive bending part, an active bending part and a laser deflector. It can be placed in advance in a designated area to be ablated (such as the laryngeal tumor in the figure), and then the laser ablation optical fiber of the laser ablation system is driven to deflect by controlling the movement of the pneumatic soft robot to achieve tissue ablation.

[0029] Furthermore, Figure 2 A feasible integration method of OCT imaging fiber, laser ablation fiber, and indicator laser fiber is demonstrated. The laser beam is irradiated to the sample through the OCT imaging fiber, and the tiny structure inside the sample reflects the beam back. The fiber receives the reflected light signal and transmits it to the imaging system, eventually forming a tomographic image of the sample. The laser ablation fiber transmits the laser beam to the target tissue, and through the thermal effect of the laser, the temperature of the tissue is increased to a level sufficient to destroy the diseased tissue. The indicator laser fiber emits a visible red laser beam to clearly identify the operation area or the location of the diseased tissue, thereby ensuring accuracy and safety during the treatment process.

[0030] In addition, Figure 2 It is also shown that the light beams emitted by the ablation laser fiber and the indicator laser fiber intersect at a certain point. The intersection is the position where the laser and the indicator laser act together, which ensures that the ablation operation can accurately act on the area to be ablated.

[0031] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0032] Embodiment 1: like Figure 3 As shown, an embodiment of the present invention provides an autonomous three-dimensional tissue ablation control system based on a soft robot, wherein the soft robot includes a soft robot body, a laser ablation system, and an OCT imaging system; including: A construction module for pre-constructing a laser tissue interaction model to describe the interaction relationship between the blue laser and the tissue and determine the target ablation depth of the area to be ablated; A planning module, used for receiving a plurality of two-dimensional ablation points planned by a doctor on an endoscopic image and generating a laser ablation path; A tracking module, used to control the soft robot body to track the laser ablation path; comprising: Establishing a unit for establishing a linearized model of the soft robot body using the Koopman method; A control unit, used to optimize the robot control strategy based on the linearized model of the soft robot body and using the MPC method to ensure the tracking accuracy of the laser ablation path; The ablation module is used to adjust the energy output of the laser ablation system during the path tracking process, combining the interaction between the blue laser and the tissue and the real-time feedback of the OCT imaging system, so that the tissue ablation depth of the to-be-ablated area is consistent with the target ablation depth.

[0033] In order to achieve precise ablation of diseased tissue in a narrow environment in the body by the laser ablation system, an embodiment of the present invention pre-constructs a laser-tissue interaction model to describe the interaction between the blue laser and the tissue, thereby developing a robust ablation depth control method; in addition, an innovative adaptive laser ablation path tracking method combining a soft robot body with an OCT imaging system is proposed; ultimately, a surgical robot system with autonomous and precise three-dimensional laser ablation capabilities is constructed, which can significantly improve the accuracy, safety and automation level of laser ablation surgery.

[0034] Next, we will introduce the various components of the above system and their working principles in detail: As for the construction module, it is used to pre-build a laser tissue interaction model to describe the interaction relationship between the blue laser and the tissue and determine the target ablation depth of the area to be ablated.

[0035] The robust laser ablation depth control algorithm based on learning OCT real-time feedback needs to update the system model in real time to predict the ablation depth through online learning of the interaction between tissue and laser, thereby dynamically adjusting the laser parameters to achieve robust ablation depth control.

[0036] Specifically, the steps of pre-building the laser-tissue interaction model include: During the laser irradiation process, the state change of the tissue is divided into three stages: light absorption, heat diffusion and tissue damage. The pre-construction method of the laser tissue interaction model is as follows: (1) For the light absorption stage, the dynamic Monte Carlo method is used to construct the laser-tissue interaction model.

[0037] The simulation results of the absorption and scattering of a large number of photons in the tissue are statistically analyzed to obtain the distribution of laser energy in the tissue S(r,z,t). The absorption and scattering process of photons is equivalent to the photon energy at the collision point in proportion to μ a / (μ a +μ s ), μ s / (μ a +μ s ) is absorbed and scattered by the node area. Among them, r and z are the radial and axial coordinates based on the cylindrical coordinate system, t is the time, and μ a , μ s They are tissue absorption and scattering parameters, respectively, which represent the absorption and scattering abilities of tissues to laser energy.

[0038] (2) For the heat diffusion stage, the laser-tissue interaction model is constructed using the finite difference element method.

[0039] That is, according to the light absorption distribution function S(r,z,t), based on the constraint equation formula (1), the finite difference method is used to solve the heat diffusion equation to obtain the temporal and spatial distribution of the heat deposition H(r,z,t) and tissue temperature T(r,z,t) in the tissue: in, represents the partial derivative of thermal deposition H(r,z,t) with respect to time t, is the symbol of partial derivative; β is the thermal conductivity of tissue, which indicates the ability of tissue to conduct heat; is called the Laplace operator, Describes the spatial second-order derivative of tissue temperature T(r,z,t) in the cylindrical coordinate system.

[0040] (3) For the tissue damage stage, the Arrhenius integral is used to construct the laser-tissue interaction model.

[0041] That is, the Arrhenius integral Ω(r,z,t) is used to quantify the tissue damage parameter α(r,z,t): Among them, Ω(r,z,t) is used to describe the cumulative damage effect of tissue under thermal action; C(r,z,t) is the remaining proportion of undamaged tissue, which represents the proportion of undamaged tissue at position (r,z) and time t; C(r,z,0) is the proportion of undamaged tissue at the initial moment; A is the frequency factor, which represents the reaction rate; E a is the activation energy, which represents the energy required for tissue damage; R is the gas constant.

[0042] When α>1, the tissue is considered to be solidified, and when H(r,z,t) reaches the threshold, the tissue is removed and becomes air, thereby updating the tissue model. Considering the dynamic changes of tissue characteristics during laser transmission, a dynamic change model of tissue ablation process based on tissue absorption and scattering parameters is constructed to simulate the real changes of tissue characteristics. The relevant parameters are updated through in vitro experiments to approximate the real ablation process, so as to analyze the influence of different blue laser parameters.

[0043] The planning module is used to receive multiple two-dimensional ablation points planned by the doctor on the endoscopic image and generate a laser ablation path.

[0044] The external endoscope system captures the endoscopic image of the diseased tissue through a high-resolution camera and a light source. During operation, the endoscope probe can be brought close to the diseased tissue to obtain a two-dimensional plane image through a video stream or a static image mode. In addition, the path planning of the embodiment of the present invention aims to set the trajectory information on the two-dimensional plane so as to facilitate the subsequent path tracking by the soft robot body using MPC control.

[0045] Specifically, the steps of generating the laser ablation path include: First, the endoscopic image including the area to be ablated is captured by a camera and a light source.

[0046] Secondly, multiple two-dimensional ablation points planned by the doctor in the endoscopic image are received, and all points on the area to be ablated are traversed.

[0047] Finally, the laser ablation path is generated by making the energy distribution of each of the two-dimensional ablation points uniform.

[0048] Exemplarily, making the energy distribution of each two-dimensional ablation point uniform can be achieved by using the following objective function: Wherein, min is the minimization function; p is the path parameter of the laser ablation path; n is the number of two-dimensional ablation points; z i is the target ablation depth of the i-th 2D ablation point; E i is the energy distribution error of the i-th two-dimensional ablation point; α and β are the weight coefficients for adjusting the depth accuracy and energy distribution uniformity, respectively.

[0049] The tracking module is used to control the soft robot body to track the laser ablation path; it includes: Establishing a unit for establishing a linearized model of the soft robot body using the Koopman method; A control unit is used to optimize the robot control strategy based on the linearized model of the soft robot body and adopt the MPC method to ensure the tracking accuracy of the laser ablation path.

[0050] Specifically: In the establishment unit, linear modeling of the pneumatic soft robot body is performed, and specifically, the Koopman method is used to establish the linear model of the soft robot body.

[0051] The pneumatic soft robot system has a strong nonlinear characteristic. Under the premise of selecting a pneumatic soft robot, the control of the soft robot body refers to achieving the deformation of the robot by adjusting the air pressure, thereby controlling the steering of the laser.

[0052] Specifically, the relevant steps of the linear modeling include: First, a nonlinear system is used to define the state dynamics of the soft robot body: Among them, x(t) is the system state of the soft robot body at time t, corresponding to the position of the indicating laser point; is the state dynamics of the soft robot body at time t; u(t) is the control input, corresponding to the air pressure control signal, for example, it contains the motor step signal corresponding to the pressure value [P1, P2, P3] of each air cavity; f(⋅) is the nonlinear dynamic function of the system, which is used to describe the working principle of the pneumatic system.

[0053] Secondly, for nonlinear systems , using the Koopman operator Map the system's observation function g(x) from time t to time t+Δt: Again, the state dynamics of the soft robot body are approximated by a linear system: Among them, A is the state transfer matrix, which represents the evolution of the observation function; B is the control input matrix, which represents the influence of the air pressure control signal on the observation function.

[0054] Next, the least squares method is used to solve A and B to transform the evolution of the system into the following linear state space equation: Among them, g(x(t+1)) represents the observation function of the system at time t+1.

[0055] It is particularly noted that the least square method is used to solve A and B, specifically referring to: For the evolution matrix of the observation function, i.e., the observation function matrix G, we hope to estimate A and B by minimizing the following objective function: Among them, G t+1 represents the value of the observation function at the next time step, G t is the observation function matrix of the current time step, and U is the control input matrix. We hope to find A and B to minimize this error. Least squares solution: Using the least squares method to solve this optimization problem, we get: As shown in formula (10), A and B are obtained by calculating the pseudo-inverse of the observation function matrix. Specifically, represents the transpose of the matrix Gt, Represents matrix product The entire expression constructs the optimal linear mapping relationship from the observation function matrix Gt of the current time step to the observation function matrix Gt+1 of the next time step, thereby minimizing the sum of squares of the prediction error, which is the core step of the Koopman method to linearize the dynamic model of the soft robot.

[0056] Finally, the linear state space equation is used as the linearized model of the soft robot body, which describes how the state g(x(t+1)) of the system at the next time step depends on the current state g(x(t)) and the control input u(t).

[0057] In the control unit, path tracking control is performed, and the MPC (model predictive control) method is specifically used to optimize the robot control strategy to ensure the tracking accuracy of the laser ablation path. In order to implement MPC, the Koopman model is used to predict the future state x(t+k) to achieve optimal control of the system. The core idea of ​​MPC is to optimize the control input u(t) in the future so that the system state x(t) reaches the desired target and is adjusted within the control constraints.

[0058] Specifically, the relevant steps of the path tracking control include: First, based on the linearized model of the soft robot body, the future k-time states x(t+k) of the system state x(t) at time t are calculated by the following prediction equation: Among them, g(x(t+k+1)) and g(x(t+k)) are the observation functions of the system at time t+k+1 and t+k respectively; u(t+k) is the control input at time t+k.

[0059] Secondly, the deviation of the system state and the energy consumption of the control input are integrated to minimize the first cost function J in the next N moments as the objective function of the MPC problem: Where p(t+k) is the predicted position of the laser point at time t+k; p d (t+k) is the expected position of the two-dimensional ablation point on the laser ablation path at time t+k; Q path , R path They are all symmetric positive definite weight matrices in the path tracking process, which control the path tracking error and the cost of the control input respectively. The subscript path represents path tracking; is the norm of the vector.

[0060] And set the constraints of the MPC problem: Among them, u min 、u max They are the minimum and maximum values ​​of the control input, respectively, indicating that the control input is limited by the maximum air cavity pressure, for example, limiting the size of the step signal.

[0061] Again, based on the objective function and constraints of the MPC problem, it is converted into solving the following optimization problem: The objective function is: Among them, min is the minimization function; J(u) is the first cost function with respect to the variable u.

[0062] The constraints are: Finally, by solving the optimization problem, the optimal control input sequence u is obtained. * (t),u * (t+1),…,u * (t+N-1), and select the first optimal control input u * (t) Real-time adjustment of the movement of the soft robot body to ensure the tracking accuracy of the laser ablation path.

[0063] The ablation module is used to adjust the energy output of the laser ablation system during the path tracking process, combining the interaction between the blue laser and the tissue and the real-time feedback of the OCT imaging system, so that the tissue ablation depth of the area to be ablated is consistent with the target ablation depth.

[0064] In this module, based on the research on the laser-tissue interaction model, the ablation depth can be predicted through real-time OCT data combined with Gaussian process regression, and the ablation depth can be controlled while using MPC for trajectory tracking.

[0065] Specifically, the steps related to the ablation depth control include: First, define the functional mapping relationship between the laser energy output per unit time and the change in tissue depth: Among them, d t is the tissue ablation depth at time t, which is determined by the tissue depth change fed back by the OCT image in real time; the input variable of the system The tissue ablation depth d at time t t And the output energy per unit time u t Composition, Δd t =d t+1 -d t is the change in ablation depth at adjacent moments, is the output variable of the system; f is the mapping function between the laser energy output per unit time and the change in tissue depth.

[0066] Secondly, the data is obtained through offline sampling, and the dynamic relationship of tissue depth change is obtained by Gaussian process regression GPR, and a nonlinear fitting is established: in, They are training data sets of input and output variables obtained from real environments, and it is assumed that the output variables of the system conform to the Gaussian distribution with a mean of 0. ; K(X,X) is the covariance matrix; I is the identity matrix; σ 2 is the variance of Gaussian noise.

[0067] Given the training data set and the input variables of the system , construct the following joint probability distribution and predict the depth change of the system given any sequence of laser energy input : Among them, K(X,X)+Iσ 2 Add a noise term to the covariance function between the input data set X and itself; Is the covariance function between the input data set X and; For input variables The covariance function between the input data set X; For input variables The covariance function between y and itself.

[0068] Thirdly, based on the dynamic relationship of tissue depth change and the target ablation depth, a loss function is constructed to predict the tissue ablation depth in the future K moments through forward mapping: Among them, J ' is the second cost function; is the mean value of the random variable; d K is the tissue ablation depth in the next K moments; d d is the target ablation depth; Q depth , S depth are all symmetric positive definite weight matrices in ablation depth control, which control the ablation depth control error and the cost of control input respectively. The subscript depth represents ablation depth control; l(d t ) is the process loss function, h(d T ) is the terminal loss function; the superscript T indicates transposition.

[0069] Next, minimize the second cost function J ' , determine the optimal tissue ablation depth at each moment in the future K moments , and then obtain a set of optimal energy laser sequences, combined with the depth variation , to set the laser power at the current moment and adjust the energy output of the laser ablation system in real time.

[0070] Finally, the dynamic relationship of tissue depth change is updated by using the tissue depth change fed back in real time by the OCT image, and the above optimization process is repeated until the tissue ablation depth of the to-be-ablated area is consistent with the target ablation depth.

[0071] It should be noted that in the embodiment of the present invention, the two-dimensional coordinates (x, y) in the endoscope assembly coordinate system and the three-dimensional coordinates (x ' ,y ' ,d) in (x ' ,y ' ) corresponds to the same physical point. Specifically, the three-dimensional coordinates of the OCT camera can be mapped to the two-dimensional coordinates of the endoscope camera through the coordinate conversion relationship, thereby ensuring that the corresponding tissue depth control is achieved at the two-dimensional ablation point of the laser ablation path.

[0072] So far, the embodiments of the present invention have introduced the complete content of the autonomous three-dimensional tissue ablation control system based on the soft robot.

[0073] Embodiment 2: An embodiment of the present invention provides a storage medium storing a computer program for autonomous three-dimensional tissue ablation control based on a soft robot, wherein the computer program enables a computer to control the autonomous three-dimensional tissue ablation control system as described in Example 1 to perform an autonomous three-dimensional tissue ablation method.

[0074] Embodiment 3: An embodiment of the present invention provides an electronic device, including: One or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a program for controlling the autonomous three-dimensional tissue ablation control system as described in Example 1 to perform an autonomous three-dimensional tissue ablation method.

[0075] In summary, compared with the prior art, the present invention has the following beneficial effects: 1. The embodiment of the present invention integrates high-resolution imaging technology, including endoscopic imaging and OCT imaging. OCT provides high-resolution depth information, while endoscopic images can provide more intuitive surface images. It provides real-time visualization of the surgical area and combines online learning algorithms to monitor and feedback control the ablation process in real time, ensuring the controllability of ablation depth and range.

[0076] 2. The embodiments of the present invention utilize the flexibility and adaptability of soft robots to develop the hardware structure of soft robots that can freely navigate in narrow and complex natural cavities, and to establish a dynamic model of the soft robot and implement trajectory tracking control to ensure that the laser ablation system can accurately reach the designated area to be ablated.

[0077] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0078] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An autonomous three-dimensional tissue ablation control system based on a soft robot, characterized in that: The soft robot comprises a soft robot body, a laser ablation system, and an OCT imaging system; including: A construction module for pre-constructing a laser tissue interaction model to describe the interaction relationship between the blue laser and the tissue and determine the target ablation depth of the area to be ablated; A planning module, used for receiving a plurality of two-dimensional ablation points planned by a doctor on an endoscopic image and generating a laser ablation path; A tracking module, used to control the soft robot body to track the laser ablation path; comprising: Establishing a unit for establishing a linearized model of the soft robot body using the Koopman method; A control unit, used to optimize the robot control strategy based on the linearized model of the soft robot body and using the MPC method to ensure the tracking accuracy of the laser ablation path; The ablation module is used to adjust the energy output of the laser ablation system during the path tracking process, combining the interaction between the blue laser and the tissue and the real-time feedback of the OCT imaging system, so that the tissue ablation depth of the to-be-ablated area is consistent with the target ablation depth.

2. The autonomous three-dimensional tissue ablation control system according to claim 1, characterized in that: During the laser irradiation process, the state change of the tissue is divided into three stages: light absorption, heat diffusion and tissue damage. The pre-construction method of the laser tissue interaction model is as follows: For the light absorption stage, the laser-tissue interaction model is constructed using a dynamic Monte Carlo method; For the heat diffusion stage, the laser-tissue interaction model is constructed using the finite difference element method; For the tissue damage stage, the laser-tissue interaction model was constructed using Arrhenius integration.

3. The autonomous three-dimensional tissue ablation control system according to claim 1, characterized in that: The receiving of multiple two-dimensional ablation points planned by the doctor on the endoscopic image and generating a laser ablation path includes: Capturing the endoscopic image including the area to be ablated by a camera and a light source; receiving a plurality of two-dimensional ablation points planned by the doctor on the endoscopic image, and traversing all points on the area to be ablated; The laser ablation path is generated by making the energy distribution of each of the two-dimensional ablation points uniform.

4. The autonomous three-dimensional tissue ablation control system according to claim 1, characterized in that: The Koopman method is used to establish a linear model of the soft robot body, including: The state dynamics of the soft robot body are defined using a nonlinear system: Among them, x(t) is the system state of the soft robot body at time t, corresponding to the position of the indicating laser point; is the state dynamics of the soft robot body at time t; u(t) is the control input, corresponding to the air pressure control signal; f(⋅) is the nonlinear dynamic function of the system, which is used to describe the working principle of the pneumatic system; For nonlinear systems , using the Koopman operator Map the system's observation function g(x) from time t to time t+Δt: The state dynamics of the soft robot body are approximated by a linear system: Among them, A is the state transfer matrix, which represents the evolution of the observation function; B is the control input matrix, which represents the influence of the air pressure control signal on the observation function; The least squares method is used to solve A and B to transform the evolution of the system into the following linear state space equation: Among them, g(x(t+1)) represents the observation function of the system at time t+1; The linear state space equation is used as the linearized model of the soft robot body.

5. The autonomous three-dimensional tissue ablation control system according to claim 4, characterized in that: The method of optimizing the robot control strategy based on the linearized model of the soft robot body and using the MPC method to ensure the tracking accuracy of the laser ablation path includes: Based on the linearized model of the soft robot body, the future k-time states x(t+k) of the system state x(t) at time t are calculated by the following prediction equation: Among them, g(x(t+k+1)) and g(x(t+k)) are the observation functions of the system at time t+k+1 and t+k respectively; u(t+k) is the control input at time t+k; The deviation of the system state and the energy consumption of the control input are integrated to minimize the first cost function J in the next N moments as the objective function of the MPC problem: Where p(t+k) is the predicted position of the laser point at time t+k; p d (t+k) is the expected position of the two-dimensional ablation point on the laser ablation path at time t+k; Q path , R path They are all symmetric positive definite weight matrices in the path tracking process, which control the path tracking error and the cost of the control input respectively. The subscript path represents path tracking; is the norm of the vector; And set the constraints of the MPC problem: Among them, u min 、u max are the minimum and maximum values ​​of the control input respectively; Based on the objective function and constraints of the MPC problem, it is converted into solving the following optimization problem: The objective function is: Where min is the minimization function; J(u) is the first cost function with respect to variable u; The constraints are: By solving the optimization problem, we can obtain the optimal control input sequence u * (t),u * (t+1),…,u * (t+N-1), and select the first optimal control input u * (t) Real-time adjustment of the movement of the soft robot body to ensure the tracking accuracy of the laser ablation path.

6. The autonomous three-dimensional tissue ablation control system according to claim 1, characterized in that: In the path tracking process, combining the interaction between the blue laser and the tissue and the real-time feedback of the OCT imaging system, adjusting the energy output of the laser ablation system so that the tissue ablation depth of the to-be-ablated area is consistent with the target ablation depth, including: Define the functional mapping relationship between the laser energy output per unit time and the change in tissue depth: Among them, d t is the tissue ablation depth at time t, which is determined by the tissue depth change fed back by the OCT image in real time; the input variable of the system The tissue ablation depth d at time t t And the output energy per unit time u t Composition, Δd t =d t+1 -d t is the change in ablation depth at adjacent moments, is the output variable of the system; f is the mapping function between the laser energy output per unit time and the change in tissue depth; The data is obtained through offline sampling, and the dynamic relationship of tissue depth change is obtained by using Gaussian process regression GPR, and a nonlinear fitting is established: in, They are training data sets of input and output variables obtained from real environments, and it is assumed that the output variables of the system conform to the Gaussian distribution with a mean of 0. ; K(X,X) is the covariance matrix; I is the identity matrix; σ 2 is the variance of Gaussian noise; Given the training data set and the input variables of the system , construct the following joint probability distribution and predict the depth change of the system given any sequence of laser energy input : Among them, K(X,X)+Iσ 2 Add a noise term to the covariance function between the input data set X and itself; Is the covariance function between the input data set X and; For input variables The covariance function between the input data set X; For input variables The covariance function with itself; Based on the dynamic relationship of tissue depth change and the target ablation depth, a loss function is constructed to predict the tissue ablation depth in the future K moments through forward mapping: Among them, J ' is the second cost function; is the mean value of the random variable; d K is the tissue ablation depth in the next K moments; d d is the target ablation depth; Q depth , S depth are all symmetric positive definite weight matrices in ablation depth control, which control the ablation depth control error and the cost of control input respectively. The subscript depth represents ablation depth control; l(d t ) is the process loss function, h(d T ) is the terminal loss function; the superscript T indicates transposition; Minimize the second cost function J ' , determine the optimal tissue ablation depth at each moment in the future K moments , and then obtain a set of optimal energy laser sequences, combined with the depth variation , to set the laser power at the current moment and adjust the energy output of the laser ablation system in real time; The tissue depth change fed back in real time by the OCT image is used to update the dynamic relationship of the tissue depth change, and the above optimization process is repeated until the tissue ablation depth of the to-be-ablated area is consistent with the target ablation depth.

7. A storage medium, characterized in that: It stores a computer program for autonomous three-dimensional tissue ablation control based on a soft robot, wherein the computer program enables the computer to control the autonomous three-dimensional tissue ablation control system as described in any one of claims 1 to 6 to perform an autonomous three-dimensional tissue ablation method.

8. An electronic device, characterized in that: include: one or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a program for controlling the autonomous three-dimensional tissue ablation control system as described in any one of claims 1 to 6 to perform an autonomous three-dimensional tissue ablation method.

Images