Methods, apparatus, devices, and storage media of operating an interventional procedure

Abstract

Embodiments of the present application provide an interventional operation method, device, equipment and storage medium, the method comprising: obtaining a target image corresponding to an interventional operation; the target image comprising an instrument image and a user blood vessel image; inputting the instrument image and the user blood vessel image into a trained interventional operation model to obtain a target operation instruction; the interventional operation model being trained based on a sample environment image, a first operation instruction corresponding to the sample environment image, a reward value corresponding to the first operation instruction, and a second time sample image corresponding to execution of the first operation instruction; and performing an operation of the interventional operation according to the target operation instruction. The method of the embodiments of the present application obtains a target operation instruction based on an instrument image, a user blood vessel image and an interventional operation model of an interventional operation obtained, realizes autonomous delivery of an interventional operation instrument, and reduces the work burden caused by a large number of simple and repetitive operations in the interventional operation.

Description

Technical Field

[0001] This invention relates to the field of medical technology, and in particular to an interventional surgical procedure, apparatus, equipment, and storage medium. Background Technology

[0002] Interventional surgery is a minimally invasive treatment method that utilizes modern high-tech means. Guided by medical imaging equipment, it involves inserting specialized catheters and guidewires into the body to diagnose and treat internal conditions. Interventional treatment uses digital technology to expand the surgeon's field of vision and extends their reach with the help of catheters and guidewires. Interventional surgery involves smaller incisions and is characterized by being non-surgical, minimally invasive, having a rapid recovery, and offering good results.

[0003] In related technologies, doctors use digital subtraction angiography (DSA) to determine the position of instruments in blood vessels and send axial movement and rotation commands to the master end of the interventional surgical robot. The robot then controls the instruments to move within the blood vessel according to these commands. In other words, during interventional surgery, the interventional surgical robot can only passively execute the doctor's control commands, failing to reduce the workload caused by the numerous simple and repetitive operations involved, resulting in low efficiency in interventional procedures. Summary of the Invention

[0004] To address the problems in the prior art, embodiments of the present invention provide an operation method, apparatus, device, and storage medium for interventional surgery.

[0005] Specifically, the embodiments of the present invention provide the following technical solutions:

[0006] In a first aspect, embodiments of the present invention provide a method for performing an interventional procedure, comprising:

[0007] Acquire the target image corresponding to the interventional procedure; the target image includes instrument images and user vascular images;

[0008] The instrument image and the user's vascular image are input into the trained interventional surgery operation model to obtain the target operation command; the interventional surgery operation model is trained based on the sample environment image, the first operation command corresponding to the sample environment image, the reward value corresponding to the first operation command, and the sample image at the second time point after the execution of the first operation command.

[0009] Perform interventional surgery according to the target operation instructions.

[0010] Furthermore, the interventional surgical operation model is trained in the following manner:

[0011] Acquire a sample environment image at the first moment of the interventional procedure; the first moment of the sample environment image includes a sample instrument image and a sample user blood vessel image;

[0012] The sample instrument image and the sample user vascular image are input into the initial interventional surgery operation model to obtain the first operation command;

[0013] The initial interventional surgery operation model is trained based on the first moment sample environment image of the interventional surgery, the first operation command, the reward value corresponding to the first operation command, and the second moment sample image corresponding to the execution of the first operation command, to obtain the trained interventional surgery operation model.

[0014] Further, the step of training the initial interventional surgery operation model based on the first-moment sample environment image of the interventional surgery, the first operation command, the reward value corresponding to the first operation command, and the second-moment sample image corresponding to the execution of the first operation command, to obtain the trained interventional surgery operation model, includes:

[0015] Based on the first moment sample environment image of the interventional surgery, the first operation instruction, the reward value corresponding to the first operation instruction, and the second moment sample image corresponding to the execution of the first operation instruction, the operation data of the surgical instruments are obtained.

[0016] The initial interventional surgical operation model is trained based on the operation data of the surgical instruments to obtain the trained interventional surgical operation model; the acquisition of the operation data of the surgical instruments and the training of the initial interventional surgical operation model based on the operation data of the surgical instruments are executed asynchronously.

[0017] Further, the surgical instrument is operated according to the first operation instruction to obtain a first position of the surgical instrument; a reward value corresponding to the first operation instruction is determined based on the distance between the first position and the target position; the target position represents the location in the user's blood vessel where surgery needs to be performed; and / or,

[0018] Based on the initial and target positions of the surgical instruments, a target path is determined; based on the target path and the first position, a reward value corresponding to the first operation instruction is determined; the initial position represents the position of the surgical instruments in the user's blood vessel at the first moment; the target path represents the path with the shortest distance between the first position and the target position; and / or,

[0019] When the first position of the surgical instrument is located on the target path, a reward value corresponding to the first operation instruction is determined based on a first distance between the initial position and the target position, and a second distance between the first position and the target position; and / or;

[0020] The reward value corresponding to the first operation command is determined based on the contact force between the surgical instrument and the surgical robot.

[0021] Furthermore, the initial interventional surgical procedure model is trained to maximize the following training objective:

[0022]

[0023] The r t The reward value represents the operation command of the surgical instrument corresponding to the environmental image at time t; γ represents the attenuation coefficient; α represents the entropy regularization coefficient; the... Entropy represents the randomness of the operating instructions of the surgical instrument; E represents the expected computation.

[0024] Furthermore, after acquiring the sample environment image at the first moment of the interventional procedure, it also includes:

[0025] The instrument image and the user blood vessel image in the sample environment image at the first time point are binarized respectively.

[0026] Secondly, embodiments of the present invention also provide an operating device for interventional surgery, comprising:

[0027] The acquisition module is used to acquire the target image corresponding to the interventional surgery; the target image includes instrument images and user blood vessel images;

[0028] The processing module is used to input the instrument image and the user's vascular image into the trained interventional surgery operation model to obtain the target operation instruction; the interventional surgery operation model is trained based on the sample environment image, the first operation instruction corresponding to the sample environment image, the reward value corresponding to the first operation instruction, and the sample image at the second time point after the execution of the first operation instruction.

[0029] The operation module is used to perform interventional surgery according to the target operation instructions.

[0030] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the interventional surgery operation method as described in the first aspect.

[0031] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the interventional surgical operation method as described in the first aspect.

[0032] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the interventional surgery operation method as described in the first aspect.

[0033] The interventional surgery operation method, apparatus, equipment, and storage medium provided in this invention obtain target operation instructions based on the acquired instrument images, user vascular images, and interventional surgery operation model. This enables the interventional surgery robot to operate the interventional surgery instruments according to the determined target operation instructions, realizing the autonomous delivery of interventional surgery instruments, improving the efficiency of interventional surgery and the level of autonomy of vascular interventional surgery robots, reducing the workload caused by a large number of simple and repetitive operations in interventional surgery, and achieving the effect of assisting doctors in safely performing interventional treatment. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0035] Figure 1 This is a schematic flowchart of the interventional surgery operation method provided in the embodiment of the present invention;

[0036] Figure 2 This is a schematic diagram of the interventional surgery operation device provided in an embodiment of the present invention;

[0037] Figure 3 This is a training diagram of the interventional surgery operation model provided in an embodiment of the present invention;

[0038] Figure 4 This is a schematic diagram of the interventional surgery operation model provided in the embodiments of the present invention;

[0039] Figure 5 This is a flowchart of another interventional surgical procedure provided in an embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram of the operating device for interventional surgery provided in an embodiment of the present invention;

[0041] Figure 7 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] The method of this invention can be applied in medical scenarios to achieve active delivery of interventional surgical instruments.

[0044] In related technologies, doctors use digital subtraction angiography (DSA) to determine the position of instruments in blood vessels and send axial movement and rotation commands to the master end of the interventional surgical robot. The robot then controls the instruments to move within the blood vessel according to these commands. In other words, during interventional surgery, the interventional surgical robot can only passively execute the doctor's control commands, failing to reduce the workload caused by the numerous simple and repetitive operations involved, resulting in low efficiency in interventional procedures.

[0045] The interventional surgery operation method of this invention obtains target operation instructions based on the acquired images of interventional surgical instruments, user blood vessels, and interventional surgical operation model. This enables the interventional surgical robot to operate the interventional surgical instruments according to the determined target operation instructions, realizing the autonomous delivery of interventional surgical instruments, improving the efficiency of interventional surgery and the level of autonomy of vascular interventional surgical robots, reducing the workload caused by a large number of simple and repetitive operations in interventional surgery, and achieving the effect of assisting doctors to safely perform interventional treatment.

[0046] The following is combined with Figures 1-7 The technical solution of the present invention will be described in detail with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0047] Figure 1 This is a schematic flowchart of an embodiment of the interventional surgery method provided by the present invention. Figure 1 As shown, the method provided in this embodiment includes:

[0048] Step 101: Obtain the target image corresponding to the interventional procedure; the target image includes the instrument image and the user's blood vessel image;

[0049] Specifically, interventional vascular surgery is one of the main methods for treating cardiovascular and cerebrovascular diseases. In interventional vascular surgery, surgeons, guided by a digital subtraction angiography (DSA) imaging system, manipulate catheters, guidewires, and other interventional instruments through the vascular cavity to reach the lesion site for treatments such as thrombolysis and dilation of narrowed vessels. Optionally, a master-slave structured interventional vascular robot can be used. The surgeon operates the robot from the master end in a radiation-free control room, delivering interventional instruments from the slave end, thus avoiding high doses of X-ray radiation. During robot-assisted interventional surgery, the surgeon determines the position of the instruments in the blood vessel based on DSA imaging and sends axial movement and rotation commands to the master end of the interventional vascular robot. The slave end of the robot then controls the movement of the instruments in the blood vessel according to these commands. However, during the interventional procedure, the interventional vascular robot can only passively execute the surgeon's control commands. This fails to reduce the workload caused by the numerous simple and repetitive operations in interventional surgery and cannot assist inexperienced surgeons in safely performing the procedure.

[0050] To address the aforementioned issues and improve the efficiency of interventional procedures and the level of autonomy of vascular interventional surgery robots, this embodiment of the invention first acquires instrument images and user vascular images during interventional procedures. Optionally, the instrument images include images of catheters and guidewires during interventional procedures.

[0051] Step 102: Input the instrument image and the user's vascular image into the trained interventional surgery operation model to obtain the target operation command; the interventional surgery operation model is trained based on the sample environment image, the first operation command corresponding to the sample environment image, the reward value corresponding to the first operation command, and the sample image at the second time point after the execution of the first operation command.

[0052] Specifically, after acquiring the instrument images and user vascular images during interventional surgery, in this embodiment of the invention, the instrument images and user vascular images are input into the trained interventional surgery operation model to obtain the target operation command. The interventional surgery operation model is used to determine the operation command corresponding to the vascular interventional surgery robot based on the acquired instrument images and user vascular images. Optionally, the target operation command includes control commands for two degrees of freedom: axial and rotational. The axial degree of freedom has two commands: forward and backward at constant speed. The rotational degree of freedom has five commands: no rotation, two clockwise rotations at two speeds, and two counterclockwise rotations at two speeds. The control command lasts for 0.5 seconds. The reward value is used to evaluate the quality of the operation command, i.e., to evaluate the degree of matching between the operation command and the current environmental image and the contribution of the operation command to the current interventional surgery, i.e., to evaluate whether the operation command can deliver the interventional instrument to the lesion location via the vascular cavity. Optionally, the interventional surgery operation model is trained based on the sample environmental image, the first operation command corresponding to the sample environmental image, the reward value corresponding to the first operation command, and the sample image at the second moment after the execution of the first operation command.

[0053] Step 103: Perform the interventional surgery according to the target operation instructions.

[0054] Specifically, after inputting instrument images and user vascular images into the trained interventional surgery operation model to obtain target operation instructions, the vascular interventional surgery robot can operate the interventional surgery instruments according to the determined target operation instructions. That is, in this embodiment of the invention, based on the acquired image information of the instruments and user's blood vessels during the interventional surgery, the operation instructions of the vascular interventional surgery robot are determined. This allows the vascular interventional surgery robot to accurately and effectively operate the interventional surgical instruments according to the determined operation instructions, achieving autonomous delivery of interventional surgical instruments, improving the efficiency of interventional surgery and the level of autonomy of the interventional surgery robot, reducing the workload caused by a large number of simple and repetitive operations in interventional surgery, and achieving the effect of assisting doctors in safely performing interventional treatment.

[0055] The method described in the above embodiments obtains target operation instructions based on the acquired images of interventional surgical instruments, user blood vessels, and interventional surgical operation models. This enables the interventional surgical robot to operate the interventional surgical instruments according to the determined target operation instructions, achieving autonomous delivery of interventional surgical instruments, improving the efficiency of interventional surgery and the level of autonomy of vascular interventional surgical robots, reducing the workload caused by a large number of simple and repetitive operations in interventional surgery, and achieving the effect of assisting doctors in safely performing interventional treatment.

[0056] In one embodiment, the interventional surgical procedure model is trained in the following manner:

[0057] Acquire the first-moment sample environment image of the interventional procedure; the first-moment sample environment image includes the sample instrument image and the sample user's blood vessel image;

[0058] Input the sample instrument image and the sample user vascular image into the initial interventional surgery operation model to obtain the first operation command;

[0059] The initial interventional surgery operation model is trained based on the first moment sample environment image, the first operation command, the reward value corresponding to the first operation command, and the second moment sample image corresponding to the execution of the first operation command, to obtain the trained interventional surgery operation model.

[0060] Specifically, in order to enable the interventional surgery operation model to accurately output the operation instructions corresponding to the current environment image, in this embodiment of the invention, the surgical operation model is trained using sample environment images, the first operation instructions corresponding to the sample environment images, the reward value corresponding to the first operation instructions, and the sample images at the second moment after the execution of the first operation instructions. That is, the sample instrument images and sample user vascular images of the interventional surgery are first input into the initial interventional surgery operation model to obtain the operation instructions of the surgical instruments corresponding to the current environment image and determine the reward value corresponding to the operation instructions. Optionally, if the reward value of the operation instructions determined based on the sample instrument images and sample user vascular images of the interventional surgery is lower than a preset threshold, the initial interventional surgery operation model is trained until the interventional surgery robot can achieve autonomous delivery of interventional surgical instruments based on the operation instructions output by the interventional surgery operation model, and deliver interventional instruments such as catheters and guidewires to the lesion location through the vascular cavity.

[0061] For example, such as Figure 2 The diagram shows a training device for an interventional surgical operation model. The training device includes a simulated surgical environment module, an operation data acquisition module (6), a data output display module (7), an operation data storage module (8), and a parameter update module (9). The simulated surgical environment module consists of a vascular interventional surgical robot (1), a catheter (2), a guidewire (3), a vascular model (4), and a camera (5).

[0062] The vascular interventional surgery robot (1) receives control commands from the operation data acquisition module and manipulates the guidewire (3) to move in the vascular model (4) according to the control commands; optionally, the vascular model (4) is made by 3D printing technology, and the camera (5) is located directly above the vascular model (4) and its position is fixed.

[0063] The operation data acquisition module acquires images from the camera (5), sends control commands to the vascular interventional surgery robot (1) according to the currently learned operation skills, and records operation data. The operation skills are represented by a neural network. The operation data acquisition module periodically copies the latest neural network parameters from the parameter update module to update the operation skills. The operation data storage module is used to store the operation data recorded by the operation data acquisition module. The parameter update module is used to learn vascular interventional surgery operation skills and uses the operation data sampled from the operation data storage module to update the neural network parameters representing the vascular interventional surgery operation skills. The data output display module is used to display the images obtained by the simulation surgical environment module, the real-time information of the parameter update module, etc., on the computer monitor. Optionally, the simulation surgical environment module is used to simulate a clinical surgical scenario, including a vascular interventional surgery robot, a 3D printed vascular model, interventional instruments, and a camera; the operation data acquisition module is used to send control commands to the vascular interventional surgery robot in the simulation surgical environment module according to the currently learned operation skills and record operation data. The operation data storage module is used to store the operation data recorded by the operation data acquisition module. The parameter update module is used to learn vascular interventional surgery operation skills. It updates the neural network parameters representing these skills using operation data sampled from the operation data storage module and periodically sends the latest neural network parameters to the operation data acquisition module. The data output display module displays images obtained by the simulated surgical environment module and real-time information from the parameter update module on a computer monitor. Optionally, considering that both operation data acquisition and parameter updates are time-consuming, this embodiment of the invention adopts a distributed deployment, meaning that operation data acquisition, operation data storage, and parameter updates are executed asynchronously in different processes.

[0064] Optionally, the training method for the interventional surgery operation device is as follows:

[0065] Step 1: The operation data acquisition module randomly selects control commands and sends them to the robot, records the operation data in the simulated surgical environment, and stores it in the operation data storage module;

[0066] Step 2: The parameter update module uses reinforcement learning to update the robot's operational skills. The operation data acquisition module selects control commands based on the learned operational skills and sends them to the robot, records the operational data in the simulated surgical environment, and stores it in the operation data storage module. The parameter update module and the operation data acquisition module run asynchronously.

[0067] Step 3: Repeat step 2 until the device can be delivered autonomously.

[0068] The advantage of this invention is that it allows for the learning of minimally invasive vascular interventional surgery skills without human supervision and guidance, enabling the selection of appropriate control commands at different surgical stages and thus achieving autonomous instrument delivery.

[0069] like Figure 3 As shown, the operation data acquisition module sends operation instructions to the simulated surgical environment. The surgical robot operates the surgical instruments according to the operation instructions, obtains the reward value corresponding to the operation instructions, and sends the environmental image, the operation instructions corresponding to the environmental image, the reward value corresponding to the operation instructions, and the environmental image after the operation instructions are executed to the operation data acquisition module for training the interventional surgery operation model.

[0070] An exemplary structural diagram of the interventional surgical procedure model is shown below. Figure 4 As shown, the system includes an encoder, decoder, policy network, and value function network. The encoder consists of a convolutional neural network, the decoder consists of a deconvolutional neural network, and both the policy network and value function network are composed of fully connected neural networks. The input o, a binarized image of the blood vessel and instruments, is encoded by the encoder to obtain the encoded H(o). The encoded H(o) is then processed by the decoder, policy network, and value function network to output the reconstruction result R(o), the probability π(o) of selecting each action, and the value function Q(o), respectively. R(o) is an image of the same size as the input o, where the value of each pixel represents the probability that the pixel is 1 at the corresponding position in the input o. π(o) and Q(o) are both vectors with a dimension equal to the number of control commands, where each dimension represents the probability π(o,a) of selecting the corresponding control policy a under the input and the value function Q(o,a), respectively. This means that the training efficiency of the interventional surgery operation model can be improved through reconstruction tasks and distributed deployment, enabling the model to learn vascular interventional surgery operation skills in a shorter time, achieve autonomous control of guidewire advancement and rotation, and reduce the workload of doctors.

[0071] During each update step, the decision network utilizes n sets of operational data sampled from the database {o t ,a t ,o t+1 ,r t} t Update the parameters using gradient descent.

[0072] The reconstruction loss function J(R) is used to update the encoder and reconstruct the network, and it is defined as follows:

[0073]

[0074] Where FL(·,·) represents the focal loss, and the specific calculation formula is as follows:

[0075] FL[R(o t ),o t]=SUM[-|R(o t )-o t | τ log(1-|R(o t )-o t |)]

[0076] SUM(·) represents summation pixel by pixel.

[0077] The value function loss function J(Q) is used to update the encoder and the value function network, and it is defined as follows:

[0078]

[0079] Where V(o) t+1 ) is defined as

[0080]

[0081] Among them, the objective value function network The parameters are the exponential average of the parameters of the value function network Q(·).

[0082] The policy loss function J(π) is used to update the policy network, and it is defined as follows:

[0083]

[0084] The method described above trains an initial surgical operation model using a sample environment image, a first operation instruction corresponding to the sample environment image, a reward value corresponding to the first operation instruction, and a sample image at a second time after the execution of the first operation instruction. In other words, it optimizes the initial interventional surgical operation model based on the reward value corresponding to the operation instruction output by the initial interventional surgical operation model. This enables the initial interventional surgical operation model to accurately output the operation instruction corresponding to the current environment image, improving the efficiency of interventional surgery, reducing the workload caused by a large number of simple and repetitive operations in interventional surgery, and assisting doctors in safely implementing interventional treatment.

[0085] In one embodiment, an initial interventional surgery operation model is trained based on a first-moment sample environment image of the interventional surgery, a first operation command, a reward value corresponding to the first operation command, and a second-moment sample image corresponding to the execution of the first operation command, to obtain a trained interventional surgery operation model, including:

[0086] Based on the first moment of the interventional surgery sample environment image, the first operation command, the reward value corresponding to the first operation command, and the second moment sample image corresponding to the execution of the first operation command, the operation data of the surgical instruments are obtained.

[0087] The initial interventional surgical operation model is trained based on the operation data of the surgical instruments to obtain the trained interventional surgical operation model; the acquisition of the operation data of the surgical instruments and the training of the initial interventional surgical operation model based on the operation data of the surgical instruments are executed asynchronously.

[0088] Specifically, in this embodiment of the invention, the training of the initial interventional surgery operation model is divided into two steps. The first step is the operation data acquisition stage. Optionally, the operation data of the surgical instruments includes the environmental image corresponding to the interventional surgery, the operation instructions output by the initial interventional surgery operation model based on the environmental image, the position of the surgical instruments after the operation according to the operation instructions, and the reward value corresponding to the operation instructions. The second step is the initial interventional surgery operation model training stage, that is, training the initial interventional surgery operation model based on the acquired environmental image corresponding to the interventional surgery, the operation instructions output by the initial interventional surgery operation model based on the environmental image, the position of the surgical instruments after the operation according to the operation instructions, and the reward value corresponding to the operation instructions. Optionally, the two steps of the initial interventional surgery operation model training can be executed asynchronously, thereby solving the problem that both data acquisition and model training are time-consuming, and avoiding mutual interference caused by simultaneous and overlapping operation data acquisition and model training. That is, after the first operation data acquisition is completed, it is necessary to wait for the operation data to be input into the initial interventional surgery operation model for model training before the next operation data can be acquired, thereby improving the efficiency of model training.

[0089] The method described above solves the problem of long processing times for both data acquisition and model training by executing the two steps of operational data acquisition and model training asynchronously, thus avoiding mutual interference caused by simultaneous or overlapping operations of operational data acquisition and model training, and effectively improving the efficiency of interventional surgical operation model training.

[0090] In one embodiment, a surgical instrument is operated according to a first operation instruction to obtain a first position of the surgical instrument; a reward value corresponding to the first operation instruction is determined based on the distance between the first position and a target position; the target position represents the location in the user's blood vessel where surgery needs to be performed; and / or,

[0091] Based on the initial and target positions of the surgical instruments, determine the target path; based on the target path and the first position, determine the reward value corresponding to the first operation instruction; the initial position represents the position of the surgical instruments in the user's blood vessel at the first moment; the target path represents the path with the shortest distance between the first position and the target position; and / or,

[0092] When the first position of the surgical instrument is located on the target path, the reward value corresponding to the first operation command is determined based on the first distance between the initial position and the target position, and the second distance between the first position and the target position; and / or;

[0093] The reward value corresponding to the first operation command is determined based on the contact force between the surgical instruments and the surgical robot.

[0094] Specifically, in this embodiment of the invention, the reward value corresponding to the operation instruction is used to evaluate the quality of the operation instruction, that is, to evaluate the degree of matching between the operation instruction and the current environmental image and the degree of contribution of the operation instruction to the current interventional procedure, that is, to evaluate whether the operation instruction can deliver the interventional device to the lesion location via the vascular cavity; optionally, the reward value of the operation instruction is determined by at least one of the following:

[0095] 1. Whether the target is achieved, that is, to operate the surgical instrument according to the first operation instruction and obtain the first position of the surgical instrument; to determine the reward value corresponding to the first operation instruction based on the distance between the first position and the target position;

[0096] Optionally, when the distal end of the guidewire is within 5 pixels of the target lesion location, the guidewire is considered to have reached the target location, and a reward value is obtained.

[0097] 2. Rewards for adhering to the correct delivery route, which means determining the target path based on the initial and target positions of the surgical instruments; and determining the reward value corresponding to the first operation instruction based on the target path and the first position.

[0098] Optionally, the target path is the shortest path from the starting position corresponding to the surgical instrument to the target lesion position; if the guidewire deviates from the target path and enters the wrong blood vessel branch, a penalty will be imposed; conversely, if the guidewire leaves the wrong blood vessel branch and returns to the target path, the robot will reward the user and grant a reward value.

[0099] 3. Dense reward for shortening the target distance, that is, when the first position of the surgical instrument is on the target path, the reward value corresponding to the first operation command is determined based on the first distance between the initial position and the target position and the second distance between the first position and the target position;

[0100] Optionally, the reward only applies when the guidewire is on the target path, where the distance to each pixel can be obtained simultaneously, and the reward is set to the number of pixels that reduce the observation distance.

[0101] 4. Penalty for contact force exceeding the safety threshold, that is, determining the reward value corresponding to the first operation command based on the contact force between the surgical instrument and the surgical robot. Optionally, the contact force between the surgical instrument and the surgical robot is estimated by the motor current.

[0102] The method described above determines the reward value corresponding to the operation command from multiple dimensions, such as the position of the surgical instrument and the target lesion after the operation command is executed, whether the surgical instrument is on the target path after the operation command is executed, whether the distance between the surgical instrument on the target path and the target lesion is reduced after the operation command is executed, and the contact force between the surgical instrument and the surgical robot. This allows the determined reward value to accurately reflect the degree of matching between the operation command output by the initial interventional surgery operation model and the current environmental image, as well as the contribution of the operation command to the current interventional surgery. This enables accurate training of the initial interventional surgery operation model, thereby allowing the trained interventional surgery operation model to accurately output the operation command of the interventional surgery, improving the operation efficiency and accuracy of the interventional surgery.

[0103] In one embodiment, the initial interventional surgical procedure model is trained to maximize the following training objective:

[0104]

[0105] r t γ represents the reward value of the surgical instrument operation command corresponding to the environmental image at time t; γ represents the decay coefficient; α represents the entropy regularization coefficient. Entropy represents the randomness of the operating instructions of surgical instruments; E represents the expected computation.

[0106] Specifically, in order to train the initial interventional surgical procedure model, the initial interventional surgical procedure model can be modeled as a Markov decision process, consisting of six tuples.<S,O,A,P,R,γ> Let S, O, and A represent the state space, respectively. Optionally, the state space includes the position of surgical instruments, the environmental image of the interventional surgery, and the operation instructions. P: S×A×S→[0,1] represents the state transition function, R: S×A→R represents the reward function, and γ represents the decay coefficient. At time t, based on the observed value o of the obtained environmental image... t ∈O selects an operation instruction action a t ∈A, then the state changes from s t ∈S with state transition probability P(s) t ,a t ,s t+1 ) transform into s t+1 ∈S, receive the reward (value) r corresponding to the operation instruction. t =R(s) t ,a t Optionally, the operation data (o) t ,a t ,o t+1 ,r t The data is stored in the database. The training objective of the initial surgical procedure model is to achieve a cumulative reward that is entropy-regularized. The largest, of which Let α represent entropy, and let α represent the entropy regularity coefficient.

[0107] The method described in the above embodiments maximizes the training objective by training an initial interventional surgery operation model, thereby maximizing or exceeding the cumulative reward of the entropy regularization of the operation instructions corresponding to the environmental image of the interventional surgery. This enables the trained interventional surgery operation model to accurately and effectively output operation instructions that match the environmental image, thus improving the efficiency and accuracy of interventional surgery.

[0108] In one embodiment, after acquiring the sample environment image at the first moment of the interventional procedure, the method further includes:

[0109] The instrument image and the user's blood vessel image in the first-time sample environment image are binarized respectively.

[0110] Specifically, after acquiring the sample environment image for the interventional surgery, the instrument image and the user's vascular image in the sample environment image can be binarized separately. This improves the initial interventional surgery operation model's ability to discriminate the environment image, making the trained interventional surgery operation model more accurate, and thus leading to more accurate interventional surgery operation commands. Specifically, in the user's vascular image, the vascular portion and the background portion have different colors; a pixel threshold can be used to obtain the binarized image of the vascular portion. The instrument image can be obtained by calculating the difference between the current image and the initial image without instruments. The binarized image of the instrument needs to undergo a closing operation to eliminate possible breaks and then be centered.

[0111] The method described in the above embodiments binarizes the instrument image and the user's blood vessel image in the sample environment image to improve the initial interventional surgery operation model's ability to discriminate the environment image. This makes the trained interventional surgery operation model more accurate, thereby obtaining more accurate interventional surgery operation instructions and improving the efficiency and accuracy of interventional surgery.

[0112] For example, a flowchart of the interventional surgery procedure is shown below. Figure 5 As shown, the operation data acquisition module receives the observed values ​​of the environmental images sent by the simulated surgical environment module. t Select control command a t It sends the data to the simulated surgery module and then receives a response from the simulated surgery environment module. t and new environmental image observations o t+1 Packaged into operation data (o t ,a t ,o t+1 ,r tThe data is sent to the operation data storage module. In the initial phase (the first 2000 operation data points), the operation data acquisition module randomly selects control commands; thereafter, it selects commands based on the learned operation skills. The operation data storage module stores operation data according to a certain capacity; when the capacity is full, the oldest operation data is deleted, and the newest operation data is saved first. The parameter update module randomly samples operation data from the operation data storage module and updates the parameters of the neural network representing the operation skills, periodically sending the neural network parameters to the operation data acquisition module to update its operation skills.

[0113] The operating device for interventional surgery provided by the present invention is described below. The operating device for interventional surgery described below can be referred to in correspondence with the operating method for interventional surgery described above.

[0114] Figure 6 This is a schematic diagram of the operating device for interventional surgery provided by the present invention. The operating device for interventional surgery provided in this embodiment includes:

[0115] The acquisition module 710 is used to acquire the target image corresponding to the interventional surgery; the target image includes instrument images and user vascular images.

[0116] Processing module 720 is used to input instrument images and user vascular images into the trained interventional surgery operation model to obtain target operation instructions; the interventional surgery operation model is trained based on sample environment images, the first operation instructions corresponding to the sample environment images, the reward value corresponding to the first operation instructions, and the sample images at the second time point after the execution of the first operation instructions.

[0117] The operation module 730 is used to perform interventional surgery according to the target operation instructions.

[0118] Optionally, the processing module 720 is specifically used to: acquire a sample environment image at the first moment of the interventional surgery; the sample environment image at the first moment includes a sample instrument image and a sample user blood vessel image;

[0119] Input the sample instrument image and the sample user vascular image into the initial interventional surgery operation model to obtain the first operation command;

[0120] The initial interventional surgery operation model is trained based on the first moment sample environment image, the first operation command, the reward value corresponding to the first operation command, and the second moment sample image corresponding to the execution of the first operation command, to obtain the trained interventional surgery operation model.

[0121] Optionally, the processing module 720 is specifically used to: obtain the operation data of the surgical instruments based on the first moment sample environment image of the interventional surgery, the first operation instruction, the reward value corresponding to the first operation instruction, and the second moment sample image corresponding to the execution of the first operation instruction;

[0122] The initial interventional surgical operation model is trained based on the operation data of the surgical instruments to obtain the trained interventional surgical operation model; the acquisition of the operation data of the surgical instruments and the training of the initial interventional surgical operation model based on the operation data of the surgical instruments are executed asynchronously.

[0123] Optionally, the processing module 720 is specifically configured to: operate the surgical instrument according to a first operation instruction to obtain a first position of the surgical instrument; determine a reward value corresponding to the first operation instruction based on the distance between the first position and a target position; the target position represents the location in the user's blood vessel where surgery needs to be performed; and / or,

[0124] Based on the initial and target positions of the surgical instruments, determine the target path; based on the target path and the first position, determine the reward value corresponding to the first operation instruction; the initial position represents the position of the surgical instruments in the user's blood vessel at the first moment; the target path represents the path with the shortest distance between the first position and the target position; and / or,

[0125] When the first position of the surgical instrument is located on the target path, the reward value corresponding to the first operation command is determined based on the first distance between the initial position and the target position, and the second distance between the first position and the target position; and / or;

[0126] The reward value corresponding to the first operation command is determined based on the contact force between the surgical instruments and the surgical robot.

[0127] Optionally, the processing module 720 is specifically used to: train the initial interventional surgical operation model to maximize the following training objective:

[0128]

[0129] r t γ represents the reward value of the surgical instrument operation command corresponding to the environmental image at time t; γ represents the decay coefficient; α represents the entropy regularization coefficient. Entropy represents the randomness of the operating instructions of surgical instruments.

[0130] Optionally, the processing module 720 is specifically used to: after acquiring the first-moment sample environment image of the interventional surgery, perform binarization processing on the instrument image and the user blood vessel image in the first-moment sample environment image respectively.

[0131] The apparatus of this invention is used to execute the method in any of the foregoing method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0132] Figure 7 A schematic diagram of the physical structure of an electronic device is provided. This electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions stored in the memory 830 to execute an interventional surgery operation method. This method includes: acquiring a target image corresponding to the interventional surgery; the target image includes an instrument image and a user vascular image; inputting the instrument image and user vascular image into a trained interventional surgery operation model to obtain a target operation instruction; the interventional surgery operation model is trained based on a sample environment image, a first operation instruction corresponding to the sample environment image, a reward value corresponding to the first operation instruction, and a sample image at a second time step after the execution of the first operation instruction; and performing the interventional surgery operation according to the target operation instruction.

[0133] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to perform the interventional surgery operation method provided by the above methods, the method comprising: acquiring a target image corresponding to the interventional surgery; the target image comprising an instrument image and a user vascular image; inputting the instrument image and the user vascular image into a trained interventional surgery operation model to obtain a target operation instruction; the interventional surgery operation model being trained based on a sample environment image, a first operation instruction corresponding to the sample environment image, a reward value corresponding to the first operation instruction, and a sample image at a second time point corresponding to the execution of the first operation instruction; and performing the interventional surgery operation according to the target operation instruction.

[0135] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the operation methods for the aforementioned interventional surgeries. The method includes: acquiring a target image corresponding to the interventional surgery; the target image including an instrument image and a user vascular image; inputting the instrument image and the user vascular image into a trained interventional surgery operation model to obtain a target operation instruction; the interventional surgery operation model being trained based on a sample environment image, a first operation instruction corresponding to the sample environment image, a reward value corresponding to the first operation instruction, and a sample image at a second time step after the execution of the first operation instruction; and performing the interventional surgery operation according to the target operation instruction.

[0136] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An operating device for interventional surgery, characterized in that, include: The acquisition module is used to acquire the target image corresponding to the interventional surgery; the target image includes instrument images and user blood vessel images; The processing module is used to input the instrument image and the user's vascular image into the trained interventional surgery operation model to obtain the target operation instruction; the interventional surgery operation model is trained based on the sample environment image, the first operation instruction corresponding to the sample environment image, the reward value corresponding to the first operation instruction, and the sample image at the second time point after the execution of the first operation instruction. The operation module is used to perform interventional surgery according to the target operation instructions; The interventional surgery operation model was trained based on the following method: Acquire a sample environment image at the first moment of the interventional procedure; the first moment of the sample environment image includes a sample instrument image and a sample user blood vessel image; The sample instrument image and the sample user vascular image are input into the initial interventional surgery operation model to obtain the first operation command; Based on the first moment sample environment image of the interventional surgery, the first operation command, the reward value corresponding to the first operation command, and the second moment sample image corresponding to the execution of the first operation command, the initial interventional surgery operation model is trained to obtain the trained interventional surgery operation model. The processing module is also used for: The surgical instrument is operated according to the first operation instruction to obtain a first position of the surgical instrument; a reward value corresponding to the first operation instruction is determined based on the distance between the first position and the target position; the target position represents the location in the user's blood vessel where surgery needs to be performed; and, Based on the initial and target positions of the surgical instruments, a target path is determined; based on the target path and the first position, a reward value corresponding to the first operation instruction is determined; the initial position represents the position of the surgical instruments in the user's blood vessel at the first moment; the target path represents the path with the shortest distance between the first position and the target position. and, When the first position of the surgical instrument is located on the target path, the reward value corresponding to the first operation instruction is determined based on the first distance between the initial position and the target position and the second distance between the first position and the target position; and, The reward value corresponding to the first operation command is determined based on the contact force between the surgical instrument and the surgical robot.

2. The interventional surgery operating device according to claim 1, characterized in that, The step of training the initial interventional surgery operation model based on the first-moment sample environment image of the interventional surgery, the first operation command, the reward value corresponding to the first operation command, and the second-moment sample image corresponding to the execution of the first operation command, to obtain the trained interventional surgery operation model, includes: Based on the first moment sample environment image of the interventional surgery, the first operation instruction, the reward value corresponding to the first operation instruction, and the second moment sample image corresponding to the execution of the first operation instruction, the operation data of the surgical instruments are obtained. The initial interventional surgical operation model is trained based on the operation data of the surgical instruments to obtain the trained interventional surgical operation model; the acquisition of the operation data of the surgical instruments and the training of the initial interventional surgical operation model based on the operation data of the surgical instruments are executed asynchronously.

3. The interventional surgery operating device according to claim 1, characterized in that, The initial interventional surgical procedure model is trained to maximize the following training objective: ； The The reward value represents the operation command of the surgical instrument corresponding to the environmental image at time t; Indicates the attenuation coefficient; the Represents the entropy regularity coefficient; the Entropy represents the randomness of the operating instructions of the surgical instrument; E represents the expected computation.

4. The interventional surgery operating device according to claim 1, characterized in that, The acquisition module is also used for: The instrument image and the user blood vessel image in the sample environment image at the first time point are binarized respectively.

Images