Method for training microcatheter delivery model and related product

Abstract

The invention discloses a method for training a microcatheter delivery model and a related product. The method comprises the following steps: constructing a physical simulation interventional operation environment and an in-vitro simulation interventional operation environment; in a physical simulation interventional operation environment, taking the first image and the first force feedback as training data, taking the micro-catheter conveying model as a strategy network of a reinforcement learning agent, and training the micro-catheter conveying model by adopting a deep reinforcement learning algorithm to obtain a trained micro-catheter conveying model; and deploying the trained micro-catheter conveying model in an in-vitro simulation interventional operation environment, and performing fine adjustment on the trained micro-catheter conveying model based on the second image, the second force feedback and an expected result. By utilizing the scheme disclosed by the invention, the microcatheter can be more accurately conveyed to a focus position in an actual operation, and the navigation success rate of the catheter in a complex cerebrovascular path can be improved.

Description

Technical Field

[0001] The present disclosure generally relates to the field of computer vision technology. More specifically, the present disclosure relates to a method, electronic device, and computer-readable storage medium for training a microcatheter delivery model. Further, the present disclosure also relates to a method, electronic device, and computer-readable storage medium for delivering a microcatheter. Background Art

[0002] In the field of neurointervention, microcatheters are key devices for treating cerebrovascular diseases such as aneurysms, arteriovenous malformations, vascular stenosis, and acute stroke. However, due to the characteristics of cerebral blood vessels, which are large curvature, multiple branches, and small diameter, microcatheters face many limitations during delivery. For example, when entering branches such as the middle cerebral artery or posterior communicating artery, microcatheters need to be highly flexible and precise in operation. At present, in the process of delivering microcatheters to the lesion site, doctors not only spend a lot of time, but also are exposed to digital subtraction angiography (DSA) radiation, causing irreversible damage to the doctor's health.

[0003] Although existing interventional robots can perform microcatheter delivery via remote control, there are still some problems. First, interventional robots require doctors to operate outside the operating room. Since doctors cannot directly touch the microcatheter and the robot lacks fine force feedback, doctors cannot sense the tension generated when the microcatheter moves, and it is difficult to accurately control the force of microcatheter delivery. This not only prolongs the operation time, but may also expose the patient to more radiation. Secondly, inaccurate control may also lead to unexpected situations during the operation. For example, excessive force may cause vascular spasm and consumables to fall off, endangering the patient's life safety.

[0004] In view of this, there is an urgent need to provide a solution for training microcatheter delivery models in order to achieve precise control of microcatheter delivery, improve the degree of automation of interventional robots in surgical applications, improve the success rate of catheter navigation in complex cerebrovascular pathways, reduce the risk of errors and complications caused by human operation, and improve the accuracy of microcatheter positioning and treatment material delivery. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes a solution for training a microcatheter delivery model in the following aspects.

[0006] In a first aspect, the present disclosure provides a method for training a microcatheter delivery model, comprising: constructing a physical simulation interventional surgery environment, the physical simulation interventional surgery environment comprising a blood vessel simulation model, a microcatheter simulation model and a mechanical model, wherein the mechanical model is used to perform mechanical control simulation on the microcatheter simulation model so that the microcatheter simulation model moves in the blood vessel simulation model and obtains a first force feedback in real time, and the first image in motion is simulated in real time using computer vision technology; constructing an in vitro simulated interventional surgery environment, the in vitro simulated interventional surgery environment comprising a blood vessel entity model, a surgical microcatheter, a camera, a control module and a mechanical module, wherein a force sensor is provided at the delivery end of the surgical microcatheter, and the force sensor is provided at the delivery end of the surgical microcatheter. The device is used to obtain a second force feedback in real time, the control module and the mechanical module are used to operate the surgical microcatheter so that the surgical microcatheter moves in the vascular entity model, and the camera is used to collect the second image in motion in real time; in the physical simulation interventional surgery environment, the first image and the first force feedback are used as training data, the microcatheter delivery model is used as a policy network of a reinforcement learning agent, and a deep reinforcement learning algorithm is used to train it to obtain a trained microcatheter delivery model; and the trained microcatheter delivery model is deployed in the in vitro simulated interventional surgery environment, and the trained microcatheter delivery model is fine-tuned based on the second image, the second force feedback and the expected result.

[0007] In some embodiments, the microcatheter delivery model is used as a policy network of a reinforcement learning agent and trained using a deep reinforcement learning algorithm, including: extracting visual features from the first image to obtain visual features contained in the first image, wherein the visual features include direction and position information of the microcatheter, vascular path, and geometric information and position information of the lesion target; determining a reward value using a reward function based on the visual features and the first force feedback; and optimizing the parameters of the policy network according to the reward value.

[0008] In some embodiments, the reward function is used to reward a state of being close to the lesion target and safely advanced, and to punish a state of being deviated from the lesion target and dangerously advanced, and based on the visual feature and the first force feedback, determining the reward value using the reward function includes: determining the position state of the microcatheter and the lesion target based on the visual feature; determining the advancement state of the microcatheter based on the first force feedback; and determining the reward value based on the position state and the advancement state.

[0009] In some embodiments, determining the propulsion state of the microcatheter based on the first force feedback includes: determining whether the first force feedback is greater than or equal to a preset threshold; when the first force feedback is greater than or equal to the preset threshold, determining that the propulsion state of the microcatheter is dangerous propulsion; and when the first force feedback is less than the preset threshold, determining that the propulsion state of the microcatheter is safe propulsion.

[0010] In some embodiments, the microcatheter delivery model includes an input layer and an output layer; the input layer receives the first image and the first force feedback, and performs feature extraction on the first image and the first force feedback to obtain image features; the output layer generates an operation plan for the microcatheter simulation model based on the image features, the operation plan includes an operation action and an operation force, and the operation action includes any one of forward, backward, twisting or stillness.

[0011] In some embodiments, the input layer includes a visual feature extraction module, a force feedback feature extraction module and a feature fusion module; the visual feature extraction module uses a convolutional neural network to extract visual features of the first image to obtain a feature map of the first image; the force feedback feature extraction module uses a fully connected network to extract geometric features of the first force feedback and the first image to obtain a feature vector; and the feature fusion module performs feature fusion on the feature map and the feature vector to obtain a feature vector in a unified state as the image feature.

[0012] In some embodiments, fine-tuning the trained microcatheter delivery model based on the second image, the second force feedback and the expected result includes: inputting the second image and the second force feedback into the trained microcatheter delivery model to generate an operation plan to obtain an operation plan for the surgical microcatheter; determining the difference between the operation plan and the expected result; and fine-tuning the parameters of the trained microcatheter delivery model according to the difference.

[0013] In a second aspect, the present disclosure provides a method for delivering a microcatheter, comprising: acquiring vascular images and force feedback to be processed in a real interventional surgery environment; and inputting the vascular images and force feedback to be processed into a microcatheter delivery model trained by the method described in the first aspect and its multiple embodiments to generate an operation plan, so as to output an operation plan for the microcatheter in the real interventional surgery environment.

[0014] In a third aspect, the present disclosure provides an electronic device comprising: a processor; and a memory storing program instructions for training a microcatheter delivery model and / or for delivering a microcatheter, wherein when the program instructions are executed by the processor, the method described in the first aspect and its multiple embodiments and / or the method described in the second aspect are implemented.

[0015] In a fourth aspect, the present disclosure provides a computer-readable storage medium having stored thereon program instructions for training a microcatheter delivery model and / or for delivering a microcatheter, wherein when the program instructions are executed by a processor, the method described in the aforementioned first aspect and its multiple embodiments and / or the method described in the aforementioned second aspect and its multiple embodiments are implemented.

[0016] The solution for training the microcatheter delivery model provided above, by constructing a physical simulation interventional surgery environment and an in vitro simulation interventional surgery environment, and combining a deep reinforcement learning algorithm, trains and fine-tunes the microcatheter delivery model, so that the microcatheter delivery model can learn how to accurately move forward, backward, and rotate according to the movement of the microcatheter in the blood vessel and force feedback information during the training process, so that in actual surgery, the microcatheter can be more accurately delivered to the lesion location, which can improve the navigation success rate of the catheter in complex cerebrovascular pathways and reduce the error and complication risk of human operation. In addition, by combining deep learning technology with surgical robots, more intelligent and automated surgical operations can be achieved, and the accuracy and safety of the surgery can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0018] Figure 1 An exemplary flow chart of a method for training a microcatheter delivery model according to an embodiment of the present disclosure is shown;

[0019] Figure 2 An exemplary framework diagram of a system for training a microcatheter delivery model using a deep reinforcement learning algorithm according to an embodiment of the present disclosure is shown;

[0020] Figure 3 An exemplary flow chart showing a process of training a microcatheter delivery model using a deep reinforcement learning algorithm according to an embodiment of the present disclosure;

[0021] Figure 4An exemplary flow chart showing a method for delivering a microcatheter according to an embodiment of the present disclosure;

[0022] Figure 5 An exemplary structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0024] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0025] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0026] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0027] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.

[0028] In the process of obtaining the disclosed solution, the inventors found that the cost of directly training the microcatheter delivery model in a real interventional surgery environment is too high. In order to save training costs, the microcatheter input model can be first trained in a physical simulation interventional surgery environment, and then supervised fine-tuned in an in vitro simulation interventional surgery environment. After the microcatheter input model is verified to be stable, it can be used for in vivo experiments and real interventional surgeries.

[0029] Figure 1 An exemplary flow chart of a method 100 for training a microcatheter delivery model according to an embodiment of the present disclosure is shown. It is understood that the method 100 may be executed by any appropriate device with data processing capabilities, such as but not limited to a terminal device, a processor, and a server.

[0030] Based on this, Figure 1 As shown, at step S101, method 100 can construct a physical simulation interventional surgery environment and an in vitro simulated interventional surgery environment. Specifically, the physical simulation interventional surgery environment may include a vascular simulation model (also referred to as a vascular anatomical model), a microcatheter simulation model, and a mechanical model, wherein the mechanical model is used to perform mechanical control simulation on the microcatheter simulation model so that the microcatheter simulation model moves in the vascular simulation model, and obtains force feedback (in order to distinguish, it can be referred to as the first force feedback) of the microcatheter simulation model in real time. Under the physical simulation interventional surgery environment, the virtual vascular image (in order to distinguish, it can be referred to as the first image) in motion (i.e., the microcatheter simulation model moves in the vascular simulation model) can be simulated in real time using computer vision technology.

[0031] Here, computer vision technology is used to obtain a virtual vascular image in motion, which can be achieved through the following operations: for the three-dimensional virtual scene of the microcatheter simulation model moving in the vascular simulation model, different projection angles are selected for projection to generate a DSA two-dimensional silhouette image similar to that during surgery. Specifically, the projection can be to project the vascular simulation model as white and the microcatheter simulation model as black.

[0032] The aforementioned vascular simulation model is a three-dimensional anatomical model used to simulate real cerebral blood vessels, which can be obtained by using three-dimensional tomographic images, such as computed tomography (CT), magnetic resonance imaging (MRI) and computed tomography angiography (CTA) to perform three-dimensional reconstruction and aneurysm segmentation. Here, aneurysm segmentation can obtain geometric information and position information of the lesion target.

[0033] The geometric information of the lesion target may include the morphological characteristics of the lesion, the diameter of the blood vessel at the lesion, and the degree of stenosis. Here, the morphological characteristics of the lesion may include the size, shape, and volume of the lesion. The position information of the lesion target may include one or more of spatial positioning, vascular branch positioning, and vascular path tracking. Spatial positioning refers to the specific position of the lesion target in three-dimensional space, which can be represented by a coordinate system; vascular branch positioning refers to the vascular branch where the lesion is located; vascular path tracking refers to the path and distribution of the lesion along the blood vessel. The path of the lesion along the blood vessel refers to the specific direction and position of the lesion in the vascular network, which includes the starting point of the lesion, the vascular branches passed through, and the end point. The distribution of the lesion refers to the distribution characteristics of the lesion in the vascular network, including the length, width, density, and curvature of the lesion.

[0034] The aforementioned microcatheter simulation model can be a model established using discrete elastic rod technology to simulate the bending force, torsion force and tensile force of the microcatheter. Discrete elastic rod technology is a physical model used to simulate deformable linear objects. The technology discretizes continuous elastic rods into a series of nodes and rod units, and the nodes are connected by rod units. In the simulation, the deformation and movement of the object can be simulated by introducing elastic forces and constraints between these rod units.

[0035] Specifically, by setting the bending stiffness, torsional stiffness, and tensile stiffness between the rod units, the deformation behavior of the microcatheter during bending, torsion, and stretching can be simulated respectively. Later, during the simulation process, by setting the fixed end and free end of the microcatheter and applying bending force, the deformation of the microcatheter can be observed; by applying torque at one end of the microcatheter, the torsion angle and stress distribution of the microcatheter can be observed; by applying tension at both ends of the microcatheter, the elongation and stress distribution of the microcatheter can be observed.

[0036] In the simulation, each node can be regarded as a rack, and the number of racks depends on the degree of discretization. The more racks there are, the higher the accuracy of the simulation, but the greater the amount of calculation. When the elastic rod is subjected to external forces (such as wall pressure, thrust, friction and blood flow impact), these forces will act on the nodes of the rod and be transmitted to adjacent nodes through the nodes, gradually establishing tension inside the rod. The aforementioned mechanical model can be used to calculate the tension generated by the microcatheter due to external forces (wall pressure, thrust, friction and blood flow impact between the blood vessel wall during transportation), thereby obtaining the force returned to the microcatheter delivery end, which is the first force feedback. In practical applications, the first force feedback is used to evaluate the safety of microcatheter advancement.

[0037] The aforementioned in vitro simulated interventional surgery environment may include a vascular entity model, a surgical microcatheter, a camera, a control module, and a mechanical module, wherein a force sensor is provided at the delivery end of the surgical microcatheter, and the force sensor is used to obtain force feedback in real time (for the sake of distinction, it can be referred to as the second force feedback). The control module and the mechanical module are used to operate the surgical microcatheter so that the surgical microcatheter moves in the vascular entity model. The camera is used to collect the vascular image in motion (i.e., the surgical microcatheter moves in the vascular entity model) in real time (for the sake of distinction, it can be referred to as the second image).

[0038] In the disclosed embodiment, the vascular entity model can use a vascular model made of materials such as silicone, gelatin or agar gel, and can also add a simulated blood circulation system. The surgical microcatheter is a microcatheter used for real surgery. When the control module and the mechanical module are used to operate the surgical microcatheter, the control module is specifically used to convert the operation scheme output by the microcatheter delivery model into a signal that can control the movement of the mechanical module, so as to control the operation of the mechanical module. The mechanical module is specifically used to mechanically control the delivery end of the surgical microcatheter, including forward, backward and rotation. In actual operation, two sets of roller units with opposite directions can be used. By driving the roller units to rotate under the drive of the motor, the friction force is used to push the guide wire forward or withdraw backward, so that the surgical microcatheter moves forward or backward, and the surgical microcatheter is rotated by moving the two sets of roller units up and down.

[0039] Then, at step S102, method 100 can use the aforementioned first image and first force feedback as training data in a physical simulation interventional surgery environment, use the microcatheter delivery model as a policy network of a reinforcement learning agent, and train it using a deep reinforcement learning algorithm to obtain a trained microcatheter delivery model.

[0040] In the disclosed embodiment, the microcatheter delivery model may include but is not limited to an input layer and an output layer. The input layer may receive the aforementioned first image and first force feedback, and perform feature extraction on the first image and first force feedback to obtain image features. The output layer generates an operation scheme for the microcatheter simulation model based on the image features. The operation scheme may include an operation action and an operation force. The operation action may include any one of forward, backward, twisting or stillness, and the operation force may include strength and direction.

[0041] In the disclosed embodiment, the input layer may include a visual feature extraction module, a force feedback feature extraction module, and a feature fusion module. The visual feature extraction module uses a convolutional neural network to extract visual features of the first image to obtain a feature map of the first image. These feature maps can be regarded as abstract representations of the first image at different levels and scales, and the pixel value in each feature map represents the feature response intensity of the corresponding position. The multi-layer structure of the convolutional neural network makes the feature extraction hierarchical. The low-level convolutional layer extracts low-level features of the first image, such as edges, textures, etc.; the high-level convolutional layer extracts higher-level features, such as the global morphology of the blood vessels, the overall posture of the microcatheter, etc.

[0042] The force feedback feature extraction module uses a fully connected network to extract geometric features from the first force feedback and the first image to obtain a feature vector of fixed length. This feature vector contains information related to the geometric features in the first force feedback and the first image, such as the magnitude, direction, and position of the point of application of the force. The force feedback feature extraction module not only considers the first force feedback itself, but also combines the geometric information in the first image, making the extracted geometric features more comprehensive and comprehensive, and can better reflect the force conditions and geometric state of the microcatheter inside the blood vessel. The feature fusion module performs feature fusion on the feature map and feature vector to obtain a feature vector in a unified state as an image feature.

[0043] The aforementioned reinforcement learning agent is an artificial intelligence agent that learns the optimal behavior strategy by interacting with the environment. It receives observations from the environment, takes actions, obtains rewards, and updates its strategy to maximize the cumulative rewards. The workflow of a reinforcement learning agent usually includes the steps of observation, action, reward, and learning update.

[0044] Specifically, observation refers to the agent observing the current state of the environment and obtaining the necessary information to make decisions. These observations can be a combination of multiple data types. Action refers to the agent selecting an action based on its current strategy based on the observed state. This action aims to maximize the expected reward. Reward refers to the feedback provided by the environment based on the agent's actions, usually in the form of rewards. The reward quantifies the degree of success of the agent's actions in achieving the goal. Learning updates refer to the agent using reward information to update its strategy to improve future actions. The goal is to maximize the cumulative reward based on past results. To make it easier to understand how, we will later combine Figure 2 and Figure 3 The system and process of training the microcatheter delivery model using deep reinforcement learning algorithm are described in detail respectively.

[0045] Next, at step S103, method 100 may deploy the trained microcatheter delivery model in an in vitro simulated interventional surgery environment, and fine-tune the trained microcatheter delivery model based on the second image, the second force feedback, and the expected result.

[0046] Specifically, the second image and the second force feedback can be first input into the trained microcatheter delivery model to generate an operation plan to obtain an operation plan for the surgical microcatheter. Then, the difference between the operation plan and the expected result can be determined. In practical applications, the expected result can be an operation plan for the surgical microcatheter formulated by a senior neurointerventional physician based on the second image and the second force feedback. Thus, the difference between the operation plan generated by the microcatheter delivery model and the expected result may include differences in operating actions and differences in operating forces (intensity and direction). Finally, based on the differences, an optimization algorithm (such as a gradient descent method) can be used to fine-tune the parameters of the trained microcatheter delivery model so that the output of the microcatheter delivery model is as accurate as possible and the loss function is minimized. Thus, the obtained microcatheter delivery model is more versatile, safe and stable in the real world.

[0047] Combination of the above Figure 1 The method for training the microcatheter delivery model is described. By constructing a physical simulation interventional surgery environment and an in vitro simulation interventional surgery environment, and combining a deep reinforcement learning algorithm, the microcatheter delivery model is trained and fine-tuned, so that the microcatheter delivery model can learn how to accurately move forward, backward, and rotate according to the movement of the microcatheter in the blood vessel and force feedback information during the training process, so that the microcatheter can be more accurately delivered to the lesion location in actual surgery, which can improve the navigation success rate of the microcatheter in complex cerebrovascular pathways and reduce the error and complication risk of human operation. In addition, by combining deep learning technology with surgical robots, more intelligent and automated surgical operations can be achieved, improving the accuracy and safety of surgery.

[0048] Figure 2 An exemplary framework diagram of a system 200 for training a microcatheter delivery model using a deep reinforcement learning algorithm according to an embodiment of the present disclosure is shown. Figure 2 As shown, system 200 may include a reinforcement learning environment 201 , an observation module 202 , a reward module 203 , an agent module 204 , a control module 205 , and a mechanical module 206 .

[0049] The aforementioned reinforcement learning environment 201 may include blood vessels (such as the aforementioned blood vessel simulation model) and microcatheters (such as the aforementioned microcatheter simulation model), and the observation module 202 may include a visual feedback collection module 2021 and a force feedback collection module 2022. The visual feedback collection module 2021 is used to collect real-time intraoperative image information (such as digital subtraction angiography), and the direction and position information of the microcatheter, the blood vessel path, the geometric information and position information of the lesion target can be extracted by analyzing the image information later, and this information can be used to evaluate the current position, posture and position state of the microcatheter with respect to the lesion target. The force feedback collection module 2022 is used to collect the force of the microcatheter returning to the microcatheter delivery end due to the wall contact pressure, thrust, friction force and blood flow impact, and the force is used to evaluate the safety of microcatheter advancement.

[0050] The aforementioned reward module 203 can calculate the reward value using the information returned by the visual feedback collection module 2021 and the force feedback collection module 2022. Generally, the reward value increases when the microcatheter reaches the target area and approaches the lesion target, and decreases when the microcatheter does not reach the target area and is far away from the lesion target. In addition, the reward module 203 also punishes unsafe operations, and the reward value decreases when the microcatheter touches the blood vessel wall or the pressure of the wall exceeds a preset threshold.

[0051] The aforementioned agent module 204 adopts a deep reinforcement learning model, and uses a policy network and a value network to realize control decisions. The model input is visual features (i.e., real-time image information collected by the visual feedback collection module 2021) and mechanical signals (force collected by the force feedback collection module 2022), and the model output is the operation plan of the microcatheter. In addition, the control module 205 is used to convert the operation plan output by the model into a signal that can control the movement of the mechanical module, so as to control the operation of the mechanical module. The mechanical module 206 is used to mechanically control the delivery end of the microcatheter, including forward, backward and rotation.

[0052] Figure 3 An exemplary flow chart of the process 300 of training a microcatheter delivery model using a deep reinforcement learning algorithm according to an embodiment of the present disclosure is shown. Figure 3 The description is a specific implementation of the aforementioned step S102. Figure 1 The features described can apply analogously here.

[0053] like Figure 3As shown, at step S301, visual features can be extracted from the first image to obtain visual features contained in the first image. The visual features here may include but are not limited to the direction and position information of the microcatheter, the vascular path, and the geometric information and position information of the lesion target. Then, at step S302, a reward value can be determined using a reward function based on the visual features and the first force feedback. Finally, at step S303, the parameters of the policy network can be optimized according to the reward value.

[0054] In order to improve the efficiency and safety of microcatheter delivery, in the disclosed embodiment, the reward function is used to reward the state of being close to the lesion target and advancing safely, and to punish the state of being deviated from the lesion target and advancing dangerously. In other words, for the state of being close to the lesion target and advancing safely, the reward function can determine a positive reward value, and for the state of being deviated from the lesion target and advancing dangerously, the reward function can determine a negative reward value.

[0055] In actual operation, at the aforementioned step S302, the following operations may be specifically performed to determine the reward value using the reward function based on the visual feature and the first force feedback: determining the position state of the microcatheter and the lesion target based on the visual feature; determining the propulsion state of the microcatheter based on the first force feedback; and determining the reward value based on the position state and the propulsion state. Here, the position state may include a close state and a deviated state, and the recommended state may include a safe propulsion and a dangerous propulsion.

[0056] In the disclosed embodiment, the position information of the microcatheter may include one or more of spatial positioning and vascular branch positioning. Spatial positioning refers to the specific position of the microcatheter in three-dimensional space, which can be represented by a coordinate system; vascular branch positioning refers to the vascular branch where the microcatheter is located. Thus, when determining the position state of the microcatheter and the lesion target based on visual features, it can be determined according to the direction and position information of the microcatheter and the position information of the lesion target. Specifically, when it is determined according to the direction and position information of the microcatheter and the position information of the lesion target that the microcatheter moves in a direction away from the lesion target, it can be determined that the position state of the microcatheter and the lesion target is a deviation state; when it is determined according to the direction and position information of the microcatheter and the position information of the lesion target that the microcatheter moves in a direction close to the lesion target, it can be determined that the position state of the microcatheter and the lesion target is a close state.

[0057] In the disclosed embodiment, the propulsion state of the microcatheter is determined based on the first force feedback, and the following operations can be performed specifically: determine whether the microcatheter contacts the blood vessel wall. When the microcatheter does not contact the blood vessel wall, the propulsion state of the microcatheter can be determined to be safe propulsion. Conversely, when the microcatheter contacts the blood vessel wall, it can be determined whether the first force feedback is greater than or equal to a preset threshold. It can be understood that although it is difficult to directly obtain the wall-touching pressure on the microcatheter during the actual pushing process, there is a proportional relationship between the wall-touching pressure and the first force feedback. Therefore, we can indirectly evaluate the size of the wall-touching pressure on the microcatheter inside the blood vessel through the size of the first force feedback, thereby determining whether the recommended state of the microcatheter is safe propulsion or dangerous recommendation.

[0058] Further, when the first force feedback is less than the preset threshold, the propulsion state of the microcatheter is determined to be safe propulsion. On the contrary, when the first force feedback is greater than or equal to the preset threshold, the propulsion state of the microcatheter is determined to be dangerous propulsion. It is understood that those skilled in the art can select the specific value of the preset threshold according to actual needs, and this disclosure does not specifically limit this.

[0059] Combination of the above Figures 1 to 3 A method for training a microcatheter delivery model is described. Accordingly, the present disclosure also provides a method for delivering a microcatheter. Figure 4 , which shows an exemplary flow chart of a method 400 for delivering a microcatheter according to an embodiment of the present disclosure. It is understood that the method 400 can be executed by any appropriate device with data processing capabilities, such as but not limited to a processor, a terminal device, and a server.

[0060] like Figure 4 As shown, at step S401, method 400 can obtain the vascular image to be processed and force feedback in a real interventional surgery environment. Then, at step S402, method 400 can input the vascular image to be processed (such as digital subtraction angiography) and force feedback into the microcatheter delivery model to generate an operation plan, so as to output an operation plan for the microcatheter in the real interventional surgery environment. The microcatheter delivery model here is trained and generated according to the aforementioned method for training a microcatheter delivery model and its multiple embodiments.

[0061] In actual interventional surgery, by using the microcatheter delivery model to generate the microcatheter operation plan, the microcatheter can be delivered to the lesion more accurately, the navigation success rate of the microcatheter in complex cerebrovascular pathways can be improved, and the error and complication risk of human operation can be reduced. This helps to improve the safety of surgery and reduce the patient's postoperative recovery time.

[0062] Next, combine Figure 5An electronic device 500 provided in an embodiment of the present application is exemplarily introduced. Figure 5 As shown, the electronic device 500 of the embodiment of the present application may include a processor 501 , a memory 502 and a communication bus 503 .

[0063] In the process of a specific embodiment, the processor 501 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing image processing device (DSPD), a programmable logic image processing device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It is understandable that for different devices, the electronic device used to implement the function of the processor may also be other, which is not specifically limited in this embodiment.

[0064] In the embodiment of the present application, the communication bus 503 is used to realize the connection and communication between the processor 501 and the memory 502; the memory 502 stores program instructions for training the microcatheter delivery model and / or for delivering the microcatheter; when the processor 501 executes the program instructions stored in the memory 502, the present application is realized. Figures 1 to 3 The method described herein for training a microcatheter delivery model or a combination thereof Figure 4 A method for delivering a microcatheter is described.

[0065] Combination of the above Figure 5 An electronic device for training a microcatheter delivery model and / or for delivering a microcatheter that can be used to execute the present application is described. It should be understood that the device structure or architecture here is merely exemplary, and the implementation method and implementation entity of the present application are not limited thereto, but can be changed without departing from the spirit of the present application. It is understandable that the description of each embodiment in the present disclosure emphasizes the differences between the various embodiments, and the same or corresponding parts can be referenced to each other. For the purpose of brevity, the present disclosure will not repeat them one by one.

[0066] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by a software program. Therefore, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores program instructions for training a microcatheter delivery model and / or for delivering a microcatheter, and the program instructions can be used to implement the present application in combination with the present application. Figures 1 to 3 The method described herein for training a microcatheter delivery model or a combination thereof Figure 4 A method for delivering a microcatheter is described.

[0067] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0068] Although multiple embodiments of the present application have been shown and described herein, it is obvious to those skilled in the art that such embodiments are provided only by way of example. Those skilled in the art can think of many changes, modifications and alternatives without departing from the thought and spirit of the present application. It should be understood that in the process of practicing the present application, various alternatives to the embodiments of the present application described herein can be adopted. The attached claims are intended to limit the scope of protection of the present application, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A method for training a microcatheter delivery model, comprising: Constructing a physical simulation interventional surgery environment, the physical simulation interventional surgery environment comprising a blood vessel simulation model, a microcatheter simulation model and a mechanical model, wherein the mechanical model is used to perform mechanical control simulation on the microcatheter simulation model, so that the microcatheter simulation model moves in the blood vessel simulation model and obtains a first force feedback in real time, and the first image in motion is simulated in real time using computer vision technology; Constructing an in vitro simulated interventional surgery environment, the in vitro simulated interventional surgery environment comprising a vascular entity model, a surgical microcatheter, a camera, a control module and a mechanical module, wherein a force sensor is provided at the delivery end of the surgical microcatheter, the force sensor is used to obtain a second force feedback in real time, the control module and the mechanical module are used to operate the surgical microcatheter to make the surgical microcatheter move in the vascular entity model, and the camera is used to collect a second image in real time during the movement; In the physical simulation interventional surgery environment, the first image and the first force feedback are used as training data, the microcatheter delivery model is used as a policy network of a reinforcement learning agent, and a deep reinforcement learning algorithm is used to train the microcatheter delivery model to obtain a trained microcatheter delivery model; and The trained microcatheter delivery model is deployed in the in vitro simulated interventional surgery environment, and the trained microcatheter delivery model is fine-tuned based on the second image, the second force feedback and the expected result.

2. The method according to claim 1, wherein: Using the microcatheter delivery model as a policy network of a reinforcement learning agent and training it using a deep reinforcement learning algorithm includes: Extracting visual features from the first image to obtain visual features contained in the first image, wherein the visual features include direction and position information of the microcatheter, a blood vessel path, and geometric information and position information of a lesion target; determining a reward value using a reward function based on the visual feature and the first force feedback; and Optimizing the parameters of the policy network according to the reward value.

3. The method according to claim 2, wherein: The reward function is used to reward a state of being close to the lesion target and advancing safely, and to punish a state of being deviated from the lesion target and advancing dangerously, and based on the visual feature and the first force feedback, using the reward function to determine the reward value includes: Determine the position status of the microcatheter and the lesion target based on the visual features; determining an advancement state of the microcatheter based on the first force feedback; and The reward value is determined based on the position state and the propulsion state.

4. The method according to claim 3, wherein: Determining the advancement state of the microcatheter based on the first force feedback includes: Determining whether the first force feedback is greater than or equal to a preset threshold; When the first force feedback is greater than or equal to a preset threshold, determining that the advancement state of the microcatheter is dangerous advancement; and When the first force feedback is less than the preset threshold, it is determined that the advancement state of the microcatheter is safe advancement.

5. The method according to any one of claims 1 to 4, wherein: The microcatheter delivery model includes an input layer and an output layer; The input layer receives the first image and the first force feedback, and performs feature extraction on the first image and the first force feedback to obtain image features; The output layer generates an operation plan for the microcatheter simulation model based on the image features, wherein the operation plan includes an operation action and an operation force, and the operation action includes any one of advancing, retreating, twisting or staying still.

6. The method according to claim 5, wherein: The input layer includes a visual feature extraction module, a force feedback feature extraction module and a feature fusion module; The visual feature extraction module uses a convolutional neural network to extract visual features from the first image to obtain a feature map of the first image; The force feedback feature extraction module uses a fully connected network to extract geometric features from the first force feedback and the first image to obtain a feature vector; and The feature fusion module performs feature fusion on the feature map and the feature vector to obtain a feature vector in a unified state as the image feature.

7. The method according to claim 1, wherein: Fine-tuning the trained microcatheter delivery model based on the second image, the second force feedback, and the expected result includes: Inputting the second image and the second force feedback into the trained microcatheter delivery model to generate an operation plan to obtain an operation plan for the surgical microcatheter; determining a discrepancy between the operational scenario and the expected outcome; and The parameters of the trained microcatheter delivery model are fine-tuned according to the differences.

8. A method for delivering a microcatheter, comprising: Obtain the vascular images and force feedback to be processed in a real interventional surgery environment; as well as The vascular image to be processed and the force feedback are input into the microcatheter delivery model trained according to the method described in any one of claims 1 to 7 to generate an operation plan, so as to output an operation plan for the microcatheter in the real interventional surgery environment.

9. An electronic device, comprising: processor; as well as A memory storing program instructions for training a microcatheter delivery model and / or for delivering a microcatheter, wherein when the program instructions are executed by a processor, the method according to any one of claims 1 to 7 and / or the method according to claim 8 are implemented.

10. A computer-readable storage medium having stored thereon program instructions for training a microcatheter delivery model and / or for delivering a microcatheter, wherein when the program instructions are executed by a processor, the method according to any one of claims 1 to 7 and / or the method according to claim 8 are implemented.

Images