Interventional instrument distal end force prediction method, apparatus, and program product
By constructing a predictive model in vascular interventional surgery to extract the deformation information of interventional instruments from imaging data, and using an attention mechanism to generate weighted features and perform multi-task analysis, the accuracy and safety issues of distal force sensing in existing technologies are solved. This achieves low-cost, high-precision mechanical state feedback and real-time early warning, thereby improving surgical safety.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI UNITED IMAGING RES INST OF INTELLIGENT IMAGING
- Filing Date
- 2026-03-13
- Publication Date
- 2026-07-21
Smart Images
Figure CN122423967A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of medical device technology, and in particular relates to a method, device and procedure for predicting distal force of interventional devices. Background Technology
[0002] The clinical application of vascular interventional surgical robots is limited by the lack of force-tactile feedback mechanisms. Operators find it difficult to perceive the interaction between the distal end of the interventional device and the vessel wall, which can easily lead to serious complications such as vascular perforation. Existing solutions all have significant limitations and cannot meet clinical needs: for example, the indirect force measurement method using proximal sensors is affected by nonlinear frictional forces caused by vascular tortuosity, resulting in a low signal-to-noise ratio and distorted force transmission; the physical model visual-mechanical analysis method is highly dependent on the precise physical parameters of the device and the vessel wall, and has a large computational load, which cannot meet the real-time requirements of surgery; the invasive solution of integrating micro-sensors at the device tip will significantly increase the cost of consumables, change the mechanical properties of the device tip, and also pose clinical safety risks such as signal line breakage and biocompatibility; existing deep learning methods can only achieve device shape segmentation or motion posture prediction, without establishing a deep quantitative mapping between visual features and contact force values and contact states, and are limited in function and cannot provide effective mechanical feedback. Summary of the Invention
[0003] This application provides a method, device, and program product for predicting distal force of interventional devices, which can solve the technical problems existing in the prior art, such as proximal measurement distortion, poor real-time performance of physical modeling, high cost and safety risks of invasive sensors, and limited functionality of existing deep learning methods.
[0004] In a first aspect, embodiments of this application provide a method for predicting the distal force of an interventional device, the method comprising: Acquire imaging data of interventional devices within biological lumen; The image data is input into a prediction model; wherein the prediction model includes an encoder and a decoder; Based on the encoder, a first feature containing information about the deformation of interventional instruments is extracted from the image data; Based on the decoder, the first feature is processed to obtain attention-weighted features focused on the distal deformation region of the interventional device; Based on the decoder, the attention-weighted features are analyzed in a multi-task manner to output the predicted value of the distal contact force and the distal contact state level.
[0005] In one possible implementation of the first aspect, the processing of the first feature based on the decoder to obtain attention-weighted features focused on the distal deformation region of the interventional device includes: The first feature is processed based on the decoder to generate an attention weight mask for characterizing the distal position of the interventional device. The attention-weighted feature is generated based on the attention weight mask and the first feature.
[0006] In one possible implementation of the first aspect, the decoder includes a decoder localization head sub-network; the processing of the first feature based on the decoder to generate an attention weight mask for characterizing the distal position of the interventional device includes: The first feature is upsampled based on the decoder positioning head sub-network to obtain the second feature; The second feature is used for feature mapping to generate an attention weight mask for characterizing the distal position of the interventional device.
[0007] In one possible implementation of the first aspect, the decoder includes an attention fusion layer; generating the attention-weighted feature based on the attention weight mask and the first feature includes: The attention weight mask is resized based on the attention fusion layer to obtain an attention weight mask with the same size as the first feature. The attention weight mask, after size adjustment, is used as the weight mask and multiplied element-wise with the first feature to generate the attention weighted feature.
[0008] In one possible implementation of the first aspect, the decoder includes a multi-task execution sub-network; the multi-task execution sub-network includes a regression sub-network and a classification sub-network; the step of performing multi-task analysis on the attention-weighted features based on the decoder to output a predicted value of the distal contact force and a distal contact state level includes: The attention-weighted features are input in parallel into the regression subnetwork and the classification subnetwork; Based on the regression subnetwork, force regression analysis is performed on the attention-weighted features to output the predicted value of the distal contact force. Based on the classification subnetwork, the attention-weighted features are analyzed to determine the remote contact state, and the remote contact state level is output.
[0009] In one possible implementation of the first aspect, the training process of the prediction model includes: Obtain a training dataset; wherein the training dataset includes synchronously acquired image data of interventional devices, labels of the true values of distal contact force, and labels of the true levels of distal contact status. The training dataset is input into the initial prediction model, and iterative optimization is performed on the network parameters of the initial prediction model. By minimizing the multi-task joint loss function, the predicted value of the distal contact force output by the initial prediction model is made consistent with the label of the true value of the distal contact force, and the label of the distal contact state level output by the initial prediction model is made consistent with the label of the true value of the distal contact state.
[0010] In one possible implementation of the first aspect, the multi-task joint loss function is a weighted sum of the localization loss, the force regression loss, and the contact state classification loss; The positioning loss is used to monitor the accuracy of the attention weight mask generation for the distal position of the interventional device; the force regression loss is used to monitor the fit between the predicted distal contact force value and the label of the actual distal contact force value; and the contact state classification loss is used to monitor the matching degree between the label of the distal contact state level and the label of the actual distal contact state level value.
[0011] In one possible implementation of the first aspect, the method further includes: The output of the predicted remote contact force value is sent to the force feedback device so that the force feedback device generates corresponding force tactile feedback based on the predicted remote contact force value. The output of the remote contact status level is sent to the surgical robot so that when the remote contact status level reaches a preset threshold, the surgical robot executes a preset safety strategy.
[0012] Secondly, embodiments of this application provide a distal force prediction device for interventional devices, the device comprising: The acquisition module is used to acquire imaging data of interventional instruments within biological lumen; An input module is used to input the image data into a prediction model; wherein the prediction model includes an encoder and a decoder; The extraction module is used to extract a first feature containing interventional instrument deformation information from the image data based on the encoder; The feature focusing module is used to process the first feature based on the decoder to obtain attention-weighted features focused on the distal deformation region of the interventional device; The output module is used to perform multi-task analysis on the attention-weighted features based on the decoder, and output the predicted value of the far-end contact force and the far-end contact state level.
[0013] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the interventional device distal force prediction method described in any of the first aspects above.
[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the interventional device distal force prediction method described in any one of the first aspects.
[0015] Fifthly, embodiments of this application provide a computer program product that, when run on a computer device, causes the computer device to execute the interventional device distal force prediction method described in any of the first aspects above.
[0016] In this embodiment, a prediction model directly extracts a first feature containing information about the deformation of the interventional device from image data. An attention mechanism is then used to generate attention-weighted features focused on key areas. Multi-task analysis simultaneously outputs quantitative force values and qualitative state levels. By constructing an end-to-end prediction model from image to mechanical state, non-contact distal force measurement is achieved, avoiding the high cost and safety risks associated with invasive sensors. It also overcomes the signal distortion caused by friction in proximal measurements and the computational burden and poor real-time performance of physical modeling methods. The introduction of an attention mechanism allows the model to automatically focus on key areas where distal deformation of the interventional device occurs, effectively suppressing interference from complex vascular backgrounds and imaging noise, thereby improving the robustness and accuracy of the prediction. Multi-task analysis not only outputs continuous distal contact force predictions for precise force feedback but also simultaneously outputs discrete distal contact state levels for collision warning, solving the problem of limited functionality in existing deep learning methods. This provides more comprehensive mechanical state information for surgical robot systems, improving surgical safety.
[0017] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 This is a flowchart illustrating a method for predicting distal force of interventional devices according to an embodiment of this application; Figure 2 This is a schematic diagram of the internal structure and data flow of a prediction model provided in an embodiment of this application; Figure 3A schematic diagram of an experimental data acquisition platform provided in an embodiment of this application; Figure 4 This is a schematic diagram of the application system architecture for interaction with external devices during distal force prediction of interventional devices, provided in one embodiment of this application. Figure 5 This is a schematic diagram illustrating the visualization effect of overlaying a spatial attention heatmap with the original image data according to an embodiment of this application; Figure 6 This is a visual schematic diagram of the predicted effect of interventional device segmentation provided in an embodiment of this application; Figure 7 This is a schematic diagram showing the comparison of continuous multi-sample force values (predicted force value, actual force value, and propulsion force value) provided in an embodiment of this application; Figure 8 This is a schematic diagram comparing continuous multi-sample force values (predicted force value, actual force value, propulsion force value) of a blood vessel path according to an embodiment of this application; Figure 9 This is a schematic diagram comparing continuous multi-sample force values (predicted force value, actual force value, propulsion force value) of another vascular path provided in an embodiment of this application; Figure 10 This is a schematic diagram of the module structure of the distal force prediction device for interventional devices provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation
[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0021] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.
[0022] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0023] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."
[0024] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0025] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0026] Please see Figure 1 This application provides a method for predicting the distal force of interventional devices. This method achieves non-contact, high-precision force sensing by constructing a direct mapping between image features and mechanical states.
[0027] S101, acquire imaging data of interventional devices within the biological lumen.
[0028] The aforementioned image data can be generated by various medical imaging devices, such as Digital Subtraction Angiography (DSA), Intravascular Ultrasound (IVUS) probes, Optical Coherence Tomography (OCT) devices, or conventional endoscopic cameras. This image data exists in the form of digital images or video streams, reflecting the morphology and position of interventional devices in complex anatomical structures in real time.
[0029] Interventional devices refer to a class of flexible, slender medical devices used for interventional diagnostic and therapeutic procedures within biological lumens (such as blood vessels, the digestive tract, and the respiratory tract). These devices possess bendable and pushable mechanical properties, and during intervention, they undergo visually observable deformation when interacting with human tissues (such as blood vessel walls and organ mucosa). The contact state between the distal end and the tissue directly affects the safety of the treatment. Interventional devices include, but are not limited to: interventional guidewires, interventional microcatheters, balloon catheters, stent delivery systems, embolization microcatheters, biopsy catheters, and other flexible interventional devices that require advancement, retraction, and redirection within biological lumens.
[0030] S102, the above image data is input into the prediction model; wherein the prediction model includes an encoder and a decoder.
[0031] The prediction model refers to a pre-trained neural network with an encoder-decoder architecture, trained on a large amount of data, used to predict the distal mechanical state of an interventional device from its imaging data. In the embodiments of this application, the model can be specifically embodied as follows: Figure 2 The prediction model shown is logically composed of an encoder and a decoder.
[0032] S103, based on the encoder, extract a first feature containing information about the deformation of the interventional device from the image data.
[0033] The first feature refers to the high-dimensional feature map output by the encoder, which contains a compact and abstract representation of information about the shape, curvature, and position of the interventional device in the input image.
[0034] In the embodiments of this application, the encoder is typically composed of a series of convolutional layers, pooling layers, etc. It abstracts and reduces the dimensionality of high-resolution input image data layer by layer, and finally generates a low-resolution but high-dimensional feature map containing rich semantic information, namely the first feature.
[0035] S104, Based on the above decoder, the first feature is processed to obtain attention-weighted features focused on the distal deformation region of the interventional device.
[0036] Among them, attention-weighted features refer to features obtained after weighting the first feature. These features enhance the feature response of key areas related to the deformation of the distal end of the interventional device and suppress noise interference from the background and irrelevant areas, thereby making the model analysis more targeted and robust.
[0037] In this embodiment, the decoder analyzes the first feature, identifies the image region most relevant to the distal contact force (i.e., the area where the device tip and its vicinity deform), and generates an internal attention weight. This weight is used to adjust the first feature, enabling the model to focus on these key regions in subsequent analysis while ignoring interference from background information such as the blood vessel wall and other tissues.
[0038] S105, based on the above decoder, performs multi-task analysis on the above attention-weighted features, and outputs the predicted value of the far-end contact force and the far-end contact state level.
[0039] The predicted distal contact force is a continuous value (e.g., 0.15 N) that can be used for precise force feedback.
[0040] The remote contact status level is a classification result (e.g., safe, warning, danger) that can be used for intuitive risk warnings. Together, these two outputs constitute a comprehensive description of the remote mechanical state.
[0041] In this embodiment of the application, the attention-weighted features after attention focusing are analyzed by the above decoder for multi-task analysis, and two key results are output at the same time: a quantitative prediction value of the distal contact force and a qualitative level of the distal contact status.
[0042] This application establishes an end-to-end prediction pathway from pixel to force, eliminating the need for complex physical modeling or installing any physical sensors on the instrument. The prediction model autonomously learns the physical relationship between minute deformations of the instrument in the image and the force applied to the distal end, thereby achieving visual measurement of contact force. This approach is not only low-cost but also fast-responding, meeting the real-time feedback requirements during surgery.
[0043] In an optional embodiment, S104 processes the first feature based on the decoder described above to obtain attention-weighted features focused on the distal deformation region of the interventional device, including: Step a1: The first feature is processed based on the decoder to generate an attention weight mask for characterizing the distal position of the interventional device.
[0044] The attention weight mask refers to a single-channel feature map generated by the decoder after processing the first feature, which is used to accurately characterize the distal position of the interventional device. Its spatial dimension corresponds to the key area of the interventional device image.
[0045] Specifically, one branch within the decoder is dedicated to the localization task, which involves identifying the tip of the interventional device at the feature level. The output of this process is the attention weight mask, which acts like a transparent mask. Values close to 1 (indicating high attention) are applied to the location corresponding to the device tip, while values close to 0 (indicating neglect) are applied to other background areas. Its core function is to provide the attention weights for subsequent attention fusion layers. Through weighted fusion with the first feature, it enhances the distal deformation features of the device and filters background noise.
[0046] Optionally, this attention weight mask, also known as a spatial attention heatmap, is a single-channel image corresponding to the spatial dimension of the input image. The values at each location (usually between 0 and 1) represent the level of attention the model pays to that location. High-value areas correspond to regions that the model focuses on, such as the distal end of an interventional device. The spatial attention heatmap can accurately mark the location of the device tip in a highlighted manner.
[0047] Step a2: Based on the attention weight mask and the first feature, generate the attention weighted feature.
[0048] In this embodiment, after generating the attention weight mask, the model overlays this mask onto the original first feature. Through specific mathematical operations (e.g., element-wise multiplication), the portion of the first feature corresponding to the high-value region of the mask is preserved or even enhanced, while the portion corresponding to the low-value region is suppressed. The resulting new feature map is the attention-weighted feature. Compared to the first feature, it contains more information focused on the deformation of the distal end of the device.
[0049] In this embodiment, the aforementioned attention weight mask technique improves the model's predictive robustness. In clinical settings, the imaging background is extremely complex, potentially containing a beating heart, flowing contrast agents, other medical instruments, etc. Through the attention mechanism, the model can actively learn to ignore these disturbances, focusing its analysis on the instrument tip region, thereby ensuring stable and accurate mechanical state judgments even in environments with low signal-to-noise ratios.
[0050] In an optional embodiment, please refer to Figure 2 The decoder described above includes a decoder localization head sub-network. The step of generating the attention weight mask is correspondingly specified as follows: Step b1: Upsample the first feature based on the decoder positioning head sub-network to obtain the second feature.
[0051] The decoder localization head network is a specially designed lightweight neural network branch. Since the first feature is highly compressed by the encoder, its spatial resolution is low, while the attention weight mask needs to have a good spatial correspondence with the original image data. Therefore, one of the functions of the decoder localization head network is to restore the low-resolution first feature to a higher resolution through upsampling operations (such as transposed convolution or interpolation) to obtain the second feature.
[0052] Step b2: Perform feature mapping on the second feature to generate an attention weight mask for characterizing the distal position of the interventional device.
[0053] In this embodiment, after obtaining the second feature, the multi-channel second feature is converted into a single-channel attention weight mask with values between 0 and 1 through one or more convolutional layers (feature mapping) and an activation function (such as sigmoid). This network structure design makes the generation process of the attention weight mask more efficient and accurate.
[0054] In an optional embodiment, please refer to Figure 2 The decoder described above includes an attention fusion layer. The step of generating attention-weighted features based on the attention weight mask and the first feature is specifically defined as follows: Step c1: Adjust the size of the attention weight mask based on the attention fusion layer to obtain an attention weight mask with the same size as the first feature.
[0055] In this embodiment, the attention fusion layer is responsible for effectively combining the attention weight mask generated by the decoder localization head network with the first feature from the encoder. Since the two features are located in different positions within the network, their feature map sizes may differ. Therefore, the attention fusion layer first uses methods such as bilinear interpolation to precisely adjust the size of the attention weight mask to be exactly the same as the first feature.
[0056] Step c2: The attention weight mask with adjusted size is used as the weight mask and multiplied element-wise with the first feature to generate the attention weighted feature.
[0057] Subsequently, a weighting operation is performed, whereby at each spatial location of the feature map, the channel vector of the first feature is multiplied by the corresponding attention weight value (a scalar). This process is equivalent to broadcasting an attention weight mask, which is then multiplied element-wise with the multi-channel first feature. In this way, attention is precisely applied to the feature level, achieving effective filtering and enhancement of key information.
[0058] In order for the prediction model to perform the above prediction tasks, it needs to be trained. In an optional embodiment, the model training process includes the following steps: Step d1: Obtain the training dataset.
[0059] The training dataset includes synchronously acquired image data of interventional devices, labels of the true values of distal contact force, and labels of the true levels of distal contact status.
[0060] In this embodiment, a data acquisition platform can be built, which includes an image acquisition system, an interventional robot, a vascular phantom simulating a blood vessel, an interventional device to be tested, and a master control system. Through this data acquisition platform, image data of the interventional device moving within the vascular phantom, labels of the true value of the distal contact force measured by high-precision sensors, and labels of the true level of distal contact status based on the force value can be acquired simultaneously.
[0061] To train the initial prediction model, a high-quality training dataset is required. (Refer to...) Figure 3 This implementation constructs a data acquisition platform for generating strictly aligned training samples. The platform simulates a real surgical environment, including a slave surgical environment 1 and a master control system 2.
[0062] In the surgical environment 1, a transparent vascular phantom (14, 16-18) simulating the anatomy of the human coronary artery is set up. A support arm 12 is mounted next to the vascular phantom to mount and fix the image acquisition device 11, ensuring a stable shooting angle and precise alignment with the effective operating area of the vascular phantom. An interventional robot 13 is also configured, with interventional instruments 19 mounted on it. High-sensitivity pressure sensors 15 are pre-installed in key interactive areas of the vascular phantom, such as key locations on the vessel wall, to measure the normal force generated when the distal end of the interventional instrument 19 contacts the vessel wall in real time. After being fixed by the support arm 12, the image acquisition device 11 captures images of the vascular phantom from the outside, capturing the motion posture and deformation characteristics of the interventional instrument 19 within it in real time.
[0063] In the main control system 2, the main control console 22 receives the synchronization data from the multi-source data acquisition and synchronization module 24, sends instructions to the intervention robot 13, completes the millisecond-level clock synchronization setting, adjusts the image acquisition frame rate / sensor sampling frequency, and coordinates the hardware communication between the display 21, the operator 23, and the multi-source data acquisition and synchronization module 24 to ensure the timing consistency of each module.
[0064] Operator 23 controls the interventional robot 13 via the main control console 22, driving the interventional instrument 19 to perform various movements such as advancement, retraction, and turning within the vascular phantom. Simultaneously, the operator observes real-time surgical images provided by the image acquisition device 11 on the display 21. The host computer linked to the main control console 22 synchronously controls the image acquisition device 11 and the pressure sensor acquisition card, ensuring that each frame in the recorded video stream corresponds one-to-one with the force value measured by the sensor at the same moment. This generates a training dataset containing triples of image data, distal contact force true value labels, and distal contact state true level value labels. The distal contact state true level value labels can be categorized based on the distal contact force true value labels using preset thresholds. For example, a force value of 0N is defined as a safe state; a force value between 0N and 0.3N is defined as a warning state; and a force value greater than 0.3N is defined as a dangerous state. This data acquisition method provides accurate ground truth annotations for subsequent supervised learning of the model.
[0065] Step d2: Input the above training dataset into the initial prediction model, perform iterative optimization on the network parameters of the initial prediction model, minimize the multi-task joint loss function, so that the predicted value of the distal contact force output by the initial prediction model is consistent with the label of the true value of the distal contact force, and make the label of the distal contact state level output by the initial prediction model consistent with the label of the true value of the distal contact state.
[0066] In this embodiment, after obtaining the training dataset, it is input into an initial, parameter-randomized prediction model. The training process is an iterative optimization process. In each iteration, the model makes predictions based on the input image data and compares the prediction results with the corresponding ground truth labels in the dataset to calculate the value of a multi-task joint loss function. This loss value measures the difference between the current prediction and the actual situation.
[0067] Then, the system employs backpropagation and an optimizer (such as AdamW) to fine-tune all network parameters within the model (e.g., the weights of convolutional kernels) based on the loss value. The goal of this iterative optimization is to minimize the joint loss function across multiple tasks. Through thousands of iterations, the model's output of the predicted distal contact force gradually approaches the true distal contact force label, while the output of the distal contact state level becomes increasingly consistent with the true distal contact state level label. After training, the model possesses the ability to accurately predict the mechanical state from new imagery.
[0068] In an optional implementation, the multi-task joint loss function is a weighted sum of the localization loss, force regression loss, and contact state classification loss; the localization loss is used to supervise the generation accuracy of the attention weight mask for the distal position of the interventional device; the force regression loss is used to supervise the fit between the predicted distal contact force value and the label of the actual distal contact force value; and the contact state classification loss is used to supervise the matching degree between the label of the distal contact state level and the label of the actual distal contact state level.
[0069] The multi-task joint loss function refers to the overall objective function used to simultaneously optimize multiple prediction tasks (such as localization, force regression, and state classification) during model training. It is typically a weighted sum of the loss functions of each sub-task. By minimizing this function, the model's performance on each task can be balanced and synergistically improved.
[0070] In this embodiment, the multi-task joint loss function can be written as: Where Loss represents the total joint loss across multiple tasks; It is the localization loss, calculated using Dice loss or cross-entropy loss, used to supervise whether the attention weight mask generated by the model can accurately locate the distal end of the instrument. Its ground truth label can be a Gaussian heatmap centered on the tip of the instrument. It is the force regression loss, calculated using mean squared error (MSE) or mean absolute error (MAE) loss, used to supervise the fit between the model output's predicted far-end contact force and the label of the true far-end contact force. The contact state classification loss, calculated using methods such as cross-entropy loss, is used to supervise the matching degree between the model's output distant contact state level and the true distant contact state level label. α, β, and γ are the weighting coefficients of each loss term; these are hyperparameters that can be adjusted experimentally to balance the importance of different tasks during training. This design ensures that while learning force regression and state classification, the model must also learn accurate localization. These three tasks complement each other, jointly improving the overall performance of the model.
[0071] In an optional implementation, the multi-task analysis in S105 is described in detail. Please refer to... Figure 2The decoder described above includes a multi-task execution subnetwork, which in turn includes a regression subnetwork and a classification subnetwork. Attention-weighted features are input into these two subnetworks in parallel. The regression subnetwork performs force regression analysis on the attention-weighted features, outputting a scalar value—the predicted far-end contact force—through several fully connected layers. Simultaneously, the classification subnetwork performs far-end contact state analysis on the same attention-weighted features, outputting a vector representing the probability distribution of different states (e.g., safe, warning, danger) through fully connected layers and a softmax activation function. The class with the highest probability is the final far-end contact state level. This parallel network structure design allows the force prediction and state determination tasks to be performed independently based on the same focused features, without interference, ensuring the efficiency and accuracy of each prediction.
[0072] In a specific embodiment, the structure of the prediction model and its data processing flow can be clearly defined as follows. The model is configured to receive 3-channel RGB image data with a resolution of, for example, 640×640, as input. Through layer-by-layer calculations on the input image data, the model ultimately outputs a predicted far-end contact force value and a far-end contact state level. The internal data flow and the specific functions of each sub-network are detailed below: Step e1: The input is RGB image data of 3×640×640 (3 channels, resolution 640×640), and the target output is the predicted value of the far-end contact force and the far-end contact status level.
[0073] Step e2: The backbone network of the encoder performs multi-layer convolution, activation, and downsampling operations on the input image data to extract high-dimensional semantic features and output the first feature. This first feature can be a feature map with 1280 channels and a resolution of 20×20. This feature contains information related to the deformation of the interventional device.
[0074] In step e3, the decoder localization head sub-network upsamples the first feature and generates an attention weight mask to characterize the distal position of the interventional device through feature mapping. Optionally, this attention weight mask can be a single-channel feature map with a resolution of 160×160. After activation by the Sigmoid function, the pixel values in the distal region of the interventional device tend to be close to 1, while the pixel values in the background region tend to be close to 0.
[0075] Step e4: The attention fusion layer performs a size adjustment operation on the spatial attention heatmap based on the attention mechanism. It downsamples the heatmap to the same size as the first feature using methods such as bilinear interpolation. Then, it performs element-wise multiplication with the first feature to generate attention-weighted features, thereby suppressing invalid features such as vascular background and imaging noise, while enhancing the effective features of the deformed area at the distal end of the device.
[0076] Step e5: The multi-task execution sub-network receives attention-weighted features. This multi-task execution sub-network includes parallel regression and classification sub-networks. The specific processing flow is as follows: the attention-weighted features are first transformed into a one-dimensional feature vector through adaptive average pooling and flattening; then, the one-dimensional feature vector is input in parallel to the regression and classification sub-networks, both of which are processed through fully connected layers, etc. Specifically, the regression sub-network outputs continuous far-end contact force prediction values, and the classification sub-network outputs the far-end contact state level after passing through a Softmax activation function.
[0077] During the training phase, force regression loss (such as mean squared error MSE and mean absolute error MAE) and contact state classification loss (such as cross-entropy loss) need to be calculated separately. Together with the localization loss in step e3, they constitute a multi-task joint loss function. The model network parameters are iteratively optimized through backpropagation algorithm and optimizer to make the prediction results consistent with the true values.
[0078] Step e6: The output remote contact force prediction value is sent to the force feedback device. The force feedback device generates corresponding force tactile feedback (such as damping and vibration) based on the prediction value to provide the operator with physical intuitive perception. At the same time, the remote contact status level is sent to the surgical robot. When the status level reaches the preset danger threshold, the surgical robot automatically executes the preset safety strategy (such as stopping instrument advancement, automatic retraction, and audible and visual alarms) to achieve a closed loop of surgical safety protection.
[0079] It should be noted that the technical solution of this application is flexible in the selection of network architecture and attention mechanism, and is not limited by specific implementation form. Specifically, it may include the following variations: Backbone network replacement: The backbone network in the prediction model is not limited to EfficientNet, ResNet, MobileNet or ViT based on the Transformer architecture, which have feature extraction capabilities. As long as it can effectively extract high-dimensional semantic features containing deformation information from interventional device image data, it falls within the technical scope of this application.
[0080] Attention mechanism replacement: The attention mechanism is not limited to the attention weight mask generated by the positioning head (which can be represented as a spatial attention heatmap). It can also be replaced by convolutional block attention modules, compressed excitation modules, or self-attention mechanisms, as long as its logic is to use auxiliary tasks or attention weights to enhance the regression task's attention to the key deformation area of the interventional device, and can achieve background feature suppression and effective deformation feature enhancement, all of which fall within the protection scope of this application.
[0081] In one optional implementation, the intraluminal image data captured in real time by the image acquisition device is first transmitted to the distal force prediction device of the interventional device. Optionally, the distal force prediction device of the interventional device is deployed entirely within the edge-side intelligent computing module (…). Figure 4 (Not shown in the image), the edge-side intelligent computing module has a pre-trained and stored prediction model, and the interventional device distal force prediction device relies on this model to complete real-time inference of image data.
[0082] Optionally, the method also includes subsequent application steps. For example... Figure 4 As shown, in a complete application system, after completing the prediction, the distal force prediction device of the interventional device can send the output distal contact force prediction value to a force feedback device via a network or bus. This force feedback device (e.g., a handle operated by the physician) can generate corresponding force-tactile feedback (such as generating force, vibration, or damping changes of appropriate intensity) based on the received distal contact force prediction value, providing the operator with realistic force-tactile feedback, making it feel as if they can touch the distal blood vessel wall.
[0083] Simultaneously, the distal force prediction device for the interventional device can send the output distal contact status level to the surgical robot. A safety threshold can be preset in the surgical robot, and when the distal contact status level reaches the preset threshold, the surgical robot executes a preset safety strategy. For example, when the received status level is "dangerous," it is considered that a dangerous contact has occurred or is about to occur. At this time, the surgical robot can automatically execute the preset safety strategy, such as immediately stopping the forward movement of the interventional device or automatically retracting a short distance, thereby proactively avoiding damage to the blood vessel wall. From image acquisition to the robot executing the safety strategy, the entire closed-loop control process can be completed within milliseconds, achieving real-time proactive safety protection.
[0084] Optionally, after completing the forward propagation calculation and outputting the predicted values of the remote contact force and the remote contact status level in parallel, these prediction results (such as the predicted values of the remote contact force and the remote contact status level) can be visualized on the display screen in real time in the form of numerical values, dashboards, or color alerts.
[0085] The application scenarios of this application are not limited to the embodiments described above. For example, regarding input image modalities, in addition to visible light and X-ray, this method is also applicable to intravascular ultrasound or optical coherence tomography images, and the model can learn to extract the contact information between the instrument and the vessel wall from these higher resolution tomographic images. Regarding the network architecture, in addition to using a convolutional neural network, the encoder can also use an architecture such as ViT to extract features, while the core ideas of attention guidance and multi-task analysis remain unchanged.
[0086] In terms of clinical applications, this application can be used not only for coronary interventional surgery, but also extensively in neurovascular interventions (such as cerebral aneurysm embolization), peripheral vascular interventions, and flexible endoscopic surgeries of the digestive and respiratory tracts. Wherever flexible instruments are used in surgery and distal force sensing is required, this application has application potential. Furthermore, the mechanical information output by this application can also be used for surgical skills assessment and training, or as an important channel for transmitting remote environmental information in remote surgical systems.
[0087] To further verify the reliability and physical consistency of the method proposed in this application under complex clinical environments, this embodiment conducted multi-dimensional in-depth verification experiments, as follows: To further verify the reliability and physical consistency of the proposed method in complex clinical environments, this embodiment conducted multi-dimensional deep validation experiments. The collected dataset was divided into training, validation, and test sets. The AdamW optimizer was used for model training, and data augmentation strategies (such as random rotation and brightness adjustment) were introduced to improve the model's generalization ability. The model was evaluated on an independent test set, and the results showed that the proposed method has excellent performance. In terms of accuracy, the mean absolute error (MAE) of the model's prediction power was only 0.0150N, and the prediction accuracy reached 95.12% with an error tolerance of ±0.05N. In the ablation experiment, compared with the baseline model without the attention mechanism, the proposed method reduced the proportion of samples with large errors (≥0.1N) by 97.9%, demonstrating the effectiveness of the attention mechanism in focusing on key deformation features. Deploying the quantized model on an embedded edge computing platform, the inference speed can reach over 100 FPS, fully meeting the real-time feedback requirements of clinical surgery.
[0088] To verify that the model truly learns the morphological and mechanical characteristics of the interventional device, rather than relying on specific vascular background textures, this embodiment designed a background stripping robustness experiment. An image segmentation algorithm was introduced to completely remove the vascular background, light reflection, and shadows from the original input image, retaining only the device's own pixels to generate a clean device mask image as model input. Experimental data shows that even when the input image contains no vascular background information, the model can still accurately locate the device tip and output high-precision force prediction results. Compared to the original input, the prediction error did not significantly deteriorate (performance fluctuation <5%). This result strongly demonstrates that the attention-guided mechanism of this application has extremely strong generalization ability, automatically ignoring complex background noise and focusing on extracting the deformation features of the device itself. This means that the system has stable performance under different patients, different angiographic conditions, and even different vascular textures.
[0089] To verify the system's dynamic performance during continuous surgical procedures, this embodiment conducted a long-term continuous push-pull experiment and analyzed the relationship between the predicted force, the actual distal force, and the proximal advancement force. During continuous and rapid advancement and retraction of the interventional device, the predicted force curve output by the model and the true value curve measured by the sensor showed extremely high consistency. Whether in the rapid rise or steady holding phase of the force, the predicted curve closely and in real time matched the true value curve, without significant hysteresis or drift, demonstrating the system's excellent dynamic response characteristics. This verification proves that this application not only fits the data but also successfully learns the physical transmission laws of the interventional device under complex mechanical environments. A clear and reasonable physical causal relationship exists between the predicted distal force and the operator's proximal input.
[0090] To further demonstrate the precise focusing effect of the attention mechanism of this application on the tip region of the interventional device, this embodiment provides a visualization diagram showing the superimposed effect of the spatial attention heatmap generated by the decoder positioning head sub-network and the original image data. Please refer to [link / reference]. Figure 5 The figure uses a two-column layout: the upper image shows the original input image data (3×640×640 RGB image data) of the interventional device within the biological lumen, fully presenting the original morphology of the interventional device, the vascular background, and surrounding tissues; the lower image is the superposition effect of the spatial attention heatmap (i.e., a visualization of the attention weight mask) and the original input image data, where the tip area of the interventional device is highlighted with a high-brightness heatmap (pixel value close to 1), while the vascular background and irrelevant areas show low attention (pixel value close to 0), and the model's output distal contact force prediction value (e.g., 0.18N) is also labeled. The visualization results clearly demonstrate that the decoder localization head sub-network of this application can effectively guide the model to focus on the critical deformation area of the interventional device tip, accurately filter background noise interference, and the distal contact force prediction value output based on this focusing result has high accuracy.
[0091] To visually demonstrate the predictive reliability of the proposed model in a scenario without background interference, this embodiment provides a visualization of the prediction effect after interventional device segmentation. Please refer to [link / reference]. Figure 6The figure uses a three-column layout: the left side shows the original image data of the interventional device within the biological lumen (3×640×640 RGB image data), fully presenting the original morphology of the device, vascular background, and surrounding tissues; the middle side shows the image of the interventional device itself after image segmentation algorithm processing, completely removing irrelevant information such as vascular background, light reflection, and shadows, retaining only the pixels of the interventional device itself; the right side shows the model output results, including the spatial attention heatmap (i.e., a visualization of the attention weight mask) and the superposition effect of the segmented interventional device image (the tip area of the interventional device is highlighted in a high-brightness heatmap), as well as the corresponding predicted value of distal contact force (e.g., 0.21N). The visualization results clearly demonstrate that even when the input image contains only the interventional device itself and no background information, the decoder positioning head sub-network of this application can still accurately locate the tip area of the interventional device, and the predicted value of distal contact force output by the model still has high accuracy.
[0092] To quantify the model's prediction accuracy on a continuous batch of samples, this embodiment provides a comparative diagram of force values (predicted force value, actual force value, and propulsion force value) across multiple consecutive samples. Please refer to [link / reference]. Figure 7 The figure uses a two-column layout: the top column shows the comparison curve between the predicted and actual distal contact force values, with the horizontal axis representing the sample index and the vertical axis representing the force value (unit: N). It is evident that the predicted force curve and the actual force curve almost completely overlap, visually confirming the previously stated quantification result that the mean absolute error (MAE) is only 0.0150 N. The bottom column shows the proximal thrust force curve for the corresponding sample, with the horizontal axis consistent with the top column and the vertical axis representing the proximal thrust force value (unit: N). The comparison shows that the trend of proximal thrust force changes is highly synchronized with the distal contact force, but the peak value is significantly higher. This difference conforms to physical laws—when the interventional device moves within the blood vessel, the proximal input thrust is attenuated due to internal friction of the device and contact friction with the vessel wall, further corroborating the physical consistency of the model's prediction results. Figure 8 and Figure 9 These are continuous batches of samples from different blood vessel locations, and the patterns they exhibit are the same. Figure 7 .
[0093] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0094] Corresponding to the distal force prediction method for interventional devices described in the above embodiments, this application also provides a distal force prediction device for interventional devices. Please refer to... Figure 10In one embodiment, the distal force prediction device for interventional instruments can be a hardware device integrating a high-performance computing unit (such as a GPU or AI chip). The device includes multiple logic modules, the functions of which are implemented by software programs running on the computing unit.
[0095] Specifically, the distal force prediction device for interventional devices includes an acquisition module for acquiring image data of the interventional device within a biological lumen and transmitting the data to an input module.
[0096] The input module is used to input the aforementioned image data into the prediction model; wherein the aforementioned prediction model includes an encoder and a decoder.
[0097] The extraction module is used to receive the data processed by the input module and extract the first feature containing the deformation information of the interventional device from the above image data.
[0098] The feature focusing module is used to receive the first feature output by the extraction module, process the first feature, and obtain the attention-weighted feature focused on the deformation region of the distal end of the interventional device.
[0099] The output module receives the attention-weighted features output by the feature focusing module, performs multi-task analysis on these features, and outputs the predicted value of the distal contact force and the distal contact state level. Furthermore, the output module is responsible for sending these results to external force feedback devices or surgical robots via communication interfaces (such as Ethernet or CAN bus).
[0100] In a specific physical implementation, the aforementioned distal force prediction device for interventional instruments can be an edge computing box, which can house one or more embedded AI computing platforms. This device connects to the image acquisition equipment via a USB interface or high-speed network port, and to the force feedback device and surgical robot via Ethernet or serial port, forming a compact and efficient intelligent sensing and control hub.
[0101] To more comprehensively demonstrate the technical solution of this application, an embodiment integrating the aforementioned optional technical features is described below. This embodiment aims to implement a closed-loop force feedback system with active safety protection function. The system uses... Figure 10 The distal force prediction device of the interventional instrument shown is the core component.
[0102] In this embodiment, firstly, the training method described in the embodiment is used to collect data from the data acquisition platform to fully train a prediction model that includes all optional structures (i.e., a decoder localization head sub-network, an attention fusion layer, and a multi-task execution sub-network). The training objective is to minimize the multi-task joint loss function composed of localization loss, force regression loss, and contact state classification loss.
[0103] After training, the model is deployed on the distal force prediction device for interventional instruments. During the procedure, the image acquisition equipment captures real-time X-ray image sequences containing the interventional instruments and transmits them to the device. The device's acquisition and input modules acquire and process the images.
[0104] Next, the extraction module (encoder) extracts the first feature from the image. The feature focusing module (decoder localization head network and attention fusion layer) generates an accurate attention weight mask based on the first feature. The highlighted area of this mask will be focused on the instrument tip as expected, and then attention-weighted features are generated.
[0105] The output module (multi-task execution sub-network) receives attention-weighted features and quickly calculates the predicted value of the distal contact force (e.g., 0.35N) and the distal contact status level (e.g., danger).
[0106] The device immediately distributes the results through two channels: on the one hand, it sends a force value of 0.35N to the force feedback device, allowing the operator to feel a strong vibration or resistance from the handle, clearly perceiving excessive distal contact force; on the other hand, it sends the danger level to the surgical robot's controller. Upon receiving this signal, the controller triggers preset safety strategies, such as immediately stopping the motor driving the instrument forward and automatically retracting 2 millimeters. The entire closed loop of perception-decision-execution can be completed quickly, effectively preventing the risk of vascular perforation caused by operator delays or misjudgments.
[0107] This embodiment combines attention mechanisms, multi-task learning, specific training strategies, and closed-loop control with external devices to provide not only both quantitative and qualitative feedback information, but more importantly, to enable the system to have proactive safety protection capabilities. Its prediction accuracy is achieved through the attention mechanism and joint loss function, with the predicted curve closely matching the actual curve; its real-time performance is achieved through an efficient network structure and hardware platform; and its safety is achieved through closed-loop linkage with the robot controller.
[0108] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.
[0109] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0110] This application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.
[0111] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0112] This application provides a computer program product that, when run on a computer device, enables the computer device to execute the steps described in the various method embodiments above.
[0113] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 11 As shown, the computer device of this embodiment includes: at least one processor 120 ( Figure 11 (Only one is shown in the diagram), memory 121, and computer program 122 stored in said memory 121 and executable on said at least one processor 120, wherein said processor 120 executes said computer program 122 to implement the steps in any of the above embodiments of the distal force prediction method for interventional devices.
[0114] The computer device may include, but is not limited to, a processor 120 and a memory 121. Those skilled in the art will understand that... Figure 11The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, they may also include input / output devices, network access devices, etc.
[0115] The processor 120 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.
[0116] In some embodiments, the memory 121 may be an internal storage unit of the computer device, such as a hard disk or memory. In other embodiments, the memory 121 may be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 121 may include both internal and external storage units of the computer device. The memory 121 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 121 can also be used to temporarily store data that has been output or will be output.
[0117] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / computer equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.
[0118] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0119] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0120] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0121] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0122] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for predicting distal force of interventional devices, characterized in that, The method includes: Acquire imaging data of interventional devices within biological lumen; The image data is input into a prediction model; wherein the prediction model includes an encoder and a decoder; Based on the encoder, a first feature containing information about the deformation of interventional instruments is extracted from the image data; Based on the decoder, the first feature is processed to obtain attention-weighted features focused on the distal deformation region of the interventional device; Based on the decoder, the attention-weighted features are analyzed in a multi-task manner to output the predicted value of the distal contact force and the distal contact state level.
2. The method as described in claim 1, characterized in that, The process of processing the first feature based on the decoder to obtain attention-weighted features focused on the distal deformation region of the interventional device includes: The first feature is processed based on the decoder to generate an attention weight mask for characterizing the distal position of the interventional device. The attention-weighted feature is generated based on the attention weight mask and the first feature.
3. The method as described in claim 2, characterized in that, The decoder includes a decoder localization head sub-network; the processing of the first feature based on the decoder to generate an attention weight mask for characterizing the distal position of the interventional device includes: The first feature is upsampled based on the decoder positioning head sub-network to obtain the second feature; The second feature is used for feature mapping to generate an attention weight mask for characterizing the distal position of the interventional device.
4. The method as described in claim 2, characterized in that, The decoder includes an attention fusion layer; generating the attention-weighted feature based on the attention weight mask and the first feature includes: The attention weight mask is resized based on the attention fusion layer to obtain an attention weight mask with the same size as the first feature. The attention weight mask, after size adjustment, is used as the weight mask and multiplied element-wise with the first feature to generate the attention weighted feature.
5. The method as described in claim 1, characterized in that, The decoder includes a multi-task execution sub-network; the multi-task execution sub-network includes a regression sub-network and a classification sub-network; the multi-task analysis based on the attention-weighted features of the decoder, outputting the predicted value of the distal contact force and the distal contact state level, includes: The attention-weighted features are input in parallel into the regression subnetwork and the classification subnetwork; Based on the regression subnetwork, force regression analysis is performed on the attention-weighted features to output the predicted value of the distal contact force. Based on the classification subnetwork, the attention-weighted features are analyzed to determine the remote contact state, and the remote contact state level is output.
6. The method as described in claim 2, characterized in that, The training process of the prediction model includes: Obtain a training dataset; wherein the training dataset includes synchronously acquired image data of interventional devices, labels of the true values of distal contact force, and labels of the true levels of distal contact status. The training dataset is input into the initial prediction model, and iterative optimization is performed on the network parameters of the initial prediction model. By minimizing the multi-task joint loss function, the predicted value of the distal contact force output by the initial prediction model is made consistent with the label of the true value of the distal contact force, and the label of the distal contact state level output by the initial prediction model is made consistent with the label of the true value of the distal contact state.
7. The method as described in claim 6, characterized in that, The multi-task joint loss function is a weighted sum of the localization loss, force regression loss, and contact state classification loss; The positioning loss is used to monitor the accuracy of the attention weight mask generation for the distal position of the interventional device; the force regression loss is used to monitor the fit between the predicted distal contact force value and the label of the actual distal contact force value; and the contact state classification loss is used to monitor the matching degree between the label of the distal contact state level and the label of the actual distal contact state level value.
8. The method as described in claim 1, characterized in that, The method further includes: The output of the predicted remote contact force value is sent to the force feedback device so that the force feedback device generates corresponding force tactile feedback based on the predicted remote contact force value. The output of the remote contact status level is sent to the surgical robot so that when the remote contact status level reaches a preset threshold, the surgical robot executes a preset safety strategy.
9. A distal force prediction device for interventional instruments, characterized in that, The device includes: The acquisition module is used to acquire imaging data of interventional instruments within biological lumen; An input module is used to input the image data into a prediction model; wherein the prediction model includes an encoder and a decoder; The extraction module is used to extract a first feature containing interventional instrument deformation information from the image data based on the encoder; The feature focusing module is used to process the first feature based on the decoder to obtain attention-weighted features focused on the distal deformation region of the interventional device; The output module is used to perform multi-task analysis on the attention-weighted features based on the decoder, and output the predicted value of the far-end contact force and the far-end contact state level.
10. A computer program product, characterized in that, Includes a computer program that, when run, implements the method as described in any one of claims 1 to 8.