A guidance path planning method and system

Abstract

The application discloses a kind of guided path planning method and system.Method includes following steps:S1.MRI image and ultrasound image are obtained, multi-modal image registration and segmentation are carried out, and MRI image and ultrasound image are fused.S2.real-time modeling of electric field distribution, based on dynamic conductivity model and finite element method, the electric field intensity distribution of cartilage and surrounding tissue after electrode applied pulse is calculated in real time, to provide input for subsequent pulse parameter optimization and path planning.S3.pulse parameter adaptive optimization, through model predictive control algorithm, real-time adjustment pulse voltage and pulse width, so that the electric field intensity of ablation area accurately covers target cartilage, while protecting blood vessel / nerve sensitive area from damage.S4.restrained path search is carried out, and guided path planning is realized.

Description

Technical Field

[0001] This invention relates to the field of industrial robot technology, specifically to a guided path planning method and system. Background Technology

[0002] When applying irreversible electroporation (IRE) ablation technology, the guidance path of the IRE needs to be planned; otherwise, vascular and / or nerve damage can easily occur. Current technologies lack dynamic guidance methods based on electric field coverage requirements and cannot provide optimized guidance paths. Summary of the Invention

[0003] This invention provides a guided path planning method, the method comprising:

[0004] S1. Acquire MRI and ultrasound images, perform multimodal image registration and segmentation, and fuse MRI and ultrasound images;

[0005] S2. Real-time modeling of electric field distribution: Based on the dynamic conductivity model and finite element method, the electric field intensity distribution of cartilage and surrounding tissues after the application of pulses from the electrodes is calculated in real time, providing input for subsequent pulse parameter optimization and path planning;

[0006] S3. Pulse parameter adaptive optimization: Through model predictive control algorithm, the pulse voltage and pulse width are adjusted in real time to ensure that the electric field intensity of the ablation area accurately covers the target cartilage, while protecting the vascular / nerve sensitive areas from damage.

[0007] S4. Perform constrained path search to achieve guided path planning.

[0008] Furthermore, step S1 specifically includes:

[0009] S11. Multimodal image registration aligns MRI images to ultrasound images, ensuring precise alignment of the two anatomically and eliminating modal differences.

[0010]

[0011] in, The registration loss function is used to drive non-rigid registration of multimodal images by optimizing the registration loss function; I MRI For MRI images; I US The image is an ultrasound image; T represents the three-dimensional displacement field, i.e., the non-rigid deformation field; T(I) MRI The result after not applying a non-rigid deformation field T to the MRI images; λ1 is the squared norm of the non-rigid deformation field T, SSIM() is the structural similarity exponential function used to measure the structural similarity between the registered MRI and ultrasound images; λ1 and λ2 are weighting parameters.

[0012] S12. Cross-modal image fusion: The registered MRI and ultrasound images are fused and used as input to the segmentation network.

[0013] I f =concat(T(I MRI ),I US )

[0014] Among them, I f For the fused image, I MRI For MRI images; I US For ultrasound images; T(I) MRI The image is the result of applying a non-rigid deformation field T to the MRI image, i.e., the registered image; concat() is the stitching function;

[0015] S13. Based on the U-Net++ segmentation network, a multi-task segmentation network with a cross-modal attention mechanism is introduced to segment images:

[0016]

[0017] Among them, M cartilage For cartilage masking, a value of 1 indicates a cartilage region; M avoid This is a vascular / nerve mask, where a value of 1 indicates a vascular / nerve-sensitive area; W c W a Let b be the weight matrix. c b a For bias terms; τ c τ a These are the cartilage segmentation threshold and the blood vessel / nerve segmentation threshold, respectively; F decoder For the fused image I f The feature image obtained after internal processing by the U-Net++ network.

[0018] Furthermore, step S2 specifically includes:

[0019]

[0020] Where φ represents the electric potential distribution; The potential gradient is E; the electric field strength is E. σ0 is the initial conductivity; α is the conductivity correction coefficient; ||E|| is the electric field strength modulus; σ(E) is the dynamic conductivity, that is, the conductivity after the electric field strength changes.

[0021] Furthermore, step S3 specifically includes:

[0022]

[0023] Among them, t p,k V is the pulse width at time k; k E is the pulse voltage at time k; k Let f be the electric field at time k; f() is the nonlinear dynamic model; E k+1 The electric field at time k+1 is predicted; ||E k+1 || represents the magnitude of the electric field intensity at time k+1; E th Ω represents the minimum electric field strength for irreversible electroporation; Ω represents the target cartilage tissue region to be ablated; β represents the time penalty coefficient.

[0024] Dynamically adjust pulse parameters to select the optimal pulse voltage V and pulse width t. p Make the electric field strength ||E|| in the target cartilage region as close as possible to E th Pulse width t p Keep it as short as possible.

[0025] Furthermore, step S4 specifically includes:

[0026]

[0027] Where C(p) is the cost required to reach the target node p, w1, w3, w3 are weight parameters, d is the distance to node p, ||E|| is the magnitude of the electric field intensity, and E safe The electric field threshold is exp(), which is an exponential function, and Curvature is the path curvature to node p, used to control the path smoothness.

[0028] The present invention also relates to a system using the above-described guided path planning method, the system comprising:

[0029] The multimodal image fusion unit is used to acquire MRI and ultrasound images, perform multimodal image registration and segmentation, and fuse MRI and ultrasound images.

[0030] The modeling unit is used for real-time modeling of electric field distribution. Based on the dynamic conductivity model and the finite element method, it calculates the electric field intensity distribution of cartilage and surrounding tissues in real time after the electrode pulse is applied, providing input for subsequent pulse parameter optimization and path planning.

[0031] The optimization unit is used for adaptive optimization of pulse parameters. Through model predictive control algorithm, it adjusts pulse voltage and pulse width in real time so that the electric field intensity of the ablation area accurately covers the target cartilage, while protecting the vascular / nerve sensitive areas from damage.

[0032] The path planning unit is used to perform constrained path search and realize guided path planning.

[0033] The present invention also relates to a computer program product comprising a computer program that is executed by a processor for performing the method described above.

[0034] The present invention also relates to a computer-readable storage medium for storing a computer program that is executed by a processor for performing the method described above.

[0035] The technical solution of this invention integrates MRI and ultrasound images, and achieves cross-modal registration through deep learning to solve the problem of blurred cartilage boundaries; based on real-time tissue conductivity feedback, a dynamic finite element model is constructed to automatically adjust pulse parameters and optimize electric field distribution; combined with anatomical constraints and electric field coverage requirements, an optimized guidance path is generated to avoid vascular or nerve damage. Attached Figure Description

[0036] Figure 1 This is a system structure block diagram of the present invention. Detailed Implementation

[0037] The present invention will now be further described with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention. It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention.

[0038] Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0039] Embodiment 1 of the present invention relates to a guided path planning method, the method comprising the following steps:

[0040] S1. Acquire MRI and ultrasound images, perform multimodal image registration and segmentation, and fuse the MRI and ultrasound images.

[0041] S2. Real-time modeling of electric field distribution: Based on the dynamic conductivity model and finite element method, the electric field intensity distribution of cartilage and surrounding tissues after the electrode pulse is applied is calculated in real time, providing input for subsequent pulse parameter optimization and path planning.

[0042] S3. Pulse parameter adaptive optimization: Through model predictive control algorithm, the pulse voltage and pulse width are adjusted in real time to ensure that the electric field intensity of the ablation area accurately covers the target cartilage, while protecting the vascular / nerve sensitive areas from damage.

[0043] S4. Perform constrained path search to achieve guided path planning.

[0044] Step S1 specifically includes:

[0045] S11. Multimodal image registration: Aligning MRI images to ultrasound images to ensure precise anatomical alignment and eliminate modal differences, including performing the following processes:

[0046]

[0047] in, The registration loss function is used to drive non-rigid registration of multimodal images by optimizing the registration loss function; I MRI For MRI images; I US The image is an ultrasound image; T represents the three-dimensional displacement field, i.e., the non-rigid deformation field; T(I) MRI The result after not applying a non-rigid deformation field T to the MRI images; λ1 is the gradient squared norm of the non-rigid deformation field T, SSIM() is the structural similarity exponential function used to measure the structural similarity between the registered MRI image and the ultrasound image; λ1 and λ2 are weight parameters.

[0048] S12. Cross-modal image fusion: The registered MRI and ultrasound images are fused and used as input to the segmentation network. This includes performing the following processes:

[0049] I f =concat(T(I MRI ),I US )

[0050] Among them, I f For the fused image, I MRI For MRI images; I US For ultrasound images; T(I) MRI The result of applying a non-rigid deformation field T to the MRI image is the registered image; concat() is the stitching function.

[0051] S13. Based on the U-Net++ segmentation network, a multi-task segmentation network with a cross-modal attention mechanism is introduced to segment images. This includes performing the following processes:

[0052]

[0053] Among them, M cartilage For cartilage masking, a value of 1 indicates a cartilage region; M avoid This is a vascular / nerve mask, where a value of 1 indicates a vascular / nerve-sensitive area; W c W a Let b be the weight matrix. c b a For bias terms; τ c τ a These are the cartilage segmentation threshold and the blood vessel / nerve segmentation threshold, respectively; F decoder For the fused image I f The feature image obtained after internal processing by the U-Net++ network.

[0054] Step S2 specifically includes performing the following processes:

[0055]

[0056] Where φ represents the electric potential distribution; The potential gradient is E; the electric field strength is E. σ0 is the initial conductivity; α is the conductivity correction coefficient; ||E|| is the electric field strength modulus; σ(E) is the dynamic conductivity, that is, the conductivity after the electric field strength changes.

[0057] Step S3 specifically includes performing the following processes:

[0058]

[0059] Among them, t p,k V is the pulse width at time k; k E is the pulse voltage at time k; k Let f be the electric field at time k; f() is the nonlinear dynamic model; E k+1 The electric field at time k+1 is predicted; ||E k+1 || represents the magnitude of the electric field intensity at time k+1; E th Ω represents the minimum electric field strength for irreversible electroporation; Ω represents the target cartilage tissue region to be ablated, i.e., the cartilage mask M obtained in step S1. cartilage The defined region; β is the time penalty coefficient. The optimal pulse voltage V and pulse width t are selected by dynamically adjusting the pulse parameters. p Make the electric field strength ||E|| in the target cartilage region as close as possible to E th Pulse width t p Keep it as short as possible.

[0060] Step S4 specifically includes performing the following processes:

[0061]

[0062] Where C(p) is the cost required to reach the target node p, w1, w3, w3 are weight parameters, d is the distance to node p, ||E|| is the magnitude of the electric field intensity, and E safe The safe electric field threshold is defined by exp(), which is an exponential function. Curvature is the path curvature to node p, used to control the smoothness of the path and avoid sharp turns that could cause electrode deflection or tissue tearing.

[0063] Embodiment 2 of the present invention relates to a system using the guided path planning method of Embodiment 1, the system comprising:

[0064] The multimodal image fusion unit is used to acquire MRI and ultrasound images, perform multimodal image registration and segmentation, and fuse MRI and ultrasound images.

[0065] The modeling unit is used for real-time modeling of electric field distribution. Based on the dynamic conductivity model and the finite element method, it calculates the electric field intensity distribution of cartilage and surrounding tissues in real time after the electrode pulse is applied, providing input for subsequent pulse parameter optimization and path planning.

[0066] The optimization unit is used for adaptive optimization of pulse parameters. Through model predictive control algorithm, it adjusts pulse voltage and pulse width in real time to ensure that the electric field intensity of the ablation area accurately covers the target cartilage, while protecting vascular / nerve sensitive areas from damage.

[0067] The path planning unit is used to perform constrained path search and realize guided path planning.

[0068] Embodiment 3 of the present invention relates to a computer program product, the computer program product comprising a computer program executed by a processor for performing the method of Embodiment 1.

[0069] Embodiment 4 of the present invention relates to a computer-readable storage medium for storing a computer program, which is executed by a processor to perform the method of Embodiment 1.

[0070] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A guided path planning system, characterized in that, The system includes: The multimodal image fusion unit is used to acquire MRI and ultrasound images, perform multimodal image registration and segmentation, and fuse MRI and ultrasound images. The modeling unit is used for real-time modeling of electric field distribution. Based on the dynamic conductivity model and the finite element method, it calculates the electric field intensity distribution of cartilage and surrounding tissues in real time after the electrode pulse is applied, providing input for subsequent pulse parameter optimization and path planning. The optimization unit is used for adaptive optimization of pulse parameters. Through model predictive control algorithm, it adjusts pulse voltage and pulse width in real time so that the electric field intensity of the ablation area accurately covers the target cartilage, while protecting the vascular / nerve sensitive areas from damage. The path planning unit is used to perform constrained path search and realize guided path planning.

2. The guided path planning system according to claim 1, characterized in that, The system performs the following method: S1. Acquire MRI and ultrasound images, perform multimodal image registration and segmentation, and fuse MRI and ultrasound images; S2. Real-time modeling of electric field distribution: Based on the dynamic conductivity model and finite element method, the electric field intensity distribution of cartilage and surrounding tissues after the application of pulses from the electrodes is calculated in real time, providing input for subsequent adaptive optimization of pulse parameters and guidance path planning; S3. Pulse parameter adaptive optimization: Through model predictive control algorithm, the pulse voltage and pulse width are adjusted in real time to ensure that the electric field intensity of the ablation area accurately covers the target cartilage, while protecting the vascular / nerve sensitive areas from damage. S4. Perform constrained path search to achieve guided path planning.

3. The guided path planning system according to claim 2, characterized in that, Step S1 specifically includes: S11. Multimodal image registration aligns MRI images to ultrasound images, ensuring precise alignment of the two anatomically and eliminating modal differences: ； in, The registration loss function is used to drive non-rigid registration of multimodal images by optimizing the registration loss function; For MRI images; An ultrasound image; It is a three-dimensional displacement field, i.e., a non-rigid deformation field; To apply a non-rigid deformation field to MRI images The result; Non-rigid deformation field The gradient square norm, This is a structural similarity exponential function used to measure the structural similarity between registered MRI and ultrasound images. These are weight parameters; S12. Cross-modal image fusion: The registered MRI and ultrasound images are fused and used as input to the segmentation network. ； in, For the merged image, For MRI images; An ultrasound image; To apply a non-rigid deformation field to MRI images The result is the registered image; This is a concatenation function; S13. Based on the U-Net++ segmentation network, a multi-task segmentation network with a cross-modal attention mechanism is introduced to segment images: ； ； in, For cartilage masking, a value of 1 indicates the cartilage region; This is a vascular / nerve mask, where a value of 1 indicates a vascular / nerve sensitive area; This is the weight matrix. For bias terms; These are the cartilage segmentation threshold and the blood vessel / nerve segmentation threshold, respectively. For the merged image The feature image obtained after internal processing by the U-Net++ network.

4. The guided path planning system according to claim 2, characterized in that, Step S2 specifically includes: ； ； in, The potential distribution; The potential gradient; electric field strength ( ); The initial conductivity; |E|| is the conductivity correction factor; ||E|| is the electric field strength modulus. Dynamic conductivity refers to the conductivity that changes with the electric field strength.

5. A guided path planning system according to claim 2, characterized in that, Step S3 specifically includes: ； in, Let be the pulse width at time k; Let be the pulse voltage at time k; Let be the electric field at time k; It is a nonlinear dynamic model; The electric field at time k+1 is predicted. Let be the magnitude of the electric field intensity at time k+1; The minimum electric field strength for irreversible electroporation; The target cartilage tissue area that needs to be ablated; This is the time penalty coefficient; Dynamically adjust pulse parameters to select the optimal pulse voltage V and pulse width. Make the electric field strength ||E|| in the target cartilage region as close as possible to Pulse width Keep it as short as possible.

6. A guided path planning system according to claim 2, characterized in that, Step S4 specifically includes: ； in, To reach the target node The cost required, For weight parameters, To reach the node distance, The magnitude of the electric field intensity. The safe electric field threshold is given by exp(), which is an exponential function. To reach the node The path curvature is used to control the smoothness of the path.

Images