A tumor ablation image fusion modeling method based on deep learning

Abstract

The present invention discloses a tumor ablation image fusion modeling method based on deep learning, which relates to the field of intelligent medical technology. The method comprises: obtaining multiple preoperative tumor images of a tumor patient before performing ablation therapy puncture surgery on the tumor; fusing a first ultrasound tumor image, a first CT tumor image and a first MRI tumor image to obtain a preoperative fused tumor image; obtaining multiple intraoperative tumor images of the tumor patient during ablation therapy puncture surgery on the tumor; obtaining an intraoperative fused tumor image from a second ultrasound tumor image, a second CT tumor image and a second MRI tumor image; constructing a deep learning model based on extracted spatial pattern data and deep learning training data, and training and optimizing the constructed deep learning model to obtain a tumor ablation image fusion model. The present application realizes accurate evaluation of the therapeutic effect of the ablation therapy puncture surgery based on tumor ablation images.

Description

Technical Field

[0001] The present application relates to the field of intelligent medical technology, and in particular to a tumor ablation image fusion modeling method based on deep learning. Background Art

[0002] Tumor ablation therapy is a minimally invasive interventional therapy that uses physical or chemical means to directly act on tumor tissue to cause necrosis, thereby achieving the purpose of treating the tumor. This treatment method is usually carried out under the guidance of modern imaging technology to ensure the safety of the treatment.

[0003] In related technologies, after the tumor ablation treatment is completed, the attending physician usually evaluates the surgical effect based on the information observed during the operation. However, under this method, when the attending physician's judgment is wrong or the attending physician needs to be replaced, the evaluation results of the surgical effect may be biased.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiment of the present application provides a tumor ablation image fusion modeling method based on deep learning to solve the above technical problems.

[0006] The present application provides a tumor ablation image fusion modeling method based on deep learning, including:

[0007] Before performing ablation therapy puncture surgery on a tumor, a plurality of preoperative tumor images of the tumor patient are obtained; wherein the plurality of preoperative tumor images include a first ultrasound tumor image, a first CT tumor image, and a first MRI tumor image;

[0008] Performing image fusion on the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image to obtain a preoperative fused tumor image;

[0009] During the ablation therapy puncture surgery on the tumor, a plurality of intraoperative tumor images of the tumor patient are obtained; wherein the plurality of intraoperative tumor images include a second ultrasound tumor image, a second CT tumor image, and a second MRI tumor image; the plurality of intraoperative tumor images include an image at the time when the ablation therapy puncture surgery is being performed and an image at the time when the ablation therapy puncture surgery is finished;

[0010] Obtaining an intraoperative fused tumor image from the second ultrasound tumor image, the second CT tumor image, and the second MRI tumor image;

[0011] Extracting spatial pattern data and deep learning training data from both the preoperative fused tumor image and the intraoperative fused tumor image;

[0012] A deep learning model is constructed based on the extracted spatial pattern data and the deep learning training data, and the constructed deep learning model is trained and optimized to obtain a tumor ablation image fusion model; wherein the tumor ablation image fusion model is used to evaluate the therapeutic effect of the ablation treatment puncture surgery.

[0013] Furthermore, the performing image fusion on the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image to obtain a preoperative fused tumor image includes:

[0014] Performing spatial registration processing on the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image to obtain the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image after the registration processing;

[0015] The first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image after the registration processing are respectively input into the context dynamic extraction module for multi-scale decomposition to obtain a multi-scale feature atlas;

[0016] Downsampling the multi-scale feature map set in sequence through a downsampling structure of an image fusion neural network to obtain a first-scale coded feature map, a second-scale coded feature map, and a third-scale coded feature map;

[0017] The first-scale coded feature map, the second-scale coded feature map, and the third-scale coded feature map are fused to obtain the preoperative fused tumor image.

[0018] Furthermore, the fusing the first scale coding feature map, the second scale coding feature map and the third scale coding feature map to obtain the preoperative fused tumor image includes:

[0019] The first scale coding feature map, the second scale coding feature map and the third scale coding feature map are processed respectively using gray level co-occurrence matrices of different pixel scale components to obtain a first scale gray level co-occurrence map, a second scale gray level co-occurrence map and a third scale gray level co-occurrence map;

[0020] Performing feature point detection and matching on the first-scale grayscale symbiosis map, the second-scale grayscale symbiosis map, and the third-scale grayscale symbiosis map to obtain matching feature points;

[0021] Calculating the deviation values ​​of the coordinate values ​​of the matching feature points on the x-axis, y-axis and z-axis respectively to obtain the deviation basis function;

[0022]

[0023] Wherein, F(N,div1,div2,div3) represents the deviation basis function; N represents the number of the matching feature points; div1 represents the deviation value of the coordinate value of the matching feature point on the x-axis; div2 represents the deviation value of the coordinate value of the matching feature point on the y-axis; div3 represents the deviation value of the coordinate value of the matching feature point on the z-axis; α, β, and γ are all weight parameters;

[0024] The first-scale grayscale symbiosis map, the second-scale grayscale symbiosis map, and the third-scale grayscale symbiosis map are fused based on the deviation basis function to obtain the preoperative fused tumor image.

[0025] Furthermore, extracting spatial pattern data and deep learning training data from both the preoperative fused tumor image and the intraoperative fused tumor image includes:

[0026] Preprocessing the preoperative fused tumor image and the intraoperative fused tumor image;

[0027] Inputting the pre-processed preoperative fused tumor image into the tumor position determination model, marking the first position information of the tumor and the first position relationship between the tumor and its surrounding structures;

[0028] Inputting the preprocessed intraoperative fused tumor image into the tumor position determination model, marking the second position information of the tumor and the second position relationship between the tumor and its surrounding structures;

[0029] The spatial pattern data and the deep learning training data are extracted from the first position information of the tumor, the first position relationship between the tumor and its surrounding structures, the second position information of the tumor, and the second position relationship between the tumor and its surrounding structures.

[0030] Furthermore, the preprocessing of the preoperative fused tumor image and the intraoperative fused tumor image includes:

[0031] rigidly matching the preoperative fused tumor image and the intraoperative fused tumor image so that the physical positions of the preoperative fused tumor image and the intraoperative fused tumor image are the same;

[0032] The preoperative fused tumor image and the intraoperative fused tumor image after rigid matching are resampled to the same size.

[0033] Further, extracting the spatial pattern data and the deep learning training data from the first position information of the tumor, the first position relationship between the tumor and its surrounding structures, the second position information of the tumor, and the second position relationship between the tumor and its surrounding structures includes:

[0034] Performing joint registration on the first position information of the tumor and the second position information of the tumor, and outputting a first ablation position linkage image and a second ablation position linkage image;

[0035] generating a linkage attention feature map according to the first ablation position linkage image and the second ablation position linkage image;

[0036]

[0037] Among them, p ij represents the linkage attention feature map, and the linkage attention feature map is used to represent the correlation influence of the i-th key position in the first ablation position linkage image on the j-th key position in the second ablation position linkage image; f i represents the ith key position in the first ablation position linkage image; f j represents the jth key position in the second ablation position linkage image; exp() is an exponential function; M1 is the number of key positions in the first ablation position linkage image; M2 is the number of key positions in the second ablation position linkage image;

[0038] Based on the linkage attention feature map, the first positional relationship between the tumor and its surrounding structures, and the second positional relationship between the tumor and its surrounding structures, the spatial pattern data and the deep learning training data are generated.

[0039] Furthermore, the generating the spatial pattern data and the deep learning training data based on the linkage attention feature map, the first positional relationship between the tumor and its surrounding structures, and the second positional relationship between the tumor and its surrounding structures includes:

[0040] Performing joint registration on a first positional relationship between the tumor and its surrounding structures and a second positional relationship between the tumor and its surrounding structures, and outputting a positional relationship registration feature map;

[0041] The spatial pattern data and the deep learning training data containing spatial feature information are generated by utilizing the positional relationship registration feature map and the linkage attention feature map.

[0042] Furthermore, constructing a deep learning model according to the extracted spatial pattern data and the deep learning training data, and training and optimizing the constructed deep learning model to obtain a tumor ablation image fusion model includes:

[0043] Performing Gaussian denoising and data standardization on the spatial pattern data containing spatial feature information and the deep learning training data;

[0044] Constructing the deep learning model based on the spatial pattern data after Gaussian denoising and data standardization and the network architecture of KDBNet; wherein the network architecture of KDBNet includes a first graph neural network and a second graph neural network;

[0045] Using the spatial pattern data and deep learning training data after Gaussian denoising and data standardization, the constructed deep learning model is trained and optimized, and the model parameters are updated based on the following formula until the loss function converges to obtain a tumor ablation image fusion model;

[0046]

[0047] Among them, θ update represents the parameters of the deep learning model after update; θ0 represents the parameters of the deep learning model before update, ω1 represents the preset learning rate, represents the gradient of the loss function; represents the deep learning model before adjustment, and data represents the spatial pattern data and deep learning training data after Gaussian denoising and data standardization.

[0048] Furthermore, the location information of the tumor includes the location of the tumor, the morphology of the tumor and the size of the tumor.

[0049] Furthermore, the surrounding structures of the tumor include blood vessels, nerves and organs around the tumor.

[0050] Based on the embodiments provided by the present application, before performing ablation therapy puncture surgery on a tumor, multiple preoperative tumor images of the tumor patient are obtained; wherein the multiple preoperative tumor images include a first ultrasound tumor image, a first CT tumor image, and a first MRI tumor image; the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image are fused to obtain a preoperative fused tumor image; during the ablation therapy puncture surgery on the tumor, multiple intraoperative tumor images of the tumor patient are obtained; wherein the multiple intraoperative tumor images include a second ultrasound tumor image, a second CT tumor image, and a second MRI tumor image; the multiple intraoperative tumor images The tumor image includes an image of the ablation therapy puncture operation at the time of the operation and an image of the ablation therapy puncture operation at the time of the operation completion; the second ultrasound tumor image, the second CT tumor image and the second MRI tumor image are used to obtain an intraoperative fusion tumor image; spatial pattern data and deep learning training data are extracted from both the preoperative fusion tumor image and the intraoperative fusion tumor image; a deep learning model is constructed based on the extracted spatial pattern data and deep learning training data, and the constructed deep learning model is trained and optimized to obtain a tumor ablation image fusion model; wherein the tumor ablation image fusion model is used to evaluate the therapeutic effect of the ablation therapy puncture operation. Thus, the therapeutic effect of the ablation therapy puncture operation is accurately evaluated based on the tumor ablation image.

[0051] Specifically, it has the following beneficial effects: Comprehensive information acquisition: By acquiring the patient's multimodal preoperative and intraoperative tumor images (ultrasound, CT, MRI), more comprehensive tumor information can be obtained from different angles and different imaging technologies, providing more precise guidance for surgery; Enhanced effect of image fusion: Image fusion technology can integrate the advantages of different modal images, improve the contrast and resolution of the image, make the tumor boundary and internal structure clearer, and help doctors better identify and locate the tumor; Real-time monitoring and evaluation: Intraoperative images obtained during the operation can monitor the progress of tumor ablation treatment in real time, adjust the treatment plan in time, and improve the safety and effectiveness of the operation; Optimize surgical planning: Preoperative fusion images can help doctors make detailed surgical plans before surgery, select appropriate puncture paths, and predict possible risks and complications, thereby improving surgery. Improve surgical accuracy: Through the fusion of preoperative and intraoperative images, doctors can locate tumors more accurately, reduce damage to surrounding normal tissues, and improve the accuracy of ablation therapy; Evaluation of treatment effect: The constructed deep learning model can evaluate the effect of ablation therapy, and predict the treatment effect and possible recurrence risk by analyzing the changes in preoperative and postoperative images; Data-driven decision support: The extracted spatial pattern data and deep learning training data provide doctors with data-based decision support, which helps to achieve personalized and precise treatment; Improve surgical efficiency: The rapid analysis capability of the deep learning model helps to shorten the operation time, improve the utilization rate of the operating room and the efficiency of the allocation of medical resources; Reduce patient risks: Through precise image fusion and the assistance of deep learning models, uncertainty and potential errors in surgery can be reduced, thereby reducing patient risks and postoperative complications. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings described herein are used to provide a further understanding of the embodiments of the present invention and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0053] Figure 1 is a flowchart of an optional deep learning-based tumor ablation image fusion modeling method according to an embodiment of the present application;

[0054] Figure 2 The present invention is a flowchart of another optional deep learning-based tumor ablation image fusion modeling method according to an embodiment of the present application.

[0055] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0056] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0057] Alternatively, if Figure 1 As shown, the present application provides a tumor ablation image fusion modeling method based on deep learning, comprising:

[0058] S101, before performing a puncture surgery for ablation treatment of a tumor, obtaining a plurality of preoperative tumor images of a tumor patient; wherein the plurality of preoperative tumor images include a first ultrasound tumor image, a first CT tumor image, and a first MRI tumor image;

[0059] S102, performing image fusion on the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image to obtain a preoperative fused tumor image;

[0060] S103, during the process of performing ablation therapy puncture surgery on the tumor, obtaining a plurality of intraoperative tumor images of the tumor patient; wherein the plurality of intraoperative tumor images include a second ultrasound tumor image, a second CT tumor image, and a second MRI tumor image; the plurality of intraoperative tumor images include an image at the time when the ablation therapy puncture surgery is being performed and an image at the time when the ablation therapy puncture surgery is finished;

[0061] S104, obtaining an intraoperative fused tumor image from the second ultrasound tumor image, the second CT tumor image, and the second MRI tumor image;

[0062] S105, extracting spatial pattern data and deep learning training data from both the preoperative fused tumor image and the intraoperative fused tumor image;

[0063] S106, constructing a deep learning model based on the extracted spatial pattern data and deep learning training data, and training and optimizing the constructed deep learning model to obtain a tumor ablation image fusion model; wherein the tumor ablation image fusion model is used to evaluate the therapeutic effect of ablation treatment puncture surgery.

[0064] Based on the embodiments provided by the present application, before performing ablation therapy puncture surgery on a tumor, multiple preoperative tumor images of the tumor patient are obtained; wherein the multiple preoperative tumor images include a first ultrasound tumor image, a first CT tumor image, and a first MRI tumor image; the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image are fused to obtain a preoperative fused tumor image; during the ablation therapy puncture surgery on the tumor, multiple intraoperative tumor images of the tumor patient are obtained; wherein the multiple intraoperative tumor images include a second ultrasound tumor image, a second CT tumor image, and a second MRI tumor image; the multiple intraoperative tumor images The tumor image includes an image of the ablation therapy puncture operation at the time of the operation and an image of the ablation therapy puncture operation at the time of the operation completion; the second ultrasound tumor image, the second CT tumor image and the second MRI tumor image are used to obtain an intraoperative fusion tumor image; spatial pattern data and deep learning training data are extracted from both the preoperative fusion tumor image and the intraoperative fusion tumor image; a deep learning model is constructed based on the extracted spatial pattern data and deep learning training data, and the constructed deep learning model is trained and optimized to obtain a tumor ablation image fusion model; wherein the tumor ablation image fusion model is used to evaluate the therapeutic effect of the ablation therapy puncture operation. Thus, the therapeutic effect of the ablation therapy puncture operation is accurately evaluated based on the tumor ablation image.

[0065] Specifically, it has the following beneficial effects: Comprehensive information acquisition: By acquiring the patient's multimodal preoperative and intraoperative tumor images (ultrasound, CT, MRI), more comprehensive tumor information can be obtained from different angles and different imaging technologies, providing more precise guidance for surgery; Enhanced effect of image fusion: Image fusion technology can integrate the advantages of different modal images, improve the contrast and resolution of the image, make the tumor boundary and internal structure clearer, and help doctors better identify and locate the tumor; Real-time monitoring and evaluation: Intraoperative images obtained during the operation can monitor the progress of tumor ablation treatment in real time, adjust the treatment plan in time, and improve the safety and effectiveness of the operation; Optimize surgical planning: Preoperative fusion images can help doctors make detailed surgical plans before surgery, select appropriate puncture paths, and predict possible risks and complications, thereby improving surgery. Improve surgical accuracy: Through the fusion of preoperative and intraoperative images, doctors can locate tumors more accurately, reduce damage to surrounding normal tissues, and improve the accuracy of ablation therapy; Evaluation of treatment effect: The constructed deep learning model can evaluate the effect of ablation therapy, and predict the treatment effect and possible recurrence risk by analyzing the changes in preoperative and postoperative images; Data-driven decision support: The extracted spatial pattern data and deep learning training data provide doctors with data-based decision support, which helps to achieve personalized and precise treatment; Improve surgical efficiency: The rapid analysis capability of the deep learning model helps to shorten the operation time, improve the utilization rate of the operating room and the efficiency of the allocation of medical resources; Reduce patient risks: Through precise image fusion and the assistance of deep learning models, uncertainty and potential errors in surgery can be reduced, thereby reducing patient risks and postoperative complications.

[0066] Further, the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image are fused to obtain a preoperative fused tumor image, including:

[0067] Performing spatial registration processing on the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image to obtain the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image after the registration processing;

[0068] The first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image after the registration processing are respectively input into the context dynamic extraction module for multi-scale decomposition to obtain a multi-scale feature atlas;

[0069] The multi-scale feature map set is sequentially downsampled through the downsampling structure of the image fusion neural network to obtain a first-scale coding feature map, a second-scale coding feature map, and a third-scale coding feature map;

[0070] The first-scale coded feature map, the second-scale coded feature map, and the third-scale coded feature map are fused to obtain a preoperative fused tumor image.

[0071] Furthermore, the first scale coding feature map, the second scale coding feature map and the third scale coding feature map are fused to obtain a preoperative fused tumor image, including:

[0072] The first scale coding feature map, the second scale coding feature map and the third scale coding feature map are processed respectively using the gray level co-occurrence matrices of different pixel scale components to obtain a first scale gray level co-occurrence map, a second scale gray level co-occurrence map and a third scale gray level co-occurrence map;

[0073] Performing feature point detection and matching on the first-scale grayscale symbiosis map, the second-scale grayscale symbiosis map, and the third-scale grayscale symbiosis map to obtain matching feature points;

[0074] Calculate the deviation values ​​of the coordinate values ​​of the matching feature points on the x-axis, y-axis and z-axis respectively to obtain the deviation basis function;

[0075]

[0076] Among them, F(N,div1,div2,div3) represents the deviation basis function; N represents the number of matching feature points; div1 represents the deviation value of the coordinate value of the matching feature point on the x-axis; div2 represents the deviation value of the coordinate value of the matching feature point on the y-axis; div3 represents the deviation value of the coordinate value of the matching feature point on the z-axis; α, β, and γ are all weight parameters;

[0077] The first scale grayscale symbiosis map, the second scale grayscale symbiosis map and the third scale grayscale symbiosis map are fused based on the deviation basis function to obtain a preoperative fused tumor image.

[0078] It should be noted that the process of obtaining an intraoperative fused tumor image by using the second ultrasound tumor image, the second CT tumor image, and the second MRI tumor image can refer to the process of obtaining a preoperative fused tumor image by performing image fusion on the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image.

[0079] Alternatively, if Figure 2 As shown, spatial pattern data and deep learning training data are extracted from both preoperative fused tumor images and intraoperative fused tumor images, including:

[0080] S201, preprocessing the preoperative fusion tumor image and the intraoperative fusion tumor image;

[0081] S202, inputting the pre-processed preoperative fused tumor image into a tumor position determination model, marking first position information of the tumor and a first position relationship between the tumor and its surrounding structures;

[0082] S203, inputting the preprocessed intraoperative fusion tumor image into the tumor position determination model, marking the second position information of the tumor and the second position relationship between the tumor and its surrounding structures;

[0083] S204, extracting spatial pattern data and deep learning training data from the first position information of the tumor, the first position relationship between the tumor and its surrounding structures, the second position information of the tumor, and the second position relationship between the tumor and its surrounding structures.

[0084] Furthermore, the preoperative fused tumor image and the intraoperative fused tumor image are preprocessed, including:

[0085] Rigid matching is performed on the preoperative fused tumor image and the intraoperative fused tumor image, so that the physical positions of the preoperative fused tumor image and the intraoperative fused tumor image are the same;

[0086] The preoperative fused tumor image and the intraoperative fused tumor image after rigid matching are resampled to the same size.

[0087] Further, extracting spatial pattern data and deep learning training data from the first position information of the tumor, the first position relationship between the tumor and its surrounding structures, the second position information of the tumor, and the second position relationship between the tumor and its surrounding structures includes:

[0088] Performing joint registration on the first position information of the tumor and the second position information of the tumor, and outputting a first ablation position linkage image and a second ablation position linkage image;

[0089] generating a linkage attention feature map according to the first ablation position linkage image and the second ablation position linkage image;

[0090]

[0091] Among them, p ij represents the linkage attention feature map, which is used to represent the correlation influence of the i-th key position in the linkage image of the first ablation position on the j-th key position in the linkage image of the second ablation position; f i represents the i-th key position in the first ablation position linkage image; f j represents the jth key position in the second ablation position linkage image; exp() is an exponential function; M1 is the number of key positions in the first ablation position linkage image; M2 is the number of key positions in the second ablation position linkage image;

[0092] Based on the linkage attention feature map, the first positional relationship between the tumor and its surrounding structures, and the second positional relationship between the tumor and its surrounding structures, spatial pattern data and deep learning training data are generated.

[0093] Based on the embodiments provided in the present application, the following beneficial effects are achieved: accurate tumor positioning: by jointly registering the anterior-posterior position information of the tumor, an accurate ablation position linkage image can be generated to provide support for accurate positioning and tracking of the tumor; enhanced feature representation: the generated linkage attention feature map can highlight the key positions of the tumor and its surrounding structures, enhance the representation capability of image features, and help improve the accuracy of subsequent analysis; quantification of correlation effects: the correlation effects between the key positions of the tumor at different time points can be quantified to provide quantitative indicators for understanding the dynamic changes of the tumor; introduction of attention mechanism: the introduction of linkage attention feature maps enables the model to automatically focus on the most important areas for evaluating the tumor ablation effect, thereby improving the model's sensitivity to key information; enhanced data fusion: combining the anterior-posterior position relationship between the tumor and its surrounding structures, more comprehensive spatial pattern data can be generated, providing richer training information for the deep learning model; improved model performance: using these data to train the deep learning model can improve the accuracy of the model's evaluation of the tumor ablation treatment effect, thereby optimizing the treatment plan; realization of personalized medicine: by analyzing the individual tumor position changes and structural relationships, personalized treatment plans can be provided to patients to improve the treatment effect.

[0094] Further, based on the linkage attention feature map, the first positional relationship between the tumor and its surrounding structures, and the second positional relationship between the tumor and its surrounding structures, spatial pattern data and deep learning training data are generated, including:

[0095] Performing joint registration on a first positional relationship between the tumor and its surrounding structures and a second positional relationship between the tumor and its surrounding structures, and outputting a positional relationship registration feature map;

[0096] The spatial pattern data and deep learning training data containing spatial feature information are generated by utilizing the positional relationship registration feature map and the linkage attention feature map.

[0097] Furthermore, a deep learning model is constructed based on the extracted spatial pattern data and deep learning training data, and the constructed deep learning model is trained and optimized to obtain a tumor ablation image fusion model, including:

[0098] Perform Gaussian denoising and data standardization on spatial pattern data containing spatial feature information and deep learning training data;

[0099] A deep learning model is constructed based on the spatial pattern data after Gaussian denoising and data standardization and the network architecture of KDBNet; wherein the network architecture of KDBNet includes a first graph neural network and a second graph neural network;

[0100] The constructed deep learning model is trained and optimized using the spatial pattern data and deep learning training data after Gaussian denoising and data standardization, and the model parameters are updated based on the following formula until the loss function converges to obtain a tumor ablation image fusion model;

[0101]

[0102] Among them, θ update represents the parameters of the updated deep learning model; θ0 represents the parameters of the deep learning model before the update, ω1 represents the preset learning rate, represents the gradient of the loss function; represents the deep learning model before adjustment, and data represents the spatial pattern data and deep learning training data after Gaussian denoising and data standardization.

[0103] Furthermore, the location information of the tumor includes the location of the tumor, the morphology of the tumor, and the size of the tumor.

[0104] Furthermore, the surrounding structures of the tumor include blood vessels, nerves, and organs surrounding the tumor.

[0105] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A tumor ablation image fusion modeling method based on deep learning, characterized in that: include: Before performing ablation therapy puncture surgery on a tumor, a plurality of preoperative tumor images of the tumor patient are obtained; wherein the plurality of preoperative tumor images include a first ultrasound tumor image, a first CT tumor image, and a first MRI tumor image; Performing image fusion on the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image to obtain a preoperative fused tumor image; During the ablation therapy puncture surgery on the tumor, a plurality of intraoperative tumor images of the tumor patient are obtained; wherein the plurality of intraoperative tumor images include a second ultrasound tumor image, a second CT tumor image, and a second MRI tumor image; the plurality of intraoperative tumor images include an image at the time when the ablation therapy puncture surgery is being performed and an image at the time when the ablation therapy puncture surgery is finished; Obtaining an intraoperative fused tumor image from the second ultrasound tumor image, the second CT tumor image, and the second MRI tumor image; Extracting spatial pattern data and deep learning training data from both the preoperative fused tumor image and the intraoperative fused tumor image; A deep learning model is constructed according to the extracted spatial pattern data and the deep learning training data, and the constructed deep learning model is trained and optimized to obtain a tumor ablation image fusion model; wherein the tumor ablation image fusion model is used to evaluate the therapeutic effect of the ablation treatment puncture surgery; The performing image fusion on the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image to obtain a preoperative fused tumor image includes: Performing spatial registration processing on the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image to obtain the first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image after the registration processing; The first ultrasound tumor image, the first CT tumor image, and the first MRI tumor image after the registration processing are respectively input into the context dynamic extraction module for multi-scale decomposition to obtain a multi-scale feature atlas; Downsampling the multi-scale feature map set in sequence through a downsampling structure of an image fusion neural network to obtain a first-scale coded feature map, a second-scale coded feature map, and a third-scale coded feature map; fusing the first-scale coded feature map, the second-scale coded feature map, and the third-scale coded feature map to obtain the preoperative fused tumor image; The fusing the first scale coding feature map, the second scale coding feature map and the third scale coding feature map to obtain the preoperative fused tumor image includes: The first scale coding feature map, the second scale coding feature map and the third scale coding feature map are processed respectively using gray level co-occurrence matrices of different pixel scale components to obtain a first scale gray level co-occurrence map, a second scale gray level co-occurrence map and a third scale gray level co-occurrence map; Performing feature point detection and matching on the first-scale grayscale symbiosis map, the second-scale grayscale symbiosis map, and the third-scale grayscale symbiosis map to obtain matching feature points; Calculating the deviation values ​​of the coordinate values ​​of the matching feature points on the x-axis, y-axis and z-axis respectively to obtain the deviation basis function; ； in, represents the deviation basis function; N represents the number of matching feature points; Indicates the deviation value of the coordinate value of the matching feature point on the x-axis; Represents the deviation value of the coordinate value of the matching feature point on the y-axis; Indicates the deviation value of the coordinate value of the matching feature point on the z-axis; , , They are all weight parameters; The first-scale grayscale symbiosis map, the second-scale grayscale symbiosis map, and the third-scale grayscale symbiosis map are fused based on the deviation basis function to obtain the preoperative fused tumor image.

2. The deep learning-based tumor ablation image fusion modeling method according to claim 1, characterized in that: The extracting of spatial pattern data and deep learning training data from both the preoperative fused tumor image and the intraoperative fused tumor image comprises: Preprocessing the preoperative fused tumor image and the intraoperative fused tumor image; Inputting the pre-processed preoperative fused tumor image into the tumor position determination model, marking the first position information of the tumor and the first position relationship between the tumor and its surrounding structures; Inputting the preprocessed intraoperative fused tumor image into the tumor position determination model, marking the second position information of the tumor and the second position relationship between the tumor and its surrounding structures; The spatial pattern data and the deep learning training data are extracted from the first position information of the tumor, the first position relationship between the tumor and its surrounding structures, the second position information of the tumor, and the second position relationship between the tumor and its surrounding structures.

3. The deep learning-based tumor ablation image fusion modeling method according to claim 2, characterized in that: The preprocessing of the preoperative fused tumor image and the intraoperative fused tumor image includes: rigidly matching the preoperative fused tumor image and the intraoperative fused tumor image so that the physical positions of the preoperative fused tumor image and the intraoperative fused tumor image are the same; The preoperative fused tumor image and the intraoperative fused tumor image after rigid matching are resampled to the same size.

4. The deep learning-based tumor ablation image fusion modeling method according to claim 3, characterized in that: The extracting the spatial pattern data and the deep learning training data from the first position information of the tumor, the first position relationship between the tumor and its surrounding structures, the second position information of the tumor, and the second position relationship between the tumor and its surrounding structures comprises: Performing joint registration on the first position information of the tumor and the second position information of the tumor, and outputting a first ablation position linkage image and a second ablation position linkage image; generating a linkage attention feature map according to the first ablation position linkage image and the second ablation position linkage image; ； in, represents the linkage attention feature map, the linkage attention feature map is used to represent the correlation influence of the i-th key position in the first ablation position linkage image on the j-th key position in the second ablation position linkage image; represents the i-th key position in the first ablation position linkage image; represents the jth key position in the second ablation position linkage image; is an exponential function; is the number of key positions in the first ablation position linkage image; is the number of key positions in the second ablation position linkage image; Based on the linkage attention feature map, the first positional relationship between the tumor and its surrounding structures, and the second positional relationship between the tumor and its surrounding structures, the spatial pattern data and the deep learning training data are generated.

5. The deep learning-based tumor ablation image fusion modeling method according to claim 4, characterized in that: The generating the spatial pattern data and the deep learning training data based on the linkage attention feature map, the first positional relationship between the tumor and its surrounding structures, and the second positional relationship between the tumor and its surrounding structures includes: Performing joint registration on a first positional relationship between the tumor and its surrounding structures and a second positional relationship between the tumor and its surrounding structures, and outputting a positional relationship registration feature map; The spatial pattern data and the deep learning training data containing spatial feature information are generated by utilizing the positional relationship registration feature map and the linkage attention feature map.

6. The deep learning-based tumor ablation image fusion modeling method according to claim 5, characterized in that: The step of constructing a deep learning model based on the extracted spatial pattern data and the deep learning training data, and training and optimizing the constructed deep learning model to obtain a tumor ablation image fusion model includes: Performing Gaussian denoising and data standardization on the spatial pattern data containing spatial feature information and the deep learning training data; Constructing the deep learning model based on the spatial pattern data after Gaussian denoising and data standardization and the network architecture of KDBNet; wherein the network architecture of KDBNet includes a first graph neural network and a second graph neural network; Using the spatial pattern data and deep learning training data after Gaussian denoising and data standardization, the constructed deep learning model is trained and optimized, and the model parameters are updated based on the following formula until the loss function converges to obtain a tumor ablation image fusion model; ； in, Represents the updated parameters of the deep learning model; represents the parameters of the deep learning model before updating, represents the preset learning rate, represents the gradient of the loss function; represents the deep learning model before adjustment, Represents the spatial pattern data and deep learning training data after Gaussian denoising and data normalization.

7. The deep learning-based tumor ablation image fusion modeling method according to claim 6, characterized in that: The location information of the tumor includes the location of the tumor, the morphology of the tumor and the size of the tumor.

8. The deep learning-based tumor ablation image fusion modeling method according to claim 7, characterized in that: The surrounding structures of the tumor include blood vessels, nerves and organs around the tumor.

Images