Ablation impedance dynamic prediction method based on multi-modal time sequence fusion and related device

Abstract

The application discloses an ablation impedance dynamic prediction method based on multi-modal time sequence fusion and related devices. The method comprises the following steps: collecting multi-modal time sequence data of an ablation electrode in real time, and constructing a multi-dimensional input vector containing a temperature change rate, an ultrasonic echo gradient and a standardized timestamp; using a pre-trained double-channel long short-term memory neural network to fine-tune parameters, extracting feature vectors of physical parameters and image parameters respectively and fusing the feature vectors, and outputting impedance prediction values in a future preset time window; based on the deviation of the predicted impedance and the target impedance, a proportional control strategy is used to dynamically adjust the ablation output power. Through the time sequence fusion of multi-modal data and the prediction ability of the deep learning model, the application improves the real-time monitoring and power regulation accuracy of the ablation process, effectively improves the safety and effect stability of the ablation treatment.

Description

Technical Field

[0001] The present invention relates to the field of medical surgical robots and intelligent control, and in particular to a method for dynamic prediction of ablation impedance based on multimodal time series fusion and related devices. Background Art

[0002] Thermal ablation is a common minimally invasive method for tumor treatment. It involves inserting an ablation electrode and releasing energy, heating the target tissue to achieve coagulation and necrosis. During surgery, the electrical impedance of the tissue changes dynamically with temperature, water content, and degree of coagulation. Accurately predicting and controlling impedance in real time is crucial to ensuring the ablation range and avoiding tissue carbonization or perforation.

[0003] Existing technologies typically rely solely on the instantaneous voltage and current signals provided by the ablation host to estimate impedance and adjust output power based on simple proportional-integral control. This approach has the following shortcomings:

[0004] Single signal source: Unable to utilize multimodal features such as temperature, tissue conductivity, and imaging information, resulting in delayed response to changes in tissue status;

[0005] Lack of predictive capability: Existing methods can only provide feedback on current impedance and cannot predict future impedance changes, making it difficult to adjust power in a timely manner when tissue rapidly heats up or coagulates.

[0006] Limited control accuracy: Simple feedback control can easily lead to uneven ablation range or overtreatment in complex tissue heterogeneity environments.

[0007] With the development of medical artificial intelligence and deep learning, prediction and adaptive control based on multi-source data have become possible. However, there is currently a lack of dynamic impedance prediction technology that can simultaneously integrate temperature, power, tissue electrical properties, and intraoperative imaging information. There is also no mature dual-channel neural network architecture that can achieve feature extraction and impedance prediction from multimodal time series signals.

[0008] Therefore, there is an urgent need for a dynamic prediction method and device for ablation impedance based on multimodal time series fusion, which can collect multi-source surgical data in real time, realize short-term prediction of impedance through neural network modeling, and dynamically adjust the ablation power based on the prediction results, thereby improving surgical safety and ablation accuracy. Summary of the Invention

[0009] To address the above technical issues, the present invention provides a method and related apparatus for dynamic prediction of ablation impedance based on multimodal time series fusion, including but not limited to a device for dynamic prediction of ablation impedance based on multimodal time series fusion, an electronic device, a computer-readable storage medium, and a computer program product.

[0010] A first aspect provides a method for dynamic prediction of ablation impedance based on multimodal time series fusion, the method comprising:

[0011] Step S1, real-time acquisition of multimodal time series data of the ablation electrode, wherein the multimodal time series data includes electrode temperature signal, output power, tissue conductivity, average ultrasonic echo intensity, and current impedance;

[0012] Step S2, constructing a multidimensional input vector including temperature change rate, ultrasonic echo gradient and normalized timestamp based on the multimodal time series data;

[0013] Step S3, based on the pre-trained dual-channel long short-term memory neural network, transfer learning is used to fine-tune parameters on a small amount of annotated impedance historical data to obtain impedance prediction weights;

[0014] Step S4: Based on the impedance prediction weight, the multidimensional input vector is input into the dual-channel long short-term memory neural network, physical feature vectors and image feature vectors are extracted respectively, and the physical feature vectors and the image feature vectors are spliced ​​in the feature dimension to output the predicted impedance for a preset time period in the future;

[0015] Step S5: dynamically adjusting the ablation output power according to a proportional control strategy based on the deviation between the predicted impedance and the target impedance.

[0016] In combination with any embodiment of the present application, step S1 includes:

[0017] The electrode temperature signal is collected by the needle tip temperature sensor with a period of 100ms;

[0018] Obtain output power through the ablation host power detection module;

[0019] Calculate tissue conductivity and current impedance based on ablation circuit current and voltage;

[0020] The average intensity of ultrasound echoes in the ablation area was collected and calculated using the intraoperative ultrasound probe.

[0021] In combination with any embodiment of the present application, step S2 includes:

[0022] Acquire multimodal time series data, including electrode temperature signals , output power , tissue conductivity , mean ultrasound echo intensity and current impedance ;

[0023] Acquire ultrasound echo gradient, the ultrasound echo gradient It is calculated by performing gradient filtering on the echo image;

[0024] Calculate the rate of temperature change:

[0025] ,

[0026] in, is the electrode temperature difference between adjacent sampling points, is the fixed time interval between adjacent sampling points;

[0027] Constructing a multidimensional input vector :

[0028] ,

[0029] in, It is a standardized timestamp, indicating the relative time position of the current sampling point in the entire ablation process.

[0030] In conjunction with any embodiment of this application, the standardized timestamp , standardized using the following formula:

[0031] ,

[0032] in, is the current sampling moment, The moment when the ablation begins, is the estimated maximum ablation time.

[0033] In combination with any embodiment of the present application, step S3 includes:

[0034] Freezing the weights of the underlying feature extraction module of the dual-channel long short-term memory neural network during the pre-training process, and optimizing the loss function based on the historical impedance true value to obtain the pre-training weights;

[0035] Using a small amount of local multimodal time series data collected by the target surgical device, fine-tuning the high-level feature fusion module and output module of the dual-channel long short-term memory neural network to obtain fusion weights and output weights;

[0036] The pre-trained weights, fusion weights and output weights are combined to form the impedance prediction weights for impedance prediction.

[0037] In combination with any embodiment of the present application, the dual-channel long short-term memory neural network includes:

[0038] The underlying feature extraction module is used to extract basic features from the input multimodal time series data. It includes a physical parameter feature extraction channel and an image parameter feature extraction channel, each of which is composed of a multi-layer bidirectional long short-term memory network.

[0039] Feature fusion module: used to splice physical feature vectors and image feature vectors, and perform high-dimensional feature fusion through a fully connected network;

[0040] Output module: used to receive fused features and output the predicted impedance for a preset time period in the future.

[0041] In combination with any embodiment of the present application, step S5 includes:

[0042] An adjustment factor is calculated based on the deviation between the predicted impedance and the target impedance;

[0043] Calculate the initial adjustment power;

[0044] The calculated results are safely limited and the final power is output.

[0045] In a second aspect, a device for dynamic prediction of ablation impedance based on multimodal time series fusion is provided, comprising:

[0046] An acquisition unit, configured to acquire multimodal time series data of the ablation electrode in real time, wherein the multimodal time series data includes electrode temperature signal, output power, tissue conductivity, average ultrasonic echo intensity, and current impedance;

[0047] A construction unit, configured to construct a multidimensional input vector including a temperature change rate, an ultrasonic echo gradient, and a normalized timestamp based on the multimodal time series data;

[0048] A training unit is used to fine-tune the parameters of a pre-trained dual-channel long short-term memory neural network using transfer learning on a small amount of annotated impedance historical data to obtain impedance prediction weights;

[0049] A prediction unit, configured to predict the impedance weight, inputs the multidimensional input vector into the dual-channel long short-term memory neural network, extracts physical feature vectors and image feature vectors respectively, concatenates the physical feature vectors and image feature vectors in the feature dimension, and outputs the predicted impedance for a preset time period in the future;

[0050] A control unit is used to dynamically adjust the ablation output power according to a proportional control strategy based on the deviation between the predicted impedance and the target impedance.

[0051] In a third aspect, an electronic device is provided, comprising: a processor, a communication module, a sensor, a user interface, and a storage unit, wherein the storage unit is configured to store computer program code, wherein the program code comprises computer instructions. When the processor executes these instructions, the electronic device performs the method described in the second aspect and any embodiment thereof.

[0052] In a fourth aspect, another electronic device is provided, comprising: a processor, a wireless communication module, a touch screen, a speaker, and a storage unit, wherein the storage unit is configured to store computer program code, wherein the program code comprises computer instructions. When the processor executes these instructions, the electronic device performs the method described in the second aspect and any embodiment thereof.

[0053] In a fifth aspect, a computer-readable storage medium is provided, wherein a computer program is stored, wherein the program includes program instructions. When these instructions are executed by a processor, the processor will perform the method described in the second aspect and any embodiment thereof.

[0054] In a sixth aspect, a computer program product is provided, wherein the computer program product comprises a computer program or instructions. When the computer program or instructions are run on a computer, the computer will execute the method described in the second aspect and any embodiment thereof.

[0055] It should be understood that the above general description and the following detailed description are only used as examples and explanations and do not limit the present application in any way.

[0056] In this application, compared with the prior art, the present invention first collects multimodal time series data during the ablation procedure in real time, including electrode temperature, output power, tissue conductivity, ultrasonic echo intensity and impedance information, with an acquisition period of 100ms; then constructs a dual-channel long short-term memory (LSTM) neural network containing physical parameter channels and imaging parameter channels, extracts thermal and electrical dynamic features in the physical channel, extracts tissue state evolution features in the imaging channel, and performs time series fusion at the feature layer; then, based on the pre-trained dual-channel long short-term memory neural network, uses transfer learning to fine-tune parameters on a small amount of annotated impedance historical data, and utilizes standardized timestamps and ultrasonic gradient enhancement models to explore the sensitivity of nonlinear tissue changes; finally, the trained dual-channel neural network is deployed in the surgical device to predict tissue impedance changes in a short time window in the future, and dynamically adjusts the ablation power based on the prediction results to achieve closed-loop precise control of the ablation process.

[0057] This invention utilizes multimodal time-series data acquisition technology, dual-channel long short-term memory (LSTM) neural network modeling, and closed-loop adaptive power regulation to achieve short-term dynamic prediction of tissue impedance during ablation procedures. Based on the prediction results, it adjusts power output in real time, significantly improving the safety and accuracy of the ablation process. The invention integrates multi-source information, including temperature, power, conductivity, impedance, and ultrasound imaging, to comprehensively reflect changes in tissue state, predict impedance trends in advance, and effectively avoid problems such as excessive carbonization or insufficient ablation caused by single-parameter control. It also possesses strong adaptability and generalization capabilities, operating stably under diverse lesions and tissue environments, improving surgical energy utilization and overall therapeutic efficacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0059] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0060] FIG1 is a flow chart of a method for dynamic prediction of ablation impedance based on multimodal time series fusion according to an embodiment of the present application;

[0061] FIG2 is a schematic structural diagram of a device for dynamic prediction of ablation impedance based on multimodal time series fusion according to an embodiment of the present application;

[0062] FIG3 is a schematic diagram of the hardware architecture of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to allow professionals in this technical field to more fully understand the technical solution of the present application, the technical solution of the present application will be explained in detail and clearly with the help of the accompanying drawings. It should be noted that the described embodiments are only some examples of the present application and do not represent all. Based on these embodiments, those skilled in the art can directly deduce all other possible implementation plans without engaging in creative thinking, and these are also included in the scope of protection of the present application.

[0064] In the specification, claims, and related drawings of this application, the terms "first," "second," and the like are used solely to distinguish between different elements and do not imply any particular order. Furthermore, the use of "including," "having," and their variations denotes non-exclusive inclusion. This means that if a process, method, system, product, or apparatus includes a series of steps or components, the process, method, system, product, or apparatus is not limited to the enumerated steps or components and may include other steps or components not listed, or other steps or units inherent to the process, method, system, product, or apparatus.

[0065] The “embodiment” mentioned in this document refers to any instance in which a particular feature, structure or characteristic is combined, and these instances may belong to at least one embodiment of the present application. The “embodiment” mentioned in this document does not necessarily refer to the same specific case, nor does it mean that they are independent or exclusive alternatives. It should be understood by those skilled in the art that the embodiments described herein can be used in conjunction with other embodiments. It should be understood that in this application, “at least one” includes one or more instances, “a plurality” means two or more instances, and “at least two” means two or more instances.

[0066] It should be understood that the method embodiment of the present application can also be implemented by a processor executing computer program code. The embodiment of the present application is described below in conjunction with the drawings in the embodiment of the present application.

[0067] See also Figure 1 FIG1 is a flow chart of a method for dynamic prediction of ablation impedance based on multimodal time series fusion provided in an embodiment of the present application. The specific steps are as follows:

[0068] 101. Data Collection:

[0069] In this embodiment, during the operation, multiple sensors integrated into the ablation electrode and the intraoperative monitoring system collect multimodal time series data including temperature signals, output power, tissue conductivity, current impedance, and ultrasonic echo intensity in real time at intervals of 100 ms.

[0070] In this embodiment, the temperature signal is obtained by a needle tip temperature sensor;

[0071] The output power is directly read by the power monitoring module of the ablation host;

[0072] The tissue conductivity is obtained by calculating the instantaneous voltage and current;

[0073] The impedance value is obtained by the impedance detection circuit of the ablation host;

[0074] The ultrasound echo intensity is acquired by an ultrasound probe placed beside the operating bed, and the average intensity of the region of interest is calculated at each sampling point.

[0075] In this embodiment, the data from the above sensors are synchronously recorded by a unified acquisition module and marked with a timestamp for subsequent model training and prediction.

[0076] In another possible implementation method, the temperature sensor may use a fiber Bragg grating thermometer to improve the ability to resist electromagnetic interference.

[0077] In another possible implementation, the conductivity can be measured using a four-electrode method to improve accuracy.

[0078] In another possible implementation method, impedance acquisition can use multi-frequency scanning or complex impedance measurement instead of single-frequency impedance measurement to obtain more tissue electrical characteristics.

[0079] In another possible implementation method, image information can not only come from ultrasound, but also be obtained through intraoperative CT or near-infrared optical imaging modules to adapt to different surgical environments.

[0080] 102. Input vector construction:

[0081] In this embodiment, for the multimodal time series data collected in step 101, an input vector containing multidimensional features is constructed for each sampling point. The following steps are taken:

[0082] First, calculate the temperature change rate ΔT / Δt:

[0083] ,

[0084] in, is the electrode temperature difference between adjacent sampling points, Δt is the fixed interval between adjacent sampling points;

[0085] Secondly, the ultrasound echo image is gradient filtered to obtain the echo gradient ∇E; at the same time, the timestamp t is normalized to the relative value of the ablation start time:

[0086] ,

[0087] in, is the current sampling moment, The moment when the ablation begins, is the estimated maximum ablation time;

[0088] Finally, construct the multidimensional input vector:

[0089] ,

[0090] This input vector can fully characterize the physical state and imaging characteristics of the current tissue, providing complete temporal information for the neural network.

[0091] In another possible implementation method, the temperature change rate can be calculated by linear regression using sliding window fitting, instead of the direct difference method, to reduce the influence of noise.

[0092] In another possible implementation method, the ultrasound gradient feature may be replaced by a Laplacian filter or a texture feature based on a Histogram of Oriented Gradients (HOG).

[0093] In another possible implementation method, the normalization method may use Min-Max normalization or Z-score normalization instead of linear time normalization.

[0094] In another possible implementation method, additional features may be added to the input vector, such as energy integral value, ablation needle tip position coordinates, tissue water content parameters, etc., to improve the prediction capability.

[0095] In another possible implementation method, the ablation needle tip pressure sensor or near-infrared spectral signal can be further introduced as input features to enhance the perception of tissue state changes, thereby improving the robustness of the prediction.

[0096] 103. Model training:

[0097] In this embodiment, the impedance prediction weights of a dual-channel long short-term memory (LSTM) neural network are obtained through transfer learning and parameter fine-tuning.

[0098] After pre-training is completed, freeze the weight matrix of the underlying feature extraction module , only the weight matrix of the feature fusion module is retained and the weight matrix of the output module Renewable;

[0099] Next, the feature fusion module and the output module are fine-tuned using a small amount of local multimodal time series data collected by the target surgical device.

[0100] Finally, the impedance prediction weight is formed by the following parameter set:

[0101] ,

[0102] in, is a frozen weight matrix set of an underlying feature extraction module, wherein the underlying feature extraction module includes a physical parameter channel and an image parameter channel;

[0103] is the fine-tuning weight matrix of the feature fusion module;

[0104] is the fine-tuning weight matrix of the output module;

[0105] The impedance prediction weight is used to predict tissue impedance changes within a short time window in the future, thereby improving the adaptability of the model to different devices and individual differences, and playing a role in dynamic power control.

[0106] In this embodiment, the dual-channel long short-term memory neural network is composed of the following modules:

[0107] The underlying feature extraction module is used to extract basic features from the input multimodal time series data. It includes a physical parameter feature extraction channel and an image parameter feature extraction channel, each of which is composed of a multi-layer bidirectional long short-term memory network.

[0108] Feature fusion module: used to splice physical feature vectors and image feature vectors, and perform high-dimensional feature fusion through a fully connected network;

[0109] Output module: used to receive fused features and output the predicted impedance for a preset time period in the future.

[0110] In another possible implementation, the dual-channel network can be replaced with a Transformer encoder structure or a Temporal Convolutional Network (TCN) to process temporal features.

[0111] 104. Impedance prediction:

[0112] In this embodiment, the input vectors collected in real time are forward propagated through a trained dual-channel long short-term memory neural network. The dual-channel network includes a physical parameter channel and an image parameter channel for feature extraction from data of different modalities.

[0113] In this embodiment, the physical parameter channel consists of two stacked bidirectional LSTM networks:

[0114] The first bidirectional LSTM layer contains 128 hidden units, which can capture the forward and backward time-dependent features of the input sequence; the second bidirectional LSTM layer contains 64 hidden units, which is used to further integrate the feature information at different time steps; after the output of the LSTM layer, a fully connected (FC) network is used to compress the features into a fixed-dimensional physical feature vector. , whose dimension is set to 32 in this embodiment.

[0115] In this embodiment, the image parameter channel uses a single-layer unidirectional LSTM network for feature extraction. The LSTM layer contains 64 hidden units to learn the temporal variation of image features. The output of this channel is compressed by a fully connected network layer to obtain the image feature vector , whose dimension is set to 16 in this embodiment.

[0116] In the feature fusion stage, the physical feature vector and image feature vector Merge into fused feature vector through vector concatenation operation The above fusion feature vector is input into a fully connected neural network for nonlinear mapping, and the predicted impedance value is output. .

[0117] In this embodiment, the preset prediction time window Δt\Delta tΔt is 3 seconds, so the output result can predict the impedance change trend within the next 3 seconds for closed-loop control of the dynamic power regulation module.

[0118] In another possible implementation, the physical parameter channel may include only a single-layer LSTM to reduce computational complexity, or a gated recurrent unit (GRU) may be used instead of the LSTM. The image parameter channel may also use a convolutional neural network (CNN) or a temporal convolutional network (TCN) to extract image features. In addition to the splicing method, the feature fusion module may also use an attention mechanism for weighted fusion to enhance the selectivity of key features.

[0119] 105. Dynamic Power Control:

[0120] In this embodiment, a proportional control strategy is used to dynamically adjust the ablation power based on the deviation between the predicted impedance and the target impedance.

[0121] First, calculate the impedance deviation:

[0122] ,

[0123] in, is the proportionality coefficient, To predict impedance; is the preset target impedance value; The upper limit of impedance preset for the system. The lower impedance limit preset for the system;

[0124] Then calculate the preliminary adjustment power:

[0125] ,

[0126] Perform safety limiting on the calculated results and output the final power:

[0127] ,

[0128] in is the preset minimum safe power value, is the preset maximum safe power value;

[0129] Ultimately, the controller feeds the power value back to the ablation host, enabling real-time dynamic adjustment of the output power, thereby maintaining the impedance value close to the target impedance and avoiding tissue overheating or insufficient ablation.

[0130] In another possible implementation, the control strategy can be replaced by fuzzy control, PID control, or adaptive control based on reinforcement learning.

[0131] In another possible implementation method, a safety mechanism may be added, such as immediately reducing the power or triggering an automatic ablation pause function when impedance suddenly changes.

[0132] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0133] The above describes in detail the method of the embodiment of the present application, and the following provides an apparatus of the embodiment of the present application.

[0134] See also Figure 2 FIG2 is a schematic diagram of the structure of a dynamic prediction device for ablation impedance based on multimodal time series fusion according to an embodiment of the present application. The prediction device 1 includes: an acquisition unit 11, a construction unit 12, a training unit 13, a prediction unit 14, and a control unit 15. Specifically:

[0135] An acquisition unit 11 is configured to acquire multimodal time series data of the ablation electrode in real time, wherein the multimodal time series data includes electrode temperature signal, output power, tissue conductivity, average ultrasonic echo intensity, and current impedance;

[0136] A construction unit 12 is configured to construct a multidimensional input vector including a temperature change rate, an ultrasound echo gradient, and a normalized timestamp based on the multimodal time series data;

[0137] Training unit 13, based on a pre-trained dual-channel long short-term memory neural network, uses transfer learning to fine-tune parameters on a small amount of annotated impedance historical data to obtain impedance prediction weights;

[0138] The prediction unit 14 is used for the impedance prediction weight, inputs the multi-dimensional input vector into the dual-channel long short-term memory neural network, extracts the physical feature vector and the image feature vector respectively, and splices the physical feature vector and the image feature vector in the feature dimension to output the predicted impedance for a preset time period in the future;

[0139] The control unit 15 is configured to dynamically adjust the ablation output power according to a proportional control strategy based on the deviation between the predicted impedance and the target impedance.

[0140] See also Figure 3 Figure 3 shows a schematic diagram of the hardware architecture of an electronic device described in an embodiment of the present application. The electronic device 2 is primarily composed of a processor 21 and a memory 22. Furthermore, the device may also include an input device 23 and an output device 24. The processor 21, memory 22, input device 23, and output device 24 are interconnected via connecting components, which can be various interfaces, data cables, or communication buses, and are not specifically specified in the present embodiment.

[0141] Processor 21 may be one or more graphics processing units (GPUs). If processor 21 is a GPU, the GPU may be single-core or multi-core. Optionally, processor 21 may comprise a processor group consisting of multiple GPUs, interconnected via one or more buses. Furthermore, the processor may be other types of processors, which are not specifically limited in this embodiment of the present application.

[0142] The memory 22 is designed to store computer program instructions and various program codes required to execute the solution of the present application. Optionally, the memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which are used to store relevant instructions and data.

[0143] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The input device 23 and the output device 24 can be independent devices or an integrated device.

[0144] It should be appreciated that in the embodiment of the present application, the memory 22 can store not only relevant instructions but also relevant data. The embodiment of the present application does not specify the specific data content stored in the memory.

[0145] It should be understood that FIG3 merely illustrates a simplified design of an electronic device. In actual use, the electronic device may also include other necessary components, such as varying numbers of input / output devices, processors, and memories. All electronic devices capable of implementing the embodiments of this application are within the scope of protection of this application.

[0146] Those skilled in the art will recognize that, according to the components and algorithm steps of each example described in the embodiments disclosed herein, these functions can be implemented by electronic hardware or by combining computer software and electronic hardware. Whether these functions are performed by hardware or software will be determined based on the specific application requirements and design limitations of the technical solution. Technicians can adopt different implementation methods according to the requirements of each specific application, but such implementation methods should not be considered to exceed the scope of protection of this application.

[0147] Professionals should understand that, for the sake of ease of description and simplification, the specific operating procedures of the above-mentioned systems, devices, and components can refer to the corresponding steps in the previous method embodiments and will not be repeated here. At the same time, professionals should also understand that each embodiment in this application has its own focus. For the sake of ease of description and simplification, the same or similar content may not be repeated in different embodiments. Therefore, if a part is not mentioned or not explained in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0148] In the several embodiments provided in this application, it should be recognized that the disclosed systems, devices and methods can also be implemented in other ways. For example, the device embodiments described are only exemplary, in which the division of the units is only a division of logical functions, and there may be different division methods in actual implementation. For example, multiple units or components may be merged or integrated into another system, or certain features may be omitted, or certain steps may not be performed. In addition, the connections between each other shown or discussed, whether direct or indirect, whether coupling or communication connection, may be implemented in electrical, mechanical or other forms through interfaces, devices or units.

[0149] Units described as independent components may or may not actually be physically separate; parts presented as units may or may not be physical entities; that is, they may be centralized in one location or distributed across multiple network nodes. Depending on actual needs, some or all of these units may be selected to achieve the objectives of this embodiment.

[0150] Furthermore, in the various embodiments of the present application, the various functional units may be integrated into a single processing unit, physically exist independently, or two or more units may be combined into a single unit. In the aforementioned embodiments, the relevant functions may be implemented in whole or in part through software, hardware, firmware, or any combination thereof. If software implementation is chosen, it may be implemented in whole or in part in the form of a computer program product. This computer program product comprises one or more computer instructions. When these instructions are loaded and executed on a computer, they will generate, in whole or in part, the processes or functions described in the embodiments of this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. These computer instructions may be stored in a computer-readable storage medium or transmitted via such a medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic cable, DSL) or wireless (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium may be any computer-accessible, usable medium, or a data storage facility such as a server or data center that integrates one or more usable media. These available media may include magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), semiconductor media (e.g., SSDs), etc. Those skilled in the art will appreciate that all or part of the process steps for implementing the above-described method embodiments can be accomplished through hardware associated with computer program instructions. These programs can be stored on computer-readable storage media. When executed, these programs will contain the processes for each of the above-described method embodiments. These storage media include, but are not limited to, various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An electronic device, characterized in that: include: A processor and a storage unit, wherein the storage unit stores program instructions. When the processor executes the program instructions, the electronic device performs a method for dynamic prediction of ablation impedance based on multimodal time series fusion, the method comprising: Step S1, real-time acquisition of multimodal time series data of the ablation electrode, the multimodal time series data including electrode temperature signal, output power, tissue conductivity, average ultrasonic echo intensity and current impedance; specifically, the electrode temperature signal is acquired with a needle tip temperature sensor at a period of 100ms; the output power is acquired through the ablation host power detection module; the tissue conductivity and the current impedance are calculated based on the ablation circuit current and voltage; the average ultrasonic echo intensity of the ablation area is acquired and calculated through the intraoperative ultrasonic probe; Step S2, constructing a multidimensional input vector including temperature change rate, ultrasonic echo gradient and normalized timestamp based on the multimodal time series data; Step S3, based on the pre-trained dual-channel long short-term memory neural network, transfer learning is used to fine-tune parameters on a small amount of annotated impedance historical data to obtain impedance prediction weights; Step S4: Based on the impedance prediction weight, the multidimensional input vector is input into the dual-channel long short-term memory neural network, physical feature vectors and image feature vectors are extracted respectively, and the physical feature vectors and the image feature vectors are spliced ​​in the feature dimension to output the predicted impedance for a preset time period in the future; Step S5: dynamically adjusting the ablation output power according to a proportional control strategy based on the deviation between the predicted impedance and the target impedance.

2. The electronic device according to claim 1, wherein The step S2 comprises: Acquire multimodal time series data, including electrode temperature signals , output power , tissue conductivity , mean ultrasound echo intensity and current impedance ; Acquire ultrasound echo gradient, the ultrasound echo gradient It is calculated by performing gradient filtering on the echo image; Calculate the rate of temperature change: , in, is the electrode temperature difference between adjacent sampling points, is the fixed time interval between adjacent sampling points; Constructing a multidimensional input vector : ， in, It is a standardized timestamp, indicating the relative time position of the current sampling point in the entire ablation process.

3. The electronic device according to claim 2, wherein: The standardized timestamp , standardized using the following formula: ， in, is the current sampling moment, The moment when the ablation begins, is the estimated maximum ablation time.

4. The electronic device according to claim 1, wherein: The step S3 comprises: Freezing the weights of the underlying feature extraction module of the dual-channel long short-term memory neural network during the pre-training process, and optimizing the loss function based on the historical impedance true value to obtain the pre-training weights; Using a small amount of local multimodal time series data collected by the target surgical device, fine-tuning the high-level feature fusion module and output module of the dual-channel long short-term memory neural network to obtain fusion weights and output weights; The pre-trained weights, fusion weights and output weights are combined to form the impedance prediction weights for impedance prediction.

5. The electronic device according to claim 4, characterized in that The dual-channel long short-term memory neural network comprises: The underlying feature extraction module is used to extract basic features from the input multimodal time series data. It includes a physical parameter feature extraction channel and an image parameter feature extraction channel, each of which is composed of a multi-layer bidirectional long short-term memory network. Feature fusion module: used to splice physical feature vectors and image feature vectors, and perform high-dimensional feature fusion through a fully connected network; Output module: used to receive fused features and output the predicted impedance for a preset time period in the future.

6. The electronic device according to claim 1, wherein: The step S5 comprises: An adjustment factor is calculated based on the deviation between the predicted impedance and the target impedance; Calculate the initial adjustment power; The calculated results are safely limited and the final power is output.

7. A dynamic prediction device for ablation impedance based on multimodal time series fusion, characterized in that: include: An acquisition unit is configured to acquire multimodal time series data of the ablation electrode in real time, the multimodal time series data including the electrode temperature signal, output power, tissue conductivity, average ultrasonic echo intensity, and current impedance; specifically, the electrode temperature signal is acquired at a 100ms cycle by a needle tip temperature sensor; the output power is acquired by an ablation host power detection module; the tissue conductivity and current impedance are calculated based on the ablation circuit current and voltage; and the average ultrasonic echo intensity of the ablation area is acquired and calculated by an intraoperative ultrasonic probe; A construction unit, configured to construct a multidimensional input vector including a temperature change rate, an ultrasonic echo gradient, and a normalized timestamp based on the multimodal time series data; A training unit is used to fine-tune the parameters of a pre-trained dual-channel long short-term memory neural network using transfer learning on a small amount of annotated impedance historical data to obtain impedance prediction weights; A prediction unit, configured to predict the impedance weight, inputs the multidimensional input vector into the dual-channel long short-term memory neural network, extracts physical feature vectors and image feature vectors respectively, concatenates the physical feature vectors and image feature vectors in the feature dimension, and outputs the predicted impedance for a preset time period in the future; A control unit is used to dynamically adjust the ablation output power according to a proportional control strategy based on the deviation between the predicted impedance and the target impedance.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 6.

Images