Soft tissue deformation simulation method and device, equipment and medium

Abstract

The invention discloses a soft tissue deformation simulation method, a soft tissue deformation simulation device, soft tissue deformation simulation equipment and a medium, and relates to the technical field of image processing. According to the method, energy increment of a previous frame is uniquely mapped to displacement field increment of a next frame, accumulative errors in a time sequence are processed by adopting a dimension reduction method, and stable and accurate deformation prediction is ensured. In addition, a method using Gaussian diffusion energy distribution constraint and prior input is also introduced, so that the overall loss of the network can be minimized through global deformation instead of local adjustment near a collision point. According to the method, the prediction accuracy of the whole organ is remarkably improved. Simulation results show that the provided method is superior to a traditional finite element model in the aspect of speed, and meanwhile the precision close to that of the traditional finite element model is achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technologies, and particularly to a soft tissue deformation simulation method, device, equipment and medium based on energy increment self-circulation and time series. Background Art

[0002] The real-time prediction of tissue biomechanical behavior is crucial for surgical training simulators and intraoperative virtual navigation systems. Although the demand for real-time processing has decreased in some fields, there is still a need to quickly and accurately predict biomechanical properties, such as preoperative planning, surgical training, and surgical robots. Biological soft tissues are essentially nonlinear materials that exhibit anisotropy, large deformations, viscoelasticity, and near-incompressibility when interacting with other tissues or external instruments. The biomechanical properties of these tissues are usually represented by time series data, where phenomena such as creep and stress relaxation describe the relationship between stress and strain over time. To accurately capture these nonlinear and time-dependent biomechanical properties, current methods mainly rely on numerical simulation techniques. Among them, the finite element method (FEM) is considered the most accurate and can provide a scenario close to the real world. However, due to the large amount of computation, FEM is often not suitable for tasks that require real-time interactive display.

[0003] Deep learning methods have shown great potential in simulating complex biomechanical deformations but still face many challenges. One important limitation is the difficulty of real-time simulation using time series data. Soft tissues (such as the liver) exhibit complex properties, such as viscoelasticity and nonlinear elasticity, which cause deformations to change over time in response to external forces. Deep learning methods are usually conceptualized as establishing a mapping between deformation-induced conditions (inputs) and the resulting deformation displacements (outputs). These conditions can include velocity, force, displacement field, energy, and other related factors.

[0004] However, traditional deep learning methods face a key limitation: if the same deformation-induced conditions are repeatedly used as inputs, the network will always output the same deformation field. In sharp contrast to the real-world scenario, continuously applying the same force will result in different deformation fields. Therefore, traditional networks cannot incorporate time series information in their predictions.

[0005] To address this issue, some methods have been proposed for real-time simulation of deep learning time series. For example, some studies have introduced an additional time variable into the input of the neural network, and others have proposed steps towards interactive soft tissue simulation. In addition, some studies have proposed a new deep learning-driven method that uses recurrent neural networks to predict pelvic soft tissue deformations and developed and evaluated classical deep neural networks.

[0006] However, although these methods introduce a time variable, most of them are still applicable to simple scenarios and do not establish a real-time simulation model that can adapt to different forces from multiple angles. Summary of the Invention

[0007] In view of the above problems, the present invention provides a soft tissue deformation simulation method, device, equipment and medium for overcoming the above problems or at least partially solving the above problems. By multiplying the displacement field data of the previous frame by the force data, an energy metric with a direction is generated, effectively solving the time series problem in soft tissue deformation. This enables the recursive framework to convert a long time series into a series of single-frame cyclic predictions, thus alleviating the problem of generating the same displacement field from the same force or velocity input.

[0008] The present invention provides the following solutions:

[0009] A soft tissue deformation simulation method, comprising:

[0010] Obtain the displacement field increment of the previous frame input for the force deformation prediction of the target point of the soft tissue simulation model;

[0011] Calculate the centroid displacement of the previous frame of the target point by using the displacement field increment of the previous frame;

[0012] Obtain the external force received by the target point in the previous frame; apply Gaussian diffusion to the external force to obtain the Gaussian diffusion force point cloud of the previous frame;

[0013] Multiply the Gaussian diffusion force points of the previous frame by the centroid displacement of the previous frame to obtain the vector energy of the previous frame;

[0014] Input the vector energy of the previous frame into the Transformer-KAN self-recurrent neural network prediction model, so that the Transformer-KAN self-recurrent neural network prediction model outputs the displacement field increment of the current frame;

[0015] Apply the displacement field increment of the current frame to the soft tissue simulation model to predict the deformation of the biological tissue;

[0016] Wherein, the Transformer-KAN self-recurrent neural network prediction model includes a Transformer Neck module and a KAN network; the Transformer Neck module is used to extract comprehensive features from the input data and input the extracted features into the KAN network; the KAN network is used to route the processed data to a separate TBR decoder to obtain the displacement field increment of the current frame.

[0017] Preferably, the soft tissue simulation model is constructed based on the Comsol Multiphysics simulation software.

[0018] Preferably, the soft tissue simulation model includes a liver simulation model. The liver parenchyma follows the Neo-Hookean model and obtains viscoelasticity from the Kelvin-Voigt model; blood vessels follow the Holzapfel l-gasser-ogden model; liver tumors follow the St-Venant-Kirchhoff model.

[0019] Preferably, at the beginning of the training of the Transformer-KAN self-recurrent neural network prediction model, initial values are assigned to each vertex in the model according to the distance from the collision point; and Gaussian weighting is applied to the loss function to increase the importance of points far from the collision center during the training process.

[0020] Preferably, the loss function for training the Transformer-KAN self-recurrent neural network prediction model includes the MSE loss function and a weighted loss function based on the Gaussian distribution.

[0021] Preferably, the MSE loss function is represented by the following formula:

[0022]

[0023] In the formula: represents the predicted displacement, U n represents the finite element deformation field result, n represents the nodes of the hexahedron model;

[0024] The weighted loss function based on the Gaussian distribution is represented by the following formula:

[0025]

[0026] In the formula: W n represents the Gaussian distribution weight.

[0027] Preferably, the Gaussian distribution weight W n is represented by the following formula:

[0028]

[0029] In the formula: D n =‖v i -V res ‖ represents a matrix, v i represents the centroid of the point, V res represents the Euclidean distance of the point not directly affected by force, and α = 1, β = 0.2 represent parameters.

[0030] A soft tissue deformation simulation device for performing the above-mentioned soft tissue deformation simulation method, the device comprising:

[0031] A previous frame displacement field increment acquisition unit for acquiring the previous frame displacement field increment input for the force deformation prediction of the target point of the soft tissue simulation model;

[0032] A previous frame center of gravity displacement calculation unit for calculating the previous frame center of gravity displacement of the target point by using the previous frame displacement field increment;

[0033] An external force Gaussian diffusion unit for acquiring the external force received by the target point in the previous frame; applying Gaussian diffusion to the external force to obtain the previous frame Gaussian diffusion force point cloud;

[0034] A previous frame vector energy calculation unit for multiplying the previous frame Gaussian diffusion force point by the previous frame center of gravity displacement to obtain the previous frame vector energy;

[0035] A current frame displacement field increment acquisition unit for inputting the previous frame vector energy into a Transformer-KAN self-recurrent neural network prediction model, so that the Transformer-KAN self-recurrent neural network prediction model outputs the current frame displacement field increment;

[0036] A deformation prediction unit for applying the current frame displacement field increment to the soft tissue simulation model to predict the deformation condition of the biological tissue;

[0037] Wherein, the Transformer-KAN self-recurrent neural network prediction model includes a Transformer Neck module and a KAN network; the Transformer Neck module is used to extract comprehensive features from the input data and input the extracted features into the KAN network; the KAN network is used to route the processed data to a separate TBR decoder to obtain the current frame displacement field increment.

[0038] A soft tissue deformation simulation device, the device comprising a processor and a memory:

[0039] The memory is used to store program code and transmit the program code to the processor;

[0040] The processor is used to execute the above-mentioned soft tissue deformation simulation method according to the instructions in the program code.

[0041] A computer-readable storage medium, the computer-readable storage medium is used to store program code, and the program code is used to execute the above-mentioned soft tissue deformation simulation method.

[0042] According to the specific embodiments provided by the present invention, the following technical effects are disclosed by the present invention:

[0043] A soft tissue deformation simulation method, device, equipment and medium provided by an embodiment of the present application. This method uniquely maps the energy increment of the previous frame to the displacement field increment of the next frame, and uses a dimensionality reduction method to process the cumulative error in the time series to ensure stable and accurate deformation prediction. In addition, it also introduces a method using Gaussian diffusion energy distribution constraint and prior input, enabling the network to minimize the overall loss through global deformation rather than local adjustment near the collision point. This method significantly improves the accuracy of the entire organ prediction. The simulation results show that the proposed method is superior to the traditional finite element model in terms of speed, while achieving an accuracy close to that of the traditional finite element model.

[0044] Of course, it is not necessary for any product implementing the present invention to simultaneously achieve all the above-mentioned advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0046] Figure 1 is a flowchart of the soft tissue deformation simulation method provided by an embodiment of the present invention;

[0047] Figure 2 is a data processing flowchart of the self-recurrent neural network prediction model provided by an embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of the training process of the self-recurrent neural network prediction model provided by an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of the structure of the self-recurrent neural network prediction model provided by an embodiment of the present invention;

[0050] Figure 5 Table 2 provided by an embodiment of the present invention shows the comparison results of different time series deep learning methods;

[0051] Figure 6 Table 3 provided by an embodiment of the present invention shows the comparison results of short time series test data of different methods;

[0052] Figure 7 is a schematic diagram of the cumulative nearest neighbor distance error exceeding 3s, with average acting forces of 2.3N and 5.7N provided by an embodiment of the present invention;

[0053] Figure 8 It is a schematic diagram provided by an embodiment of the present invention when the average force is 1.9 N and 3.9 N, and the cumulative error of the nearest neighbor distance exceeds 10 s;

[0054] Figure 9 It is a schematic diagram of the simulation results of the ground truth and the networks trained with loss and without loss provided by an embodiment of the present invention;

[0055] Figure 10 It is a schematic diagram of the loss change under the training of loss 2 provided by an embodiment of the present invention;

[0056] Figure 11 It is a schematic diagram of a soft tissue deformation simulation device provided by an embodiment of the present invention;

[0057] Figure 12 It is a schematic diagram of a soft tissue deformation simulation device provided by an embodiment of the present invention. Specific embodiments

[0058] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present invention.

[0059] See Figure 1 , a soft tissue deformation simulation method provided by an embodiment of the present invention, as Figure 1 shown, the method may include:

[0060] S101: Obtain the displacement field increment of the previous frame input for the force deformation prediction of the target point of the soft tissue simulation model; The method provided by the embodiments of the present application is applicable to the use of various soft tissue simulation models. For example, in one implementation, the embodiments of the present application may also provide that the soft tissue simulation model is constructed based on Comsol Multiphysics simulation software. By using Comsol Multiphysics simulation software to construct a finite element model of soft tissue, during deformation prediction, the obtained displacement field increment can be directly applied to the finite element model to achieve deformation prediction.

[0061] In addition, the soft tissue can be a variety of human organs. For example, in one implementation, the embodiments of the present application can provide that the soft tissue simulation model includes a liver simulation model, where the liver parenchyma follows the Neo-Hookean model and obtains viscoelasticity from the Kelvin-Voigt model; blood vessels follow the Holzapfel l-gasser-ogden model; and liver tumors follow the St-Venant-Kirchhoff model.

[0062] S102: Calculate the centroid displacement of the previous frame of the target point by using the increment of the displacement field of the previous frame;

[0063] S103: Obtain the external force received by the target point in the previous frame; apply Gaussian diffusion to the external force to obtain the point cloud of the Gaussian diffusion force in the previous frame;

[0064] S104: Multiply the Gaussian diffusion force point in the previous frame by the centroid displacement of the previous frame to obtain the vector energy of the previous frame;

[0065] S105: Input the vector energy of the previous frame into the Transformer-KAN self-recurrent neural network prediction model so that the Transformer-KAN self-recurrent neural network prediction model outputs the increment of the displacement field of the current frame;

[0066] S106: Apply the increment of the displacement field of the current frame to the soft tissue simulation model to predict the deformation of the biological tissue;

[0067] Among them, the Transformer-KAN self-recurrent neural network prediction model includes a Transformer Neck module and a KAN network; the Transformer Neck module is used to extract comprehensive features from the input data and input the extracted features into the KAN network; the KAN network is used to route the processed data to a separate TBR decoder to obtain the increment of the displacement field of the current frame.

[0068] The method provided by the embodiments of the present application adopts a self-loop real-time prediction model constructed based on the Transformer and the KAN module. By multiplying the displacement field data of the previous frame by the force data, an energy metric with a direction is generated, and finally, the increment of the displacement field of the current frame is obtained through the Transformer-KAN self-recurrent neural network prediction model. Specifically, when implemented, the embodiments of the present application can provide that at the beginning of the training of the Transformer-KAN self-recurrent neural network prediction model, initial values are assigned to each vertex in the model according to the distance from the collision point; and Gaussian weighting is applied to the loss function to increase the importance of points far from the collision center during the training process.

[0069] The loss function for training the Transformer-KAN self-recurrent neural network prediction model includes the MSE loss function and a weighted loss function based on the Gaussian distribution.

[0070] The MSE loss function is expressed by the following formula:

[0071]

[0072] In the formula: represents the predicted displacement, U n represents the finite element deformation field result, and n represents the nodes of the hexahedron model;

[0073] The weighted loss function based on the Gaussian distribution is expressed by the following formula:

[0074]

[0075] In the formula: W n represents the Gaussian distribution weight.

[0076] The Gaussian distribution weight W n is expressed by the following formula:

[0077]

[0078] In the formula: D n =‖v i -V res ‖ represents a matrix, v i represents the centroid of the point, and V res represents the Euclidean distance of the point not directly affected by the force, and α = 1, β = 0.2 represent parameters.

[0079] Next, taking the liver as soft tissue and the finite element model as an example, the method provided by the embodiments of the present application will be introduced in detail.

[0080] For data generation, the first step is to construct the fem for liver biomechanical analysis. The next step is to apply external forces of different magnitudes to different angles and directions of the finite element model and record the displacement increments at corresponding times. Then a self-loop real-time prediction system based on the Transformer and KAN modules is constructed. The Transformer-KAN model is constructed for training and prediction. Finally, the loss based on the Gaussian diffusion distribution and the implementation details of the network are given.

[0081] A composite hyperelastic nearly incompressible liver with parenchyma, tumor, and blood vessels was constructed using Comsol Multiphysics simulation software. Specifically, the liver parenchyma follows the Neo-Hookean model and obtains viscoelasticity from the Kelvin-Voigt model; the blood vessels follow the Holzapfell-Gasser-Ogden model; the liver tumor follows the St-Venant-Kirchhoff model (the number of vertices in the liver parenchyma mesh model is 5004, the number of vertices in the blood vessel mesh model is 2030, and the number of tumor vertices is 375).

[0082] The material property parameters used in the finite element model were taken from Zou and Schiavone. A deformable liver model was established through volume hexahedral meshing. The total number of elements in the hexahedral liver is 2843. The points where a force is applied have been uniformly selected on the liver surface.

[0083] The total number of frames in each calculation process of the FEM is 300 frames, the time interval is 0.01 s, and the corresponding total time is 3 s. By repeatedly selecting a point on the model surface in a randomly applied force direction 3100 times, the displacements caused by the corresponding applied forces were obtained and simulated, resulting in 930,000 sets of data corresponding to external forces and displacements in different directions. The dataset of applied forces and corresponding displacements from the composite model is considered the gold standard. Among them, 10 of the randomly selected force application processes were used as a short-time series validation dataset containing 3000 pairs of force and displacement fields, and the rest were used as training data.

[0084] At the same time, as Figure 2 shown, to further verify the performance of the proposed method on long-time series, based on the same finite element model, 4 directions were randomly selected to generate long-time series data with a 10 s force application process. Then, the Gaussian diffusion force point cloud (with direction) was multiplied by the displacement of the center of gravity (the force multiplied by the displacement in the force direction represents energy) to represent the energy of the previous frame, denoted as E t-1 = FG t-1 * CD t-1 . Let this application assume that the vector energy E t is the only increment of this framework, so U t = f(E t-1 , physical properties) This energy is used as the input U t for predicting the deformation field of the current frame. The new node position vector to be calculated is denoted as X t+1 = X t + U t . The simulation time step is t = 0.01 s.

[0085] Since the initial state of the model is known, obtaining the increment for each frame is sufficient to derive all coordinate values through an iterative method. Note that for the loop input-output structure, each X is saved at each frame and updated in the next frame. The entire real-time simulation process is regarded as a loop structure where the previous frame is related to the next frame. Denote the current node position vector at each simulation instant as X t , U t as the displacement field increment. Before applying the external force F t-1 in the current frame, it is necessary to process both the force and the displacement field simultaneously. Specifically, the transformation from F t-1 to FG t-1 is achieved by applying Gaussian diffusion to F t-1 . This enables the force to be dispersed over the entire surface of the model according to the distance from the central collision point, rather than being concentrated only near the force application point. Meanwhile, U t-1 representing the deformation field at each point is converted to scalar data according to the centroid displacement and CD t-1 . Table 1 shows the pseudo-program illustrating this process.

[0086]

[0087] Essentially, the network in the architecture of this application utilizes the energy increment of the previous frame to predict the displacement increment of the next frame. Its main function is to distinguish the subtle changes in energy and establish an independent correspondence with the displacement increment.

[0088] This application converts the hexahedron model into a regular rectangular matrix, surrounds the original 2843 points with zeros, and expands it to 7106 points. To ensure that the length, width, and height input to the convolutional network are equal, the completed 7106 (17×19×22) rectangular matrix is further padded with zeros to reach 13824 (24×24×24). The shape of the network input is 24×24×24×3, where 3 represents the length, width, and height. The output shape is the same, representing the displacement matrix of the model.

[0089] In the prediction output stage, the output result is back-filled to its original form and the hexahedron model is mapped to the mesh model in real time through centroid mapping. Considering that the data captures the action of uniformly varying external forces on specific points of the model, there is an inherent connection between the input of each frame and the previous frame, resulting in temporal dependence. This relationship requires a training and testing method that is both continuous and cyclic.

[0090] Specifically, as Figure 3As shown, the training process is carried out in cycles. Each cycle starts from a certain point, experiences the moment of stress, and continues until the moment when the stress is completely relieved. This cyclic method ensures that the model comprehensively learns the dynamic changes of force over time. This process is continuously repeated until the entire dataset is traversed, including various force directions and magnitudes. By exposing the model to such a wide range of scenarios, it can better generalize under different stress conditions and effectively capture the nuances of the deformation process over time.

[0091] Figure 4 The network structure of this application is shown. This network is based on an encoder-decoder architecture derived from Transformer-Unet, and its transformation modules facilitate robust multi-channel feature extraction. Initially, these modules extract comprehensive features from the input data, and then the extracted features are input into the KAN network structure, which serves as a key intermediary to convert the features into a format suitable for subsequent processing.

[0092] KAN plays a key role in bridging the extracted multi-dimensional information and effectively transmitting this information through a series of skip connection structures. Finally, the processed data is routed to a separate TBR decoder. Each decoder consists of three interconnected components: a transposed convolution module for adjusting the spatial resolution of the feature map, a batch normalization module for stabilizing the learning process, and a ReLU activation module for introducing non-linearity.

[0093] The integration of KAN with the convolutional neural network follows the specific implementation details outlined in the literature. This hybrid architecture starts with a transformation bottleneck and performs initial feature extraction through five encoder modules to ensure the preservation of key features. The extracted features are then reshaped and transformed to seamlessly integrate with the KAN structure. The main goal of this network is to accelerate the finite element method, which requires the use of a lightweight architecture. However, lightweight networks often face a major challenge known as the "curse of dimensionality". In this case, the dimensions of the input and output are determined by the number of vertices in the model and the complexity of the applied forces. Specifically, a large number of model vertices and different force conditions result in different sizes, directions, and time frames, causing the search space of the network to explode exponentially. This high-dimensional complexity gives rise to problems such as overfitting, extended training time, and an increased need for large datasets.

[0094] The KAN structure addresses these challenges by learning to decompose complex high-dimensional functions into simpler one-dimensional combined functions. By focusing on optimizing these one-dimensional functions rather than the entire multi-dimensional space, KAN significantly reduces the dependence on the weight matrix, thereby improving the inference speed while maintaining the fitting accuracy. Another key advantage of the KAN structure lies in its difference from the traditional multi-layer perceptron (MLP) method. Different from the MLP that relies on fixed node activation functions, KAN uses adjustable activation functions at its edges. These methods replace the traditional linear weight matrix with one-dimensional spline functions, enabling KAN to flexibly aggregate input signals without additional non-linear functions. This design improves the theoretical ability of KAN to approximate geometric functions.

[0095] In addition, the use of smooth activation functions provides a more refined and finer-grained approach to processing data signals, making KAN particularly effective in a wide range of applications. In summary, the KAN structure is particularly suitable for the tasks of this application because it allows this application to address the curse of dimensionality caused by a large number of model vertices and various force conditions while retaining a lightweight architecture. This enables this application to improve the inference speed and efficiency without sacrificing the ability to handle the high-dimensional complexity inherent in the problem. The main purpose of the first loss function is to ensure that the overall result is consistent with the result of the finite element simulation. After testing, it is found that the mean squared error (MSE) is a simple and effective loss function.

[0096] However, in some cases, the output shows local deformation rather than global deformation. This problem occurs because the input consists of a matrix whose shape corresponds to the number of vertices in the model, usually containing non-zero values only in the local area under the action of force, while the input values of most points not directly under the action of force are zero. In addition, due to the relatively small applied force, the model tends to show significant local deformation, while the deformation of other unaffected areas is minimal. This may cause the prediction of the network to focus too much on the deformation in the force-applied area while ignoring the deformation in other areas.

[0097] To solve this problem, this application first applies Gaussian diffusion to the input, assigning initial values (which decrease with the increase of distance) to each vertex in the model according to the distance from the collision point at the beginning of training. On this basis, a similar Gaussian weighting is applied to the loss function to increase the importance of points far from the collision center during the training process. Experiments show that this method effectively alleviates the static prediction problem in the non-stress area of the deformation field.

[0098] The loss of this application is a combination of two losses, one is the regular MSE loss, and the other is the weighted loss based on the Gaussian distribution. The formula is as follows:

[0099] Loss = Loss1 + Loss2 #(1)

[0100] The first loss function is the classical MSE loss function:

[0101]

[0102] where \(\hat{U}\) is the predicted displacement, \(U\) n is the finite element deformation field result, and \(n\) is the node of the hexahedron model.

[0103] Specifically, first calculate the centroid \(v\) of these points i , and then calculate the Euclidean distance \(V\) from this coordinate to the remaining points that are not directly stressed res to obtain a matrix \(D\) n as shown in Equation (3):

[0104] \(D\) n =\(\|v\) i -V\) res \|\) #(3)

[0105] By converting the distance matrix into a Gaussian distribution and setting the parameters \(\alpha = 1\) and \(\beta = 0.2\). According to the distance between the stress points and the non-stress points, the Gaussian distribution weights are obtained as follows:

[0106]

[0107] Finally, the second loss function is as shown in Equation (5):

[0108]

[0109] Effect verification: All experiments were conducted on a computer with an Intel i7-13700K CPU and an NVIDIA RTX 4090 GPU. The learning rate was \(e - 5\), the optimizer was Adam, the training epoch of the Transformer-KAN model was 1000, and the training time was approximately 20 hours.

[0110] Experiments and Results: First, a detailed introduction to the evaluation metrics is given. Subsequently, the time efficiency and accuracy are verified, and comparisons are made with three state-of-the-art methods. Finally, the effectiveness of the proposed method is verified through three ablation experiments.

[0111] Verification Metrics: To evaluate the accuracy of the proposed method, four statistical analyses of the average norm error of the test dataset are performed and denoted by \(e\). Let \(u\) n denote the ground truth displacement tensor of the sample \(n\) generated by FEM, and \(h(f\) m ) denote the predicted value. \(u\) m and \(h(f\) m) The error between them can be measured by four metrics: mean error, mean squared error (MSE), chamfer distance, and Hausdorff distance.

[0112] Comparison with time - series deep - learning methods. The main method of data - driven deep - learning for real - time soft - body deformation simulation is characterized by its time - series nature and relies on a recurrent framework.

[0113] Current methods focus on changes in the network model, input / output parameter adjustment, and data pre - processing to improve efficiency. Meister et al. proposed an encoder - decoder architecture with convolutional layers (256, 128, 64, 32 filters), ReLU activation, batch normalization, and max - pooling in the encoder, and up - sampling, skip connections, and a 64 - filter layer in the decoder for recursive time - series displacement prediction. Hien et al. used LSTM and BiLSTM models to simulate childbirth, integrating velocity, displacement, and external forces while reducing error through dimensionality reduction. Similarly, here LSTM and BiLSTM networks are used to predict the deformation field based on external forces and previous displacements. The LSTM has three stacked layers (256 units per layer) with sigmoid and tanh activation, outputting a sequence, except for the last layer which produces a deformation field reduced by PCA. The BiLSTM processes the input bidirectionally with 512 units per layer and a similar activation function.

[0114] As Figure 5 shown in Table 2, it summarizes the time - series - based deep - learning methods used for comparison purposes in this application.

[0115] Through tests on long - time - series and short - time - series data, the effectiveness of the method is confirmed. For short - time - series data, as Figure 6 shown in Table 3, it illustrates how the error propagates with each iteration, resulting in an increase in cumulative error over time. The method of this application demonstrates a significant ability to reduce error accumulation, making it more effective in deformation prediction. This advantage is particularly evident when comparing the results of this application with those of other methods.

[0116] In addition, Figure 7Further demonstrates the superior performance of the method of the present application. It shows that although the error accumulation within different force magnitudes and time step ranges can gradually amplify locally in the model over time to an unacceptable level. On the other hand, the method of the present application, while not completely eliminating the time series error accumulation, always has relatively small associated errors within a limited time, indicating that the method of the present application can maintain accuracy over time even under different conditions. These findings highlight the robustness and reliability of the method of the present application, making it highly suitable for tasks that require accurate prediction in dynamic environments.

[0117] To evaluate the effectiveness of the method under extended time conditions, experimental verification was carried out using a newly generated test dataset containing four different deformation processes. This dataset applied a consistent force over a duration of 10 s, collected samples at intervals of 0.01 s, resulting in a total of 1000 frames. As Figure 8 shown, the extended cyclic prediction process led to significant error accumulation, which in turn led to obvious local deformations within the model in the later stages of the simulation.

[0118] Due to time limitations, it was not possible to generate a large dataset with a long time series, which may lead to larger prediction errors compared to short time series results. Despite the limitation of lacking sufficient data, the method of the present application effectively maintained the smoothness and relative accuracy of the model in long-term prediction. The results clearly show that the method of the present application is superior to other methods in predicting long time series data. Compared with other techniques, the method of the present application demonstrated superior performance on the long-time test dataset. This advantage is attributed to the strategy of the present application, which extremely reduces the dimensionality of the output displacement field by using the centroid displacement in each frame. By utilizing the reduced displacement field data as the input for predicting the next frame, the present application significantly reduced the error accumulation rate during the network iterative prediction process.

[0119] Compared with other time series deep learning deformation methods, the deformation trend predicted by the method of the present application shows improvement. For the 3 s interval, the movement predicted by the method of the present application has a certain degree of consistency and accuracy, maintaining the expected physical behavior of the points. When extended to the 10 s interval, although there is naturally some error accumulation, the method of the present application still provides a reliable deformation representation. The results indicate that the method of the present application can handle longer time frames with reasonable accuracy, making it a useful tool for modeling time-dependent deformations.

[0120] In the case of applying a consistent unidirectional force to a specific area of the liver for a duration of 3 s. Due to the constant external force, the non-time series deep learning simulation method produces the same output for each frame, and the model reaches a static state after the initial movement, which results in an unchanged sitting value. In contrast, the time series deep learning simulation method incorporates the information of the previous frame, ensuring the difference in the input for each frame. This enables the model to generate different deformation fields even under a constant force of the same magnitude and direction.

[0121] To improve the time efficiency, the proposed method needs to downscale the output displacement field data and generate a new energy increment input for each frame. It also applies Gaussian diffusion energy distribution input preprocessing before the next input. This preprocessing method makes the method slower in efficiency than other relatively simple time series models. However, the computational speed is the natural advantage of the network because they do not require Newton iteration. The prediction time for a single frame is 0.01 s. The efficiency is significantly greater than that of the traditional finite element algorithm. The proposed method is significantly faster than the FEM for multi-component biomechanical liver, which means that the proposed method meets the real-time requirement.

[0122] Input preprocessing with Gaussian diffusion and Gaussian distribution loss.

[0123] At the beginning of training, the deformation in the collision area is small, and the deformation of the edge points is even smaller. Therefore, the model tends to focus on the learning values near the collision area because the resulting inertial bias will affect further training. The ablation experiment aims to test the effect of the Gaussian operations (Gaussian distribution loss and Gaussian diffusion force) of this application on reducing the inertial bias.

[0124] The first operation is to assign Gaussian distribution weights to the forces, ensuring that at the early stage of the training process, the non-stress points have a certain force value rather than zero. This allows the network to reduce the loss by adjusting the values of the edge points that are not directly stressed at the beginning of the training, thereby increasing the attention to them. In addition, by combining the Gaussian distribution loss 2, additional weights are assigned to the edge points. Figure 9 Shows the ground truth and time series simulation results (from 1 s to 3 s) of the network trained with and without loss 2. The arrow points to the collision area, and the red box represents the difference between the ground truth of the simulation with / without loss 2 and the deformed non-direct collision area. As Figure 9 shown, in the non-collision area, the overall magnitude of the deformation of the simulation with loss 2 is significantly greater than that of the simulation without loss 2, and it is closer to the ground truth, especially for the long-term simulation in the time series.

[0125] Two operations of this application solve this problem. The first operation is to weight the Gaussian distribution of the force, ensuring a certain force value rather than zero. This allows the network to reduce the loss by adjusting the values of the marginal points that were not directly emphasized at the beginning of training, thereby increasing the attention to them. In addition, by incorporating Loss 2, this application effectively provides additional weights for the marginal points. As Figure 9 shown, after adding Loss 2, the overall amplitude of deformation in the prediction is significantly larger and closer to the gold standard.

[0126] Figure 10 The red curve and the blue curve in [reference] respectively represent the loss changes under training with Loss 2 and without Loss 2. In each epoch, the data of each frame is sequentially input into the autoregressive network, and the network is iteratively trained to output the ground truth close to the tissue deformation in the time series. Without considering the Gaussian distribution Loss 2, when the deformation in the non-direct collision area is small at the initial iteration, using the mse-based Loss 1 for training can obtain a lower loss value. However, using only the MSE loss for initial training will cause the network to over-focus on the direct collision area, thus introducing an inertial bias for further training. As Figure 10 shown, compared with the training without Loss 2, the training with Loss 2 can reach the relatively overall optimal solution faster and with a lower value, indicating that the inertial bias reduces the stability, accuracy, and efficiency of the time series simulation training.

[0127] The second operation disperses the force to be dispersed over the entire surface of the model according to the distance from the central collision point, rather than only concentrating near the force application point. The four obtained error evaluation results show that the input diffusion force enables the network to deform the entire organ, rather than just the area near the collision point, thereby improving the time series simulation accuracy of the entire organ. Adopting two Gaussian-based processes can obtain the best evaluation results. Using the Gaussian distribution loss and the Gaussian diffusion force simultaneously can improve the sensitivity of the network to the overall deformation of the direct and non-direct collision areas during the iteration process.

[0128] In summary, the soft tissue deformation simulation method provided by this application uniquely maps the energy increment of the previous frame to the displacement field increment of the next frame, and uses a dimensionality reduction method to process the cumulative error in the time series to ensure stable and accurate deformation prediction. In addition, it also introduces a method using Gaussian diffusion energy distribution constraints and prior inputs, enabling the network to minimize the overall loss through global deformation rather than local adjustment near the collision point. This method significantly improves the accuracy of the entire organ prediction. The simulation results show that the proposed method is superior to the traditional finite element model in terms of speed, while achieving an accuracy close to that of the traditional finite element model.

[0129] See Figure 11, embodiments of the present application can also provide a soft tissue deformation simulation device, such as Figure 11 shown, for performing the above-mentioned soft tissue deformation simulation method. The device may include:

[0130] The previous frame displacement field increment acquisition unit 1101 is configured to acquire the previous frame displacement field increment input for the force deformation prediction of the target point of the soft tissue simulation model;

[0131] The previous frame centroid displacement calculation unit 1102 is configured to calculate the previous frame centroid displacement of the target point by using the previous frame displacement field increment;

[0132] The external force Gaussian diffusion unit 1103 is configured to acquire the external force received by the target point in the previous frame; apply Gaussian diffusion to the external force to obtain the previous frame Gaussian diffusion force point cloud;

[0133] The previous frame vector energy calculation unit 1104 is configured to multiply the previous frame Gaussian diffusion force points by the previous frame centroid displacement to obtain the previous frame vector energy;

[0134] The current frame displacement field increment acquisition unit 1105 is configured to input the previous frame vector energy into the Transformer-KAN self-recurrent neural network prediction model, so that the Transformer-KAN self-recurrent neural network prediction model outputs the current frame displacement field increment;

[0135] The deformation prediction unit 1106 is configured to apply the current frame displacement field increment to the soft tissue simulation model to predict the deformation condition of the biological tissue;

[0136] Wherein, the Transformer-KAN self-recurrent neural network prediction model includes a Transformer Neck module and a KAN network; the Transformer Neck module is configured to extract comprehensive features from the input data and input the extracted features into the KAN network; the KAN network is configured to route the processed data to a separate TBR decoder to obtain the current frame displacement field increment.

[0137] Embodiments of the present application can also provide a soft tissue deformation simulation device. The device includes a processor and a memory:

[0138] The memory is configured to store program codes and transmit the program codes to the processor;

[0139] The processor is configured to execute the steps of the above-mentioned soft tissue deformation simulation method according to the instructions in the program codes.

[0140] Such as Figure 12As shown in the figure, a soft tissue deformation simulation device provided by an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 all complete mutual communication through the communication bus 13.

[0141] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array, or other programmable logic devices, etc.

[0142] The processor 10 may call the program stored in the memory 11. Specifically, the processor 10 may execute the operations in the embodiment of the soft tissue deformation simulation method.

[0143] The memory 11 is used to store one or more programs. The program may include program codes, and the program codes include computer operation instructions. In the embodiment of the present application, the memory 11 stores at least programs for implementing the following functions:

[0144] Obtain the incremental displacement field of the previous frame input for the force deformation prediction of the target point of the soft tissue simulation model;

[0145] Calculate the centroid displacement of the previous frame of the target point by using the incremental displacement field of the previous frame;

[0146] Obtain the external force received by the target point in the previous frame; apply Gaussian diffusion to the external force to obtain the Gaussian diffusion force point cloud of the previous frame;

[0147] Multiply the Gaussian diffusion force point of the previous frame by the centroid displacement of the previous frame to obtain the vector energy of the previous frame;

[0148] Input the vector energy of the previous frame into the Transformer-KAN self-recurrent neural network prediction model so that the Transformer-KAN self-recurrent neural network prediction model outputs the incremental displacement field of the current frame;

[0149] Apply the incremental displacement field of the current frame to the soft tissue simulation model to predict the deformation of biological tissue;

[0150] Wherein, the Transformer-KAN self-recurrent neural network prediction model includes a Transformer Neck module and a KAN network; the Transformer Neck module is used to extract comprehensive features from the input data and input the extracted features into the KAN network; the KAN network is used to route the processed data to a separate TBR decoder to obtain the incremental displacement field of the current frame.

[0151] In a possible implementation, the memory 11 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function (such as a file creation function and a data reading and writing function). The data storage area may store data created during use, such as initialization data.

[0152] In addition, the memory 11 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid-state storage devices.

[0153] The communication interface 12 may be an interface of a communication module for connecting to other devices or systems.

[0154] Of course, it should be noted that Figure 12 the structure shown does not limit the soft tissue deformation simulation device in the embodiments of the present application. In actual applications, the soft tissue deformation simulation device may include more or fewer components than Figure 12 those shown, or combine certain components.

[0155] The embodiments of the present application may also provide a computer-readable storage medium for storing program code for executing the steps of the above-mentioned soft tissue deformation simulation method.

[0156] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0157] As can be understood from the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0158] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments. In particular, for a system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, reference can be made to the partial description of the method embodiment. The systems and system embodiments described above are only illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0159] The above is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A soft tissue deformation simulation method, characterized in that: include: Obtaining the displacement field increment of the previous frame input by the previous frame force deformation prediction of the target point of the soft tissue simulation model; The displacement of the center of gravity of the target point in the previous frame is obtained by using the displacement field increment of the previous frame; Obtaining the external force applied to the target point in the previous frame; applying Gaussian diffusion to the external force to obtain the Gaussian diffusion force point cloud of the previous frame; Multiplying the Gaussian diffusion force point of the previous frame by the center of gravity displacement of the previous frame to obtain the vector energy of the previous frame; Inputting the previous frame vector energy into a Transformer-KAN self-circulatory neural network prediction model so that the Transformer-KAN self-circulatory neural network prediction model outputs a current frame displacement field increment; Applying the current frame displacement field increment to the soft tissue simulation model to predict the deformation of the biological tissue; Among them, the Transformer-KAN self-circulatory neural network prediction model includes a Transformer Neck module and a KAN network; the Transformer Neck module is used to extract comprehensive features from the input data and input the extracted features into the KAN network; the KAN network is used to route the processed data to a separate TBR decoder to obtain the current frame displacement field increment.

2. The soft tissue deformation simulation method according to claim 1, characterized in that: The soft tissue simulation model is constructed based on Comsol Multiphysics simulation software.

3. The soft tissue deformation simulation method according to claim 2, characterized in that: The soft tissue simulation model includes a liver simulation model, the liver parenchyma follows the Neo-Hookean model and obtains viscoelasticity from the Kelvin-Voigt model; the blood vessel follows the Holzapfell-gasser-ogden model; and the liver tumor follows the St-Venant-Kirchhoff model.

4. The soft tissue deformation simulation method according to claim 1, characterized in that: The Transformer-KAN self-recurrent neural network prediction model assigns an initial value to each vertex in the model according to the distance from the collision point at the beginning of training; and applies Gaussian weighting to the loss function to increase the importance of points far from the collision center during training.

5. The soft tissue deformation simulation method according to claim 4, characterized in that: The loss function of the Transformer-KAN self-circulatory neural network prediction model training includes an MSE loss function and a weighted loss function based on Gaussian distribution.

6. The soft tissue deformation simulation method according to claim 5, characterized in that: The MSE loss function is expressed as follows: Where: Denotes the predicted displacement, U n represents the finite element deformation field result, n represents the hexahedral model node; The weighted loss function based on Gaussian distribution is expressed as follows: Where: W n represents the Gaussian distribution weight.

7. The soft tissue deformation simulation method according to claim 6, characterized in that: Gaussian distribution weight W n It is expressed by the following formula: Where: D n =‖v i -V res ‖ represents the matrix, v i represents the centroid of the point, V res It represents the Euclidean distance of the point not directly subjected to force, and α=1 and β=0.2 represent parameters.

8. A soft tissue deformation simulation device, characterized in that: For executing the soft tissue deformation simulation method according to any one of claims 1 to 7, the device comprises: A previous frame displacement field increment acquisition unit, used to acquire a previous frame displacement field increment input by a previous frame force deformation prediction of a target point of a soft tissue simulation model; A previous frame centroid displacement calculation unit, used for obtaining the previous frame centroid displacement of the target point by using the previous frame displacement field increment calculation; An external force Gaussian diffusion unit is used to obtain the external force applied to the target point in the previous frame; Gaussian diffusion is applied to the external force to obtain a Gaussian diffusion force point cloud of the previous frame; A previous frame vector energy calculation unit, used for multiplying the previous frame Gaussian diffusion force point by the previous frame gravity center displacement to obtain the previous frame vector energy; A current frame displacement field increment acquisition unit, used for inputting the previous frame vector energy into a Transformer-KAN self-circulatory neural network prediction model, so that the Transformer-KAN self-circulatory neural network prediction model outputs a current frame displacement field increment; A deformation prediction unit, used for applying the current frame displacement field increment to the soft tissue simulation model to predict the deformation of the biological tissue; Among them, the Transformer-KAN self-circulatory neural network prediction model includes a Transformer Neck module and a KAN network; the Transformer Neck module is used to extract comprehensive features from the input data and input the extracted features into the KAN network; the KAN network is used to route the processed data to a separate TBR decoder to obtain the current frame displacement field increment.

9. A soft tissue deformation simulation device, characterized in that: The device comprises a processor and a memory: The memory is used to store program codes and transmit the program codes to the processor; The processor is used to execute the soft tissue deformation simulation method according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the soft tissue deformation simulation method according to any one of claims 1 to 7.

Images