Non-rigid medical image registration method and system based on physical information neural network

Abstract

The application relates to the technical field of medical image processing, and provides a non-rigid medical image registration method and system based on a physical information neural network, which comprises the following steps: based on a source point set and a target point set, a registration result is obtained through a physical information neural network; wherein the physical information neural network respectively extracts features of the source point set and the target point set, then splices the features of the source point set and the features of the target point set, splices the features with a down-sampled source point set, obtains fusion features, inputs the fusion features into a displacement feature regression branch of a multilayer perception structure, regresses a non-rigid displacement field, adds the non-rigid displacement field and the source point set point by point, and then carries out inverse normalization processing to obtain the registration result; the physical information neural network utilizes a coupling relationship among displacement, stress and / or strain to construct a back propagation total loss. Efficient, high-precision and physically reasonable non-rigid registration is realized.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and particularly relates to a non-rigid medical image registration method and system based on physical information neural networks. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Medical image registration is one of the core technologies in multimodal image fusion and surgical navigation. Taking the prostate as an example, in prostate cancer biopsy or particle implantation surgery, the registration of preoperative MRI and intraoperative TRUS can help doctors accurately locate the lesion.

[0004] Due to external factors such as changes in patient posture, respiratory movements, and probe pressure, soft tissues like the prostate undergo significant non-rigid deformation, leading to decreased accuracy in complex scenarios for traditional registration methods based on feature point matching or gray-level similarity measurements. To address this issue, researchers have introduced biomechanical constraints, simulating the physical properties of tissues (such as elastic modulus and Poisson's ratio) to constrain the rationality of the deformation field. The finite element method (FEM) is a classic approach, achieving registration by discretizing the tissue mesh and solving partial differential equations (PDEs). For example, the paper "MR to ultrasound registration for image-guided prostate interventions" used FEM to simulate prostate deformation with good results. However, the FEM method requires precise boundary conditions and material parameters, resulting in a complex modeling process and high computational complexity, making it difficult to meet the real-time requirements of intraoperative procedures. Furthermore, FEM has poor adaptability to individual patient differences, limiting its clinical application.

[0005] In recent years, with the development of deep learning technology, PINN-based registration methods have become a research hotspot. PINN, by embedding physical laws, such as the equations of elasticity, into the training process of neural networks, can ensure the physical consistency of deformation on a data-driven basis. The papers "Non-rigid medical image registration using physics-informed neural networks" and "Biomechanics-informed Non-rigid Medical Image Registration and its Inverse Material Property Estimation with Linear and Nonlinear Elasticity" propose a stress prediction-based PINN method, which improves the accuracy of prostate point set registration by predicting the stress field and combining it with force balance constraints. The paper "WarpPINN: Cine-MR image registration with physics-informed neural networks" proposes the WarpPINN method, which introduces a Neo-Hookean nonlinear hyperelastic model to achieve registration of cardiac Cine-MR images by predicting the displacement field. However, these methods have the following shortcomings:

[0006] (1) The physical quantity prediction is singular. Existing methods are mostly focused on the prediction of a single physical quantity, such as only predicting displacement or stress. They have failed to systematically explore the potential impact of strain prediction or joint prediction of multiple physical quantities on the registration effect. For example, the PINN method based on stress prediction predicts stress, but does not consider the force constraint error that may be caused by directly predicting the strain field.

[0007] (2) Computational efficiency issues: Nonlinear models such as WarpPINN introduce complex energy functions, resulting in complex computational graphs and high gradient computation overhead, making them unsuitable for real-time applications.

[0008] (3) Insufficient robustness: Existing methods perform well in simulation data, but their performance declines in clinical data due to differences in tissue characteristics among patients, such as elastic modulus ranging from tens of kPa to hundreds of kPa.

[0009] (4) Insufficient theoretical verification. The literature lacks a systematic comparison of “which physical quantity is better to predict”, which leads to a lack of theoretical basis for the selection of technical solutions. Summary of the Invention

[0010] To address the technical problems mentioned above, this invention provides a non-rigid medical image registration method and system based on a physical information neural network. This method jointly models the global description vectors of the source and target points with the local information of the source point, and uses a multilayer perceptron to predict deformation displacement point by point, thereby achieving efficient and stable non-rigid deformation fitting. Furthermore, it comprehensively utilizes the coupling relationships of multiple physical quantities such as displacement, stress, and strain, while also considering computational efficiency and robustness to clinical data, to achieve efficient, high-precision, and physically reasonable non-rigid registration.

[0011] To achieve the above objectives, the present invention adopts the following technical solution:

[0012] The first aspect of the present invention provides a non-rigid medical image registration method based on a physical information neural network, comprising:

[0013] Obtain the source point set extracted from the MRI image and the target point set extracted from the TRUS image;

[0014] Based on the source point set and the target point set, the registration result is obtained through a physical information neural network;

[0015] In this process, the physical information neural network extracts features from the source point set and the target point set respectively, concatenates the features of the source point set and the target point set, and then concatenates them with the features of the downsampled source point set to obtain fused features. The fused features are then input into the displacement feature regression branch of the multilayer perceptron structure to regress the non-rigid displacement field. The non-rigid displacement field is then added point by point to the source point set and inversely normalized to obtain the registration result. The physical information neural network utilizes the coupling relationship between displacement, stress and / or strain to construct the backpropagation total loss.

[0016] Furthermore, if the physical information neural network utilizes the coupling relationship between displacement and strain, the medical image registration network also includes a strain feature regression branch. The strain feature regression branch predicts the strain field by fusing feature regression and obtains the stress field through the stress field calculation formula.

[0017] Furthermore, the total backpropagation loss includes registration loss, the difference between the source strain field and the predicted strain field, the change in internal material energy caused by the stress field, and the stress field satisfying the physical and mechanical equilibrium condition.

[0018] Furthermore, if the physical information neural network utilizes the coupling relationship between displacement, stress, and strain, then the medical image registration network also includes a strain feature regression branch and a stress feature regression branch; the strain feature regression branch predicts the strain field by fusing feature regression and obtains the stress field through the stress field calculation formula; the stress feature regression branch predicts the stress field by fusing feature regression and obtains the strain field by solving the inverse process of the source stress field calculation.

[0019] Furthermore, the total backpropagation loss includes registration loss, the difference between the source strain field and the predicted strain field, the difference between the source stress field and the predicted stress field, the change in internal energy of the material caused by the stress field, the change in internal energy of the material caused by the strain field, and the fact that the stress field and the strain field satisfy the physical and mechanical equilibrium condition.

[0020] Furthermore, if the physical information neural network only predicts displacement, then the total backpropagation loss includes registration loss, elastic deformation internal energy constraint, and force balance loss constraint.

[0021] Furthermore, the registration loss is the error between the registration result and the target point set.

[0022] A second aspect of the present invention provides a non-rigid medical image registration system based on a physical information neural network, comprising:

[0023] The data acquisition module is configured to acquire the source point set extracted from the MRI image and the target point set extracted from the TRUS image.

[0024] The registration module is configured to obtain the registration result based on the source point set and the target point set through a physical information neural network.

[0025] In this process, the physical information neural network extracts features from the source point set and the target point set respectively. The features from the source point set and the target point set are then concatenated, and finally concatenated with the features from the downsampled source point set to obtain fused features. These fused features are input to the displacement feature regression branch of the multilayer perceptron structure to regress the non-rigid displacement field. The non-rigid displacement field is then added point-by-point to the source point set and inversely normalized to obtain the registration result. The physical information neural network utilizes the coupling relationship between displacement, stress, and / or strain to construct the backpropagation total loss.

[0026] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the non-rigid medical image registration method based on a physical information neural network as described above.

[0027] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps in the non-rigid medical image registration method based on a physical information neural network as described above.

[0028] Compared with the prior art, the beneficial effects of the present invention are:

[0029] This invention achieves efficient and stable non-rigid deformation fitting by jointly modeling the global description vectors of the source and target points with the local information of the source point, and by using a multilayer perceptron to predict the deformation displacement point by point.

[0030] This invention can flexibly select the predicted physical quantities, comprehensively utilize the coupling relationship of multiple physical quantities such as displacement, stress, and strain, and take into account both computational efficiency and robustness of clinical data, so as to achieve efficient, high-precision and physically reasonable non-rigid registration.

[0031] This invention designs three physical quantity prediction strategies: a strain prediction method, a unified prediction method, and a coupling method. The methods cover the use of deep neural networks to predict non-rigid deformation and regularized constraint physical quantities, solving the problems of single physical quantity selection, low computational efficiency, and insufficient robustness in the prior art.

[0032] This invention effectively improves the ability to model the spatial distribution of point clouds by combining global context with a point-by-point regression mechanism, enabling stable fitting even when the deformation scale is large or there are local missing parts.

[0033] This invention can achieve efficient, high-precision and physically reasonable registration. It can maintain a high registration accuracy under different reconstruction accuracies, different noise levels and large deformation conditions, and shows good versatility and engineering practical value.

[0034] This invention has good scalability and can be used as a core module in various non-rigid point cloud processing systems. It can also be combined with other deep learning methods to further expand its application prospects in fields such as 3D reconstruction, surgical navigation, and human deformation modeling. Attached Figure Description

[0035] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0036] Figure 1 This is a flowchart of the non-rigid medical image registration method based on a physical information neural network according to Embodiment 1 of the present invention;

[0037] Figure 2 This is a flowchart of the feature extraction process for the registration method in Embodiment 1 of the present invention;

[0038] Figure 3 This is a flowchart of the predicted strain constraint registration process according to Embodiment 1 of the present invention;

[0039] Figure 4 This is a flowchart of the collaborative prediction stress and strain constraint registration process according to Embodiment 1 of the present invention;

[0040] Figure 5 This is a flowchart of the coupling method constraint registration of Embodiment 1 of the present invention;

[0041] Figure 6 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0043] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0044] Example 1

[0045] This embodiment provides a non-rigid medical image registration method based on a physical information neural network.

[0046] The non-rigid medical image registration method based on physical information neural networks provided in this embodiment is particularly suitable for point set registration of soft tissues such as the prostate, lungs, and liver. It can achieve high-precision, efficient, and physically reasonable registration between multimodal medical images, such as magnetic resonance imaging (MRI) and transrectal ultrasound (TRUS).

[0047] The non-rigid medical image registration method based on physical information neural networks provided in this embodiment aims to solve the problem of insufficient accuracy of soft tissues (such as prostate, lung, liver, etc.) in multimodal image (such as MRI and TRUS) registration due to complex non-rigid deformation. It is applicable to fields such as clinical surgical navigation, disease diagnosis and treatment planning.

[0048] The non-rigid medical image registration method based on physical information neural networks provided in this embodiment preprocesses the source point set and the target point set, designs three physical quantity prediction strategies (strain prediction, unified prediction and coupling method), uses deep neural networks to predict deformation and combines biomechanical constraints (such as elasticity equations, force balance and energy constraints) to achieve efficient, high-precision and physically reasonable registration, which is applicable to fields such as clinical surgical navigation and disease diagnosis.

[0049] The non-rigid medical image registration method based on physical information neural networks provided in this embodiment includes the following steps:

[0050] Step 1: Process the source point set data separately. and target point set data Preprocessing, such as Figure 2As shown, it includes the following steps:

[0051] Step 101: Obtain the source point set and target point set from the medical imaging equipment. Taking the prostate as an example, the source point set is extracted from the MRI image before the operation, and the target point set is extracted from the TRUS image during the operation.

[0052] Step 102: Downsample the original point set using the farthest point sampling (i.e., farthest distance downsampling) algorithm. The source point set for downsampling is... The downsampling target point set is This reduces the number of point clouds and improves subsequent processing efficiency.

[0053] Preferably, the downsampled point set includes surface points and internal points, which respectively preserve the geometric features and internal structural information of the tissue.

[0054] Step 103: Normalize the downsampled point set by scaling the coordinate range from -1 to 1 to make the point cloud consistent in scale and distribution, thereby enhancing the adaptability of the physical information neural network model to different samples and improving the stability and efficiency of physical information neural network training.

[0055] In this implementation, the physical information neural network includes a feature extraction module and a non-rigid registration deformation regression module.

[0056] Step 2, Feature Extraction Module.

[0057] like Figure 2 As shown, the feature extraction module extracts features from the normalized source point set to obtain source point set features, which are high-dimensional representations reflecting preoperative anatomical structures or spatial distribution features; the feature extraction module extracts features from the normalized target point set to obtain target point set features, which are high-dimensional feature vectors that can characterize the intraoperative organ structural state or deformation features.

[0058] The feature extraction module uses PointNet as the backbone network and extracts global features through two layers of multilayer perceptron (MLP) and max pooling operation, with a preset scaling factor. Each layer is a neural network module with shared weights.

[0059] The feature extraction module includes the following steps: The perceptron network in the first stage is set to two layers, and the number of hidden units in each layer is dynamically adjusted. The number of neurons in each layer is divided by the integer value of the scaling factor. The perceptron network in the second stage is set to three layers, with the number of neurons being 64, 128 and 1024 respectively. The output dimensions are also normalized according to the scaling factor.

[0060] Preferably, each layer of the perceptron network uses the LeakyReLU activation function, and the computational efficiency and model generalization ability are improved by sharing parameters.

[0061] Preferably, the PointNet network architecture may optionally integrate an attention mechanism to further enhance its responsiveness to key regions or features.

[0062] This invention adopts an end-to-end structure and integrates an attention mechanism in the feature extraction module to enhance the responsiveness to key point regions. It can still maintain high matching robustness in complex scenarios such as point cloud occlusion, uneven density, or structural degradation.

[0063] Preferably, the feature extraction module symmetrically aggregates the point set, and the features of all points are integrated by dimensionality reduction through a max pooling function to generate a global feature descriptor, ensuring the robustness of the physical information neural network model in terms of the order of input points.

[0064] Step 3: Non-rigid registration deformation regression module, including feature fusion module and displacement feature regression module.

[0065] Step 301: The non-rigid registration deformation regression module concatenates the features of the source point set and the target point set, and then concatenates them with the features of the downsampled source point set to obtain the fused features.

[0066] The feature fusion module concatenates the two global feature vectors, source point set features and target point set features, along the feature dimension to form a joint feature representation that describes the source-target registration relationship, thereby enhancing the model's comprehensive understanding of preoperative and intraoperative information. The feature fusion module then replicates the concatenated global features along the number of points dimension, making its dimension consistent with the number of source points. Subsequently, this replicated global feature is connected to the original coordinates of each source point to form a joint feature tensor, resulting in fused features, which are used for subsequent registration, localization, or deformation analysis tasks to improve robustness and accuracy.

[0067] Step 302, as follows Figure 1 As shown, the fused features are input into the displacement feature regression module (displacement feature regression branch) to regress a non-rigid displacement field that describes the deformation trend of the source point set in space. This process reflects the spatial transformation information of the registration from the source point set to the target point set. Subsequently, the predicted non-rigid displacement field is added point by point to the downsampled source point set to generate the deformed point set, which serves as the preliminary registration result. To achieve spatial consistency between the source and target point clouds, the preliminary registration results are inversely normalized to restore them to their actual scale and spatial distribution in the original coordinate system. The final output is the registration point set. It can be used in various clinical scenarios such as anatomical alignment and preoperative and intraoperative image fusion.

[0068] The center point and scale of the target point cloud are used as inverse normalization conditions.

[0069] Preferably, the displacement feature regression module inputs the aforementioned joint feature tensor into a multilayer perceptron structure, with hidden layer dimensions of 1024, 512, 256, 128, and 64 respectively, progressively mapping high-dimensional features to low-dimensional deformation space, and predicting the displacement amount point by point in the non-rigid registration process. Each perceptron layer uses a consistent activation function, and optional shared parameter settings can be used to improve efficiency. The output of the multilayer perceptron is input into the last layer MLP, further reducing the features to three dimensions using a fully connected layer, serving as the displacement vector for each source point. .

[0070] Among them, the medical image registration network uses physical quantity registration constraints for backpropagation loss. During the training process of the medical image registration network, a multilayer perceptron with hidden layer dimensions of 1024, 512, 256, 128, 64, and 3 is constructed to predict stress or strain, and the total backpropagation loss is constructed.

[0071] (1) As one implementation method, such as Figure 3 As shown, the medical image registration network also includes a strain feature regression module.

[0072] The displacement feature regression module generates a non-rigid displacement field. This displacement field reflects the non-rigid deformation relationship of the point cloud in three-dimensional space from preoperative to intraoperative; the registration result is generated through step 302. The obtained registration results are compared with the actual target point set, and the registration loss is calculated to optimize the network; based on the non-rigid displacement field The source strain field was obtained by partial differential calculation. The strain feature regression module predicts the strain field by fusing feature regression, and the predicted strain tensor... It has 6 independent components, namely , , , , , The strain field predicted by the strain characteristic regression module is used to obtain the stress field through the source stress field calculation formula. This stress field is used to monitor and evaluate the physical plausibility of predicted deformation.

[0073] The total backpropagation loss (total registration constraint loss) is the registration loss. The sum of the predicted strain regularization constraints:

[0074]

[0075] Among them, the predictive strain regularization constraint term is the field residual constraint. Force balance constraints and energy loss (internal energy of elastic deformation) sum.

[0076] Strain residual loss is used to assess the difference between the predicted strain field and the differential strain field; elastic deformation internal energy is used to measure the change in internal energy of the material caused by the strain field; force balance loss is used to ensure that the predicted stress field satisfies the physical and mechanical equilibrium conditions.

[0077] (2) As one implementation method, such as Figure 4 As shown, the medical image registration network also includes a strain feature regression module and a stress feature regression module.

[0078] The displacement feature regression module generates a non-rigid displacement field. The registration result is generated through step 302. Based on non-rigid displacement field The source strain field was obtained by partial differential calculation. The source stress field is obtained through stress field calculation. The strain tensor predicted by the strain characteristic regression module It has 6 independent components, namely , , , , , Based on the strain field predicted by the strain characteristic regression module, the stress field is obtained through the source stress field calculation formula. The stress tensor predicted by the stress characteristic regression module It has 6 independent components, namely , , , , , Based on the stress tensor predicted by the stress characteristic regression module, the strain field is obtained by solving the inverse process of the source stress field calculation. .

[0079] The total backpropagation loss (total registration constraint loss) is the registration loss. The sum of the predicted strain regularization constraint term, the predicted strain regularization constraint term, and the stress-strain energy regularization term:

[0080]

[0081] Among them, the predictive stress regularization constraint term is the field residual constraint. Force balance constraints The sum of these terms, along with the predicted strain regularization constraint term, constitutes the field residual constraint. Force balance constraints The sum of the stress-strain energy regularization term is: and The result is obtained by substituting the energy constraint formula.

[0082] Stress residual loss is used to constrain the consistency between the derived stress field and the predicted stress field by comparing them; strain residual loss is used to compare the difference between the strain field obtained by the displacement derivative and the predicted strain field; elastic deformation internal energy measures the deformation cost in the form of physical energy and participates in training as a regularization term; force balance loss ensures that the predicted stress field satisfies the physical and mechanical equilibrium conditions. By co-predicting stress and strain fields and introducing registration accuracy loss and multiple physical consistency loss terms, a non-rigid registration framework with enhanced physical constraints is realized, which is suitable for high-precision, physically interpretable medical image point cloud registration tasks.

[0083] (3) As one implementation method, such as Figure 5 As shown, the medical image registration network does not include strain feature regression module and stress feature regression module.

[0084] The displacement feature regression module generates a non-rigid displacement field. The registration result is generated through step 302. Based on non-rigid displacement field The source strain field was obtained by partial differential calculation. The source stress field is obtained through stress field calculation. .

[0085] The total backpropagation loss (total registration constraint loss) is the registration loss. The sum of coupling regularization constraints: .

[0086] Based on the stress field generation, an elastic deformation internal energy constraint is introduced to measure the physical energy required for overall deformation, serving as a regularization term to prevent excessive deformation. A force balance loss constraint is also introduced, calculating the residual based on the mechanical equilibrium equations to guide the model in generating a stress distribution that better conforms to physical laws. By coupling non-rigid registration with the physical field, and introducing elastic deformation energy and force balance conditions as physical regularization terms on top of the standard registration loss, the registration accuracy is improved while ensuring its physical consistency and clinical applicability, making it suitable for scenarios such as medical image registration and personalized tissue modeling.

[0087] In this embodiment, the source strain field is obtained by partial differential calculation. Specifically, the source strain field is calculated from the predicted displacement field using a linear strain-displacement relationship. For each point The non-rigid displacement is The strain components are defined as:

[0088]

[0089] Preferably, and The gradient is solved by computational graph; and A non-rigid displacement reference mesh can also be constructed and approximated using the finite difference method.

[0090] In this embodiment, the source stress field is obtained through stress field calculation. Specifically, it is due to The source stress field was calculated using the constitutive equation of an isotropic structure. :

[0091]

[0092] Among them, stiffness tensor By Young's modulus Compared to Poisson Confirmed. Similarly, can be inverse matrix and Multiplying them together yields the result, which is called the inverse process of calculating the source stress field.

[0093] For field residual constraints, for the predicted physical quantities ( ), calculate its relationship with the source physical field Residual loss:

[0094]

[0095] Preferably, this constraint is applied in strain prediction and unified prediction methods.

[0096] For force balance constraints, apply body force balance conditions. Assuming no external force is acting, that is Calculate the loss:

[0097]

[0098] in, Through the or Calculation of spatial partial derivatives.

[0099] For coupling force balance constraints:

[0100]

[0101] in, The number of sampling points, Let n be the deformation at the nth sampling point. Lamé constant, which is related to volume change. Shear modulus For the Laplace operator, To reflect the gradient-divergence term that reflects volume compression / expansion, This is the weighting term for the integral.

[0102] For energy constraints, the internal energy of elastic deformation is calculated to constrain the stability of the deformation field:

[0103]

[0104] Among them, double dot product Defined as .

[0105] In this embodiment, unsupervised Chamfer distance is used to establish the registration loss. The calculation formula is:

[0106]

[0107] Preferably, the Chamfer distance can be improved using the Laplace function to reduce the point set registration error. The calculation formula is as follows:

[0108]

[0109] in, For Gaussian kernels or similarity measures, For the downsampling source point set after non-rigid deformation, This is the target point set after downsampling.

[0110] The non-rigid medical image registration method based on physical information neural networks provided in this embodiment has a simple overall structure, high computational efficiency, and good application value and deployment adaptability.

[0111] Example 2

[0112] This embodiment provides a non-rigid medical image registration system based on a physical information neural network, which specifically includes:

[0113] The data acquisition module is configured to acquire the source point set extracted from the MRI image and the target point set extracted from the TRUS image.

[0114] The registration module is configured to obtain the registration result based on the source point set and the target point set through a physical information neural network.

[0115] In this process, the physical information neural network extracts features from the source point set and the target point set respectively, concatenates the features of the source point set and the target point set, and then concatenates them with the features of the downsampled source point set to obtain fused features. The fused features are then input into the displacement feature regression branch of the multilayer perceptron structure to regress the non-rigid displacement field. The non-rigid displacement field is then added point by point to the source point set and inversely normalized to obtain the registration result. The physical information neural network utilizes the coupling relationship between displacement, stress and / or strain to construct the backpropagation total loss.

[0116] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0117] Example 3

[0118] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the non-rigid medical image registration method based on a physical information neural network as described in Embodiment 1 above.

[0119] Example 4

[0120] This embodiment provides a computer device, such as... Figure 6 As shown, the device includes a display device, an input device, a computer-readable storage medium (volatile memory and non-volatile storage medium), a processor, a communication interface (i.e., a network interface), and a computer program stored on the computer-readable storage medium and executable on the processor. The processor, communication interface, and computer-readable storage medium can be connected via a bus or other means. The communication interface is used to receive and transmit data, and when the processor executes the program, it implements the steps in the non-rigid medical image registration method based on a physical information neural network as described in Embodiment 1 above.

[0121] Any references to memory, storage, database, or other media used in this application and embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0122] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A non-rigid medical image registration method based on physical information neural networks, characterized in that, include: Obtain the source point set extracted from the MRI image and the target point set extracted from the TRUS image; Based on the source point set and the target point set, the registration result is obtained through a physical information neural network; In this process, the physical information neural network extracts features from the source point set and the target point set respectively, concatenates the features of the source point set and the target point set, and then concatenates them with the features of the downsampled source point set to obtain fused features. The fused features are then input into the displacement feature regression branch of the multilayer perceptron structure to regress the non-rigid displacement field. The non-rigid displacement field is then added point by point to the source point set and inverse normalization is performed to obtain the registration result. The physical information neural network utilizes the coupling relationship between displacement and strain to construct the backpropagation total loss, or utilizes the coupling relationship between displacement, stress, and strain to construct the backpropagation total loss. If the physical information neural network utilizes the coupling relationship between displacement and strain, then the physical information neural network also includes a strain feature regression branch. The strain feature regression branch predicts the strain field by fusing feature regression and obtains the stress field through the stress field calculation formula. The total backpropagation loss includes registration loss, the difference between the source strain field and the predicted strain field, the change in internal energy of the material caused by the stress field, and the stress field satisfying the physical and mechanical equilibrium condition. If the physical information neural network utilizes the coupling relationship between displacement, stress, and strain, then the physical information neural network further includes a strain feature regression branch and a stress feature regression branch; the strain feature regression branch predicts the strain field by fusing feature regression and obtains the stress field through the stress field calculation formula; the stress feature regression branch predicts the stress field by fusing feature regression and obtains the strain field by solving the inverse process of the source stress field calculation; the total backpropagation loss includes registration loss, the difference between the source strain field and the predicted strain field, the difference between the source stress field and the predicted stress field, the change in internal energy of the material caused by the stress field, the change in internal energy of the material caused by the strain field, and the fact that the stress field and the strain field satisfy the physical and mechanical equilibrium condition.

2. The non-rigid medical image registration method based on a physical information neural network as described in claim 1, characterized in that, The registration loss is the error between the registration result and the target point set.

3. A non-rigid medical image registration system based on a physical information neural network, employing the non-rigid medical image registration method based on a physical information neural network as described in any one of claims 1-2, characterized in that, include: The data acquisition module is configured to acquire the source point set extracted from the MRI image and the target point set extracted from the TRUS image. The registration module is configured to obtain the registration result based on the source point set and the target point set through a physical information neural network. In this process, the physical information neural network extracts features from the source point set and the target point set respectively, concatenates the features of the source point set and the target point set, and then concatenates them with the features of the downsampled source point set to obtain fused features. The fused features are then input into the displacement feature regression branch of the multilayer perceptron structure to regress the non-rigid displacement field. The non-rigid displacement field is then added point by point to the source point set and inversely normalized to obtain the registration result. The physical information neural network utilizes the coupling relationship between displacement, stress and / or strain to construct the backpropagation total loss.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the non-rigid medical image registration method based on a physical information neural network as described in any one of claims 1-2.

5. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the non-rigid medical image registration method based on a physical information neural network as described in any one of claims 1-2.

Images