A surface point cloud driven non-rigid medical image registration method and system based on crowd self-adaptive sparse constraint

Abstract

The application relates to a surface point cloud driven non-rigid medical image registration method and system based on crowd self-adaptive sparse constraint, and relates to the technical field of medical image processing. The method comprises the following steps: acquiring a fixed image, a moving image and a body surface point cloud of a target object; performing coarse registration on the body surface point cloud by using the moving image to obtain a simplified body surface point cloud; calculating a signed distance field of the fixed image; acquiring historical crowd deformation statistical priori, constructing a sub-crowd deformation base dictionary based on the statistical priori; constructing an estimated non-rigid deformation field based on the base dictionary; constructing a joint energy function based on the estimated non-rigid deformation field, the fixed image, the moving image, the simplified body surface point cloud and the signed distance field and solving the joint energy function to obtain an optimal non-rigid deformation field; and processing the moving image and the simplified body surface point cloud by using the optimal non-rigid deformation field to obtain registered medical image data. Compared with the prior art, the application has strong adaptability, high registration accuracy and good robustness.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a non-rigid medical image registration method and system driven by surface point cloud based on population adaptive sparse constraints. Background Technology

[0002] Non-rigid registration is a core task in medical image analysis, aiming to establish voxel-by-voxel correspondences between fixed and moving images and to estimate the deformation vector field (DVF) driving deformation in the moving image. Existing main techniques can be categorized into three main types, each with its own limitations: (i) Voxel registration methods, represented by B-spline free-form deformation (FFD) and the Demons algorithm, solve for the deformation field by maximizing image gray-level similarity metrics (mutual information MI, mean square error MSE, etc.). These methods rely on internal volume gray-level information, have limited ability to perceive image boundary and contour geometric deformation, and have difficulty accurately defining cross-modal similarity metrics in multimodal registration.

[0003] (ii) Point cloud registration methods, represented by Iterative Closest Point (ICP) and its variants, estimate the transformation by establishing correspondences between point sets. These methods lack internal volume constraints; the estimated transformation only applies to surface sampling points, failing to directly provide the dense deformation field in voxel space or constrain the continuous deformation of internal organs. Existing research (such as Sem-ICP) has introduced semantic labels to improve matching robustness, but the sparse-dense fusion problem remains unresolved.

[0004] (III) Deep learning-based registration methods, represented by VoxelMorph (Balakrishnan et al., 2019, CVPR), utilize convolutional neural networks to directly regress the deformation field, achieving real-time registration during inference. However, these methods use voxel data as the sole input, requiring labeled training data or unsupervised similarity loss, and neither utilizes statistical priors on the deformation distribution of the patient population. Recent work (such as Chasing Clouds) proposes voxelizing point clouds and constructing a differentiable rasterization loss, but this approach is essentially still a pure voxel registration framework, failing to achieve true coupling between point cloud geometric constraints and voxel optimization.

[0005] In addition, existing registration schemes lack deformation priors based on historical samples, and are not robust enough to cases with large individual differences. Summary of the Invention

[0006] Therefore, it is necessary to address the technical problems of the existing methods mentioned above in terms of multimodal adaptability, complex deformation modeling, and robustness to individual differences by providing a non-rigid medical image registration method and system based on population adaptive sparse constraints driven by surface point cloud, which has strong adaptability, high registration accuracy, and good robustness.

[0007] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: In a first aspect, the present invention provides a non-rigid medical image registration method driven by surface point clouds based on crowd adaptive sparse constraints, comprising: Acquire fixed images, moving images, and surface point clouds of the target object; The moving image is used to perform coarse registration on the body surface point cloud to obtain a simplified body surface point cloud. Calculate the signed distance field for the fixed image; Obtain historical population deformation statistics priors, and construct a sub-population deformation basis dictionary based on the historical population deformation statistics priors; Based on the deformation basis dictionary of the sub-group, the non-rigid deformation field to be estimated for the moving image is constructed; Based on the non-rigid deformation field to be estimated, the fixed image, the moving image, the simplified surface point cloud, and the signed distance field, a joint energy function is constructed. Solving the joint energy function yields the optimal non-rigid deformation field; The moving image and the simplified body surface point cloud are processed using the optimal non-rigid deformation field to obtain registered medical image data.

[0008] Secondly, the present invention also provides a non-rigid medical image registration system driven by surface point clouds based on crowd adaptive sparse constraints, utilizing the non-rigid medical image registration method driven by surface point clouds based on crowd adaptive sparse constraints as described above, including: The multimodal medical image acquisition module is used to acquire fixed images, moving images, and surface point clouds of the target object. The body surface point cloud coarse registration module is used to coarsely register the body surface point cloud with the moving image to obtain a simplified body surface point cloud. A signed distance field calculation module is used to calculate the signed distance field of the fixed image; The sub-population deformation basis dictionary construction module is used to obtain historical population deformation statistical priors and construct sub-population deformation basis dictionaries based on the historical population deformation statistical priors. The module for constructing the non-rigid deformation field to be estimated is used to construct the non-rigid deformation field to be estimated of the moving image based on the deformation basis dictionary of the sub-group. A joint energy function construction module is used to construct a joint energy function based on the non-rigid deformation field to be estimated, the fixed image, the moving image, the simplified surface point cloud, and the signed distance field. The joint energy function solving module is used to solve the joint energy function to obtain the optimal non-rigid deformation field; The medical image data registration module is used to process the moving image and the simplified body surface point cloud using the optimal non-rigid deformation field to obtain registered medical image data.

[0009] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention constructs an estimated non-rigid deformation field for a moving image based on a sub-population deformation basis dictionary. It then constructs a joint energy function based on the estimated non-rigid deformation field, a fixed image, a moving image, a simplified surface point cloud, and a signed distance field. This function integrates the surface point cloud of the target object, the voxel intensity of the medical image, and the sub-population deformation statistical prior into the same framework, solving the problem in existing methods where multimodal medical image data are independent and cannot be complementary. This invention constrains the surface point cloud using a signed distance field and maps it to the voxel space of the medical image, eliminating the need for explicit point correspondences and avoiding the local extremum problem of ICP (Internal Capacity Points). It also preserves geometric information, resulting in high registration accuracy and overcoming the boundary insensitivity of pure intensity registration. Furthermore, by constructing a sub-population deformation basis dictionary and linearly combining historical deformation fields, this invention embeds the sub-population statistical prior into individual registration optimization, significantly improving individual robustness and supporting continuous learning to expand the basis dictionary. Furthermore, this invention utilizes real-time, low-cost surface point clouds to constrain the estimation of internal organ deformation fields, reducing reliance on synchronous volumetric scanning and making it suitable for real-time organ tracking scenarios. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating a non-rigid medical image registration method driven by surface point cloud based on population adaptive sparse constraints in some embodiments of this application. Figure 2 This is a moving image of a non-rigid medical image registration method driven by surface point cloud based on crowd adaptive sparse constraints, as described in some embodiments of this application. Figure 3 This is a fixed image for a non-rigid medical image registration method driven by surface point cloud based on crowd adaptive sparse constraints in some embodiments of this application; Figure 4 This is a registered medical image data of a non-rigid medical image registration method driven by surface point cloud based on crowd adaptive sparse constraints in some embodiments of this application. Figure 5This is a schematic diagram of the framework of a non-rigid medical image registration system driven by surface point cloud based on population adaptive sparse constraints in some embodiments of this application. Detailed Implementation

[0011] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the description of embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses. The term "determine" broadly covers a wide variety of actions, including acquiring, calculating, processing, deriving, investigating, searching (e.g., searching in a table, database, or other data structure), probing, and similar actions; it may also include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), and similar actions; it may also include generating, creating, establishing, and similar actions; and parsing, selecting, choosing, and similar actions, etc. Definitions of other terms will be given in the following description.

[0012] It should be noted that when one element is considered to be "connected" to another element, it can be directly connected to the other element or connected to the other element through an intermediary element. Furthermore, in the following embodiments, "connection" should be understood as "electrical connection," "communication connection," etc., if there is transmission of electrical signals or data between the connected objects.

[0013] It should be emphasized that the acquisition, transmission, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of national laws and regulations.

[0014] In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0015] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0016] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Example 1 This embodiment provides a non-rigid medical image registration method driven by surface point clouds based on crowd-adaptive sparse constraints. (See also...) Figure 1 ,include: Acquire fixed images, moving images, and surface point clouds of the target object; The moving image is used to perform coarse registration on the body surface point cloud to obtain a simplified body surface point cloud. Calculate the signed distance field for the fixed image; Obtain historical population deformation statistics priors, and construct a sub-population deformation basis dictionary based on the historical population deformation statistics priors; Based on the deformation basis dictionary of the sub-group, the non-rigid deformation field to be estimated for the moving image is constructed; Based on the non-rigid deformation field to be estimated, the fixed image, the moving image, the simplified surface point cloud, and the signed distance field, a joint energy function is constructed. Solving the joint energy function yields the optimal non-rigid deformation field; The moving image and the simplified body surface point cloud are processed using the optimal non-rigid deformation field to obtain registered medical image data.

[0018] Compared to existing technologies, this invention constructs an estimated non-rigid deformation field for a moving image based on a sub-population deformation basis dictionary. It then constructs a joint energy function based on the estimated non-rigid deformation field, a fixed image, a moving image, a simplified surface point cloud, and a signed distance field. This integrates the surface point cloud of the target object, the voxel intensity of the medical image, and the sub-population deformation statistical prior into the same joint energy function framework, solving the problem in existing methods where multimodal medical image data are independent and cannot complement each other. This invention constrains the surface point cloud using a signed distance field and maps it to the voxel space of the medical image, eliminating the need for explicit point correspondences and avoiding the local extremum problem of ICP (Internal Capacity Points). It also preserves geometric information, resulting in high registration accuracy and overcoming the boundary insensitivity of pure intensity registration. Furthermore, by constructing a sub-population deformation basis dictionary and linearly combining historical deformation fields, this invention embeds the sub-population statistical prior into individual registration optimization, significantly improving individual robustness and supporting continuous learning to expand the basis dictionary. Furthermore, this invention utilizes real-time, low-cost surface point clouds to constrain the estimation of internal organ deformation fields, reducing reliance on synchronous volumetric scanning and making it suitable for real-time organ tracking scenarios.

[0019] In some preferred embodiments, the step of coarsely registering the body surface point cloud with the moving image to obtain a simplified body surface point cloud includes: Voxel downsampling is performed on the surface point cloud to obtain the first point cloud data; Principal component analysis is used to estimate the normal vector of each point in the first point cloud data, and normal flipping correction is performed based on viewpoint consistency to obtain the second point cloud data; Based on the second point cloud data, a fast point feature histogram descriptor is extracted; Based on the isosurface of the moving image, extract the voxel surface point cloud; Based on the fast point feature histogram descriptor, the second point cloud data and the voxel surface point cloud are rigidly coarsely registered to obtain a simplified voxel surface point cloud.

[0020] In this embodiment, voxel downsampling is performed on the body surface point cloud to reduce the body surface point cloud to the target density, thereby obtaining the first point cloud data; Robust estimation of local normal vectors in point clouds is achieved through principal component analysis (PCA), and normal vector unification correction is completed through viewpoint consistency. This eliminates ambiguity in normal vector direction, reduces the impact of noise, and constructs a consistent local geometric representation, providing a stable and reliable geometric prior for subsequent feature construction and point cloud registration, thereby improving registration convergence speed and matching accuracy. Based on the fast point feature histogram descriptor, the second point cloud data and the voxel surface point cloud are rigidly coarsely registered to obtain a simplified voxel surface point cloud and eliminate global pose deviation.

[0021] In some preferred embodiments, obtaining historical population deformation statistical priors and constructing a sub-population deformation basis dictionary based on the historical population deformation statistical priors includes: The historical population deformation statistics prior is a set of deformation fields pre-constructed based on multiple historical medical image registration samples; Based on medical prior variables or unsupervised clustering, the historical population deformation statistics prior is divided into multiple sub-populations; Principal component analysis is performed on the deformation field set corresponding to each sub-population to extract at least one principal component to construct the deformation basis of that sub-population; Based on the deformation basis of all sub-groups, construct the deformation basis dictionary of the sub-groups.

[0022] In this embodiment, during the offline training phase, according to A priori statistical understanding of historical population deformation is constructed for each historical medical image registration sample, which is represented as follows:

[0023] in, This represents the statistical prior of historical population deformation. Indicates the first A deformation field Indicates the total number of deformation fields; Based on clinical prior variables (such as respiratory phase labels, body mass index ranges, tumor location, etc.) or unsupervised clustering (such as K-Means, spectral clustering), the statistical prior of historical population deformation is used. Divided into Each subgroup contains semantically similar deformation patterns, represented as follows:

[0024] in, Indicates the first Individuals, Indicates the total number of subgroups; For each subgroup Principal component analysis (PCA) was performed on the deformation field set within the data before extraction. Principal components are used to construct the deformable basis of this subpopulation, which is represented as follows:

[0025] in, Indicates the first Deformation base of individual population This represents a voxel space reference coordinate system defined based on a fixed image. This represents the total number of voxels in the voxel space reference coordinate system. Indicates the first Baseline size of individual populations Indicates from the first The first one extracted from the deformation field set corresponding to each subgroup One principal component; For example: the For obese breast cancer patients whose respiratory phase is at the end of inspiration, before extraction... They are respectively The characteristic is restricted downward movement of the diaphragm. Stretching of the lateral chest wall fat layer; Based on the deformation basis of all sub-groups, the deformation basis of all sub-groups is merged to construct the deformation basis dictionary of the sub-groups, which is represented as follows:

[0026]

[0027] in, A base dictionary representing the deformable subgroups. This represents the total number of bases in the subgroup's deformable base dictionary; The subgroup deformation base dictionary Each column It is represented as a basis vector, corresponding to the three-dimensional displacement mode of all points in the voxel space reference coordinate system.

[0028] For each basis vector Normalize it according to the deformation magnitude (L2 norm mean) based on the statistical prior of deformation in historical populations to ensure that it is consistent with the sparse coefficient vector. Dimensional consistency.

[0029] In some preferred embodiments, the step of constructing the estimated non-rigid deformation field of the moving image based on the deformation basis dictionary of the sub-group is expressed as follows:

[0030]

[0031]

[0032] in, The first element in the subgroup deformable basis dictionary represents the... The sparse coefficient vector corresponding to each deformable basis; This represents a voxel space reference coordinate system defined based on the fixed image; Represents a voxel space reference coordinate system any point in the middle The corresponding non-rigid deformation field to be estimated; Represented as a sparse coefficient vector; Represented as sparse substrate deformation components; This represents the individual residual deformation component.

[0033] In this embodiment, the individual residual deformation component By parameterizing the 3D B-spline control point mesh (control point spacing ≤ 10mm), the smoothness and topological continuity of deformation are ensured.

[0034] The parameterization of the non-rigid deformation field to be estimated into a two-layer structure of sparse base deformation components and individual residual deformation components not only utilizes the statistical regularity of historical data but also preserves sufficient degrees of freedom for the current individual. This is the key design of this invention to achieve a balance between accuracy and robustness.

[0035] In some preferred embodiments, constructing a joint energy function based on the non-rigid deformation field to be estimated, the fixed image, the moving image, the simplified surface point cloud, and the signed distance field includes: Based on the fixed image, the moving image, and the non-rigid deformation field to be estimated, a voxel similarity term is constructed. Based on the simplified surface point cloud, the signed distance field, and the non-rigid deformation field to be estimated, a surface point cloud geometric constraint term is constructed. Based on the non-rigid deformation field to be estimated, a deformation field smoothing regularization term is constructed. Based on the sparse coefficient vector, a sparsity regularization term is constructed; Based on the fixed image, the simplified body surface point cloud, and the multiple sub-groups, a sub-group affiliation constraint term is constructed; The joint energy function is constructed based on the voxel similarity term, the surface point cloud geometric constraint term, the deformation field smoothing regularization term, the sparsity regularization term, and the subpopulation attribution constraint term.

[0036] In this embodiment, the following joint energy function is constructed to simultaneously constrain voxel intensity uniformity, surface geometry uniformity, deformation smoothness, coefficient sparsity, and subpopulation attribution:

[0037] in, and These are the first weight hyperparameter, the second weight hyperparameter, the third weight hyperparameter, and the fourth weight hyperparameter, respectively. and All are greater than 0; Indicates voxel similarity terms; Represents the geometric constraints of the surface point cloud; This represents the smoothing regularization term for the deformation field; Represents a sparsity regularization term; This represents the subgroup affiliation constraint.

[0038] The proposed joint energy function incorporates surface point cloud geometric constraints (sparse / surface), voxel intensity constraints (dense / volume), and sub-population deformation statistical priors into the same energy function framework, solving the problem that existing methods are independent and cannot complement each other.

[0039] The voxel similarity term is used to measure the grayscale similarity between the fixed medical image data and the floating medical image data. It is the main data item driving registration, and its expression is as follows:

[0040] in, Represents normalized mutual information; Represents a fixed image. Indicates a moving image. and These represent the dimensions of the image in each dimension; Achieving this through trilinear interpolation Differentiable sampling enables For deformation field It is differentiable, thus supporting gradient backpropagation.

[0041] The surface point cloud geometric constraint term maps the geometric information of the simplified surface point cloud to the voxel space reference coordinate system through a signed distance field, and aligns the constrained and deformed simplified surface point cloud with the fixed image target contour. Its expression is as follows:

[0042] in, A signed distance field representing a fixed target region in an image, taking a value of zero at the target surface, negative inside and positive outside (or vice versa, depending on the definition convention). This represents the first point in the simplified body surface point cloud. One point; This indicates the first [unit / item] after transformation by the non-rigid deformation field to be estimated. The coordinates of the points; The Huber robust loss function suppresses the influence of outlier points caused by point cloud noise or occlusion, and is defined as follows:

[0043] This design enables point cloud constraints to be seamlessly coupled with voxel space via SDF, eliminating the need for explicit point correspondences and avoiding the bottleneck of correspondence establishment and local extremum problems in ICP-type methods.

[0044] The deformation field smoothing regularization term penalizes the first and second gradients of the parameterized deformation field to ensure the continuity and physical plausibility of the deformation (no folding, no tearing), and its expression is as follows:

[0045]

[0046] in, The integral representing the Frobenius norm of the Jacobian matrix measures the smoothness of first-order deformation. It represents a second-order differential operator term, which measures the smoothness of second-order deformation. This is represented as a second-order smoothing weight (hyperparameter), balancing the magnitudes of the first-order and second-order constraints, and ; Due to individual residual deformation components B-spline parameterization already provides a certain degree of smoothness. It mainly functions to estimate the non-rigid deformation field after connecting the sparse base deformation component and the individual residual deformation component, thus avoiding discontinuous jumps between the two.

[0047] Sparsity regularization term constrains sparse coefficient vector This allows the current deformation to be expressed by a sparse basis deformation component, which is formed by linearly combining no more than a preset number of basis vectors, thus suppressing overfitting. Furthermore, Group Lasso is used to constrain the sparse coefficient sub-vectors corresponding to each sub-population, enabling registration to utilize the deformation distribution of semantically similar historical samples. This significantly improves the robustness to individual differences caused by clinical factors such as respiratory phase and body position changes, achieving adaptive selection of sub-populations. Its expression is as follows:

[0048]

[0049] in, For the first The sparse coefficient subvectors corresponding to each subgroup; Group Lasso shrinks the entire group of coefficients to zero simultaneously, so that the optimized active basis is concentrated in 1 to 2 semantically similar subgroups, realizing implicit subgroup adaptive selection without the need to pre-specify the subgroup affiliation.

[0050] The subgroup attribution constraint uses a classification network or Bayesian estimation to predict the subgroup to which the target object belongs as a soft attribution prior. Based on this soft attribution prior, soft constraints are applied in solving the optimization problem to guide the sparse coefficient vector. The non-zero elements in the vector are concentrated in the basis vectors (guide vectors) of the subgroup to which the target object belongs. The active components are concentrated in the basis of the corresponding subpopulation, which complements the sparsity regularization term, and its expression is as follows:

[0051]

[0052] in, This indicates that the target object belongs to a subgroup. The posterior probability; The target audience is the subgroup to which it belongs.

[0053] This item and Synergistic effect: Provides explicit soft attribution guidance. Structural sparsity constraints are provided, and together they ensure the concentration of subgroups of sparse coefficients.

[0054] In some preferred embodiments, solving the joint energy function to obtain the optimal non-rigid deformation field includes: With the goal of minimizing the joint energy function, the sparse coefficient vector and the individual residual deformation components are optimized in stages. The optimal sparse coefficient vector and the optimal individual residual deformation components are obtained by solving the problem. The expression for minimizing the joint energy function is as follows:

[0055] in, Represents the optimal sparse coefficient vector; Represents the optimal individual residual deformation component; Represents the joint energy function; and These represent the first weight hyperparameter, the second weight hyperparameter, the third weight hyperparameter, and the fourth weight hyperparameter, respectively. Represented as a voxel similarity term; Represented as surface point cloud geometric constraint terms; Represented as a deformation field smoothing regularization term; Represented as a sparsity regularization term; This is represented as a subgroup affiliation constraint. Based on the optimal sparse coefficient vector and the optimal individual residual deformation components, an optimal non-rigid deformation field is constructed, which is expressed as:

[0056] in, Represented as the optimal non-rigid deformation field; This represents the optimal sparse coefficient vector. This represents the optimal sparse substrate deformation component; This represents the optimal individual residual deformation component.

[0057] In some preferred embodiments, the step of optimizing the sparse coefficient vector and the individual residual deformation components in stages with the objective of minimizing the joint energy function, and solving for the optimal sparse coefficient vector and the optimal individual residual deformation components, includes: Construct a multi-resolution optimization pyramid, which includes sequentially connected first to Lth resolution layers; The sparse coefficient vector and the individual residual deformation components are initialized, and the initialized sparse coefficient vector and the initialized individual residual deformation components are input into the multi-resolution optimization pyramid: For the first resolution layer, the initialized sparse coefficient vector and the initialized individual residual deformation components are input. The initialized individual residual deformation components are fixed, and the initialized sparse coefficient vector is updated through the first optimizer until the first preset convergence condition is met and the update stops. The sparse coefficient vector after the first resolution layer stops updating is input to the second resolution layer. For the second to the Kth resolution layers, in the current resolution layer, the individual residual deformation components are fixed after initialization. The sparse coefficient vector after the previous resolution layer stops updating is updated through the first optimizer until the first preset convergence condition is met and the update stops. The sparse coefficient vector after the current resolution layer stops updating is input to the next resolution layer. The sparse coefficient vector after the Kth resolution layer stops updating is used as the initial optimized sparse coefficient vector and input to the (K+1)th resolution layer. For the K+1 resolution layer, input the initial optimized sparse coefficient vector and the initialized individual residual deformation components. Fix the initial optimized sparse coefficient vector and update the initialized individual residual deformation components through the second optimizer until the second preset convergence condition is met and the update stops. Then input the individual residual deformation components of the K+1 resolution layer after the update stops to the K+2 resolution layer. For resolution layers K+2 to L-1, in the current resolution layer, the initial optimized sparse coefficient vector is fixed. The individual residual deformation components after the previous resolution layer stops updating are iteratively updated through the second optimizer until the second preset convergence condition is met and the update stops. The individual residual deformation components after the current resolution layer stops updating are input to the next resolution layer. The individual residual deformation components after the L-1 resolution layer stops updating are used as the initial optimized individual residual deformation components and input to the Lth resolution layer. For the Lth resolution layer, an initial optimized sparse coefficient vector and an initial optimized individual residual deformation component are input. The initial optimized sparse coefficient vector and the initial optimized individual residual deformation component are optimized using an alternating iterative optimization algorithm until the third preset convergence condition is met, and the optimal sparse coefficient vector and the optimal individual residual deformation component are obtained.

[0058] In this embodiment, both the sparse coefficient vector and the individual residual deformation component are initialized to 0; A coarse-to-fine multi-resolution optimization pyramid is used to optimize the sparse coefficient vector and the individual residual deformation components. For the first resolution layer, the input consists of an initialized sparse coefficient vector and initialized individual residual deformation components. The initialized individual residual deformation components are fixed, and the initialized sparse coefficient vector is updated using the L-BFGS optimizer. and Gradient-guided active basis Concentrate, up to the global energy change at the first resolution layer. or reaching the maximum number of iterations. Stop updating and input the sparse coefficient vector of the first resolution layer after stopping updates into the second resolution layer; By optimizing the first-level resolution, the macroscopic deformation pattern can be quickly determined, avoiding local extrema.

[0059] For resolution layers 2 through K, in each current resolution layer, the initialized individual residual deformation components are fixed. The sparse coefficient vector, after the previous resolution layer stopped updating, is updated using the L-BFGS optimizer. and Gradient-guided active basis Concentrate, up to the global energy change at the current resolution layer. or until the maximum number of iterations is reached. Stop updating and input the sparse coefficient vector of the current resolution layer after stopping the update to the next resolution layer; use the sparse coefficient vector of the Kth resolution layer after stopping the update as the initial optimized sparse coefficient vector to input to the (K+1)th resolution layer. For the (K+1)th resolution layer, the initial optimized sparse coefficient vector and the initialized individual residual deformation components are input. The initial optimized sparse coefficient vector is fixed, and the initialized individual residual deformation components are updated through the Adam optimizer. The B-spline control points of the individual residual deformation components are refined to align the local deformation with the geometric constraints of the surface point cloud and the voxel similarity term, until the global energy change of the (K+1)th resolution layer is reached. or reaching the maximum number of iterations. Stop updating and input the individual residual deformation components of the K+1 resolution layer after stopping the update to the K+2 resolution layer; For resolution layers K+2 to L-1, the initial optimized sparse coefficient vector is fixed in the current resolution layer. The individual residual deformation components after the previous resolution layer stops updating are iteratively updated using the Adam optimizer. The B-spline control points of the individual residual deformation components are refined to align the local deformation with the geometric constraints of the surface point cloud and the voxel similarity term, until the global energy change of the current resolution layer is reached. or reaching the maximum number of iterations. Stop updating, and input the individual residual deformation components after the current resolution layer stops updating to the next resolution layer; use the individual residual deformation components after the L-1 resolution layer stops updating as the initial optimized individual residual deformation components and input them to the L resolution layer; For the Lth resolution layer, an initial optimized sparse coefficient vector and an initial optimized individual residual deformation component are input. The initial optimized sparse coefficient vector and the initial optimized individual residual deformation component are optimized using an alternating iterative optimization algorithm until the third preset convergence condition is met, and the optimal sparse coefficient vector and the optimal individual residual deformation component are obtained.

[0060] At the Lth resolution layer, individual residual deformation components are refined first to ensure millimeter-level registration accuracy.

[0061] In some preferred embodiments, the process of using the optimal non-rigid deformation field to process the moving image and the simplified body surface point cloud to obtain registered medical image data includes: Based on the inverse mapping of the optimal non-rigid deformation field, the moving image is resampled to obtain a registered moving image aligned with the fixed image space, which is represented as:

[0062] in, Indicates the registered moving image; Based on the optimal non-rigid deformation field, the spatial coordinates of each point in the simplified surface point cloud are transformed to obtain a registered surface point cloud aligned with the voxel space reference coordinate system, which is represented as follows:

[0063] in, This indicates the registration of the surface point cloud. This represents a simplified surface point cloud. Indicates the index of a point in a simplified point cloud; The registered moving image and the registered body surface point cloud constitute the registered medical image data. The moving image, the fixed image, and the registered medical image data are viewed sequentially. Figure 2 , Figure 3 and Figure 4 .

[0064] In some preferred embodiments, the method further includes: Calculate the Jacobian determinant of the optimal non-rigid deformation field and count the proportion of negative voxels in it; If the percentage is greater than a preset threshold, a deformation rationality warning is triggered, and the region where the Jacobian determinant is negative is smoothed and corrected.

[0065] In this embodiment, the Jacobian determinant is calculated. .

[0066] Let the preset threshold be If the following conditions are met:

[0067] This triggers a deformation rationality warning and performs a smoothing correction on the negative value region.

[0068] Example 2 This embodiment further provides a non-rigid medical image registration system driven by surface point clouds based on crowd adaptive sparse constraints, based on the method described above. (See also...) Figure 5 ,include: The multimodal medical image acquisition module is used to acquire fixed images, moving images, and surface point clouds of the target object. The body surface point cloud coarse registration module is used to coarsely register the body surface point cloud with the moving image to obtain a simplified body surface point cloud. A signed distance field calculation module is used to calculate the signed distance field of the fixed image; The sub-population deformation basis dictionary construction module is used to obtain historical population deformation statistical priors and construct sub-population deformation basis dictionaries based on the historical population deformation statistical priors. The module for constructing the non-rigid deformation field to be estimated is used to construct the non-rigid deformation field to be estimated of the moving image based on the deformation basis dictionary of the sub-group. A joint energy function construction module is used to construct a joint energy function based on the non-rigid deformation field to be estimated, the fixed image, the moving image, the simplified surface point cloud, and the signed distance field. The joint energy function solving module is used to solve the joint energy function to obtain the optimal non-rigid deformation field; The medical image data registration module is used to process the moving image and the simplified body surface point cloud using the optimal non-rigid deformation field to obtain registered medical image data.

[0069] In this embodiment, the multimodal medical image acquisition module of this system acquires surface point clouds in real time using a low-cost depth camera, without the need for simultaneous high-dose CT scanning. It is suitable for real-time organ tracking scenarios during radiotherapy and has significant medical application value.

[0070] The subpopulation deformation base dictionary construction module of this system can store and manage the subpopulation deformation base dictionary offline, and supports incremental base updates and subpopulation expansion. As medical data accumulates, the system performance continues to improve.

[0071] The system also includes an evaluation module, which is used to calculate registration quality indicators such as Jacobian determinant, Dice coefficient (requiring segmentation map), and target registration error (TRE).

[0072] It is understood that the system in this embodiment corresponds to the method in Embodiment 1 above, and the options in Embodiment 1 above are also applicable to this embodiment, so they will not be described again here.

[0073] Example 3 This embodiment provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor, causing the processor to perform some or all of the steps of the method provided in Embodiment 1 of this application.

[0074] It is understood that the storage medium can be transient or non-transient. Exemplarily, the storage medium includes, but is not limited to, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0075] By way of example, the processor may be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0076] By way of example, the read-only memory includes, but is not limited to, MASK ROM, PROM, EPROM, EEPROM, Flash, etc.

[0077] By way of example, the random access memory includes, but is not limited to, DRAM, SRAM, SDRAM, DDR SDRAM, etc.

[0078] In some examples, a computer program product is provided, which can be implemented by hardware, software, or a combination thereof. As a non-limiting example, the computer program product can be embodied in the storage medium, or it can be embodied in a software product, such as an SDK (Software Development Kit).

[0079] As a non-limiting example, a computer program product is provided, comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer program or computer-executable instructions from the computer-readable storage medium, and executes the computer-executable instructions, causing the electronic device to perform some or all of the steps of the method described in the embodiments of this application.

[0080] In some examples, a computer program is provided, including computer-readable code, wherein, when the computer-readable code is run in a computer device, a processor in the computer device performs some or all of the steps for implementing the method.

[0081] This embodiment also proposes an electronic device, including a memory and a processor. The memory stores at least one instruction, at least one program, code set, or instruction set. When the processor executes the at least one instruction, at least one program, code set, or instruction set, it implements some or all of the steps of the method described in Embodiment 1.

[0082] In some examples, a hardware entity of the electronic device is provided, including: a processor, a memory, and a communication interface; wherein the processor typically controls the overall operation of the electronic device; the communication interface is used to enable the electronic device to communicate with other terminals or servers via a network; the memory is configured to store instructions and applications executable by the processor, and may also cache data to be processed or already processed (including but not limited to image data, audio data, voice communication data, and video communication data) to be processed by the processor and various modules in the electronic device, and may be implemented using flash memory (FLASH), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or random access memory (RAM).

[0083] A processor may include one or more processing elements. Therefore, a processor may include one or more integrated circuits (ICs) configured to perform the functions of the processor. Furthermore, each integrated circuit may include circuitry (e.g., a first circuit, a second circuit, and other circuitry) configured to perform the functions of the processor.

[0084] Furthermore, data can be transferred between the processor, communication interface, and memory via a bus, which can include any number of interconnected buses and bridges, connecting various circuits of one or more processors and memories together.

[0085] It is understood that the options in Embodiment 1 above also apply to this embodiment, so they will not be described again here.

[0086] The same or similar labels correspond to the same or similar parts; The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this application. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0087] In different specific implementations, the methods or systems described in this application can be implemented in software, hardware, or a combination thereof. Furthermore, the order of the method steps can be changed, and various elements can be added, reordered, combined, omitted, or modified.

[0088] Obviously, the above embodiments of this application are merely examples for clearly illustrating this application, and are not intended to limit the implementation of this application, nor are they intended to limit this application. For those skilled in the art, other variations or modifications can be made based on the above description. The separate structural / functional modules or units can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. The structure and function of the separate components can be implemented as a combined structure or component. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of the claims of this application.

Claims

1. A non-rigid medical image registration method driven by surface point clouds based on crowd adaptive sparse constraints, characterized in that, include: Acquire fixed images, moving images, and surface point clouds of the target object; The moving image is used to perform coarse registration on the body surface point cloud to obtain a simplified body surface point cloud. Calculate the signed distance field for the fixed image; A historical population deformation statistical prior is obtained, and a sub-population deformation basis dictionary is constructed based on the historical population deformation statistical prior. The historical population deformation statistical prior is a set of deformation fields pre-constructed based on multiple historical medical image registration samples. The historical population deformation statistical prior is divided into multiple sub-populations based on medical prior variables or unsupervised clustering. Principal component analysis is performed on the deformation field set corresponding to each sub-population to extract at least one principal component to construct the deformation basis for that sub-population. Based on the deformation basis of all sub-populations, the sub-population deformation basis dictionary is constructed. Based on the deformation basis dictionary of the sub-group, the estimated non-rigid deformation field of the moving image is constructed, which is represented as follows: in, This represents a voxel space reference coordinate system defined based on the fixed image; Represents a voxel space reference coordinate system any point in the middle The corresponding non-rigid deformation field to be estimated; Represented as a sparse coefficient vector; Represented as sparse substrate deformation components; Represents the individual residual deformation component; Based on the non-rigid deformation field to be estimated, the fixed image, the moving image, the simplified surface point cloud, and the signed distance field, a joint energy function is constructed. Solving the joint energy function yields the optimal non-rigid deformation field; The moving image and the simplified body surface point cloud are processed using the optimal non-rigid deformation field to obtain registered medical image data.

2. The non-rigid medical image registration method driven by surface point cloud based on crowd adaptive sparse constraints according to claim 1, characterized in that, The step of coarsely registering the body surface point cloud with the moving image to obtain a simplified body surface point cloud includes: Voxel downsampling is performed on the surface point cloud to obtain the first point cloud data; Principal component analysis is used to estimate the normal vector of each point in the first point cloud data, and normal flipping correction is performed based on viewpoint consistency to obtain the second point cloud data; Based on the second point cloud data, a fast point feature histogram descriptor is extracted; Based on the isosurface of the moving image, extract the voxel surface point cloud; Based on the fast point feature histogram descriptor, the second point cloud data and the voxel surface point cloud are rigidly coarsely registered to obtain a simplified voxel surface point cloud.

3. The non-rigid medical image registration method based on surface point cloud driven by crowd adaptive sparse constraints according to claim 2, characterized in that, The construction of a joint energy function based on the non-rigid deformation field to be estimated, the fixed image, the moving image, the simplified surface point cloud, and the signed distance field includes: Based on the fixed image, the moving image, and the non-rigid deformation field to be estimated, a voxel similarity term is constructed. Based on the simplified surface point cloud, the signed distance field, and the non-rigid deformation field to be estimated, a surface point cloud geometric constraint term is constructed. Based on the non-rigid deformation field to be estimated, a deformation field smoothing regularization term is constructed. Based on the sparse coefficient vector, a sparsity regularization term is constructed; Based on the fixed image, the simplified body surface point cloud, and the multiple sub-groups, a sub-group affiliation constraint term is constructed; The joint energy function is constructed based on the voxel similarity term, the surface point cloud geometric constraint term, the deformation field smoothing regularization term, the sparsity regularization term, and the subpopulation attribution constraint term.

4. The non-rigid medical image registration method based on surface point cloud driven by crowd adaptive sparse constraints according to claim 3, characterized in that, Solving the joint energy function to obtain the optimal non-rigid deformation field includes: With the goal of minimizing the joint energy function, the sparse coefficient vector and the individual residual deformation components are optimized in stages. The optimal sparse coefficient vector and the optimal individual residual deformation components are obtained by solving the problem. The expression for minimizing the joint energy function is as follows: in, Represents the optimal sparse coefficient vector; Represents the optimal individual residual deformation component; Represents the joint energy function; and These represent the first weight hyperparameter, the second weight hyperparameter, the third weight hyperparameter, and the fourth weight hyperparameter, respectively. Represented as a voxel similarity term; Represented as surface point cloud geometric constraint terms; Represented as a deformation field smoothing regularization term; Represented as a sparsity regularization term; This is represented as a subgroup affiliation constraint. Based on the optimal sparse coefficient vector and the optimal individual residual deformation components, an optimal non-rigid deformation field is constructed.

5. The non-rigid medical image registration method based on surface point cloud driven by crowd adaptive sparse constraints according to claim 4, characterized in that, The process of optimizing the sparse coefficient vector and the individual residual deformation components in stages, with the objective of minimizing the joint energy function, to obtain the optimal sparse coefficient vector and the optimal individual residual deformation components, includes: Construct a multi-resolution optimization pyramid, which includes sequentially connected first to Lth resolution layers; The sparse coefficient vector and the individual residual deformation components are initialized, and the initialized sparse coefficient vector and the initialized individual residual deformation components are input into the multi-resolution optimization pyramid: For the first resolution layer, the initialized sparse coefficient vector and the initialized individual residual deformation components are input. The initialized individual residual deformation components are fixed, and the initialized sparse coefficient vector is updated through the first optimizer until the first preset convergence condition is met and the update stops. The sparse coefficient vector after the first resolution layer stops updating is input to the second resolution layer. For the second to the Kth resolution layers, in the current resolution layer, the individual residual deformation components are fixed after initialization. The sparse coefficient vector after the previous resolution layer stops updating is updated through the first optimizer until the first preset convergence condition is met and the update stops. The sparse coefficient vector after the current resolution layer stops updating is input to the next resolution layer. The sparse coefficient vector after the Kth resolution layer stops updating is used as the initial optimized sparse coefficient vector and input to the (K+1)th resolution layer. For the K+1 resolution layer, input the initial optimized sparse coefficient vector and the initialized individual residual deformation components. Fix the initial optimized sparse coefficient vector and update the initialized individual residual deformation components through the second optimizer until the second preset convergence condition is met and the update stops. Then input the individual residual deformation components of the K+1 resolution layer after the update stops to the K+2 resolution layer. For resolution layers K+2 to L-1, in the current resolution layer, the initial optimized sparse coefficient vector is fixed. The individual residual deformation components after the previous resolution layer stops updating are iteratively updated through the second optimizer until the second preset convergence condition is met and the update stops. The individual residual deformation components after the current resolution layer stops updating are input to the next resolution layer. The individual residual deformation components after the L-1 resolution layer stops updating are used as the initial optimized individual residual deformation components and input to the Lth resolution layer. For the Lth resolution layer, an initial optimized sparse coefficient vector and an initial optimized individual residual deformation component are input. The initial optimized sparse coefficient vector and the initial optimized individual residual deformation component are optimized using an alternating iterative optimization algorithm until the third preset convergence condition is met, and the optimal sparse coefficient vector and the optimal individual residual deformation component are obtained.

6. The non-rigid medical image registration method based on surface point cloud driven by crowd adaptive sparse constraints according to claim 4, characterized in that, The optimal non-rigid deformation field is used to process the moving image and the simplified body surface point cloud respectively to obtain registered medical image data, including: Based on the inverse mapping of the optimal non-rigid deformation field, the moving image is resampled to obtain a registered moving image aligned with the fixed image space; Based on the optimal non-rigid deformation field, the spatial coordinates of each point in the simplified surface point cloud are transformed to obtain a registered surface point cloud aligned with the voxel space reference coordinate system. The registered moving image and the registered body surface point cloud constitute the registered medical image data.

7. A non-rigid medical image registration method based on surface point cloud driven by crowd adaptive sparse constraints according to any one of claims 1-6, characterized in that, The method further includes: Calculate the Jacobian determinant of the optimal non-rigid deformation field and count the proportion of negative voxels in it; If the percentage is greater than a preset threshold, a deformation rationality warning is triggered, and the region where the Jacobian determinant is negative is smoothed and corrected.

8. A non-rigid medical image registration system driven by surface point cloud based on crowd adaptive sparse constraints, utilizing the non-rigid medical image registration method driven by surface point cloud based on crowd adaptive sparse constraints as described in any one of claims 1-7, characterized in that, include: The multimodal medical image acquisition module is used to acquire fixed images, moving images, and surface point clouds of the target object. The body surface point cloud coarse registration module is used to coarsely register the body surface point cloud with the moving image to obtain a simplified body surface point cloud. A signed distance field calculation module is used to calculate the signed distance field of the fixed image; The sub-population deformation basis dictionary construction module is used to obtain historical population deformation statistical priors and construct sub-population deformation basis dictionaries based on the historical population deformation statistical priors. The module for constructing the non-rigid deformation field to be estimated is used to construct the non-rigid deformation field to be estimated of the moving image based on the deformation basis dictionary of the sub-group. A joint energy function construction module is used to construct a joint energy function based on the non-rigid deformation field to be estimated, the fixed image, the moving image, the simplified surface point cloud, and the signed distance field. The joint energy function solving module is used to solve the joint energy function to obtain the optimal non-rigid deformation field; The medical image data registration module is used to process the moving image and the simplified body surface point cloud using the optimal non-rigid deformation field to obtain registered medical image data.

Images