Multi-modal medical image registration method, system and electronic device

Abstract

The application provides a multi-modal medical image registration method and system, which comprises the following steps: performing rigid coarse registration on a target multi-modal medical image after aligning the respiratory phase, obtaining an initial deformation field based on an initial radio transformation matrix of a generated global image and an initial grid spacing of the target multi-modal medical image, and calculating a normalized cross-correlation (NCC) gradient value of the global image; then identifying a local image of the target multi-modal medical image, dynamically updating the grid spacing of the local image, updating the deformation field, and calculating an NCC gradient value of the local image; determining a local displacement field according to the NCC gradient value of the local image and a preset gradient value, and superimposing the initial radio transformation matrix on the local displacement field to obtain a final registration result. The rigid registration of the target multi-modal medical image can ensure the end-expiratory phase alignment of the image, the non-rigid registration can further optimize the deformation field, and the registration accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of big data processing technology, and in particular to a multimodal medical image registration method, system and electronic device. Background Technology

[0002] In medical diagnosis, doctors often need to combine multimodal medical images to diagnose conditions and assess treatment effectiveness. However, these images exist independently, differing in imaging principles, resolution, contrast, and other aspects, and do not have a precise correspondence. Furthermore, the patient's respiratory status at different examination times can also lead to differences in respiratory phase between medical images, resulting in registration errors. This is particularly problematic for assessing basal segment lesions in elderly patients with community-acquired pneumonia (CAP). Due to these registration errors between multimodal medical images, doctors must rely on experience and subjective judgment to match the two images, which may be influenced by subjective factors such as fatigue and differences in experience, leading to judgment errors. Summary of the Invention

[0003] In view of the above problems, this application provides a multimodal medical image registration method and system that overcomes or at least partially solves the above problems. Achieving accurate registration between multimodal medical images has important clinical significance and can assist doctors in more accurately diagnosing diseases and evaluating treatment effects.

[0004] In a first aspect, embodiments of this application provide a multimodal medical image registration method, including:

[0005] Rigid coarse registration is performed on the target multimodal medical images after respiratory phase alignment to generate the initial radiometric transformation matrix of the global image. 0( P );

[0006] Based on the initial radiometric transformation matrix 0( P The initial control mesh is generated based on the initial mesh spacing L0 of the target multimodal medical image, and the initial deformation field is obtained. init ( P According to the initial deformation field init ( P Calculate the NCC (Normalized cross-correlation) gradient value of the global image;

[0007] Step S: Identify local images of the target multimodal medical image based on the NCC gradient value, and dynamically update the grid spacing of the local image using a free-form deformation model to obtain the updated grid spacing L. init To increase mesh density; based on the initial deformation field init ( P ) and the updated grid spacing L init Generate a local control mesh to obtain the updated deformation field. update ( P According to the updated deformation field update ( P Calculate the NCC gradient value of the local image;

[0008] In response to the NCC gradient value of the local image being greater than a preset gradient value, step S continues to be executed until each NCC gradient value is within a preset range, thus obtaining the local displacement field Δ. ( P );

[0009] In response to the NCC gradient value of the local image being less than or equal to a preset gradient value, the updated deformation field is used... update ( P Determine the local displacement field Δ ( P );

[0010] The initial radiometric transformation matrix 0( P The local displacement field Δ is superimposed. ( P The registration results of global rigid alignment and local transformation of the fused target multimodal medical images are obtained.

[0011] Secondly, embodiments of this application provide a multimodal medical image registration system, which includes a rigid coarse registration module, a global image calculation module, a local image calculation module, a local displacement field determination module, and a registration result module;

[0012] The rigid coarse registration module is used to perform rigid coarse registration on the target multimodal medical image after respiratory phase alignment to generate the initial radiometric transformation matrix of the global image. 0( P );

[0013] The global image calculation module is used to calculate the initial radiometric transformation matrix generated by the rigid coarse registration module. 0( PThe initial control mesh is generated based on the initial mesh spacing L0 of the target multimodal medical image, and the initial deformation field is obtained. init ( P According to the initial deformation field init ( P Calculate the normalized cross-correlation (NCC) gradient value of the global image;

[0014] The local image calculation module is used to identify local images of the target multimodal medical image based on the NCC gradient value calculated by the global image calculation module, and dynamically update the grid spacing of the local image using a free deformation model to obtain the updated grid spacing L. init To increase mesh density; based on the initial deformation field init ( P ) and the updated grid spacing L init A local control mesh is generated to obtain the adjusted deformation field. update ( P According to the adjusted deformation field update ( P Calculate the NCC gradient value of the local image;

[0015] The local displacement field determination module is configured to continue executing step S in response to the NCC gradient value of the local image being greater than a preset gradient value, until each NCC gradient value is within a preset range, thereby obtaining the local displacement field Δ. ( P ); In response to the NCC gradient value of the local image being less than or equal to a preset gradient value, based on the updated deformation field update ( P Determine the local displacement field Δ ( P );

[0016] The registration result module is used to process the initial radiometric transformation matrix obtained by the rigid coarse registration module. 0( P The local displacement field Δ obtained by superimposing the local displacement field determination module ( P The registration results of global rigid alignment and local transformation of the fused target multimodal medical images are obtained.

[0017] Thirdly, embodiments of this application provide an electronic device, which includes: a processor, a communication interface, a memory, and a communication bus;

[0018] The processor, the communication interface, and the memory communicate with each other through the communication bus; the processor calls the logical instructions in the memory to execute the computer program to implement the multimodal medical image registration method described above.

[0019] The technical solution of this application embodiment performs rigid coarse registration on the target multimodal medical image after respiratory phase alignment to generate an initial radiometric transformation matrix for the global image. Then, based on this initial radiometric transformation matrix and the initial grid spacing of the target multimodal medical image, an initial deformation field and the NCC gradient value of the global image are generated. The deformation field is then adjusted to calculate the NCC gradient value of the local image to obtain a local displacement field. Thus, based on the initial radiometric transformation moment and the local displacement field, a registration result combining global rigid alignment and local transformation of the target multimodal medical image is obtained. In this embodiment, rigid registration of the target multimodal medical image ensures end-expiratory phase alignment, while non-rigid registration further optimizes the deformation field and improves registration accuracy. By performing global rigid alignment and local transformation on the target multimodal medical image, the registered image combines the advantages of both modalities, helping doctors more accurately determine the extent and nature of lesions and more accurately assess treatment effects. Attached Figure Description

[0020] Figure 1 This diagram illustrates the multimodal medical image registration method provided in the embodiments of this application.

[0021] Figure 2 This is another schematic diagram illustrating the multimodal medical image registration method provided in the embodiments of this application;

[0022] Figure 3 This is a schematic diagram illustrating the method for medical image registration of chest X-ray and CT images provided in an embodiment of this application;

[0023] Figure 4 This diagram illustrates another method for multimodal medical image registration provided in the embodiments of this application.

[0024] Figure 5 This diagram illustrates the multimodal medical image registration system provided in an embodiment of this application.

[0025] Figure 6 This is a schematic diagram of the electronic device structure provided in the embodiments of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Multiple embodiments in this application may include two or more.

[0028] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0029] Multimodal medical imaging differs in imaging principles, resolution, and contrast. For example, CT (Computed Tomography) is sensitive to density changes in the lungs and can clearly show small lung structures and lesions, while MRI (Magnetic Resonance Imaging) performs better in terms of soft tissue contrast and can better show structures around the lungs such as the mediastinum and chest wall.

[0030] The registered multimodal medical images in this application combine the advantages of two modalities. The large model analyzes the registered images and extracts various quantitative features, such as the size, density, and signal intensity of the lesion area. These quantitative features can provide doctors with more objective diagnostic evidence and significantly improve diagnostic accuracy. Thus, for lung inflammation, especially for elderly patients with CAP (Community-Acquired Pneumonia), CT images can provide density changes in the inflamed area, while MRI images can provide soft tissue contrast changes in the inflamed area. In CAP diagnosis, the large model can more accurately determine the nature and extent of inflammation, helping doctors differentiate between bacterial and viral pneumonia, thereby selecting more appropriate antibiotics or antiviral drugs.

[0031] like Figure 1 As shown in the figure, this application provides a multimodal medical image registration method, including the following steps:

[0032] Step 101: Perform rigid coarse registration on the target multimodal medical image after respiratory phase alignment to generate the initial radiometric transformation matrix of the global image;

[0033] Step 102: Generate an initial control grid based on the initial radiometric transformation matrix and the initial grid spacing of the target multimodal medical image to obtain the initial deformation field, and calculate the NCC gradient value of the global image based on the initial deformation field.

[0034] Step 103: Identify local images of the target multimodal medical image based on the NCC gradient value, dynamically update the grid spacing of the local image using a free deformation model to obtain the updated grid spacing, thereby improving the grid density; generate a local control grid based on the initial deformation field and the updated grid spacing to obtain the updated deformation field, and calculate the NCC gradient value of the local image based on the updated deformation field.

[0035] Step 104: If the NCC gradient value of the local image is greater than the preset gradient value, continue to execute step 103 until each NCC gradient value is within the preset range to obtain the local displacement field; if the NCC gradient value of the local image is less than or equal to the preset gradient value, determine the local displacement field based on the updated deformation field.

[0036] Step 105: Superimpose the initial radiometric transformation matrix with the local displacement field to obtain the registration result of global rigid alignment and local transformation of the fused target multimodal medical image.

[0037] In this embodiment, rigid registration of the target multimodal medical image ensures end-expiratory phase alignment, while non-rigid registration further optimizes the deformation field and improves registration accuracy. By performing global rigid alignment and local transformation on the target multimodal medical image, the registered image combines the advantages of both modalities, helping doctors to more accurately determine the extent and nature of lesions and to more accurately assess treatment effectiveness.

[0038] like Figure 2 As shown in the figure, this application provides a multimodal medical image registration method, including the following steps:

[0039] Step 201: Acquire at least two multimodal medical images and their timestamps;

[0040] In this step, the acquisition of at least two multimodal medical images is triggered by the R wave of the ECG gating signal. In this embodiment, the multimodal medical images can be ECG (Electrocardiogram), chest X-ray, or MRI.

[0041] When acquiring ECG images, ECG electrodes are placed on the patient's chest. To ensure the accuracy and stability of ECG image acquisition, a professional ECG acquisition device, such as the PAM-200 module or other compatible ECG monitoring device, is used. The ECG acquisition device and the imaging device are synchronized to ensure that their timestamps are consistent. The imaging device can be a chest X-ray machine or an MRI scanner.

[0042] When acquiring a chest X-ray, the R wave of the ECG signal is used to trigger the X-ray exposure. This is typically done at the end of diastole, approximately 300 ms after the R wave, because the heart's activity is minimal at this time. The timestamp of the chest X-ray acquisition is recorded for later alignment with the timestamp of the MRI acquisition.

[0043] When acquiring MRI data, the R wave of the ECG signal is used to trigger data acquisition. Data acquisition is performed at the end of diastole during the cardiac cycle to reduce cardiac motion artifacts. The timestamp of the MRI acquisition is recorded to ensure consistency with the timestamp of the chest X-ray acquisition.

[0044] Step 202: Calculate the time deviation between multimodal medical images and determine whether it is within the allowable range. If yes, proceed to step 203; otherwise, proceed to step 209.

[0045] This application embodiment calculates the time deviation between multimodal medical images based on the timestamps of the multimodal medical images. If the time deviation is within the allowable range, the medical image registration in step 203 is performed directly; if the time deviation exceeds the allowable range, inter-frame displacement compensation is required. Optical flow or other motion compensation algorithms are used to estimate and compensate for the inter-frame displacement to achieve medical image registration.

[0046] Step 203: Preprocess the multimodal medical images;

[0047] This step is a preferred step. The preprocessing operations in this application embodiment include, but are not limited to: using image denoising algorithms to suppress noise, using CLAHE to enhance local image contrast, using high-frequency emphasis filtering to suppress bone shadows or lung field segmentation to identify lung regions. The preprocessing operations for multimodal medical images are different.

[0048] The following is an example of preprocessing chest X-ray and CT images:

[0049] Preprocessing for chest X-rays includes: using CLAHE (Contrast Limited Adaptive Histogram Equalization) to enhance local image contrast, and using high-frequency emphasis filtering to suppress bone shadows;

[0050] The CLAHE method for enhancing local image contrast involves using parameters clip_limit=2.0 and tile_grid_size=(8,8) to perform histogram equalization on each region, thereby enhancing the contrast of the basal segment in chest X-rays, making lesions and other details clearer, and avoiding image oversaturation or artifacts.

[0051] High-frequency emphasis filtering is used to suppress bone shadows, including: using high-frequency emphasis filtering with a filter kernel=[0,-1,0; -1,5,-1; 0,-1,0] to reduce the shadow interference of skeletal structures such as ribs on chest X-rays and reduce the impact on the observation of lung lesions.

[0052] Preprocessing of CT images includes: using image denoising algorithms to suppress noise, and segmenting the lung fields to identify lung regions;

[0053] The image denoising algorithm used to suppress noise includes: based on the BM3D algorithm (Block-Matching and 3DFiltering, a three-dimensional collaborative filtering algorithm based on block matching, whose core idea is to efficiently suppress noise while preserving image details through similar block grouping and three-dimensional transform domain filtering), the noise standard deviation σ of the input image is set to 50, and noise suppression is performed on low-dose CT. While preserving the structural details of lesions, noise in low-dose CT images is effectively suppressed, thus improving image quality.

[0054] Lung field segmentation identifies lung regions by using the U-Net model to extract lung field regions from CT images as regions of interest, excluding interference from non-lung tissues such as the mediastinum and chest wall, which helps subsequent analysis focus on lung lesions.

[0055] Step 204: Align the respiratory phases of at least two multimodal medical images using the ECG timestamps.

[0056] In this step, end-expiratory phase alignment is performed based on the timestamps of multimodal medical images to ensure consistency of respiratory phases.

[0057] Step 205: Perform rigid coarse registration on the multimodal medical images after respiratory phase alignment to generate an initial radiometric transformation matrix;

[0058] Specifically, in this embodiment, SIFT (Scale-invariant feature transform) is used to extract features from multimodal medical images to obtain rigid transformation parameters. These rigid transformation parameters include key points and feature descriptors. Key points of the multimodal medical images are matched using the feature descriptors, and correct matching point pairs are selected using the RANSAC (Random Sample Consensus) algorithm. The optimal rigid transformation matrix is ​​then calculated and used as the initial radiometric transformation matrix. 0( P ).

[0059] The following is an example of rigid coarse registration of chest X-ray and CT images:

[0060] ECG Gated Signal Alignment: Aligns the end-expiratory phase of chest X-ray and CT images using ECG gated signals to ensure consistency in respiratory phase. If the time deviation exceeds 200ms, interpolation compensation is enabled.

[0061] Aligning the chest X-ray timestamp with the end-expiratory phase of the CT image involves the following steps: after the ECG detects the R wave, a trigger pulse is sent to the X-ray machine after a 300ms delay; after the pressure sensor verifies that the chest displacement is <2mm, the X-ray is exposed and the precise timestamp t_xray is recorded; and the CT scanner completes a single spiral scan within the same respiratory cycle (t_xray ±200ms).

[0062] Key point matching: Anatomical landmarks in chest X-ray and CT images are extracted as key points, including tracheal bifurcation and diaphragm top. SIFT feature descriptors are used to extract and match features from these key points, and the RANSAC algorithm is used to robustly optimize the matching results. The RANSAC algorithm is set with 5000 iterations and a threshold of 2mm to remove mismatched points, thereby achieving coarse registration of chest X-ray and CT images at the rigid transformation level, laying the foundation for further fine registration.

[0063] In this embodiment, anatomical landmarks (such as tracheal bifurcation and diaphragm top) of chest X-ray and CT images are extracted by SIFT, anatomical landmarks of chest X-ray and CT images are matched by feature descriptors, and rigid transformation parameters such as rotation matrix and translation vector are estimated by RANSAC algorithm to directly solve large-scale spatial offset caused by respiratory motion or patient position differences. In this embodiment, >15mm is set to represent large-scale spatial offset.

[0064] Existing technologies that use generators to convert multimodal images into target modalities before registration have the following limitations: 1. The generator may alter the anatomical structure of the original image (e.g., excessive smoothing of rib shadows), leading to subsequent registration based on spurious features; 2. The image output by the generator loses temporal information (e.g., respiratory phase), making it impossible to compensate for dynamic displacement using methods such as optical flow; 3. Generator training requires a large amount of pairing data, resulting in instability on low-dose CT or pathological images (e.g., pulmonary consolidation). In contrast, the embodiments of this application directly match features from the original image, avoiding such information loss.

[0065] Step 206: Generate an initial control grid based on the initial radiometric transformation matrix and the initial grid spacing of the multimodal medical image to obtain the initial deformation field, and calculate the NCC gradient value of the global image based on the initial deformation field.

[0066] In this embodiment, based on the rigid transformation parameters obtained from feature extraction of multimodal medical images, and based on the initial radiometric transformation matrix... 0( P The initial control mesh is generated based on the initial mesh spacing L0 of the multimodal medical images, and the initial deformation field is calculated. init ( P Based on the initial deformation field init ( P ) Calculate the NCC gradient value of the global image, transform the CT image to a state that is spatially aligned with the chest X-ray image, and perform spatial alignment of multimodal medical images.

[0067] In this embodiment, after preprocessing the multimodal medical images, rigid coarse registration is performed using ECG gating signals. By compensating for respiratory motion artifacts caused by time differences, the consistency of the two images in the respiratory phase is ensured, thus achieving respiratory phase alignment. Features are extracted from the multimodal medical images to obtain rigid transformation parameters, and an initial deformation field is calculated based on the rigid transformation parameters. The NCC gradient value of the global image is calculated based on the initial deformation field to ensure spatial alignment of the multimodal medical images.

[0068] Step 207: Identify local images of multimodal medical images based on NCC gradient values, dynamically update the grid spacing of local images using a free deformation model, and obtain the updated grid spacing; generate local control grids based on the initial deformation field and the updated grid spacing, and obtain the updated deformation field; calculate the NCC gradient values ​​of local images based on the updated deformation field.

[0069] This application embodiment meshes key regions of multimodal medical images, with the initial spacing of the control point grids for local images being [missing information]. L0. The non-rigid deformation of each control point is achieved based on cubic B-splines. Specifically, the displacement of each control point is determined by three-dimensional cubic B-spline basis functions. β 3 ( u Weighted superposition generates a continuous deformation field. ( P ), the continuous deformation field ( P ) By control point { c i,j,k} and cubic B-spline basis functions β 3 Define the model to obtain the B-spline free-form deformation (FFD) model.

[0070] Specifically, a series of grid points are distributed on the image, and the grid points are dynamically optimized. The positions of these grid points are used to calculate the coordinate offset of each pixel, thereby flexibly modeling and describing the non-rigid deformation between chest X-ray and CT images, adapting to the complex deformation of lung tissue under the influence of factors such as respiration.

[0071] In medical image registration, grid spacing has significant physical implications, directly affecting the resolution and computational accuracy of the deformation field. The physical meaning and representative aspects of grid spacing are described below.

[0072] 1. The physical meaning of grid spacing

[0073] Deformation field resolution: The grid spacing determines the resolution of the deformation field. A smaller grid spacing means higher resolution, which can capture local deformations more finely. For example, a grid spacing of 8 mm in the hilar region can better capture the complex anatomical structures and deformations of the hilar region.

[0074] Computational accuracy: Smaller grid spacing can improve computational accuracy, but it also increases the computational cost. Larger grid spacing, while requiring less computation, may fail to capture subtle local deformations.

[0075] Smoothness of the deformation field: A larger grid spacing can make the deformation field smoother and reduce excessive local distortion. A smaller grid spacing may make the deformation field more complex, but it can better adapt to local deformation.

[0076] 2. The physical meaning and representative content of grid spacing

[0077] The physical meaning of the global default grid spacing of 10mm: A global default grid spacing of 10mm means that initially, a grid node is set every 10mm across the entire image area. This provides a relatively sparse grid, suitable for large-scale deformation capture.

[0078] Representative content: Preliminary registration applicable to the entire lung can capture the overall deformation trend, but may not be able to capture subtle local deformations.

[0079] The physical significance of setting the mesh spacing to 8mm in the hilar region: Based on anatomical atlases, the hilar region has complex anatomical structures, including important blood vessels and bronchi, requiring higher resolution to capture the deformation of these areas. Therefore, the mesh spacing in the hilar region is set to 8mm.

[0080] Key features: More precise capture of local deformation in the hilar region, improving registration accuracy, especially effective in treating lesions in the hilar region.

[0081] In this application embodiment, an updated deformation field is provided. update ( P and based on the updated deformation field update ( P The process of calculating the NCC gradient value of a local image, specifically...

[0082] Based on the initial deformation field init ( P ) and the updated grid spacing L init Generate a local control mesh to obtain the updated deformation field. update ( P According to the updated deformation field update ( P Calculate the NCC gradient value of the local image; when the local gray-level gradient of a certain area is greater than the preset value, reduce the grid spacing of the area with the large gradient by 50% to better capture the deformation details of the area.

[0083] In this embodiment, deformation constraints are set to limit the maximum displacement of mesh nodes, avoiding physiological structural distortion due to excessive deformation, thereby improving the adaptability and accuracy of registration. In this embodiment, the region with a large gradient is ||. In the region where I‖>100HU / mm,‖ I‖ represents the local grayscale gradient, with a maximum displacement of 15mm for the grid nodes.

[0084] The objective function in this embodiment is: F=α NCC(I 1 , )+β ∥ ∥ 2 Among them, NCC is used to measure gray-level similarity, and the normalized cross-correlation weight α = 0.7; ∥ 2 This is a deformation smoothing term used to penalize excessive deformation and prevent local distortion. The weight of the deformation smoothing term is β = 0.3.

[0085]

[0086] Among them, two images (such as X-rays) and I2 (such as deformed CT scans). , Each is its own mean; outputs a scalar value in the range [-1, 1].

[0087] Sum of squares of the deformation field gradient:

[0088]

[0089] Input: Deformation field (Each component represents displacement in the corresponding direction).

[0090] Output: Scalar value, reflecting the total variation of the deformation field.

[0091] in: This represents the displacement components of the deformation field in the x, y, and z directions. ∈[ 10mm, 10mm];

[0092] ,express x right x Partial derivatives in the direction (local deformation rate);

[0093] ∑ This represents summing over all image pixels;

[0094] ∥ ∥ 2 It's a squaring operation used to amplify the penalty in areas of severe deformation, outputting a non-negative value. Specifically, it first calculates the spatial partial derivatives of the deformation field for each displacement component. Calculate the partial derivatives in each of the three directions:

[0095] Discretization method (with) (For example) ≈ ；h Pixel pitch (e.g., 1mm);

[0096] Then square and sum the squares, calculating the sum of the squares of the partial derivatives in the three directions for each voxel position:

[0097]

[0098] Finally, the sum of the squares of all voxels is accumulated globally to obtain the final smoothing term value:

[0099]

[0100] Due to deformation field Control points of B spline c i,j,k In practical calculations, the chain rule is used for transformation:

[0101]

[0102] The calculation of partial derivatives is transformed into a weighted summation of the derivatives of the B-spline basis functions; where the control point spacing... h It directly affects the gradient magnitude.

[0103] This application employs the L-BFGS solver to optimize the objective function, obtaining a non-rigid deformation field, and then performs non-rigid registration based on this field. The memory step count is set to 10, and the convergence condition is ΔNCC < 0.001 or 50 iterations. By optimizing the objective function, the smoothness of the deformation field is constrained while ensuring image grayscale matching, so that the registration result meets the similarity requirements of anatomical structures and avoids unreasonable results caused by excessive deformation. This optimization algorithm can accurately solve the optimal solution of the objective function while ensuring computational efficiency, thereby obtaining a high-quality non-rigid deformation field and achieving precise registration of chest X-ray and CT images.

[0104] This application employs a B-spline free deformation model to dynamically adjust the control grid spacing, further optimizing the deformation field and improving registration accuracy. Taking the non-rigid registration of chest X-ray and CT images of a patient with pulmonary fibrosis as an example, the spacing of the control grid is dynamically adjusted according to the characteristics of different lung regions, as detailed below:

[0105] 1. Initialization:

[0106] Current deformation field init ( P The initial deformation field has a global grid spacing of 10mm, a hilar grid spacing of 8mm, and a peripheral grid spacing of 15mm. This is to ensure registration accuracy while reasonably controlling the computational load and making the registration process more efficient.

[0107] Chest X-ray-CT image pair: Preprocessed chest X-ray and CT images.

[0108] 2. First iteration:

[0109] Calculating the NCC gradient map revealed that the gradient value in the left lower lung region was greater than 100 HU / mm.

[0110] Dynamically refined mesh: The mesh spacing in the lower left lung region was reduced from 10mm to 5mm.

[0111] Update deformation field update ( P ), calculate the gradient of the objective function F, and perform optimization.

[0112] Upon checking the displacement of the mesh nodes, it was found that the displacement of some nodes exceeded the limit (‖Δx‖>15mm). The displacement vectors of these nodes were scaled.

[0113] 3. Second iteration:

[0114] Recalculating the NCC gradient map revealed that gradient values ​​remained high in certain regions, prompting further refinement of the mesh in these areas.

[0115] For example, the grid spacing in a certain area is reduced from 8mm to 4mm.

[0116] Update the deformation field again update ( P ), calculate the gradient of the objective function F, and perform optimization.

[0117] Check the node displacement to ensure that the displacement of all nodes does not exceed 15mm.

[0118] 4. Subsequent iterations:

[0119] Repeat the above process until all gradient values ​​in the NCC gradient plot are less than 100 HU / mm and the displacements of all nodes are within a reasonable range, thus obtaining the updated deformation field. update ( P ).

[0120] The final updated deformation field update ( P It can accurately register CT images onto chest X-ray images, especially in key areas such as lesion areas and hilar regions.

[0121] 5. Output results:

[0122] The deformation field obtained from the last update is determined as the local displacement field Δ. ( P This local displacement field represents the update amount of the deformation field, which is used to adjust the current deformation field.

[0123] Mesh adjustment recommendation: Based on the gradient information obtained during the optimization process, it is recommended to further refine the mesh in the lower left lung region to improve registration accuracy.

[0124] After the above processing,

[0125] Registration accuracy: Through the objective function, the final registration error is less than 1 mm in key areas (such as the edge of the lesion), which significantly improves the registration accuracy.

[0126] Computational efficiency: Although multiple iterations and mesh refinement were performed, the overall computation time remained within an acceptable range due to reasonable control of the lower limit of mesh spacing and displacement constraints, making it suitable for real-time clinical applications.

[0127] Through the above embodiments, by comprehensively considering NCC, deformation smoothing term and edge penalty term, the deformation field can be effectively optimized and the registration accuracy can be improved. At the same time, by dynamically adjusting the mesh density, the adaptability and accuracy of registration can be further improved.

[0128] Step 208: Based on the NCC gradient value of the local image and the preset gradient value, obtain the local displacement field. Based on the initial radiometric transform moment and the local displacement field, obtain the fusion registration result.

[0129] In this step, the magnitude of the NCC gradient value of the local image is compared with the preset gradient value, and different local displacement fields Δ are provided. (P) ;

[0130] Specifically, if the NCC gradient value of a local image is greater than a preset gradient value, step 207 above is continued until each NCC gradient value is within a preset range. The deformation field obtained from the last update is then determined as the local displacement field Δ. ( P );

[0131] In response to the NCC gradient value of a local image being less than or equal to a preset gradient value, the updated deformation field will be... update ( P ) is determined to be a local displacement field Δ ( P ).

[0132] Specifically, in this embodiment of the application, the initial radiometric transformation matrix is... 0( P Superimposed local displacement field Δ ( P This yields the registration results of global rigid alignment and local transformation of the fused target multimodal medical images.

[0133] Through this step, the embodiments of this application retain global alignment while superimposing local deformations, such as lesion contraction / expansion.

[0134] Step 209: Compensate for inter-frame shift in multimodal medical images.

[0135] This application uses optical flow to estimate and compensate for inter-frame displacement. Specifically, a Gaussian pyramid is built for the multimodal medical image. Starting from the top of the Gaussian pyramid, an iterative algorithm is used to calculate the optical flow field. Based on the calculated optical flow field, the pixel values ​​of the previous frame of the multimodal medical image are mapped to the corresponding positions in the compensation frame to achieve alignment of the multimodal images.

[0136] Specifically, optical flow compensation for inter-frame displacement includes: constructing a Gaussian pyramid for each frame of the multimodal image, which consists of multiple levels, gradually refined from the highest resolution at the top to the lowest resolution at the bottom.

[0137] Starting from the top layer of the Gaussian pyramid, an iterative algorithm is used to calculate the optical flow. In each layer of the pyramid, the optical flow estimation result is used as the initial value for the next layer, gradually refining to the low-resolution layer. By minimizing the matching error in the neighborhood of each point, the optical flow of each point in the top layer image is obtained.

[0138] Based on the optical flow estimation results of the current layer, the optical flow vector is fed back to the next layer for further optical flow estimation. This process improves the accuracy of optical flow estimation through multiple iterative optimizations. For example, a coarse optical flow is calculated at the top layer first, and then a finer adjustment is made at the next layer.

[0139] Based on the calculated optical flow field, the image is warped by mapping the pixel values ​​of the previous frame to the corresponding positions in the compensation frame. This process is achieved through image transformation, thereby generating new image frames to reduce blurring or information loss caused by motion. Through multi-scale and multi-iteration optimization, the optical flow field between the two images is finally obtained, and this optical flow field is used to align the images, thus achieving precise image matching.

[0140] This application provides different registration schemes based on the temporal deviation between multimodal medical images and their timestamps, considering whether it is within an acceptable range. When the temporal deviation is within the acceptable range, preprocessing the multimodal medical images, such as noise suppression and contrast enhancement, reduces shadow interference. Rigid registration ensures end-expiratory phase alignment and spatial alignment between the two images, while non-rigid registration further improves registration accuracy. When the temporal deviation exceeds the acceptable range, optical flow is used to estimate and compensate for inter-frame displacement, achieving alignment of the multimodal images. The registered images in this application combine the advantages of both modalities; the precise spatial correspondence helps doctors more accurately determine the extent and nature of lesions and more accurately assess treatment effectiveness.

[0141] The embodiments of this application are applicable to a variety of application scenarios:

[0142] 1. Scenarios where both chest X-rays and CT scans are required.

[0143] Elderly patients with community-acquired pneumonia (CAP) often require a chest X-ray for screening, followed by a CT scan to clarify the details of the lesions. This application's embodiment precisely registers the chest X-ray and CT scan, linking the two types of information to assist doctors in more accurate diagnosis. For example, if a patient's initial chest X-ray shows a blurred shadow in the right lower lung, clinically suspecting CAP, a low-dose CT scan is performed for verification. Registering the chest X-ray and CT scan confirms a ground-glass opacity in the corresponding area on the CT scan, with a volume of 1.5 cm³.

[0144] 2. Typical scenarios of asynchronous data acquisition

[0145] If the interval between two examinations is within the same stage of the disease, such as a chest X-ray and CT scan completed within 24 hours for a CAP patient, and the morphological changes of the lesions are small, the embodiments of this application use respiratory phase synchronization technology, such as ECG gating or optical flow, to estimate and compensate for inter-frame displacement. The respiratory signal is used to synchronize the motion between image or video frames, thereby reducing image blurring or information loss caused by breathing. Effective registration is achieved by compensating for respiratory motion artifacts caused by time differences.

[0146] 3. Special scenarios for synchronous data acquisition

[0147] Completing chest X-ray and CT scans within the same respiratory cycle, such as with dual-modal imaging equipment, can completely eliminate differences in respiratory motion. The advantage of this system lies in its significantly improved accuracy, providing higher quality registration results for clinical diagnosis. However, it requires dedicated hardware support, such as a combined X-ray and CT scanning system.

[0148] The following describes the solutions of the embodiments of this application based on several practical application scenarios.

[0149] Scenario 1: Providing chest X-ray and MRI registration for CAP patients

[0150] First, acquire chest X-rays and MRI scans, along with the timestamps for the chest X-rays and MRI scans;

[0151] ECG signals were acquired using a PAM-200 module to ensure signal stability. ECG electrodes were placed on the patient's chest to record the R wave.

[0152] Chest radiographs were taken 300ms after the R-wave, and the timestamp t of the radiographs was recorded. xray .

[0153] MRI data acquisition was triggered 300ms after the R wave, and the MRI timestamp t was recorded. MRI .

[0154] Then, the time deviation between the chest X-ray and MRI information is calculated to determine whether it is within the allowable range. If it is, image registration is performed; otherwise, inter-frame shift is compensated.

[0155] Calculate the time deviation Δt=t MRI t xray The allowable range provided in this application embodiment is within 200ms.

[0156] If |Δt|≤200 ms, perform image registration directly using steps 103-104 above to obtain the registered chest X-ray and MRI, which will not be elaborated here.

[0157] If |Δt| > 200 ms, optical flow is used to compensate for inter-frame displacement. By constructing a Gaussian pyramid, calculating optical flow, feedback, and iteration, the optical flow field is finally obtained, and this optical flow field is used to warp the image, thereby achieving alignment of the two images.

[0158] In this embodiment, chest X-ray is the gold standard for diagnosing CAP (pulmonary embolism), while MRI provides more detailed information on lung structure, such as pulmonary consolidation, pulmonary edema, pulmonary infarction, and interstitial fibrosis, and is particularly advantageous in differentiating diseases such as pulmonary fibrosis and congestive heart failure. By registering MRI with chest X-ray, the characteristics of lung lesions can be more comprehensively assessed, improving diagnostic accuracy.

[0159] Scenario 2: Registration of cavitary lesions in the right upper lobe of a pneumonia patient

[0160] Step 301: Obtain CT images and chest X-rays of pneumonia patients, perform low-dose noise suppression on CT images, and perform adaptive histogram equalization and bone shadow suppression on chest X-rays.

[0161] Among them, low-dose noise suppression adopts the BM3D algorithm with σ=50, and in adaptive histogram equalization, clip_limit=2.0, tile_grid_size=(8,8) is used to suppress bone shadows by high-frequency emphasis filtering.

[0162] Step 302: Align the end-expiratory phase of the chest X-ray and CT scan using ECG gating signals to achieve rigid coarse registration for respiratory motion compensation and ensure consistency between the two in the respiratory phase.

[0163] For rigid coarse registration of respiratory motion compensation, keypoint matching technology is employed, including: extracting anatomical landmarks from chest X-rays and CT scans, such as tracheal bifurcation and diaphragm roof; matching these landmarks using SIFT feature descriptors; and eliminating mismatched points using the RANSAC algorithm to obtain rigid transformation parameters. The RANSAC algorithm has 5000 iterations and a threshold of 2 mm.

[0164] Step 303: Calculate the initial deformation field based on the rigid transformation parameters, transform the CT image to a state aligned with the chest X-ray image space, apply the deformation field to the CT image, and obtain the registered CT image aligned with the chest X-ray space.

[0165] Step 304: Dynamically adjust the control grid spacing based on the B-spline free deformation model.

[0166] First, an initial mesh is generated, with a global default mesh spacing of 10mm, and a preset mesh spacing of 8mm in the hilar region. Then, the NCC gradient map fed back from the optimizer is received. During optimization, the NCC gradient map is received. For regions with gradients greater than 100HU / mm, the mesh is dynamically refined, halving the mesh spacing, with a lower limit of 5mm. After each iteration, the node displacement is checked for exceeding the limit, preset ‖Δx‖≤15mm. If it exceeds the limit, the displacement vector is scaled to avoid distortion of the physiological structure.

[0167] Preferably, during initial mesh refinement, at the edge of the voids, if the gradient is 150HU / mm and the mesh is refined to 5mm, the optimizer reports that the NCC improvement in this area is slow. Further refinement to 3mm is then performed. After dynamic adjustment and optimization, the alignment error of the void boundary is less than 1mm, and the deformation field... (P) It can more accurately reflect the non-rigid deformation relationship between chest X-ray and CT images, thus improving registration accuracy.

[0168] Step 305: Input and output of the objective function.

[0169] Input: Current deformation field (P) Comparison of chest X-ray and CT images, current deformation field (P)This represents the current non-rigid deformation field used to register CT images to chest X-ray images; the chest X-ray-CT image pair includes the preprocessed chest X-ray image and the CT image.

[0170] Output: Deformation field update Δ And grid adjustment suggestions, deformation field update Δ This represents the update amount of the deformation field, used to adjust the current deformation field; based on the gradient information during the optimization process, it suggests whether the mesh density needs to be adjusted.

[0171] This application embodiment dynamically adjusts the control grid spacing based on the NCC gradient map fed back by the optimizer to control the deformation field. (P) Further optimization involves dynamically refining the mesh in areas with large gradients, such as the edges of voids, to better capture local deformation details. The L-BFGS optimization algorithm and deformation constraints are used to ensure the deformation field... (P) The smoothness and physiological rationality of the process ultimately yield high-precision non-rigid registration results. In this embodiment, not only is rapid alignment achieved in the rigid registration stage, but the B-spline free deformation model is also used in the non-rigid registration stage to dynamically adjust the control grid spacing, further improving registration accuracy.

[0172] Scenario 3: Registration of lung lesions in elderly CAP patients

[0173] Steps 401-403 correspond to steps 301-303 in scenario 2 above, respectively, to obtain the registered CT image and chest X-ray image, which will not be described again here.

[0174] Step 404: Input the registered CT image and chest X-ray image, as well as their edge feature images, to generate the initial mesh.

[0175] In this embodiment, the initial mesh is set to a global default mesh spacing of 10 mm. Based on the anatomical atlas, the mesh spacing in the hilar region is refined to 8 mm. This is because the hilar region contains important anatomical structures, such as blood vessels and bronchi, and deformation in these areas has a significant impact on registration accuracy.

[0176] Step 405: Receive the NCC gradient map fed back by the optimizer, and dynamically adjust the control grid spacing based on the gradient threshold.

[0177] The NCC gradient map reflects the grayscale similarity changes between CT images and chest X-ray images under the current deformation field, with a gradient threshold set at 100 HU / mm. For regions with gradients greater than 100 HU / mm, the deformation in these regions is considered to have a significant impact on registration accuracy, requiring further mesh refinement.

[0178] Dynamically adjusting the grid spacing based on gradient thresholds involves halving the grid spacing for regions with gradients greater than 100 HU / mm, while maintaining a lower limit of 5mm to prevent excessive computational overhead due to over-refinement. For example, if the initial grid spacing is 10mm, the adjusted grid spacing will be 5mm, satisfying the lower limit of 5mm. Similarly, if the initial grid spacing is 8mm, halving it to 4mm, and still maintaining the lower limit of 5mm, the adjusted grid spacing will be 5mm.

[0179] Step 406: After each iteration, check whether the displacement of each grid node exceeds the limit. If the displacement of a node exceeds the limit, scale the displacement vector of that node so that its displacement does not exceed the upper limit.

[0180] The upper limit of displacement is set at 15mm, i.e., |Δx|≤15mm, which can avoid physiological structural distortion due to excessive deformation.

[0181] This application's embodiments achieve a final registration error of less than 1mm in key areas by dynamically adjusting the grid spacing and deformation constraints, significantly improving registration accuracy. Through multiple iterations and grid refinement, the lower limit of the grid spacing and displacement constraints are reasonably controlled, while the overall computation time remains within an acceptable range, making it suitable for real-time clinical applications. When registering lung images of elderly CAP patients, the B-spline free deformation model, which dynamically adjusts and controls the grid spacing, effectively improves registration accuracy while avoiding physiological structural distortion caused by excessive deformation, demonstrating high practical value.

[0182] Step 407: Implement multi-objective optimization based on the objective function.

[0183] The input information for the objective function includes: the current deformation field. (P) Pairing chest X-ray and CT images, output deformation field update Δ (P) And grid adjustment suggestions;

[0184] Current deformation field (P) This represents the current non-rigid deformation field used to register CT images to chest X-ray images; the chest X-ray-CT image pair includes the preprocessed chest X-ray image and the CT image.

[0185] Deformation field update Δ (P) This represents the update amount of the deformation field, used to adjust the current deformation field; the mesh adjustment suggestion refers to the suggestion on whether the mesh density needs to be adjusted based on the gradient information in the optimization process. For example, the mesh adjustment suggestion could be "It is recommended to further densify the mesh in the lower left lung region to improve the registration accuracy".

[0186] In this application's embodiments, for elderly patients with community-acquired pneumonia (CAP), high-precision CT and chest X-ray registration allows doctors to more accurately locate and quantify inflammatory lesions in the lungs. During treatment, multi-temporal registration enables doctors to dynamically monitor changes in the lesions and adjust the treatment plan promptly. This high-precision registration not only improves diagnostic accuracy but also reduces the possibility of misdiagnosis and missed diagnosis, increases medical efficiency, optimizes the utilization of medical resources, supports telemedicine and multidisciplinary collaboration, and promotes the development of medical research and education.

[0187] Scenario 4: Chest X-ray-CT registration for patients with pulmonary fibrosis

[0188] 1. Initialization:

[0189] Current deformation field (P) Initial deformation field, global grid spacing is 10mm, hilar region grid spacing is 8mm.

[0190] Chest X-ray-CT image pair: Preprocessed chest X-ray and CT images.

[0191] 2. First iteration:

[0192] Calculating the NCC gradient map revealed that the gradient value in the left lower lung region was greater than 100 HU / mm.

[0193] Dynamically refined mesh: The mesh spacing in the lower left lung region was reduced from 10mm to 5mm.

[0194] Update deformation field (P) Calculate the gradient of the objective function F and perform optimization.

[0195] Upon checking the displacement of the mesh nodes, it was found that the displacement of some nodes exceeded the limit (‖Δx‖>15mm). The displacement vectors of these nodes were scaled.

[0196] 3. Second iteration:

[0197] Recalculating the NCC gradient map revealed that gradient values ​​remained high in certain regions, prompting further refinement of the mesh in these areas.

[0198] For example, the grid spacing in a certain area is reduced from 8mm to 4mm.

[0199] Update the deformation field again (P) Calculate the gradient of the objective function F and perform optimization.

[0200] Check the node displacement to ensure that the displacement of all nodes does not exceed 15mm.

[0201] 4. Subsequent iterations:

[0202] Repeat the above process until the gradient values ​​in the NCC gradient plot are all less than 100HU / mm and the displacements of all nodes are within a reasonable range.

[0203] The final deformation field (P) It can accurately register CT images onto chest X-ray images, especially in key areas such as lesion areas and hilar regions.

[0204] 5. Output results:

[0205] Deformation field update Δ : Represents the update amount of the deformation field, used to adjust the current deformation field.

[0206] Mesh adjustment recommendation: Based on the gradient information obtained during the optimization process, it is recommended to further refine the mesh in the lower left lung region to improve registration accuracy.

[0207] After the above processing,

[0208] Registration accuracy: Through the objective function, the final registration error is less than 1 mm in key areas (such as the edge of the lesion), which significantly improves the registration accuracy.

[0209] Computational efficiency: Although multiple iterations and mesh refinement were performed, the overall computation time remained within an acceptable range due to reasonable control of the lower limit of mesh spacing and displacement constraints, making it suitable for real-time clinical applications.

[0210] In this embodiment, by comprehensively considering NCC, deformation smoothing term and edge penalty term, the deformation field can be effectively optimized and the registration accuracy can be improved. At the same time, by dynamically adjusting the grid density, the adaptability and accuracy of image registration are further improved.

[0211] like Figure 5 As shown, this application provides a multimodal medical image registration system, including: a rigid coarse registration module 501, a global image calculation module 502, a local image calculation module 503, a local displacement field determination module 504, and a registration result module 505;

[0212] The rigid coarse registration module 501 is used to perform rigid coarse registration on the target multimodal medical image after respiratory phase alignment to generate the initial radiometric transformation matrix of the global image. 0( P );

[0213] Global image calculation module 502 is used to calculate the initial radiometric transformation matrix based on the rigid coarse registration module. 0( PThe initial control mesh is generated based on the initial mesh spacing L0 of the target multimodal medical image, and the initial deformation field is obtained. init ( P According to the initial deformation field init ( P Calculate the normalized cross-correlation (NCC) gradient value of the global image;

[0214] The local image calculation module 503 is used to identify local images of the target multimodal medical image based on the NCC gradient values ​​calculated by the global image calculation module, and dynamically update the grid spacing of the local image using a free deformation model to obtain the updated grid spacing L. init To increase mesh density; based on the initial deformation field init ( P ) and the updated grid spacing L init A local control mesh is generated to obtain the adjusted deformation field. update ( P According to the adjusted deformation field update ( P Calculate the NCC gradient value of the local image;

[0215] The local displacement field determination module 504 is used to continue calling the local image calculation module to adjust the deformation field in response to the NCC gradient value of the local image being greater than a preset gradient value. update ( P ), and obtain the adjusted deformation field. update ( P Calculate the NCC gradient values ​​of the local image until each NCC gradient value is within a preset range, thus obtaining the local displacement field Δ. ( P ); In response to the NCC gradient value of a local image being less than or equal to a preset gradient value, the updated deformation field is adjusted accordingly. update ( P Determine the local displacement field Δ ( P );

[0216] Registration result module 505 is used to process the initial radiometric transformation matrix obtained by the rigid coarse registration module. 0( P The local displacement field Δ obtained by superimposing the local displacement field determination module ( PThis yields the registration results of global rigid alignment and local transformation of the fused target multimodal medical images.

[0217] Furthermore, the multimodal medical image registration system provided in this application embodiment also includes: an inter-frame displacement compensation module; the inter-frame displacement compensation module is used to compensate for the inter-frame displacement by respiratory phase synchronization when the time deviation calculated by the acquisition and processing module exceeds a preset range.

[0218] The core technologies of the embodiments of this application are described in detail below:

[0219] (I) Working Principle

[0220] Specifically, the input information of this embodiment system includes: real-time ECG signal (sampling rate 500Hz), timestamp of DICOM header file of chest X-ray / CT; output: synchronization flag signal (boolean value, True indicates phase matching).

[0221] The system operates as follows: After the ECG detects the R wave, a trigger pulse is sent to the X-ray machine after a 300ms delay. Once the pressure sensor verifies that the chest displacement is <2mm, exposure is performed and a precise timestamp (t_xray) is recorded. The CT scanner completes a single spiral scan within the same respiratory cycle (t_xray ±200ms) and directly performs image calibration. If the scan is not completed within the time limit, optical flow is used to estimate inter-frame displacement compensation.

[0222] (ii) Free deformation model of dynamically adjusted mesh

[0223] The input information for this free-form deformation model includes: synchronized chest X-ray-CT image pairs and gradient plots fed back by the optimizer; the output is: deformation field. (P) The nodal displacement vector.

[0224] The free-form deformation model performs the following operations: a global default mesh spacing of 10mm, with a preset mesh density of 8mm in the hilar region; it receives the NCC gradient map sent by the optimizer, and for regions with gradients > 100HU / mm, halves the mesh spacing, setting a lower limit of 5mm. After each iteration, it checks whether the node displacement exceeds the limit, with a preset limit of ||Δx| ≤ 15mm; if it does, the displacement vector is scaled.

[0225] (III) Objective Function

[0226] The input information for the objective function includes: the current deformation field. (P) Chest X-ray to CT image pair; output deformation field update Δ Grid adjustment suggestions.

[0227] In the objective function, similarity calculation is performed as follows: NCC is calculated within a 7×7 window, and this similarity calculation is used to exclude skeletal regions; the deformation field gradient penalty term is calculated based on smoothness constraints; the deformation field is updated iteratively using L-BFGS, and the mesh density is checked every 5 iterations to see if it needs to be adjusted.

[0228] For example, Figure 6 A schematic diagram of the physical structure of an electronic device is shown. (For example...) Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions from the memory 530. The processor 610 is used to execute various processes of the multimodal medical image registration method of this application embodiment, which will not be described in detail here.

[0229] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0230] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described multimodal medical image registration method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0231] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0232] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0233] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A multimodal medical image registration method, characterized in that, include: Rigid coarse registration is performed on the target multimodal medical images after respiratory phase alignment to generate the initial radiometric transformation matrix of the global image. 0( P ), P This represents the location point of the target in the multimodal medical image space; Based on the initial radiometric transformation matrix 0( P The initial control grid is generated using the initial grid spacing L0 of the target multimodal medical image, and the initial deformation field is obtained. init ( P According to the initial deformation field init ( P Calculate the normalized cross-correlation (NCC) gradient value of the global image; Step S: Identify local images of the target multimodal medical image based on the NCC gradient value, and dynamically update the grid spacing of the local image using a free-form deformation model to obtain the updated grid spacing L. init To increase grid density; Based on the initial deformation field init ( P ) and the updated grid spacing L init Generate a local control mesh to obtain the updated deformation field. update ( P According to the updated deformation field update ( P Calculate the NCC gradient value of the local image; In response to the NCC gradient value of the local image being greater than a preset gradient value, step S continues to be executed until each NCC gradient value is within a preset range, and the deformation field obtained from the last update is determined as the local displacement field Δ. ( P ); In response to the NCC gradient value of the local image being less than or equal to a preset gradient value, the updated deformation field is... update ( P ) is determined to be a local displacement field Δ ( P ); The initial radiometric transformation matrix 0( P The local displacement field Δ is superimposed. ( P The registration results of global rigid alignment and local transformation of the fused target multimodal medical images are obtained.

2. The registration method according to claim 1, wherein rigid coarse registration is performed on the respiratory phase-aligned target multimodal medical image to generate an initial radiometric transformation matrix. 0( P ),include: Keypoints and feature descriptors of the target multimodal medical image are extracted using Scale Invariant Feature Transform (SIFT). The feature descriptors are then matched against the keypoints of the target multimodal medical image. Correct matching point pairs are selected using the Random Sample Consensus Algorithm (RANSAC). The optimal rigid transformation matrix is ​​then calculated as the initial radiometric transformation matrix. 0( P ).

3. The registration method according to claim 1 or 2, wherein dynamically updating the grid spacing of the local image using a free-form deformation model includes: The initial spacing of the control point grid where the local image is located is L 0; The displacement of each control point is determined by a three-dimensional cubic B-spline basis function. β 3 ( u Weighted superposition generates a continuous deformation field. ( P The continuous deformation field ( P ) By control point { c i,j,k } and cubic B-spline basis functions β 3 definition, Deformation field ; in: This represents the displacement components of the deformation field in the x, y, and z directions; ； ,express right x The partial derivatives in the y and z directions, and so on, are used to calculate the partial derivatives of the deformation field in the y and z directions; among them, Represents the control point displacement vector. h The grid spacing is represented by (x, y, z), which are the three-dimensional coordinates of point P. Its optimization objective function is: F = α NCC( I 1， I 2)+ β || || 2 ,in, I 1 is an X-ray image. I 2 shows the CT image after deformation; NCC uses this image to measure X-ray images. I 1 and CT images after deformation I The gray-level similarity between 2 is calculated with a normalized cross-correlation weight α = 0.7; || 2 This is a deformation smoothing term used to penalize excessive deformation and prevent local distortion. The weight of the deformation smoothing term is β = 0.

3.

4. The registration method according to claim 3, wherein identifying the local image of the target multimodal medical image based on the NCC gradient value comprises: The portion of the image corresponding to an NCC gradient value greater than an initial gradient threshold is identified as a local region of the target multimodal medical image; wherein the initial gradient threshold is the same as or different from the preset gradient value.

5. The registration method according to claim 3, wherein before performing rigid coarse registration on the target multimodal medical image after respiratory phase alignment, the method further comprises: The acquisition of at least two multimodal medical images is triggered by the R wave of the ECG gating signal; The respiratory phase is aligned using the timestamps of the electrocardiogram (ECG).

6. The registration method according to claim 5, after acquiring at least two multimodal medical images and before respiratory phase alignment, the method further includes: The multimodal medical images are preprocessed; The preprocessing includes, but is not limited to: using image denoising algorithms to suppress noise, using adaptive histogram equalization (CLAHE) to enhance local image contrast, and using high-frequency emphasis filtering to suppress bone shadows or segment and identify lung regions.

7. The multimodal medical image registration method according to claim 1, characterized in that, Performing respiratory phase alignment on the target multimodal medical image includes: Based on the timestamps of the target multimodal medical images, end-expiratory phase alignment is performed to ensure consistency of respiratory phases; The rigid transformation parameters are obtained by extracting features from the target multimodal medical image, and the initial deformation field is calculated based on the rigid transformation parameters. Based on the initial deformation field, spatial alignment of the multimodal medical images is performed.

8. The multimodal medical image registration method according to claim 1, characterized in that, The method further includes: When the time deviation between the target multimodal medical images exceeds a preset range, a Gaussian pyramid is constructed for the target multimodal medical images, and the optical flow field is calculated using an iterative algorithm starting from the top of the Gaussian pyramid. Based on the calculated optical flow field, the pixel values ​​of the previous frame of the target multimodal medical image are mapped to the corresponding positions in the compensation frame to achieve multimodal image alignment.

9. A multimodal medical image registration system, characterized in that, The system includes a rigid coarse registration module, a global image calculation module, a local image calculation module, a local displacement field determination module, and a registration result module. The rigid coarse registration module is used to perform rigid coarse registration on the target multimodal medical image after respiratory phase alignment to generate the initial radiometric transformation matrix of the global image. 0( P ); The global image calculation module is used to calculate the initial radiometric transformation matrix generated by the rigid coarse registration module. 0( P The initial control mesh is generated based on the initial mesh spacing L0 of the target multimodal medical image, and the initial deformation field is obtained. init ( P According to the initial deformation field init ( P Calculate the normalized cross-correlation (NCC) gradient value of the global image; The local image calculation module is used to identify local images of the target multimodal medical image based on the NCC gradient value calculated by the global image calculation module, and dynamically update the grid spacing of the local image using a free deformation model to obtain the updated grid spacing L. init To increase grid density; Based on the initial deformation field init ( P ) and the updated grid spacing L init A local control mesh is generated to obtain the adjusted deformation field. update ( P According to the adjusted deformation field update ( P Calculate the NCC gradient value of the local image; The local displacement field determination module is used to respond to the local image's NCC gradient value being greater than a preset gradient value by continuing to call the local image calculation module to adjust the deformation field. update ( P ), and obtain the adjusted deformation field. update ( P Calculate the NCC gradient values ​​of the local image until each NCC gradient value is within a preset range, thus obtaining the local displacement field Δ. ( P ); In response to the NCC gradient value of the local image being less than or equal to a preset gradient value, based on the updated deformation field update ( P Determine the local displacement field Δ ( P ); The registration result module is used to process the initial radiometric transformation matrix obtained by the rigid coarse registration module. 0( P The local displacement field Δ obtained by superimposing the local displacement field determination module ( P The registration results of global rigid alignment and local transformation of the fused target multimodal medical images are obtained.

10. An electronic device, the electronic device comprising: Processor, communication interface, memory, and communication bus; The processor, the communication interface, and the memory communicate with each other through the communication bus; characterized in that the processor calls logical instructions in the memory to execute a computer program to implement the multimodal medical image registration method as described in any one of claims 1-8.

Images