A method of image registration with a multi-modal contrast agent
By employing a multi-guide sequence decision engine and image sensing spatiotemporal alignment operations, the problem of unreasonable examination order of multimodal contrast agents is solved, generating a high-precision cross-modal registration parameter matrix to ensure optimal contrast agent enhancement effect, improve the accuracy and robustness of image registration, and provide more accurate imaging evidence.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- PEOPLES HOSPITAL PEKING UNIV
- Filing Date
- 2026-05-22
- Publication Date
- 2026-07-14
AI Technical Summary
In the existing technology, the injection sequence of multimodal contrast agents lacks standardized decision-making basis, resulting in unreasonable contrast agent examination sequence, mutual interference, overlapping or attenuation of enhancement effects, and affecting the accuracy of image analysis. Furthermore, the existing registration method fails to effectively compensate for the timestamp differences of contrast agent enhancement feature points, resulting in insufficient registration accuracy.
A reasonable contrast agent examination sequence plan is generated by a multi-guideline sequence decision engine, and a cross-modal registration parameter matrix is generated by using image sensing-based spatiotemporal alignment operations to compensate for timestamp differences and spatial offsets, ensuring optimal enhancement effect and improving image registration accuracy.
It enables automatic determination of the contrast agent injection sequence based on the examination site and type, reducing mutual interference, improving the accuracy and robustness of image registration, providing high-quality raw data, and providing more comprehensive and accurate imaging evidence for clinical diagnosis.
Smart Images
Figure CN122392834A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing technology, specifically to a method for image registration using multimodal contrast agents. Background Technology
[0002] In clinical multimodal imaging examinations, it is often necessary to use different contrast agents sequentially to image the same site of the same target object to obtain complementary anatomical and functional information. Different contrast agents have different enhancement dynamics and imaging mechanisms. Current techniques typically rely on operators' experience to determine the injection sequence and intervals, lacking standardized decision-making criteria. Different examination sites (such as the abdominal and pelvic regions versus the urinary tract) have significantly different requirements for contrast agent excretion pathways and imaging timing. Furthermore, different examination types (such as CT enhancement versus MRI enhancement) have different imaging principles and contrast agent interaction mechanisms. Operator reliance on individual experience can easily lead to unreasonable contrast agent injection sequences, causing contrast agent interference, overlapping or attenuation of enhancement effects, and affecting the accuracy of subsequent image analysis. In addition, registration between multimodal contrast agent images is a crucial step in fusion analysis. Existing image registration methods typically perform spatial alignment directly based on anatomical structural features or image grayscale information, ignoring the dynamic characteristics of contrast agent enhancement feature points over time. Because the two contrast agent injections are administered at different times, the enhanced feature points in the image sequence are not only spatially offset but also asynchronous in the temporal dimension. Conventional registration methods do not compensate for the timestamp differences of feature points, nor do they utilize the contrast agent enhancement signal intensity information to optimize the calculation of the transformation matrix, resulting in insufficient registration accuracy. This is especially true when there is a long interval between the two examinations or when the patient is moving, further increasing the registration error. This invention aims to solve the above problems by automatically generating a reasonable contrast agent examination sequence scheme through a multi-guide sequence decision engine and achieving accurate multimodal contrast agent image registration using image sensing-based spatiotemporal alignment operations. Summary of the Invention
[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for image registration using multimodal contrast agents.
[0004] The objective of this invention can be achieved through the following technical solutions:
[0005] A method for image registration using multimodal contrast agents is proposed. This method acquires the target subject's multimodal imaging examination request, which includes examination site labels, examination type labels, and a set of patient baseline information, providing foundational data for subsequent personalized examination plan generation. Based on the examination site and examination type labels, a pre-defined multi-guideline sequence decision engine is invoked to generate a multimodal contrast agent examination sequence plan. This plan includes a first contrast agent examination period, a second contrast agent examination period, and the time interval between the two periods. By introducing the multi-guideline sequence decision engine, the order of contrast agent examinations can be automatically determined based on clinical guidelines and site characteristics, effectively avoiding mutual interference between different contrast agents in vivo and ensuring optimal enhancement effects for each modality, thereby improving the accuracy of subsequent registration. The first modality image sequence is acquired during the first contrast agent examination period, and the second modality image sequence is acquired during the second contrast agent examination period. The peak enhancement time is determined based on the contrast agent's time decay curve, and keyframe images are acquired. This accurately captures the period with the most significant contrast agent enhancement effect, reducing image quality differences caused by contrast agent concentration fluctuations and providing high-quality raw data for feature extraction. A first set of contrast agent-enhanced feature points is extracted from a first modality image sequence, and a second set of contrast agent-enhanced feature points is extracted from a second modality image sequence. By calculating the grayscale stability score of candidate feature points across multiple consecutive frames, stable feature points less affected by the dynamic distribution of the contrast agent are selected. The value range is (0,1], and a higher score indicates more stable grayscale changes. This helps eliminate false feature points caused by rapid changes in contrast agent concentration, improving the robustness and representativeness of the feature point set. A spatiotemporal alignment operation based on image sensing is performed on the first and second contrast agent-enhanced feature point sets to generate a cross-modal registration parameter matrix. This operation maps feature points to their respective modal spatiotemporal coordinate systems and compensates for time offsets based on timestamp differences. The compensation process combines frame rate parameters and average sampling intervals to calculate the number of offset frames, ensuring accurate temporal correspondence between the two modal feature points. Then, using... The compensated spatial coordinates are used as the target point set. An iterative nearest-point algorithm is employed, and a rigid transformation matrix is calculated using point-to-point distance weights based on the contrast agent feature point signal intensity. This generates a high-precision cross-modal registration parameter matrix, compensating for spatial offsets caused by differences in contrast agent injection time and physiological motion. Based on the cross-modal registration parameter matrix, the first modal image sequence and the second modal image sequence are spatially mapped to obtain a multimodal fused registration image. This process generates the transformed image by applying the registration parameter matrix to each pixel coordinate of the second reference frame image, and adaptive fusion weights are used in the overlapping region. and Calculate the fused pixel value The weights are dynamically adjusted based on the gray-level variance within a local window, which can preserve the texture clarity and contrast agent enhancement information in both modal images, avoiding over-smoothing or information loss. After generating a fused image sequence, the image with the highest fusion quality score is selected as the multimodal fusion registration image. This score is based on the product of the overlapping area of the contrast agent enhancement region and the texture clarity of the image, ensuring that the final result achieves the optimal balance between structural alignment and the integrity of enhancement information, thereby providing more comprehensive and accurate imaging evidence for clinical diagnosis.
[0006] The beneficial effects of this invention are:
[0007] Based on the examination site labels and examination type labels, a preset multi-guideline sequence decision engine is invoked to generate a multimodal contrast agent examination sequence plan, including the first contrast agent examination time period, the second contrast agent examination time period, and the time interval between the two. This engine determines whether the examination site belongs to the abdominal / pelvic region label set or the urinary system label set through a first decision layer. If it does, it forces the output of a sequence identifier indicating that the first contrast agent examination should be performed before the second. The second decision layer matches the corresponding guideline rule base based on the examination type label, extracting the contrast agent examination sequence rules. The output sequence identifiers from the two decision layers are then logically ANDed to generate the final sequence identifier, and the time interval is read from the guideline rule base. This hierarchical decision-making and logical fusion approach can automatically output a reasonable contrast agent injection sequence and waiting interval for different combinations of examination sites and examination types, avoiding the subjectivity and uncertainty caused by operators relying on experience. Different body parts (such as the abdomen, pelvis, and urinary system) have different requirements for the contrast agent excretion time window, and different examination types (such as CT and MRI) also have different contrast agent interference mechanisms. By embedding a multi-guideline rule base, targeted sequential decisions are made to ensure that the enhancement periods of the first and second contrast agents are staggered, reducing mutual interference and making the contrast agent enhancement features in the two subsequently acquired modal image sequences clearer and more distinct, providing high-quality raw image data for subsequent registration. A spatiotemporal alignment operation based on image sensing is performed on the first and second contrast agent enhancement feature point sets to generate a cross-modal registration parameter matrix. The operation first maps two feature point sets to their respective modal spatiotemporal coordinate systems, obtaining a spatiotemporal coordinate set containing spatial coordinates and timestamps. Then, based on the time difference between the first and second timestamps, a time offset compensation is applied to the second spatiotemporal coordinate set, resulting in a time-aligned second spatiotemporal coordinate set. Next, the spatial coordinates in the first spatiotemporal coordinate set are used as the source point set, and the spatial coordinates in the time-aligned second spatiotemporal coordinate set are used as the target point set. A rigid transformation matrix from the source point set to the target point set is calculated, serving as the cross-modal registration parameter matrix. The time offset compensation step comprehensively considers the frame rate parameters, average sampling interval, and relative time difference of the two modal image sequences. After calculating the offset frame number, the timestamps are adjusted to eliminate the time dimension deviation caused by the asynchronous injection times of the two contrast agents. The calculation of the rigid transformation matrix employs an iterative nearest-point algorithm and introduces point-to-point distance weights based on the signal intensity of the contrast agent feature points. This allows enhanced regions with higher signal intensity to contribute a greater constraint weight in the registration process, improving the robustness and accuracy of the transformation matrix. By performing spatiotemporal alignment, not only is the spatial displacement between the two image acquisitions corrected, but the temporal asynchrony is also compensated, so that the registered two modal images achieve precise correspondence in both spatial and temporal dimensions, providing an accurate spatial mapping relationship for subsequent fusion. Attached Figure Description
[0008] The invention will now be further described with reference to the accompanying drawings.
[0009] Figure 1 This is a flowchart illustrating the process of an image registration method combining multimodal contrast agents as described in this invention.
[0010] Figure 2 This is a flowchart of the multi-guide sequential decision engine processing;
[0011] Figure 3 This is a flowchart of image acquisition using multimodal contrast agents. Detailed Implementation
[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0013] See Figure 1 This invention provides a method for image registration using multimodal contrast agents. The method includes: acquiring a multimodal imaging examination request for a target object, the multimodal imaging examination request including an examination site label, an examination type label, and a patient baseline information set; invoking a preset multi-guideline sequence decision engine based on the examination site label and the examination type label to generate a multimodal contrast agent examination sequence plan, the multimodal contrast agent examination sequence plan including a first contrast agent examination period, a second contrast agent examination period, and a time interval between the first and second contrast agent examination periods; in the... A first modal image sequence is acquired during the first contrast agent examination period, and a second modal image sequence is acquired during the second contrast agent examination period. A first contrast agent enhancement feature point set is extracted from the first modal image sequence, and a second contrast agent enhancement feature point set is extracted from the second modal image sequence. A spatiotemporal alignment operation based on image sensing is performed on the first and second contrast agent enhancement feature point sets to generate a cross-modal registration parameter matrix. The first modal image sequence and the second modal image sequence are spatially mapped according to the cross-modal registration parameter matrix to obtain a multimodal fusion registration image.
[0014] Example 1: In specific implementation, refer to Figure 2The examination site label is input into the first decision layer of the multi-guideline sequence decision engine. The first decision layer determines whether the examination site label belongs to either the abdominal / pelvic site label set or the urinary system site label set. The abdominal / pelvic site label set includes abdominal site labels, pelvic site labels, and abdominal / pelvic commissural site labels. The urinary system label set includes kidney site labels, ureter site labels, bladder site labels, and urethra site labels. When the examination site label belongs to either the abdominal / pelvic site label set or the urinary system label set, the first decision layer forcibly outputs a first sequence identifier, which indicates that the first contrast agent examination should be performed before the second contrast agent examination. When the examination site label does not belong to either the abdominal / pelvic site label set or the urinary system label set, the first decision layer does not output the first sequence identifier.
[0015] In specific implementation, the examination type label is input into the second decision layer of the multi-guideline order decision engine. The second decision layer matches the corresponding guideline rule base based on the examination type label. The guideline rule base contains rules governing the order of contrast agent examinations. The guideline rule base stores multiple mapping relationships between examination type labels and contrast agent examination order rules. The second decision layer searches for these mapping relationships based on the examination type label to obtain the corresponding contrast agent examination order rule and generates a second order identifier. The second order identifier indicates the order of the first and second contrast agent examinations as specified in the guideline rule base.
[0016] In specific implementation, a logical AND operation is performed on the first sequence identifier output by the first decision layer and the second sequence identifier output by the second decision layer to generate a final sequence identifier. The logical AND operation is defined as follows: the final sequence identifier is output as the sequence only when both the first sequence identifier and the second sequence identifier indicate the same order; when the first sequence identifier and the second sequence identifier indicate different orders, the final sequence identifier is output as the sequence indicated by the first sequence identifier.
[0017] In specific implementation, the order of the first and second contrast agent examination periods is determined based on the final sequence identifier. When the final sequence identifier indicates that the first contrast agent examination is performed before the second contrast agent examination, the first contrast agent examination period precedes the second contrast agent examination period in time. When the final sequence identifier indicates that the second contrast agent examination is performed before the first contrast agent examination, the second contrast agent examination period precedes the first contrast agent examination period in time. The time interval is read from the guideline rule base, where the time interval is the minimum waiting time between two contrast agent examinations specified in the guideline rule base for the examination type label.
[0018] Example 2: In specific implementation, refer to Figure 3 At the start of the first contrast agent examination period, a first contrast agent is injected into the target body, and the imaging area of the target object is continuously acquired at a first image sensor sampling rate to obtain a first contrast agent time decay curve. The first contrast agent time decay curve records a numerical sequence of contrast agent concentration changes over time within the imaging area, starting from the moment the first contrast agent injection is completed. The sampling frequency of the numerical sequence is equal to the first image sensor sampling rate.
[0019] In specific implementation, the first peak enhancement time of the contrast agent is determined according to the first contrast agent time decay curve. The first peak enhancement time is the time corresponding to when the contrast agent concentration value in the first contrast agent time decay curve reaches its maximum value. A first preset number of keyframe images are acquired with the first peak enhancement time as the center, forming the first modal image sequence. The first preset number is an integer greater than or equal to 2. The keyframe images are half of the first preset number of frame images acquired before and after the first peak enhancement time. When the first preset number is odd, the frame image containing the first peak enhancement time is used as the center frame image. Half of the frame images (the first preset number minus 1) are acquired before the center frame image, and half of the frame images (the first preset number minus 1) are acquired after the center frame image.
[0020] In specific implementation, the second contrast agent examination period begins after the specified time interval. The time interval is the waiting period between the end of the first contrast agent examination period and the start of the second contrast agent examination period, as specified in the multimodal contrast agent examination sequence scheme. At the start of the second contrast agent examination period, the second contrast agent is injected into the target body, and the imaging area of the target object is continuously acquired at the second image sensor sampling rate to obtain the second contrast agent time decay curve. The second contrast agent time decay curve records a numerical sequence of contrast agent concentration changes over time within the imaging area, starting from the moment the second contrast agent injection is completed. The sampling frequency of this numerical sequence is equal to the second image sensor sampling rate.
[0021] In specific implementation, the peak enhancement time of the second contrast agent is determined according to the second contrast agent time decay curve. The peak enhancement time of the second contrast agent is the time corresponding to when the contrast agent concentration reaches its maximum value in the second contrast agent time decay curve. A second preset number of keyframe images are acquired centered on the peak enhancement time of the second contrast agent to form the second modal image sequence. The second preset number is an integer greater than or equal to 2. The keyframe images are half of the second preset number of frame images acquired before and after the peak enhancement time of the second contrast agent. When the second preset number is odd, the frame image containing the peak enhancement time of the second contrast agent is used as the center frame image. Half of the frame images (the second preset number minus 1) are acquired before the center frame image, and half of the frame images (the second preset number minus 1) are acquired after the center frame image.
[0022] Example 3: In a specific implementation, image gradient calculation is performed on each frame of the first modality image sequence to generate a first gradient magnitude map. The image gradient calculation uses the Sobel operator, convolving each frame with a horizontal and a vertical template respectively to obtain horizontal and vertical gradient components. The square root of the sum of the squares of the horizontal and vertical gradient components is then taken to obtain the gradient magnitude of each pixel in the first gradient magnitude map. Local maxima are located on the first gradient magnitude map and used as first candidate feature points. A local maximum is defined as the pixel whose gradient magnitude is greater than the gradient magnitude of all other pixels within a 3×3 pixel neighborhood search window in the first gradient magnitude map.
[0023] In specific implementation, a grayscale change stability score for each first candidate feature point across multiple consecutive frames in the first modal image sequence is calculated. For the i-th first candidate feature point in the first modal image sequence, its grayscale value sequence in K consecutive frames is obtained. Where K is the total number of consecutive frames, and K is set to 5. The reason for this setting is that the duration covered by 5 frames is sufficient to reflect the dynamic enhancement process of the contrast agent in the tissue, while avoiding an increase in the amount of computation due to too many frames. Let be the grayscale value of the i-th first candidate feature point in the t-th frame of the image, where t ranges from 1 to K and t is the frame number. Calculate the stability score of the first candidate feature point based on the grayscale value sequence. :
[0024]
[0025] in: The time interval between two adjacent frames, the The value is equal to the frame interval duration during the acquisition of the first modality image sequence, in seconds. The stability score is the absolute value of the grayscale change rate between two adjacent frames. The value range is (0,1]. When the sum of the grayscale change rates of adjacent frames is 0, The value is 1; when the sum of the grayscale change rates of adjacent frames approaches infinity, Approaching 0. A higher score indicates more stable grayscale changes.
[0026] In practical implementation, a first stability threshold is set. The The value is 0.85. When the stability score... Greater than or equal to the first stability threshold When the first candidate feature point is determined to be the first contrast agent enhanced feature point, all the first contrast agent enhanced feature points constitute the first contrast agent enhanced feature point set. The same image gradient calculation and local maximum point localization operations are performed on each frame of the second modality image sequence to obtain the second candidate feature point. The same image gradient calculation uses the same Sobel operator parameter settings as the first modality image sequence, and the local maximum point localization uses the same 3×3 pixel neighborhood search window.
[0027] In specific implementation, a stability score for grayscale changes of each second candidate feature point across multiple consecutive frames in the second modality image sequence is calculated. The calculation method for the stability score is the same as that for the first candidate feature point, with the total number of consecutive frames set to the same value as K (5), and the time interval between two adjacent frames set to the frame interval duration during the acquisition of the second modality image sequence. A second stability threshold is set. The The value is 0.80. Select a stability score exceeding the second stability threshold. The second candidate feature points are used as the second contrast agent enhancement feature points, and all the second contrast agent enhancement feature points constitute the second contrast agent enhancement feature point set.
[0028] Example 4: In a specific implementation, each feature point in the first contrast agent-enhanced feature point set is mapped to the first modal spatiotemporal coordinate system to obtain a first spatiotemporal coordinate set. Each first spatiotemporal coordinate includes a first spatial coordinate and a first timestamp. The first spatial coordinate is the pixel coordinate of the feature point within the corresponding frame image in the first modal image sequence, and the first timestamp is the time value of the frame image corresponding to the feature point on the acquisition time axis.
[0029] In a specific implementation, each feature point in the second contrast agent-enhanced feature point set is mapped to the second modal spatiotemporal coordinate system to obtain a second spatiotemporal coordinate set. Each second spatiotemporal coordinate includes a second spatial coordinate and a second timestamp. The second spatial coordinate is the pixel coordinate of the feature point within the corresponding frame image in the second modal image sequence, and the second timestamp is the time value of the frame image corresponding to the feature point on the acquisition time axis.
[0030] In specific implementation, based on the time difference between the first and second timestamps, time offset compensation is performed on the second spatiotemporal coordinate set to obtain a time-aligned second spatiotemporal coordinate set. The time offset compensation steps are as follows: Obtain the first frame rate parameter of the first modal image sequence, where the first frame rate parameter is the number of frames per second during the acquisition of the first modal image sequence; obtain the second frame rate parameter of the second modal image sequence, where the second frame rate parameter is the number of frames per second during the acquisition of the second modal image sequence. Calculate the first average sampling interval based on the first timestamp, where the first average sampling interval is equal to the reciprocal of the first frame rate parameter, in seconds; calculate the second average sampling interval based on the second timestamp, where the second average sampling interval is equal to the reciprocal of the second frame rate parameter, in seconds. Using the smallest timestamp among the first timestamps as the reference zero point, convert the first timestamp of each first spatiotemporal coordinate into a first relative time based on the reference zero point, where the first relative time is equal to the first timestamp minus the reference zero point. Convert the second timestamp of each second spatiotemporal coordinate into a second relative time based on the reference zero point, where the second relative time is equal to the second timestamp minus the reference zero point. For each second spatiotemporal coordinate, based on the difference between the second relative time and the first relative time, and in conjunction with the second average sampling interval, the offset frame number on the time axis of the second spatiotemporal coordinate is calculated. The offset frame number is equal to the difference between the second relative time and the first relative time divided by the second average sampling interval, and the offset frame number is rounded to the nearest integer. The second timestamp of the second spatiotemporal coordinate is adjusted according to the offset frame number to generate the time-aligned second spatiotemporal coordinate set. The adjustment method is as follows: subtract the product of the offset frame number and the second average sampling interval from the second timestamp to obtain the adjusted timestamp, and use the adjusted timestamp as the second timestamp of the corresponding feature point in the time-aligned second spatiotemporal coordinate set.
[0031] In specific implementation, the first spatial coordinates in the first spatiotemporal coordinate set are used as the source point set, and the second spatial coordinates in the time-aligned second spatiotemporal coordinate set are used as the target point set. A rigid transformation matrix from the source point set to the target point set is calculated. The rigid transformation matrix is calculated using an iterative nearest-point algorithm, and a point-to-point distance weight based on the contrast agent feature point signal intensity is introduced. The initial transformation matrix of the iterative nearest-point algorithm is set as an identity matrix. In each iteration, for each first spatial coordinate in the source point set, the target point with the closest Euclidean distance is searched in the target point set as the corresponding point, forming a pair of corresponding points. Based on the first contrast agent-enhanced feature point signal intensity of the feature point corresponding to the source point in each pair of corresponding points, and the second contrast agent-enhanced feature point signal intensity of the feature point corresponding to the target point, the point-to-point distance weight is calculated. The point-to-point distance weight is equal to the product of the first contrast agent-enhanced feature point signal intensity and the second contrast agent-enhanced feature point signal intensity. The signal intensity is the normalized value of the grayscale value of the feature point in the corresponding frame image, with a normalization range of 0 to 1. The point-pair distance weights are multiplied by the Euclidean distance between the corresponding point pairs to construct a weighted objective function. The updated rigid transformation matrix is solved by minimizing this weighted objective function. This iteration is repeated until the change in the rigid transformation matrix is less than a preset threshold, which is set to 0.001. The rigid transformation matrix obtained after convergence is used as the cross-modal registration parameter matrix.
[0032] Example 5: In a specific implementation, a first reference frame image is selected from the first modal image sequence. The specific steps are as follows: Calculate the area of the first contrast agent enhancement region for each frame image in the first modal image sequence. The area of the first contrast agent enhancement region is defined as the total number of pixels in a connected region formed by all pixels whose grayscale values exceed a first grayscale threshold in the frame image. The first grayscale threshold is set to the mean of all pixel grayscale values in the frame image plus two standard deviations. Select the frame image with the largest area of the first contrast agent enhancement region as the first reference frame image. A second reference frame image is selected from the second modal image sequence. The specific steps are as follows: Calculate the area of the second contrast agent enhancement region for each frame image in the second modal image sequence. The area of the second contrast agent enhancement region is defined as the total number of pixels in a connected region formed by all pixels whose grayscale values exceed a second grayscale threshold in the frame image. The second grayscale threshold is set to the mean of all pixel grayscale values in the frame image plus two standard deviations. Select the frame image with the largest area of the second contrast agent enhancement region as a candidate second reference frame image. Obtain the first acquisition timestamp of the first reference frame image and the second acquisition timestamp of the candidate second reference frame image. Calculate the acquisition time difference between the first acquisition timestamp and the second acquisition timestamp, where the acquisition time difference is the absolute value of the difference between the first acquisition timestamp and the second acquisition timestamp. When the acquisition time difference is less than a preset time difference threshold, the candidate second reference frame image is directly used as the second reference frame image. When the acquisition time difference is greater than or equal to the time difference threshold, the frame image whose acquisition timestamp is closest to the first acquisition timestamp is reselected from the second modal image sequence as the second reference frame image. The closest frame image is defined as the frame image corresponding to the minimum absolute value of the difference between the acquisition timestamp and the first acquisition timestamp among all frame images in the second modal image sequence.
[0033] Impact on final result quality: The contrast agent enhancement process in vivo has dynamic time-dependent characteristics (e.g., concentration changes before and after peak enhancement can reach 30%-50% / minute). When the acquisition time difference between the first reference frame and the second candidate reference frame is too large, the contrast agent enhancement phase of the two images may be significantly different (e.g., the first frame is in the peak enhancement phase, and the second frame is in the decay phase), leading to the following problems:
[0034] Increased feature point matching error: The contrast agent signal intensity of the same anatomical structure in two frames differs significantly. The extracted stable feature point set may be misjudged as a false feature point due to the excessive gray-scale change rate, reducing the robustness of the feature point set. Reduced overlap of fusion region: The area and position of the contrast agent enhancement region change with the enhancement stage (e.g., the difference in the enhancement region range between the renal cortex stage and the medullary stage can be more than 20%). Excessive time difference will lead to a reduction in the overlap area of the enhancement regions in the two frames, directly reducing the "overlap area" index in the fusion quality score.
[0035] Texture information misalignment: Tissue textures (such as the sharpness of tumor enhancement edges) differ at different enhancement stages. Excessive time differences can lead to blurred or broken textures after fusion, reducing the "image texture clarity" index. Therefore, when the acquisition time difference is greater than or equal to a preset threshold, the frame with the closest acquisition timestamp to the second modality image sequence is selected as the second reference frame. This effectively reduces the enhancement stage difference between the two frames, ensuring that feature point matching accuracy, enhancement region overlap, and texture clarity are all within the optimal range, thereby improving the quality score of the final fused image. Experimental data shows that adopting this trade-off strategy improves the fusion quality score by an average of 18.7%.
[0036] In a specific implementation, the cross-modal registration parameter matrix is applied to each pixel coordinate of the second reference frame image to generate a transformed second reference frame image. The pixel coordinates of the transformed second reference frame image are in the same spatial coordinate system as the pixel coordinates of the first reference frame image. The cross-modal registration parameter matrix is a 3x3 rigid transformation matrix. For the second reference frame image with coordinates of... The coordinates of the pixels are calculated using the rigid transformation matrix. The coordinates in the second reference frame image are The pixel grayscale value is assigned to the coordinates in the transformed second reference frame image. The pixels are assigned values using bilinear interpolation to process non-integer coordinate positions.
[0037] In a specific implementation, the overlapping region between the first reference frame image and the transformed second reference frame image is calculated. The overlapping region is defined as the set of pixel positions jointly covered by the first reference frame image and the transformed second reference frame image in the pixel coordinate system. A weighted fusion operation is performed on the pixel values of the first reference frame image and the transformed second reference frame image within the overlapping region to generate a single-frame fused image. Let any pixel position in the overlapping region be... The pixel value of the first reference frame image at the pixel location is The pixel value at the pixel location of the transformed second reference frame image is... Calculate the fused pixel value at the pixel location. :
[0038]
[0039] in: and For adaptive fusion weights, satisfy The adaptive fusion weights The calculation method is as follows:
[0040]
[0041] in: In the first reference frame image The variance of pixel grayscale within a local window centered on the local area is set to a 7x7 pixel area. The window side length of 7 is set based on the fact that 7 is a common small window size, which can balance local texture details and computational efficiency. The variance of pixel grayscale values within the same local window in the transformed second reference frame image is defined as the adaptive fusion weight. equal .
[0042] In specific implementation, the cross-modal registration parameter matrix is sequentially applied to each remaining frame in the second modal image sequence, and fused with the corresponding frame image in the first modal image sequence to generate a fused image sequence. The corresponding frame image is defined as the frame image with the smallest absolute difference between the acquisition timestamp in the first modal image sequence and the acquisition timestamp of the corresponding frame in the second modal image sequence. The fusion operation is the same as the fusion operation of the single-frame fused image, using the same adaptive fusion weight calculation method. The fused image with the highest fusion quality score is selected from the fused image sequence as the multimodal fused registration image. The fusion quality score is calculated based on the product of the overlapping area of the contrast agent enhancement region of the two modal images in the fused image and the image texture sharpness. The overlapping area of the contrast agent enhancement region is defined as the total number of pixels in the intersection region between the first contrast agent enhancement region of the first reference frame image and the second contrast agent enhancement region of the transformed second reference frame image in the fused image. The determination thresholds for the first contrast agent enhancement region and the second contrast agent enhancement region are the same as the thresholds used when selecting the reference frame. The image texture sharpness is defined as the sum of the squares of the Laplacian operator response values of all pixels in the fused image, using a 3x3 template. The fusion quality score is equal to the overlapping area of the contrast agent-enhanced regions multiplied by the image texture sharpness.
[0043] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.
Claims
1. A method for image registration using a multimodal contrast agent, characterized in that, The method includes: Obtain the multimodal imaging examination request of the target object, wherein the multimodal imaging examination request includes examination site labels, examination type labels and patient baseline information set; Based on the examination site label and the examination type label, a preset multi-guide sequence decision engine is invoked to generate a multimodal contrast agent examination sequence plan. The multimodal contrast agent examination sequence plan includes a first contrast agent examination period, a second contrast agent examination period, and a time interval between the first contrast agent examination period and the second contrast agent examination period. A first modality image sequence is acquired during the first contrast agent examination period, and a second modality image sequence is acquired during the second contrast agent examination period; Extract a first set of contrast agent-enhanced feature points from the first modality image sequence, and extract a second set of contrast agent-enhanced feature points from the second modality image sequence; Perform a spatiotemporal alignment operation based on image sensing on the first contrast agent enhanced feature point set and the second contrast agent enhanced feature point set to generate a cross-modal registration parameter matrix; The first modal image sequence and the second modal image sequence are spatially mapped according to the cross-modal registration parameter matrix to obtain a multimodal fusion registration image.
2. The method for image registration using a multimodal contrast agent according to claim 1, characterized in that, The steps of generating a multimodal contrast agent examination sequence plan by invoking a preset multi-guideline sequence decision engine based on the examination site label and the examination type label specifically include: The examination site label is input into the first decision layer of the multi-guide sequence decision engine, and the first decision layer determines whether the examination site label belongs to the abdominal and pelvic site label set or the urinary system site label set. When the examination site label belongs to the abdominal and pelvic site label set or the urinary system site label set, the first decision layer forcibly outputs a first sequence identifier, which is used to indicate that the first contrast agent examination is performed first and the second contrast agent examination is performed later. The inspection type label is input into the second decision layer of the multi-guideline sequence decision engine. The second decision layer matches the corresponding guideline rule base according to the inspection type label. The guideline rule base contains rules for the order of contrast agent inspections. Perform a logical AND operation between the first sequence identifier output by the first decision layer and the second sequence identifier output by the second decision layer to generate the final sequence identifier; The order of the first and second contrast agent examination periods is determined based on the final sequence identifier, and the time interval is read from the guide rule base.
3. The method for image registration using a multimodal contrast agent according to claim 1, characterized in that, The steps of acquiring a first modality image sequence during the first contrast agent examination period and acquiring a second modality image sequence during the second contrast agent examination period specifically include: At the beginning of the first contrast agent examination period, the first contrast agent is injected into the target body, and the imaging area of the target body is continuously acquired at the first image sensor sampling rate to obtain the time decay curve of the first contrast agent. The peak enhancement time of the first contrast agent is determined based on the first contrast agent time decay curve. A first preset number of key frame images are collected with the first peak enhancement time of the first contrast agent as the center to form the first modal image sequence. After waiting for the aforementioned time interval, the second contrast agent examination period will begin; At the beginning of the second contrast agent examination period, the second contrast agent is injected into the target body, and the imaging area of the target object is continuously acquired at the second image sensor sampling rate to obtain the time decay curve of the second contrast agent. The peak enhancement time of the second contrast agent is determined based on the time decay curve of the second contrast agent. A second preset number of key frame images are collected with the peak enhancement time of the second contrast agent as the center to form the second modal image sequence.
4. The method for image registration using a multimodal contrast agent according to claim 1, characterized in that, The steps of extracting a first contrast agent enhancement feature point set from the first modality image sequence and extracting a second contrast agent enhancement feature point set from the second modality image sequence specifically include: Perform image gradient calculation on each frame of the first modality image sequence to generate a first gradient magnitude map; Locate local maxima on the first gradient magnitude map and use the local maxima as the first candidate feature points; Calculate the grayscale change stability score of each first candidate feature point in the first modal image sequence over multiple consecutive frames, select the first candidate feature point whose stability score exceeds the first stability threshold as the first contrast agent enhancement feature point, and all the first contrast agent enhancement feature points form the first contrast agent enhancement feature point set; Perform the same image gradient calculation and local maximum point localization operation on each frame of the second modality image sequence to obtain the second candidate feature points; Calculate the grayscale change stability score of each second candidate feature point in the second modal image sequence over multiple consecutive frames, select the second candidate feature points whose stability scores exceed the second stability threshold as the second contrast agent enhancement feature points, and form the second contrast agent enhancement feature point set by all the second contrast agent enhancement feature points.
5. The method for image registration using a multimodal contrast agent according to claim 1, characterized in that, The step of performing a spatiotemporal alignment operation based on image sensing on the first contrast agent-enhanced feature point set and the second contrast agent-enhanced feature point set to generate a cross-modal registration parameter matrix specifically includes: Each feature point in the first contrast agent-enhanced feature point set is mapped to the first modal spatiotemporal coordinate system to obtain a first spatiotemporal coordinate set, where each first spatiotemporal coordinate contains a first spatial coordinate and a first timestamp; Each feature point in the second contrast agent-enhanced feature point set is mapped to the second modal spatiotemporal coordinate system to obtain a second spatiotemporal coordinate set, where each second spatiotemporal coordinate contains a second spatial coordinate and a second timestamp. Based on the time difference between the first timestamp and the second timestamp, time offset compensation is performed on the second spatiotemporal coordinate set to obtain the time-aligned second spatiotemporal coordinate set; Using the first spatial coordinates in the first spatiotemporal coordinate set as the source point set and the second spatial coordinates in the time-aligned second spatiotemporal coordinate set as the target point set, calculate the rigid transformation matrix from the source point set to the target point set. The rigid transformation matrix is used as the cross-modal registration parameter matrix.
6. The method for image registration using a multimodal contrast agent according to claim 5, characterized in that, The rigid transformation matrix is calculated using an iterative nearest-point algorithm, and a point-to-point distance weight based on the signal intensity of the contrast agent feature points is introduced.
7. The method for image registration using a multimodal contrast agent according to claim 5, characterized in that, The steps of performing time offset compensation on the second spatiotemporal coordinate set based on the time difference between the first and second timestamps to obtain the time-aligned second spatiotemporal coordinate set specifically include: Obtain the first frame rate parameter of the first modal image sequence and the second frame rate parameter of the second modal image sequence; Calculate the first average sampling interval based on the first timestamp, and calculate the second average sampling interval based on the second timestamp; Using the smallest timestamp in the first timestamp as the reference zero point, the first timestamp of each first spatiotemporal coordinate is converted into a first relative time based on the reference zero point; Each second timestamp of the second spatiotemporal coordinate is converted into a second relative time based on the reference zero point; For each second spatiotemporal coordinate, the offset frame number of the second spatiotemporal coordinate on the time axis is calculated based on the difference between the second relative time and the first relative time, combined with the second average sampling interval. The second timestamp of the second spatiotemporal coordinate is adjusted according to the offset frame number to generate the time-aligned set of second spatiotemporal coordinates.
8. The method for image registration using a multimodal contrast agent according to claim 1, characterized in that, The step of spatially mapping the first modal image sequence and the second modal image sequence according to the cross-modal registration parameter matrix to obtain a multimodal fused registration image specifically includes: A first reference frame image is selected from the first modal image sequence, and a second reference frame image is selected from the second modal image sequence; The cross-modal registration parameter matrix is applied to each pixel coordinate of the second reference frame image to generate a transformed second reference frame image. The pixel coordinates of the transformed second reference frame image are in the same spatial coordinate system as the pixel coordinates of the first reference frame image. Calculate the overlapping region between the first reference frame image and the transformed second reference frame image, and perform a weighted fusion operation on the pixel values of the first reference frame image and the pixel values of the transformed second reference frame image within the overlapping region to generate a single-frame fused image; The cross-modal registration parameter matrix is applied sequentially to each remaining frame in the second modal image sequence, and then fused with the corresponding frame images in the first modal image sequence to generate a fused image sequence. The fused image with the highest fusion quality score is selected from the fused image sequence as the multimodal fusion registration image.
9. The method for image registration using a multimodal contrast agent according to claim 8, characterized in that, The fusion quality score is calculated based on the product of the overlapping area of the contrast agent-enhanced regions of the two modal images in the fused image and the image texture sharpness.
10. The method for image registration using a multimodal contrast agent according to claim 8, characterized in that, The steps of selecting a first reference frame image from the first modal image sequence and selecting a second reference frame image from the second modal image sequence specifically include: Calculate the area of the first contrast agent enhancement region in each frame of the first modal image sequence, and select the frame image with the largest area of the first contrast agent enhancement region as the first reference frame image; Calculate the area of the second contrast agent enhancement region in each frame of the second modality image sequence, and select the frame image with the largest area of the second contrast agent enhancement region as the candidate second reference frame image; Obtain the first acquisition timestamp of the first reference frame image, and obtain the second acquisition timestamp of the candidate second reference frame image; Calculate the acquisition time difference between the first acquisition timestamp and the second acquisition timestamp. When the acquisition time difference is less than a preset time difference threshold, the candidate second reference frame image is directly used as the second reference frame image. When the acquisition time difference is greater than or equal to the time difference threshold, the frame image whose acquisition timestamp is closest to the first acquisition timestamp is reselected from the second modal image sequence as the second reference frame image.