A liver image enhancement method and system based on multi-modal fusion

Abstract

The application discloses a liver image enhancement method based on multi-modal fusion and belongs to the field of medical image processing; the method comprises the following steps: extracting time scale features from a first image sequence optimized in terms of dose, aligning different modal time sequence data by adopting a time normalization method to obtain a second image sequence; processing a fourth image sequence subjected to quantization coding by adopting a compression model to retain key features and obtain a fifth image sequence; extracting report keywords from the fifth image sequence subjected to compression transmission, locating an enhancement region by utilizing a semantic guiding mechanism to obtain a sixth image sequence; implementing a region registration algorithm to cooperatively adjust a liver anatomical structure to obtain a seventh image sequence; extracting complementary boundary features by a boundary enhancement network, fusing and sharpening after judging feature consistency to obtain an eighth image sequence subjected to boundary enhancement. The method can reduce a radiation dose, improve multi-modal image time sequence consistency, data integrity and transmission efficiency, enhance key regions and anatomical boundaries and improve accuracy and reliability.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, and in particular relates to a method and system for liver image enhancement based on multimodal fusion. Background Technology

[0002] In the field of medical imaging, the ability of doctors to accurately analyze liver images is crucial for subsequent disease diagnosis and treatment planning, especially in the identification of complex lesions such as tumors or vascular abnormalities, where image quality directly impacts the doctor's judgment. However, single imaging techniques often fail to fully represent the multidimensional characteristics of the liver; therefore, multimodal fusion technology has become a key direction for improving image quality. This technology integrates the advantages of different imaging methods, striving to provide doctors with more comprehensive and clearer visual information.

[0003] Although the concept of multimodal fusion has been widely discussed, existing methods still face significant limitations in practical applications. Many approaches, when integrating different imaging data, lack a comprehensive consideration of equipment characteristics, patient condition, and diagnostic needs during the imaging process. This results in insufficient detail in certain key areas of the fused image, and may even lead to misdiagnosis due to uneven data quality. This problem cannot be solved by simply adding technologies together; rather, it requires dynamic balancing and optimization of multiple complex factors during the fusion process.

[0004] A deeper technical challenge lies in achieving a coordinated balance between dose control and region-specific enhancement across different imaging modalities during the fusion process. First, different imaging modalities, such as rapid scanning and multi-phase imaging, exhibit significant differences in radiation dose and temporal resolution. Failure to allocate dose appropriately may increase unnecessary health risks to patients while pursuing image clarity. Second, this uneven dose allocation can further affect the visualization of different regions of the liver. For example, at certain key anatomical structures or lesion boundaries, the fused image may appear blurry or distorted due to a lack of targeted enhancement strategies, thus impacting the physician's accurate assessment of the condition.

[0005] Therefore, how to dynamically optimize dose allocation in multimodal fusion, and design differentiated enhancement strategies for different regions of the liver, has become a key issue in improving image quality. Taking a specific scenario as an example, in the diagnosis of liver tumors, doctors need to clearly identify the relationship between the tumor boundary and surrounding blood vessels. However, because imaging equipment may reduce the dose to reduce radiation during rapid scanning, the details of the tumor edge are often not prominent enough. In multi-phase imaging, although the details are richer, the time difference may cause alignment deviation with the rapid scanning image, ultimately affecting the fusion effect and image quality reliability.

[0006] This issue is not only about technical optimization, but also directly related to the accuracy of clinical diagnosis and patient safety. A completely new approach is urgently needed to address these challenges and ensure that high-quality liver imaging support can still be provided under complex conditions. Summary of the Invention

[0007] To address the above deficiencies, this invention provides a liver image enhancement method and system based on multimodal fusion. Through end-to-end collaborative processing including dose optimization, temporal alignment, modal compensation, quantization compression, semantic guidance, region registration, and boundary enhancement, it significantly reduces radiation dose, improves temporal consistency, data integrity, and transmission efficiency of multimodal images, and precisely enhances key liver regions and anatomical boundaries, thereby greatly improving the accuracy and reliability of medical imaging diagnosis. Specifically, the contents of this invention are as follows.

[0008] A liver image enhancement method based on multimodal fusion includes:

[0009] Multimodal liver image data is acquired, and the acquisition parameters of each modality are adjusted through a dose optimization model. The radiation dose threshold is determined according to a preset quality mapping relationship to obtain the first image sequence with dose optimization.

[0010] Temporal scale features are extracted from the first image sequence, and time normalization is used to align time-series data of different modalities to obtain a time-aligned second image sequence.

[0011] Modal quality assessment is performed on the second image sequence. If the assessment score is lower than a preset threshold, a compensation algorithm is activated to fill in the missing modal information, resulting in a compensated third image sequence.

[0012] A non-uniform quantization strategy is applied to the third image sequence through quantization encoding, and quantization parameters are determined based on the difference in modal contribution to obtain a quantized encoded fourth image sequence.

[0013] The fourth image sequence is processed using a compression model to retain key features, resulting in a compressed and transmitted fifth image sequence.

[0014] Report keywords are extracted from the fifth image sequence, and the enhanced region is located using a semantic guidance mechanism to obtain a semantically guided sixth image sequence;

[0015] A region registration algorithm is applied to the sixth image sequence to coordinate the adjustment of the liver anatomical structure, resulting in a seventh image sequence with region registration.

[0016] Complementary boundary features are extracted from the seventh image sequence using a boundary enhancement network. After determining feature consistency, the features are fused and sharpened to obtain the eighth image sequence with boundary enhancement, thus obtaining the final liver image enhancement result.

[0017] Preferably, the process of obtaining the dose-optimized first image sequence includes:

[0018] Acquire multimodal liver image data, which includes raw liver images in multiple modalities;

[0019] The multimodal liver image data is processed by a dose optimization model, and the initial acquisition parameters corresponding to each modality are output.

[0020] The image quality score of the multimodal liver image data under the initial acquisition parameters is calculated according to the preset mapping relationship;

[0021] If the image quality score is lower than the corresponding threshold in the preset quality mapping relationship, the corresponding modal acquisition parameter is marked as a state to be adjusted;

[0022] The dose optimization model is used to iteratively adjust the acquisition parameters marked as being in an adjustment state until the image quality score satisfies the preset mapping relationship;

[0023] The radiation dose threshold is determined based on the adjusted acquisition parameters, and the upper limit of radiation dose for each mode is generated.

[0024] The first image sequence with dose optimization is generated by adjusting the acquisition parameters based on the radiation dose threshold constraint.

[0025] Preferably, the process of obtaining the time-aligned second image sequence includes:

[0026] Extract time-scale features from the first image sequence to determine the timestamp position of each image frame;

[0027] The sampling interval differences in the multimodal time series data are determined based on the timestamp position. If the sampling intervals are inconsistent, a normalization factor is calculated.

[0028] Adjust the time coordinates of the multimodal time series data according to the normalization factor to obtain a unified time grid;

[0029] The first image sequence is aligned with the multimodal time series data using the unified time grid to determine the correspondence.

[0030] Interpolate multimodal time-series data according to the correspondence to obtain time-series points consistent with the frame rate of the first image sequence;

[0031] The time points are processed using a linear interpolation method to obtain a time-aligned second image sequence.

[0032] Preferably, the process of obtaining the compensated third image sequence includes:

[0033] The modal integrity score of each frame is obtained by processing the second image sequence using a pre-trained quality assessment model.

[0034] Determine whether the modal integrity score of each frame is higher than a preset threshold;

[0035] If there are frames below a preset threshold, they are marked as missing modal frames;

[0036] Temporal context features are extracted from adjacent normal frame sequences based on the marked missing modality frame positions;

[0037] An image inpainting network is used to reconstruct the content of missing modal frames using extracted temporal context features;

[0038] The reconstructed frames are used to replace the original missing frames to obtain the compensated third image sequence.

[0039] Preferably, the process of obtaining the quantized encoded fourth image sequence includes:

[0040] A third image sequence after compensation processing is obtained, and the contribution value of each modality to the overall visual information is calculated by analyzing the statistical characteristics of each modal channel in the third image sequence.

[0041] The modal contribution values ​​are sorted in descending order to obtain a contribution ranking list.

[0042] Based on the modal ranking of the contribution ranking list, allocation is performed using a preset quantization step size value;

[0043] If the contribution of the current mode is higher than the preset boundary threshold, fine quantization interval division is used; otherwise, coarse quantization interval division is used to obtain the non-uniform quantization table corresponding to each mode.

[0044] The corresponding channel data in the third image sequence are quantized and mapped according to the non-uniform quantization table corresponding to each mode to obtain the quantized encoded fourth image sequence.

[0045] Preferably, the process of obtaining the compressed transmission of the fifth image sequence includes:

[0046] A pre-trained compressed neural network is used to perform initial feature extraction on the fourth image sequence to obtain an initial compressed feature map.

[0047] The initial compressed feature map is subjected to feature importance assessment analysis to determine the channels containing key medical diagnostic features. Channels with high diagnostic value are retained, while channels with low diagnostic value are discarded, resulting in a screened feature map.

[0048] The selected feature map is subjected to secondary quantization processing, and the data bit width is reduced by using a fixed quantization step size to obtain a quantized compressed feature map;

[0049] Based on the continuous sequence characteristics of the quantized compressed feature map, application sequence context compensation is performed to correct the loss of edge information caused by quantization, and the compensated feature sequence is obtained.

[0050] The compensated feature sequence is compressed using entropy coding, and the data volume is reduced by arithmetic coding method to obtain the bitstream of the fifth image sequence;

[0051] The bitstream of the fifth image sequence is encapsulated according to the transmission protocol, and sequence numbers and keyframe markers are added to obtain the final transmittable data packet.

[0052] Preferably, the process of obtaining the semantically guided sixth image sequence includes:

[0053] A set of key information is determined from the fifth image sequence using a pre-established feature extraction model;

[0054] Based on the set of key information, a semantic guidance mechanism is invoked to perform semantic association analysis on the content in the image sequence and obtain semantic mapping results.

[0055] If the semantic mapping result meets the preset matching conditions, the semantic mapping result is divided into regions to determine the distribution range of the target enhancement region;

[0056] By optimizing the distribution range of the target enhancement region, enhanced region data is obtained;

[0057] Based on the enhanced region data and the semantically guided output content, a corresponding sixth image sequence is generated;

[0058] If the generated sixth image sequence does not meet the preset clarity standard, the sequence content is locally corrected using a pixel-level adjustment tool to obtain the final sixth image sequence, which is then formatted and stored.

[0059] Preferably, the process of obtaining the seventh image sequence with region registration includes:

[0060] For the sixth image sequence, a semantically guided method is used to extract preliminary contour information of the liver anatomical structure, and the target region is separated by segmentation technology to obtain preliminary segmented structural data;

[0061] Based on the pre-segmented structural data, a region registration algorithm is implemented to compare the liver anatomical structure with a preset reference template. If the comparison deviation exceeds a preset threshold, the structural outline is adjusted to determine the intermediate structural data after registration.

[0062] By using the intermediate structural data and combining semantic guidance information, the detailed parts of the liver anatomical structure are collaboratively adjusted to obtain the adjusted fine structural data.

[0063] Based on the fine structural data, a registered seventh image sequence is generated, and the adjusted structure is rendered using image reconstruction technology to obtain reconstructed image data.

[0064] The reconstructed image data is subjected to a consistency check. If the check result shows that the structural boundaries are discontinuous, the boundaries are corrected through smoothing to determine the final seventh image sequence data.

[0065] Preferably, the process of obtaining the boundary-enhanced eighth image sequence includes:

[0066] The seventh image sequence is processed by region registration technology, and the image sequence is spatially aligned using a pre-established registration model to obtain the aligned ninth image sequence.

[0067] From the ninth image sequence, a boundary enhancement network is applied to extract complementary boundary information, and feature separation of the boundary region is performed on each frame of the image to determine the preliminary distribution result of the boundary information.

[0068] Based on the preliminary distribution results of the boundary information, if the continuity of the boundary region is detected to be lower than a preset threshold, the boundary is repaired through local smoothing processing to obtain the repaired tenth image sequence.

[0069] Based on the boundary restoration results of the tenth image sequence, a consistency detection method is used to compare complementary boundary information, determine the degree of matching between information, and obtain consistency evaluation data.

[0070] Using the consistency evaluation data, the boundary information with matching degree meeting the preset threshold is sharpened and fused to generate the eleventh image sequence after fusion.

[0071] From the eleventh image sequence, the fused boundary enhancement effect is obtained, and the details of each frame image are optimized to determine the final eighth image sequence.

[0072] The present invention also provides a liver image enhancement system based on multimodal fusion, comprising:

[0073] The dose optimization module is used to acquire multimodal liver image data, adjust the acquisition parameters of each modality through the dose optimization model, determine the radiation dose threshold according to the preset quality mapping relationship, and obtain the first image sequence with dose optimization.

[0074] The temporal alignment module is used to extract temporal scale features from the first image sequence and align temporal data of different modalities using a time normalization method to obtain a temporally aligned second image sequence.

[0075] The modality compensation module is used to evaluate the modality quality of the second image sequence. If the evaluation score is lower than a preset threshold, the compensation algorithm is activated to fill in the missing modality information and obtain the compensated third image sequence.

[0076] The quantization encoding module is used to apply a non-uniform quantization strategy to the third image sequence through quantization encoding, determine the quantization parameters based on the difference in modal contribution, and obtain the quantized encoded fourth image sequence.

[0077] The compression transmission module is used to process the fourth image sequence using a compression model, retain key features, and obtain a compressed transmission fifth image sequence.

[0078] The semantic guidance module is used to extract reporting keywords from the fifth image sequence, locate the enhancement region using the semantic guidance mechanism, and obtain the semantically guided sixth image sequence;

[0079] The region registration module is used to implement a region registration algorithm on the sixth image sequence and coordinately adjust the liver anatomical structure to obtain a region-registered seventh image sequence.

[0080] The boundary enhancement module is used to extract complementary boundary features from the seventh image sequence through the boundary enhancement network, determine the feature consistency, fuse and sharpen, obtain the eighth image sequence with boundary enhancement, and obtain the final liver image enhancement result.

[0081] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0082] (i) Significantly reduce radiation dose while maintaining image quality; by dynamically adjusting acquisition parameters and setting radiation dose thresholds through a dose optimization model, the radiation dose is reduced by 20%–30% while ensuring the clarity of liver images, especially tumor edges and vascular details, thus significantly reducing the radiation risk to patients and achieving a balance of "low dose + high quality".

[0083] (II) Improve multimodal temporal consistency and registration accuracy, repair missing modalities, and ensure data integrity; employ time normalization to align temporal data of different modalities, resolving issues of asynchronous multimodal scanning times and inconsistent sampling intervals, significantly improving temporal consistency, providing a reliable foundation for subsequent fusion and registration, and avoiding diagnostic misjudgments caused by temporal misalignment. Through modal quality assessment and image inpainting networks, missing modal frames are automatically compensated, filling data gaps and restoring continuous image sequences, improving multimodal data integrity, and preventing information loss due to equipment failure or scanning interruptions.

[0084] (III) Improve data transmission and storage efficiency; by adopting non-uniform quantization + preservation of key diagnostic features + arithmetic coding compression, the data transmission efficiency is improved by more than 15%, and the data volume is significantly reduced without losing the medical diagnostic value, making it suitable for medical network transmission, remote diagnosis and cloud storage scenarios.

[0085] (iv) Precise localization and enhancement of areas with clear details; accurate registration of anatomical structures with continuous and natural boundaries. Based on a semantic guidance mechanism, key liver regions (lesions, blood vessels, and anatomical structures) are automatically identified to achieve targeted enhancement, avoiding noise amplification caused by global enhancement. The contrast and recognizability of key regions are significantly improved. Through region registration and boundary enhancement networks, the liver anatomical structures are synergistically adjusted and the boundaries are sharpened and fused, eliminating the boundary blurring, discontinuities, and artifacts of traditional methods. The organ contours and lesion boundaries are clearer and more closely resemble the real anatomical structures. The final enhanced images are more accurate in lesion localization, boundary recognition, and vascular relationship judgment, providing high-quality imaging support for doctors' diagnosis, surgical planning, and efficacy evaluation, reducing the probability of missed diagnoses and misdiagnoses. Attached Figure Description

[0086] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0087] Figure 2 This is a schematic diagram of the system structure according to an embodiment of the present invention. Detailed Implementation

[0088] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in a different order than that shown here. Unless otherwise specified, the reagents, kits, consumables, and instruments used in the present invention are conventional reagents, kits, consumables, and instruments in the art.

[0089] Example 1: As Figure 1 As shown, this embodiment provides a liver image enhancement method based on multimodal fusion, including:

[0090] Multimodal liver image data is acquired, and the acquisition parameters of each modality are adjusted through a dose optimization model. The radiation dose threshold is determined according to a preset quality mapping relationship to obtain the first image sequence with dose optimization.

[0091] Temporal scale features are extracted from the first image sequence, and time normalization is used to align time-series data of different modalities to obtain a time-aligned second image sequence.

[0092] Modal quality assessment is performed on the second image sequence. If the assessment score is lower than a preset threshold, the compensation algorithm is activated to fill in the missing modal information, and the compensated third image sequence is obtained.

[0093] A non-uniform quantization strategy is applied to the third image sequence by quantization encoding, and the quantization parameters are determined based on the difference in modal contribution to obtain the quantized fourth image sequence.

[0094] The fourth image sequence is processed using a compression model, retaining key features, to obtain the fifth image sequence for compressed transmission;

[0095] Report keywords are extracted from the fifth image sequence, and the enhanced region is located using a semantic guidance mechanism to obtain the sixth image sequence guided by semantics.

[0096] The sixth image sequence was subjected to a region registration algorithm, and the liver anatomical structure was adjusted collaboratively to obtain the seventh image sequence with region registration.

[0097] Complementary boundary features are extracted from the seventh image sequence using a boundary enhancement network. After judging the consistency of the features, they are fused and sharpened to obtain the eighth image sequence with boundary enhancement, thus obtaining the final liver image enhancement result.

[0098] Furthermore, the process of obtaining the dose-optimized first image sequence includes:

[0099] Acquire multimodal liver image data, which includes raw liver images in multiple modalities;

[0100] Multimodal liver image data is processed using a dose optimization model, and the initial acquisition parameters corresponding to each modality are output.

[0101] The image quality score of the multimodal liver image data under the initial acquisition parameters is calculated based on the preset mapping relationship;

[0102] If the image quality score is lower than the corresponding threshold in the preset quality mapping relationship, the corresponding modal acquisition parameter is marked as a state to be adjusted;

[0103] A dose optimization model is used to iteratively adjust the acquisition parameters marked as being to be adjusted until the image quality score meets the preset mapping relationship;

[0104] The radiation dose threshold is determined based on the adjusted acquisition parameters, and the upper limit of radiation dose for each mode is generated.

[0105] The first image sequence with dose optimization is generated by adjusting the acquisition parameters through radiation dose threshold constraints.

[0106] Furthermore, the multi-modal liver image data described in this embodiment refers to the original liver images collected from the same patient under multiple imaging modalities such as CT and MRI. These images reflect tissue characteristics under different physical principles and are helpful for comprehensively evaluating liver lesions.

[0107] In one possible implementation, the system first obtains a liver image dataset including the enhanced CT arterial phase, portal venous phase, and MRI T1 and T2 weighted sequences as the basis for subsequent dose optimization. The dose optimization model processes multi-modal images based on a deep learning network and outputs the initial acquisition parameters corresponding to each modality.

[0108] For example, for the CT modality, after the model analyzes the image noise level and contrast, it outputs an initial tube voltage of 120 kV and a tube current of 200 mAs; for the MRI modality, it outputs parameters such as an initial flip angle of 90 degrees and a repetition time of 500 ms. These initial parameters are directly obtained from the regression prediction of the image features and effectively retain the information density required for subsequent doctor diagnosis. Calculate the image quality score according to a preset mapping relationship, where this mapping relationship is trained from a large amount of clinical annotation data and maps the acquisition parameters to image quality metrics such as signal-to-noise ratio and contrast-to-noise ratio.

[0109] Specifically, in one embodiment, the signal-to-noise ratio calculated under the initial parameters of the CT arterial phase is 28, corresponding to a quality score of 85 points, and the threshold for this parameter combination in the preset mapping relationship is 80 points, so it is determined to be qualified; however, the signal-to-noise ratio in the portal venous phase is only 22, with a score of 68 points, which is lower than the threshold of 75 points, so the acquisition parameters of this modality are marked as being in a state to be adjusted. This step ensures that only the modalities with insufficient quality are optimized specifically, avoiding unnecessary global adjustments.

[0110] It should be noted that if the score is lower than the threshold, the dose optimization model will iteratively adjust the parameters to be marked.

[0111] For example, the model gradually reduces the tube current to 150 mAs and simultaneously fine-tunes the tube voltage to 100 kV until the image quality score in the portal venous phase is increased to 82 points, meeting the preset mapping relationship. This iterative process usually converges within several times, not only improving the image quality but also controlling the adjustment range. After adjustment, determine the radiation dose threshold for each modality according to the radiation dose calculation formula.

[0112] For example, the dose after adjustment in the CT arterial phase is 8 mGy, and in the portal venous phase is 5 mGy, thus generating the corresponding upper limit value. This threshold is used as a hard constraint to avoid excessive sacrifice of dose control in subsequent optimization.

[0113] Preferably, the final parameters are constrained by the radiation dose threshold to generate the first image sequence after dose optimization.

[0114] In one embodiment, the system reduces the total CT dose from 15 mGy to 12 mGy while maintaining the clarity of liver tumor margins and the integrity of vascular visualization, all while meeting quality thresholds. This sequence can be directly used for clinical diagnosis or further multimodal registration and fusion.

[0115] For example, from a patient safety perspective, the radiation dose is reduced by 20%-30%, significantly reducing potential risks; from a scanning efficiency perspective, the scan time is shortened by about 15% after parameter adjustments, improving equipment utilization. These effects support each other, jointly achieving a balance between image quality and radiation protection, which is particularly suitable for liver cancer patients requiring multiple follow-ups.

[0116] Furthermore, the process of obtaining the temporally aligned second image sequence includes:

[0117] Extract time-scale features from the first image sequence to determine the timestamp position for each image frame;

[0118] The sampling interval differences in multimodal time series data are determined based on the timestamp position. If the sampling intervals are inconsistent, the normalization factor is calculated.

[0119] Adjust the time coordinates of multimodal time series data according to the normalization factor to obtain a unified time grid;

[0120] The correspondence between the first image sequence and the multimodal time series data was determined by aligning them using a unified time grid.

[0121] Interpolate multimodal time-series data based on the correspondence to obtain time-series points consistent with the frame rate of the first image sequence;

[0122] A linear interpolation method is used to process the time-series points to obtain a time-aligned second image sequence.

[0123] Furthermore, this embodiment describes the extraction of temporal scale features from the first image sequence. Temporal scale features refer to the time point information corresponding to each frame of the image sequence, which helps in subsequent alignment of data from different modalities. Assuming a set of CT and MRI image sequences of the liver is acquired, with CT images acquired every 2 seconds and MRI images every 3 seconds, extracting the timestamp positions clarifies the distribution of each frame on the time axis, laying the foundation for subsequent analysis of sampling interval differences.

[0124] For example, when assessing differences in sampling intervals in multimodal time-series data, it was found that the time intervals of CT and MRI images were inconsistent. In such cases, calculating a normalization factor becomes a necessary step. The normalization factor adjusts data from different sampling frequencies to a uniform time scale. Assuming a 1-second reference time unit, the normalization factor for CT images is 0.5, meaning that data from every 2 seconds needs to be interpolated to generate intermediate points. The normalization factor for MRI images is 0.33, requiring more frequent interpolation. This adjustment method ensures the accuracy of subsequent data alignment.

[0125] For example, after obtaining a unified time grid, the first image sequence is aligned with the multimodal time-series data. The unified time grid acts like a common timeline, and all modal data are matched to this axis. Assuming the unified time grid is divided into 1-second units, the time points of CT and MRI image sequences can be mapped onto this grid, clearly defining the correspondence between each image frame and the time-series data. This alignment method provides the foundation for subsequent interpolation processing.

[0126] For example, when interpolating multimodal time-series data based on correspondences, the goal is to ensure consistent frame rates across different modalities. Assuming the first image sequence has a frame rate of 1 frame per second, and some MRI data have temporal gaps due to larger sampling intervals, this embodiment uses interpolation to generate the missing time-point data. This interpolation ensures consistency across all modalities in the temporal dimension, providing complete data support for subsequent analysis.

[0127] For example, when using linear interpolation to process time-series points, this embodiment estimates intermediate values ​​between adjacent time points to generate smooth time-series data. Suppose that in an MRI image sequence, the data values ​​at two time points are 10 and 20, with a time interval of 3 seconds. Through linear interpolation, data values ​​of approximately 13.3 and 16.7 can be estimated at the middle 1 second and 2 seconds, respectively. This method is simple and efficient, effectively filling data gaps and ultimately obtaining a time-aligned second image sequence. This aligned sequence has significant value in subsequent multimodal data fusion.

[0128] For example, time alignment throughout the process ensures that liver image data from different modalities can be compared and analyzed on the same time dimension. This not only improves the coherence of data processing but also provides a reliable basis for subsequent image quality optimization and radiation dose control. Through the above series of operations, the synergy of multimodal data can be significantly improved, laying a solid foundation for the comprehensive analysis of liver images.

[0129] Furthermore, the process of obtaining the compensated third image sequence includes:

[0130] The modal integrity score of each frame is obtained by processing the second image sequence using a pre-trained quality assessment model.

[0131] Determine whether the modal integrity score of each frame is higher than a preset threshold;

[0132] If there are frames below a preset threshold, they are marked as missing modal frames;

[0133] Temporal context features are extracted from adjacent normal frame sequences based on the marked missing modality frame positions;

[0134] An image inpainting network is used to reconstruct the content of missing modal frames using extracted temporal context features;

[0135] The reconstructed frames are used to replace the original missing frames to obtain the compensated third image sequence.

[0136] Furthermore, in this embodiment, after obtaining the second image sequence with completed temporal alignment, modal integrity is evaluated for each frame in the sequence using a pre-trained quality assessment model.

[0137] Specifically, the model is trained based on a deep convolutional network and can output a multimodal channel integrity score for each frame. For example, the score ranges from 0 to 1, where 1 indicates that the modality is complete and without missing features.

[0138] In one possible implementation, for medical multimodal imaging sequences, such as simultaneously acquired ultrasound and optical images, the quality assessment model analyzes each frame to determine if there is any modal loss due to occlusion, noise, or acquisition interruption. Assuming the second image sequence contains 100 frames, the model processes the data to obtain a score for each frame. If a preset threshold is set to 0.85, frames with scores below 0.85 are marked as missing modal frames.

[0139] It should be noted that the purpose of marking missing modal frames is to accurately identify the location of problematic frames and avoid subsequent processing being affected by invalid data.

[0140] For example, frames 32 to 35 in the sequence have optical modal missing due to a brief equipment malfunction, with scores of 0.62, 0.58, 0.71 and 0.69 respectively. These frames will be uniformly marked to facilitate targeted compensation.

[0141] In one embodiment, temporal context features are extracted from adjacent normal frame sequences based on the marked missing modality frame position.

[0142] Specifically, five normal frames before and after the missing frame can be selected as the context window, and the depth feature maps and motion vectors of these frames can be extracted.

[0143] For example, for the missing frame 33, features are extracted from frames 28 to 32 and frames 34 to 38. These features contain information on the texture, edges, and temporal changes of adjacent frames, which can provide a reliable basis for repair.

[0144] For example, the temporal context features extracted in this embodiment are further encoded by a long short-term memory network to capture the dynamic correlation between consecutive frames. Such context features help to preserve the continuity of organ movement.

[0145] Preferably, this embodiment employs an image inpainting network to reconstruct the content of missing modal frames using extracted temporal contextual features. This inpainting network is based on a generative adversarial network structure, taking contextual features and masked missing regions as input to generate realistic modal content.

[0146] For example, when reconstructing the optical modality of frame 33, the network references the vascular texture and color distribution of the normal frames before and after it to generate details consistent with the surrounding frames, and finally outputs the reconstructed frame.

[0147] Understandably, replacing the original missing frames with the reconstructed frames yields the compensated third image sequence. This compensation significantly improves the overall modal integrity of the sequence. For example, if the original sequence had 10% missing frames, the compensated scores of all frames would be above 0.9. This provides doctors with continuous and reliable multimodal data for subsequent diagnostic analysis, effectively reducing the risk of misjudgment due to modal missing frames and maintaining temporal dynamic consistency.

[0148] Furthermore, the process of obtaining the quantized encoded fourth image sequence includes:

[0149] The third image sequence after compensation processing is obtained. By analyzing the statistical characteristics of each modal channel in the third image sequence, the contribution value of each modality to the overall visual information is calculated.

[0150] The modal contribution values ​​are sorted in descending order to obtain a contribution ranking list.

[0151] Based on the modal ranking of the contribution ranking list, allocation is performed using a preset quantization step size value;

[0152] If the contribution of the current mode is higher than the preset boundary threshold, fine quantization interval division is used; otherwise, coarse quantization interval division is used to obtain the non-uniform quantization table corresponding to each mode.

[0153] The corresponding channel data in the third image sequence is quantized and mapped according to the non-uniform quantization table corresponding to each mode to obtain the quantized encoded fourth image sequence.

[0154] Furthermore, in this embodiment, when processing the compensated third image sequence, the contribution of each modal channel to the overall visual information is evaluated by analyzing the statistical characteristics of each modality. A modal channel can be understood as different types of information layers in an image, such as color channels, texture information, or depth information. Statistical characteristic analysis typically focuses on the data distribution of each modality, such as the range or frequency distribution of pixel values ​​for a particular modality. For example, in a certain image sequence, the color modality has a wider range of pixel value variations, indicating a larger amount of information, while the texture modality has a narrower range and relatively smaller information. Through this analysis, the contribution of the color modality can be quantified as 0.6, the texture modality as 0.3, and the depth modality as 0.1. These contribution values ​​reflect the importance of each modality in visual presentation, providing a basis for subsequent processing.

[0155] For example, in this embodiment, when sorting and allocating quantization step sizes based on contribution, smaller quantization step sizes can be used for higher-ranking modalities to retain more detailed information. Assuming the color modality ranks first, its quantization step size can be set to 2, meaning more finely divided intervals during data mapping. Conversely, for lower-ranking depth modalities, the quantization step size can be set to 8, reducing data storage requirements. This non-uniform allocation method prioritizes the integrity of information for important modalities even with limited resources.

[0156] It should be noted that the choice of quantization step size can also be adjusted according to the actual application scenario. For example, in scenarios with higher requirements for color, the step size value can be further reduced.

[0157] For example, in this embodiment, when setting the boundary threshold and dividing the quantization intervals, the contribution threshold is set to 0.4. Modalities above this threshold, such as the color modality, use fine-grained quantization interval division, with the number of intervals potentially set to 16, to ensure minimal information loss. Modalities below the threshold, such as the depth modality, use coarse-grained quantization interval division, with the number of intervals potentially set to 4, to reduce processing complexity. This approach allows for flexible adjustment of the quantization strategy based on the importance of the modality.

[0158] For example, in this embodiment, during the quantization mapping process, the data of each channel of the third image sequence is transformed according to the generated non-uniform quantization table. Assuming the color channel data range is 0 to 255, it is mapped to 16 intervals using a fine quantization table, with each interval representing a coded value. The depth channel is mapped to 4 intervals, reducing the number of coded values. This mapping process effectively compresses the data while preserving as much key information as possible, forming the quantized and coded fourth image sequence. This method has significant advantages in storage and transmission.

[0159] Furthermore, the process of obtaining the compressed transmission of the fifth image sequence includes:

[0160] A pre-trained compressed neural network is used to perform initial feature extraction on the fourth image sequence to obtain an initial compressed feature map.

[0161] The initial compressed feature map is subjected to feature importance assessment analysis to identify channels containing key medical diagnostic features. Channels with high diagnostic value are retained, while channels with low diagnostic value are discarded, resulting in a screened feature map.

[0162] The selected feature map is subjected to secondary quantization processing, and the data bit width is reduced by using a fixed quantization step size to obtain a quantized compressed feature map.

[0163] Based on the continuous sequence characteristics of the quantized compressed feature map, application sequence context compensation is performed to correct the loss of edge information caused by quantization, and the compensated feature sequence is obtained.

[0164] Entropy coding compression is applied to the compensated feature sequence, and arithmetic coding method is used to reduce the data volume to obtain the bitstream of the fifth image sequence;

[0165] The bitstream of the fifth image sequence is encapsulated according to the transmission protocol, and sequence numbers and keyframe markers are added to obtain the final transmittable data packet.

[0166] Furthermore, this embodiment employs a pre-trained compressed neural network to perform initial feature extraction on the fourth image sequence. Understandably, this process utilizes a pre-trained convolutional neural network model to directly extract high-level semantic features from the quantized image sequence, forming an initial compressed feature map. These feature maps typically retain the core structural information of medical images, such as organ contours and texture details.

[0167] Specifically, pre-trained compressed neural networks can be based on efficient MobileNet or ResNet variants.

[0168] In one embodiment, the input fourth image sequence is processed frame by frame, generating a 256-channel feature map for each frame. This initial extraction helps to significantly reduce redundancy in the original data while maintaining the integrity of key medical information, resulting in reduced computational overhead and improved compression efficiency.

[0169] In one possible implementation, the initial compressed feature map is analyzed through a feature importance evaluation module.

[0170] It should be noted that this module typically uses an attention mechanism or gradient weighting method to calculate the contribution score of each channel to the diagnostic task.

[0171] For example, in a sequence of brain CT images, evaluation reveals that the first 50 channels primarily capture edges and density differences; these are identified as high diagnostic value channels and retained. The remaining 200 channels, mostly noise or redundant background, are discarded, forming a filter feature map. This approach further reduces data volume while prioritizing features relevant to the accuracy of the doctor's diagnosis.

[0172] For example, this embodiment performs secondary quantization on the selected feature map, using a fixed quantization step size to reduce the data bit width. It is understood that this embodiment selects 8-bit quantization, mapping floating-point feature values ​​to the range of -128 to 127.

[0173] Specifically, in one embodiment, a quantization mapping with a step size of 0.5 is uniformly applied to the 50 retained high-value channels. This not only simplifies the hardware implementation but also reduces the feature data bit width from 32 bits to 8 bits, significantly reducing storage requirements and transmission bandwidth, while keeping the quantization error within an acceptable range.

[0174] In one possible implementation, a sequence context compensation module is applied to correct the loss of edge information caused by quantization, based on the continuous sequence characteristics of the quantized compressed feature map.

[0175] For example, for feature maps between consecutive frames, optical flow estimation or adjacent frame difference prediction can be used to perform pixel-level compensation adjustments to address organ edge blurring caused by quantization. This compensation helps restore temporal consistency between sequences and improves the visual quality of the decoded image.

[0176] Specifically, this embodiment applies entropy coding compression to the compensated feature sequence and uses arithmetic coding to further reduce the data volume. It should be noted that arithmetic coding dynamically allocates codewords based on the probability distribution of feature values; for example, frequently occurring zero-value features are assigned shorter codeword lengths.

[0177] In one embodiment, applying this method to a set of compensated feature sequences can further compress the data volume by more than 30%, achieving a higher lossless compression rate, which is suitable for low-bandwidth medical network transmission.

[0178] For example, in this embodiment, the bitstream of the fifth image sequence is encapsulated according to the transmission protocol, and sequence numbers and keyframe markers are added to obtain the final transmittable data packet.

[0179] Understandably, this approach uses DICOM-compatible protocols or custom UDP packets, adding frame sequence numbers (such as 001 to 100) to the packet header and marking I-frames as keyframes. This encapsulation ensures that data packets can be reconstructed in an orderly manner and recover from errors during network transmission. The beneficial effects include improving the reliability and real-time performance of remote diagnosis by doctors and supporting the efficient distribution of multimodal medical images.

[0180] Furthermore, the process of obtaining the semantically guided sixth image sequence includes:

[0181] A pre-established feature extraction model is used to determine the set of key information from the fifth image sequence;

[0182] Based on the set of key information, the semantic guidance mechanism is invoked to perform semantic association analysis on the content in the image sequence and obtain semantic mapping results.

[0183] If the semantic mapping result meets the preset matching conditions, the semantic mapping result is divided into regions to determine the distribution range of the target enhancement region;

[0184] By optimizing the boundaries of the distribution range of the target enhancement region, enhanced region data is obtained;

[0185] Based on the enhanced regional data and the semantically guided output content, a corresponding sixth image sequence is generated;

[0186] If the generated sixth image sequence does not meet the preset clarity standard, the sequence content is locally corrected using a pixel-level adjustment tool to obtain the final sixth image sequence, which is then formatted and stored.

[0187] Furthermore, this embodiment employs a pre-established feature extraction model for processing. This model, based on a deep convolutional network, recovers the latent structure from the compressed and transmitted sequence, identifying key information sets such as lesion edges, texture density, and other relevant regions.

[0188] In one embodiment, the feature extraction model scans sequence frames through multi-layer convolutional operations, outputting a high-dimensional feature vector, where the key information set consists of regions with activation values ​​higher than a threshold of 0.75. These regions often correspond to key diagnostic features retained in previous compression, ensuring that subsequent processing focuses on clinically valuable components.

[0189] Specifically, when this embodiment invokes the semantic guidance mechanism, it performs semantic association analysis on the key information set.

[0190] For example, by using pre-trained medical knowledge embeddings, lesion regions can be matched with known anatomical structures or pathological patterns to obtain preliminary semantic mapping results. These results are presented as heatmaps, with values ​​ranging from 0 to 1, representing the strength of semantic relevance.

[0191] It should be noted that if the average relevance score of the semantic mapping results exceeds 0.8, it is considered to meet the preset matching conditions. In this case, the mapping results are divided into regions, and the distribution range of the target enhancement region is identified by the connected component algorithm. For example, consecutive pixel groups with scores higher than 0.85 are classified as the main enhancement regions.

[0192] In one possible implementation, this embodiment performs boundary optimization on the distribution range of the target enhancement region. Morphological dilation and erosion operations are used to extend the boundary by 2-3 pixels to cover potentially blurred edges, while simultaneously removing isolated noise points, thereby obtaining more accurate enhanced region data. This optimization helps recover slight boundary loss caused by previous quantization compression.

[0193] For example, in this embodiment, when generating the sixth image sequence by combining the semantically guided output content, the enhanced region data is superimposed on the original sequence frame, and the contrast of the target region is improved by weighted fusion. The weight is usually set to 1.2 to 1.5 times to ensure clearer visualization of lesions while maintaining a natural background.

[0194] Specifically, if the overall sharpness score of the generated sixth image sequence is lower than the preset standard of 0.9, local corrections will be made using pixel-level adjustment tools.

[0195] For example, applying histogram equalization to low-resolution regions adjusts the pixel distribution only within the target enhancement area, increasing local contrast by 15%-20% to obtain the final image sequence. This correction significantly improves the accuracy of diagnostic observation.

[0196] Preferably, in this embodiment, when the final image sequence is formatted and stored, metadata tags are added to confirm its integrity, such as the sequence frame number and enhanced region coordinates, to ensure its suitability for subsequent doctor's diagnosis or further transmission and processing. This process helps maintain the compatibility and diagnostic consistency of the sequence across multiple devices, improving the efficiency of the overall medical imaging workflow.

[0197] Furthermore, the process of obtaining the seventh image sequence with region registration includes:

[0198] For the sixth image sequence, a semantically guided method was used to extract preliminary contour information of the liver anatomical structure, and the target region was separated by segmentation technology to obtain preliminary segmented structural data;

[0199] Based on the preliminary segmented structural data, a region registration algorithm is implemented to compare the liver anatomical structure with a preset reference template. If the comparison deviation exceeds a preset threshold, the structural outline is adjusted to determine the intermediate structural data after registration.

[0200] By using intermediate structural data and combining semantic guidance information, the detailed parts of the liver's anatomical structure are collaboratively adjusted to obtain the adjusted fine structural data.

[0201] Based on the fine structure data, a registered seventh image sequence is generated, and the adjusted structure is rendered using image reconstruction technology to obtain the reconstructed image data.

[0202] The reconstructed image data is subjected to consistency verification. If the verification result shows that the structural boundaries are discontinuous, the boundaries are corrected through smoothing to determine the final seventh image sequence data.

[0203] Furthermore, in this embodiment, when processing the liver anatomical structure for the sixth image sequence, preliminary contour information is first extracted using a semantic-guided method. This process relies on a pre-trained semantic model to identify the semantic features of the liver region in the image.

[0204] Specifically, the model prioritizes capturing high-confidence pixels at the edge of the liver based on the enhanced semantic mapping region in the sixth image sequence to form an initial contour line.

[0205] In one possible implementation, if the image resolution is 1024×1024 and the pixel-level semantic score threshold is set to 0.75, then only areas above this threshold are included in the preliminary contour, thereby avoiding noise interference and ensuring the reliability of the contour.

[0206] It should be noted that in the following embodiment, segmentation technology is used to separate the target region. A deep learning-based instance segmentation method is used to separate the liver from surrounding tissues such as blood vessels and gallbladder.

[0207] For example, in actual processing, the segmentation model outputs a liver mask. After binarization, pixels within the mask are marked as 1, and the rest as 0. Small isolated regions are removed through morphological operations, such as defining connected components with an area less than 500 pixels as noise and removing them, making the structural data of the initial segmentation cleaner and providing high-quality input for subsequent registration.

[0208] Specifically, in this embodiment, when implementing the region registration algorithm, the initially segmented liver structure is compared with a preset reference template. The reference template is derived from a standard liver anatomical atlas or clear historical images of the same patient.

[0209] In one embodiment, this embodiment employs a method based on affine transformation combined with non-rigid registration. First, global transformation parameters are calculated through feature point matching. If the average registration error exceeds 3 mm, a local deformation field optimization stage is initiated, gradually reducing the deviation to within 1.5 mm, thereby obtaining the registered intermediate structural data. This progressive refinement method helps maintain the overall consistency of the liver's morphology while correcting local deformations.

[0210] For example, in this embodiment, when making detailed collaborative adjustments to the intermediate structural data, semantic guidance information is used to correct fine details such as the boundaries of liver segments and the direction of blood vessels.

[0211] Preferably, this embodiment also introduces a multi-scale attention mechanism, prioritizing the enhancement of structural details near the hepatic vein and portal vein, making the adjusted fine structural data visually closer to the actual anatomy. This adjustment not only improves the accuracy of the structure but also provides a more reliable basis for subsequent reconstruction.

[0212] In one possible implementation, this embodiment employs image reconstruction techniques, such as volumetric rendering or surface rendering methods based on stereomicroscopy principles, to generate the seventh image sequence. This involves performing 3D-to-2D projection rendering on the adjusted fine structure to obtain high-contrast reconstructed image data. This process helps highlight the differences in texture within the liver, improving the intuitiveness of clinical observation.

[0213] It should be noted that the consistency verification phase performs continuity checks on the structural boundaries of the reconstructed image. If boundary interruptions or jagged anomalies are detected, corrections are made through bilateral filtering or curvature-driven smoothing, for example, setting the smoothing iteration count to 5 times to ensure natural boundary transitions and ultimately obtain smooth and continuous image sequence data. This processing significantly reduces visual artifacts and improves the usability of the image quality.

[0214] Furthermore, the process of obtaining the boundary-enhanced eighth image sequence includes:

[0215] The seventh image sequence was processed using region registration technology, and the image sequence was spatially aligned using a pre-established registration model to obtain the aligned ninth image sequence.

[0216] From the ninth image sequence, a boundary enhancement network is applied to extract complementary boundary information, and feature separation of the boundary region is performed on each frame of the image to determine the preliminary distribution results of the boundary information.

[0217] Based on the preliminary distribution results of boundary information, if the continuity of the boundary region is detected to be lower than a preset threshold, the boundary is repaired through local smoothing to obtain the repaired tenth image sequence.

[0218] Based on the boundary restoration results of the tenth image sequence, a consistency detection method is used to compare complementary boundary information, determine the degree of matching between information, and obtain consistency evaluation data.

[0219] Using consistency assessment data, the boundary information that meets the preset threshold for matching degree is sharpened and fused to generate the eleventh image sequence after fusion.

[0220] From the eleventh image sequence, the fused boundary enhancement effect is obtained, and the details of each frame image are optimized to determine the final eighth image sequence.

[0221] Furthermore, this embodiment processes the seventh image sequence using region registration technology, primarily relying on a pre-trained registration model to achieve precise spatial alignment of the liver structure.

[0222] In one embodiment, the model is based on a deep learning network, such as a convolutional neural network, combined with deformation field prediction. It performs a non-rigid transformation on each frame of the seventh sequence, aligning the anatomical location of the liver with a reference template to obtain the aligned ninth image sequence. This alignment process helps compensate for deformations caused by changes in patient breathing or posture, improving the stability of subsequent processing. Complementary boundary information is extracted from the ninth image sequence using a boundary enhancement network.

[0223] Specifically, the network employs an attention mechanism or a multi-scale feature extraction module to perform feature separation for the liver edge region.

[0224] In one possible implementation, a channel attention gating unit is used to enhance the boundary gradient response, separating complementary information under low contrast, such as the blurred boundary between the liver and adjacent organs, to determine the preliminary distribution of boundary information. This extraction method can significantly improve boundary visibility, providing a reliable basis for restoration. Based on the preliminary distribution of boundary information, if the continuity of the detected boundary region is below a preset threshold, such as a continuity score below 0.85, the boundary is restored through local smoothing.

[0225] Understandably, this embodiment employs Gaussian filtering or bilateral filtering to smooth noise in local areas while preserving edge sharpness, obtaining the repaired tenth image sequence. This repair helps eliminate breaks caused by imaging artifacts, ensuring the integrity of the liver contour and thus improving overall image quality. Based on the boundary repair results of the tenth image sequence, a consistency detection method is used to compare complementary boundary information.

[0226] For example, the smoothness and topological consistency of the deformation field can be evaluated using cyclic consistency loss or landmark matching to determine the degree of matching between information and obtain consistency evaluation data.

[0227] Specifically, in liver CT sequences, a Dice coefficient exceeding 0.92 is considered high consistency. This detection effectively identifies potentially inconsistent regions and improves the reliability of registration. Based on the consistency assessment data, sharpening and fusion processing is performed on boundary information with high matching degrees.

[0228] For example, this embodiment preferably employs gradient domain fusion or multi-resolution sharpening techniques to superimpose and enhance highly matched boundaries, generating a fused eleventh image sequence. This fusion process highlights the fine edge details of the liver while suppressing background noise and improving visual clarity. The fused boundary enhancement effect is obtained from the eleventh image sequence, and detail optimization is performed on each frame.

[0229] For example, the liver texture and blood vessel boundaries are further refined through post-processing such as contrast-limited adaptive histogram equalization to determine the final eighth image sequence.

[0230] The optimized solution in this embodiment helps doctors to more accurately identify liver lesion areas in subsequent clinical diagnoses, providing more precise anatomical information to support doctors' surgical planning or treatment assessment.

[0231] Example 2: As Figure 2 As shown, based on the same inventive concept, this embodiment also provides a liver image enhancement system based on multimodal fusion, including:

[0232] The dose optimization module is used to acquire multimodal liver image data, adjust the acquisition parameters of each modality through the dose optimization model, determine the radiation dose threshold according to the preset quality mapping relationship, and obtain the first image sequence with dose optimization.

[0233] The temporal alignment module is used to extract temporal scale features from the first image sequence and align temporal data of different modalities using a time normalization method to obtain a temporally aligned second image sequence.

[0234] The modality compensation module is used to evaluate the modality quality of the second image sequence. If the evaluation score is lower than a preset threshold, the compensation algorithm is activated to fill in the missing modality information and obtain the compensated third image sequence.

[0235] The quantization encoding module is used to apply a non-uniform quantization strategy to the third image sequence through quantization encoding, determine the quantization parameters based on the difference in modal contribution, and obtain the quantized encoded fourth image sequence.

[0236] The compression and transmission module is used to process the fourth image sequence using a compression model, retain key features, and obtain the compressed and transmitted fifth image sequence.

[0237] The semantic guidance module is used to extract report keywords from the fifth image sequence, locate the enhancement region using the semantic guidance mechanism, and obtain the semantically guided sixth image sequence;

[0238] The region registration module is used to implement the region registration algorithm on the sixth image sequence and coordinate the liver anatomical structure to obtain the region-registered seventh image sequence;

[0239] The boundary enhancement module is used to extract complementary boundary features from the seventh image sequence through the boundary enhancement network, and after judging the consistency of features, it fuses and sharpens to obtain the eighth image sequence with boundary enhancement, thus obtaining the final liver image enhancement result.

[0240] The liver image enhancement system based on multimodal fusion provided in this embodiment has all the advantages of the liver image enhancement method based on multimodal fusion provided in Embodiment 1.

[0241] Example 3: This example also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in Example 1.

[0242] Example 4: This example also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in Example 1.

[0243] Example 5: This example also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in Example 1.

[0244] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A liver image enhancement method based on multimodal fusion, characterized in that, include: S1. Acquire multimodal liver image data, adjust the acquisition parameters of each modality through a dose optimization model, determine the radiation dose threshold based on a preset quality mapping relationship, and obtain the first image sequence with dose optimization. S2. Extract time scale features from the first image sequence, and use time normalization method to align time series data of different modalities to obtain a time-aligned second image sequence; S3. Perform modal quality assessment on the second image sequence. If the assessment score is lower than a preset threshold, activate the compensation algorithm to fill in the missing modal information and obtain the compensated third image sequence. S4. Apply a non-uniform quantization strategy to the third image sequence through quantization encoding, determine the quantization parameters based on the difference in modal contribution, and obtain the quantized encoded fourth image sequence. S5. The fourth image sequence is processed using a compression model to retain key medical diagnostic features, resulting in a compressed and transmitted fifth image sequence. S6. Extract report keywords from the fifth image sequence, use semantic guidance mechanism to locate the liver enhancement region, and obtain a semantically guided sixth image sequence; S7. Implement a region registration algorithm on the sixth image sequence to coordinately adjust the liver anatomical structure and obtain a seventh image sequence with region registration. S8. Complementary boundary features are extracted from the seventh image sequence through a boundary enhancement network. After judging the consistency of features, they are fused and sharpened to obtain the eighth image sequence with boundary enhancement, which is used as the final liver image enhancement result.

2. The liver image enhancement method based on multimodal fusion according to claim 1, characterized in that, Step S1 includes: S1.1 Acquire raw liver images containing multiple modalities including CT and MRI; S1.2 Output the initial acquisition parameters for each modality through the dose optimization model; S1.3 Calculate the image quality score based on the preset quality mapping relationship. If the score is lower than the threshold, mark the acquisition parameters as needing adjustment. S1.4 Iteratively optimize the parameters to be adjusted until the image quality meets the preset requirements; S1.

5. Determine the radiation dose threshold based on the optimized parameters and generate the upper limit of radiation dose for each mode; S1.

6. Constrain the acquisition parameters with radiation dose threshold to obtain the first image sequence with dose optimization.

3. The liver image enhancement method based on multimodal fusion according to claim 1, characterized in that, Step S2 includes: S2.1 Extract the timestamps of each frame from the first image sequence to determine the sampling interval differences; if the sampling intervals are inconsistent, calculate the normalization factor. S2.2 Calculate the normalization factor and adjust the time coordinates to obtain a unified time grid; S2.3 Aligning multimodal time series data based on a unified time grid; S2.

4. Linear interpolation is used to fill in the time sequence points to obtain a second image sequence with time alignment.

4. The liver image enhancement method based on multimodal fusion according to claim 1, characterized in that, Step S3 includes: S3.1 Obtain the modal integrity score of each frame of the second image sequence through a pre-trained quality assessment model; S3.2 Frames with a marked score below a preset threshold are considered missing modal frames; S3.3 Extract temporal context features from adjacent normal frames of the missing modality frame; S3.

4. Use an image inpainting network to reconstruct the content of missing modal frames; S3.5 Replace the missing frames to obtain the compensated third image sequence.

5. The liver image enhancement method based on multimodal fusion according to claim 1, characterized in that, Step S4 includes: S4.1 Analyze the statistical characteristics of each modal channel in the third image sequence and calculate the modal contribution. S4.2 Arrange in descending order of contribution, assign fine quantization intervals to high contribution modes and coarse quantization intervals to low contribution modes; S4.

3. Based on the non-uniform quantization table, perform quantization mapping on the data of each channel to obtain the quantized encoded fourth image sequence.

6. The liver image enhancement method based on multimodal fusion according to claim 1, characterized in that, Step S5 includes: S5.

1. Use a pre-trained compressed neural network to extract the initial compressed feature map of the fourth image sequence; S5.

2. Assess the importance of features, retain high diagnostic value channels and discard low diagnostic value channels to obtain the screening feature map; S5.

3. Perform secondary quantization on the selected feature map to reduce the data bit width; S5.

4. Based on the sequence context compensation for edge information loss, and then through arithmetic coding compression, the bitstream of the fifth image sequence is obtained.

7. The liver image enhancement method based on multimodal fusion according to claim 1, characterized in that, Step S6 includes: S6.1 Extract the set of key information from the fifth image sequence and perform semantic association analysis; S6.

2. Based on the semantic mapping results, the liver target enhancement region is divided and the boundary is optimized; S6.3 Generate a semantically guided sixth image sequence; if the clarity requirement is not met, perform pixel-level local correction.

8. The liver image enhancement method based on multimodal fusion according to claim 1, characterized in that, Step S7 includes: S7.

1. Use semantic guidance and segmentation techniques to extract the outline of the liver anatomical structure in the sixth image sequence; S7.2 Register the liver structure with the reference template; if the deviation exceeds the threshold, adjust the outline. S7.

3. Coordinate and adjust the details of the anatomical structure, reconstruct and verify the continuity of the boundaries, and obtain the seventh image sequence.

9. A liver image enhancement method based on multimodal fusion according to claim 1, characterized in that, Step S8 includes: S8.1 Spatial alignment of the seventh image sequence, and extraction of complementary boundary information through a boundary enhancement network; S8.2 If the boundary continuity is insufficient, local smoothing repair shall be performed; S8.3 Perform consistency detection on complementary boundary information; if the threshold is met, sharpen and fuse the data. S8.4 After detail optimization, the final boundary-enhanced eighth image sequence is obtained.

10. A liver image enhancement system based on multimodal fusion, characterized in that, include: A dose optimization module for performing step S1 of claim 1; A timing alignment module is used to perform step S2 in claim 1; A modal compensation module is used to perform step S3 in claim 1; A quantization encoding module is used to perform step S4 in claim 1; A compressed transmission module, used to perform step S5 in claim 1; A semantic guidance module is used to execute step S6 in claim 1; A region registration module is used to perform step S7 in claim 1; A boundary enhancement module is used to perform step S8 in claim 1.

Images