Liver cancer lesion segmentation method based on dynamic feature fusion

Abstract

The application discloses a liver cancer lesion segmentation method based on dynamic feature fusion, relates to the technical field of liver cancer diagnosis assistance, and comprises the steps of multi-source data acquisition, feature extraction, dynamic fusion, segmentation output and result verification. 4D images, multi-posture liver images and synchronous physiological signals are acquired through a multi-source data acquisition module, features are extracted through static and dynamic feature extraction modules, weights are allocated by a dynamic fusion module, and then the segmentation output and result verification module is processed. Through the cooperative processing of the above-mentioned multi-source data acquisition, feature extraction and other modules, the application can accurately distinguish the real lesion boundary from the motion artifact, effectively reduces the missed detection situation caused by the fuzzy boundary of the micro lesion, greatly reduces the missed detection rate caused by the atypical single posture feature, improves the identification accuracy by capturing the difference in the morphological stability of benign and malignant lesions, and finally realizes the accurate segmentation of the liver cancer lesion, thereby providing a reliable basis for clinical diagnosis and treatment decision.

Description

Technical Field

[0001] The present invention relates to the field of liver cancer diagnosis auxiliary technology, and in particular to a liver cancer lesion segmentation method based on dynamic feature fusion. Background Art

[0002] In the diagnosis and treatment of liver cancer, liver cancer lesion segmentation in medical images is a key step in assessing the location, size, morphology, and invasiveness of lesions, and is of great significance for clinical decision-making. However, existing liver cancer lesion segmentation technologies still have significant limitations and cannot meet the needs of accurate segmentation:

[0003] On the one hand, existing methods mostly rely on single-frame static CT / MRI images, which do not fully consider the physiological movement characteristics of the liver caused by breathing and heartbeat, resulting in blurred lesion boundaries in static images due to motion artifacts. Static algorithms cannot distinguish between "real lesion boundaries" and "motion artifacts", often misjudging grayscale changes caused by motion as lesion edges, or miss tiny lesions due to blurred boundaries, resulting in low segmentation boundary accuracy; on the other hand, existing technologies mostly perform segmentation based on single-body images, which cannot reflect the morphological changes of lesions in different body postures. Liver cancer lesions may show characteristics such as "grayscale close to normal liver tissue" and "atypical edges" in a single body posture, resulting in a missed detection rate of 20% to 30%. In addition, the difference in "morphological stability" between benign lesions and malignant liver cancer cannot be captured by a single body posture, further increasing the difficulty of distinguishing between benign and malignant lesions. Summary of the Invention

[0004] The purpose of the present invention is to provide a liver cancer lesion segmentation method based on dynamic feature fusion to address the shortcomings of the background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a liver cancer lesion segmentation method based on dynamic feature fusion, comprising the following steps: a multi-source data acquisition step: acquiring a 4D image sequence including respiratory / heartbeat cycles, liver images under different body movements, and synchronized physiological signals;

[0006] Feature extraction steps:

[0007] Static feature extraction sub-step: extracting static morphological features such as density / signal features, lesion edges and textures from CT / MRI images;

[0008] Dynamic feature extraction sub-steps:

[0009] Temporal motion feature extraction: This module processes 4D image sequences to capture liver displacement, the relative motion trajectory of the lesion and blood vessels, and the motion patterns of the lesion at different angles. It also constructs a motion consistency attention module that assigns higher weights to static density features during specific respiratory phases and verifies boundary continuity and adjusts regional weights based on motion trajectories during other respiratory phases.

[0010] Body posture feature extraction: Liver images in different postures are aligned using a deformable registration algorithm to extract the shape invariants of the lesion from multiple angles. A body posture adaptive gating network is constructed to enhance the weights of regions with consistent features across multiple postures and suppress the weights of regions with unstable features.

[0011] Physiological response feature extraction: Synchronously analyze physiological signals and dynamic enhanced images to extract "physiological-imaging" correlation features; design a dynamic weight allocator guided by physiological signals to increase the dynamic enhancement feature weights for areas where physiological signals indicate abnormalities;

[0012] Dynamic fusion step: Using attention mechanism or gating mechanism, the fusion weight of static features and dynamic features is dynamically allocated according to the liver movement state, the matching degree of body posture and respiratory state, the abnormality of physiological signals, etc.

[0013] Segmentation result output step: Output the liver cancer lesion segmentation boundaries at different angles and provide dynamic feature interpretation including the influence of body posture and respiratory status;

[0014] Result interpretation and verification steps:

[0015] Visualization: Generate dynamic feature heat maps to show the contribution ratio of different dynamic features to the segmentation results;

[0016] Clinical validation: The invasion range of the lesion in the postoperative pathological sections was associated with the dynamic features to verify the reliability of the segmentation results.

[0017] In a preferred embodiment, in the multi-source data acquisition step, the 4D image sequence includes breath-holding state information; different body movements include multiple postures; and the synchronized physiological signals include different breathing frequencies, breathing depths, and breath-holding state indicators.

[0018] In a preferred embodiment, in the temporal motion feature extraction:

[0019] Use spatiotemporal convolutional networks or Transformers to process 4D image sequences and distinguish liver motion characteristics under different respiratory rates and breath-holding states;

[0020] The inter-frame displacement field is calculated by optical flow method, and the motion modes dominated by breathing / heartbeat and breath holding state are extracted by principal component analysis.

[0021] The specific adjustment method of the motion consistency attention module is to give the static density feature the highest weight at the end of exhalation and in the breath-hold state, verify the boundary continuity through the motion trajectory during the inspiration period and at different breathing rates, and increase the weight of areas with significant motion differences.

[0022] The specific method of the motion consistency attention module dynamically adjusting the feature weight according to the respiratory cycle, respiratory frequency and breath holding state is:

[0023] In the breath-holding state, static density features such as CT value and boundary features are given the highest weight, with the weight accounting for ≥80%;

[0024] At the end of exhalation, a static density feature weight is assigned, accounting for 60%-70%, supplemented by motion trajectory consistency verification at low respiratory rate;

[0025] During the inhalation phase and at high respiratory rates, if the difference between the motion trajectory of a certain area and the surrounding tissue exceeds a preset threshold, and the threshold is reduced by 10%-20% at high respiratory rates, the weight of the area as a lesion is increased to 50%-60%;

[0026] For lesions at special angles such as the diaphragmatic surface of the liver, the overlap of motion trajectories at the end of inspiration and breath-holding states is analyzed in detail. When the overlap is less than 50%, the weight of motion features is enhanced.

[0027] In a preferred embodiment, in the extraction of body posture change features:

[0028] Thin plate spline interpolation is used to register multi-body images, and the deformation Jacobian matrix is ​​calculated to quantify the shape change amplitude under different states.

[0029] The screening criteria for the posture adaptive gating network are: increasing the weights of regions that exhibit "low density + irregular shape + fixed boundaries" in three or more combined posture movements and breathing / breath-holding states, and suppressing the weights of regions with significant shape changes in a single posture or breathing state.

[0030] In a preferred embodiment, in the physiological response feature extraction:

[0031] The dynamic contrast-enhanced image is DCE-MRI, and the extracted “physiological-imaging” correlation features include the correlation between the peak blood perfusion and respiratory rate in the breath-holding state;

[0032] Align the respiratory / ECG signals with the DCE-MRI enhancement curves according to the respiratory state time sequence, and calculate the dynamic correlation through the cross-correlation coefficient;

[0033] The workflow of the physiological signal-guided dynamic weight allocator is as follows: encode physiological signals of different respiratory rates and breath-holding states into low-dimensional vectors containing the time dimension; generate weight coefficients corresponding to dynamic enhancement features through a fully connected layer, in which a time attenuation factor is introduced in the breath-holding state; and increase the weight of dynamic enhancement features by 30%-50% in areas where physiological signals indicate abnormal blood flow.

[0034] In a preferred embodiment, in the result interpretation and verification step, the visualization and verification method for lesions at different angles is:

[0035] Grad-CAM is used to generate dynamic feature heat maps, which are displayed by lesion angle and the feature contribution of each area under specific posture and breath-holding conditions is marked.

[0036] During clinical verification, the focus was on comparing the dynamic characteristics matching degree of pathologically confirmed small-angle lesions in the corresponding body posture and deep inspiration state.

[0037] The present invention also provides a liver cancer lesion segmentation system based on dynamic feature fusion, comprising: a multi-source data acquisition module: configured to acquire a 4D image sequence including respiratory / heartbeat cycles, liver images of different body movements, and synchronized physiological signals;

[0038] Feature extraction module:

[0039] Static feature extraction submodule: configured to extract static features such as density / signal, edge, texture, etc.

[0040] Dynamic feature extraction submodule: including temporal motion feature extraction unit, posture change feature extraction unit, and physiological response feature extraction unit;

[0041] The temporal motion feature extraction unit is configured to: capture the motion trajectory of the liver and lesion and construct a motion consistency attention module;

[0042] The posture change feature extraction unit is configured to: analyze liver images in different postures, extract multi-angle shape invariants, and construct a posture adaptive gating network;

[0043] The physiological response feature extraction unit is configured to: associate physiological signals with dynamic enhanced image features and design a dynamic weight allocator;

[0044] Dynamic fusion module: configured to dynamically assign static and dynamic feature weights based on liver motion status, body posture and respiratory status matching, and the degree of physiological abnormality;

[0045] Segmentation result output module: configured to output liver cancer lesion segmentation boundaries at different angles and provide dynamic feature interpretation;

[0046] Result interpretation and verification module: configured to generate dynamic feature heat maps and associate pathological results with dynamic features for verification.

[0047] In a preferred embodiment, the multi-source data acquisition module is further configured to acquire a 4D image sequence including breath-holding states, liver images of different body movements, and synchronized physiological signals including different respiratory rates, respiratory depths, and breath-holding state indicators. The temporal motion feature extraction unit is further configured to:

[0048] Use spatiotemporal convolutional networks or Transformers to process 4D image sequences and distinguish liver motion characteristics under different respiratory rates and breath-holding states;

[0049] Extract motion modes through optical flow method and principal component analysis;

[0050] Built-in breath-holding state detection subunit, building a motion consistency attention module combined with breath-holding state;

[0051] The body posture change feature extraction unit is further configured as follows:

[0052] Thin plate spline interpolation is used to register multi-body images and calculate the deformation Jacobian matrix;

[0053] A variety of body movements and breathing state combination schemes are preset to build a body posture adaptive gating network for the combination state.

[0054] In a preferred embodiment, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements any of the steps of the above-mentioned method for liver cancer lesion segmentation based on dynamic feature fusion;

[0055] An electronic device comprising: a processor;

[0056] a memory for storing instructions executable by the processor;

[0057] The processor is configured to execute the instructions to implement any one of the above-mentioned liver cancer lesion segmentation methods based on dynamic feature fusion.

[0058] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0059] 1. 4D image sequences and synchronized physiological signals are acquired through multi-source data acquisition, combined with temporal motion feature extraction. The 3DResNet-50 architecture is used to capture motion trajectories, the Farneback algorithm is used to calculate the displacement field, and the motion consistency attention module dynamically adjusts weights according to the breathing phase. This allows for precise distinction between motion artifacts and true boundaries. Furthermore, the dynamic fusion step increases the weight of dynamic features during intense motion, further reducing motion interference. Ultimately, precise boundaries are output through U-Net++, addressing the issue of low segmentation boundary accuracy.

[0060] 2. By collecting images in multiple postures, such as supine and lateral positions, and after thin-plate spline interpolation and registration, the posture-adaptive gating network enhances the weights of regions with consistent characteristics across multiple postures, suppressing regions with varying characteristics in a single posture, effectively capturing the morphological regularities of lesions in different postures and reducing the missed detection rate. At the same time, dynamic feature extraction captures the "morphological stability" of lesions in multiple postures—malignant lesions usually exhibit consistent characteristics across multiple postures, while benign lesions have variable characteristics. Combined with physiological response characteristics, this provides more basis for differential diagnosis of benign and malignant lesions, solving the difficult problem of differential diagnosis in a single posture. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0062] Figure 1 Flow chart of the method of the present invention.

[0063] Figure 2 This is a system block diagram of the present invention. DETAILED DESCRIPTION

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0065] Example 1, please refer to Figure 1 As shown, the liver cancer lesion segmentation method based on dynamic feature fusion described in this embodiment includes the following steps:

[0066] S1. Multi-source data acquisition step: Acquire 4D image sequences including respiratory / cardiac cycles, liver images under different body postures and movements, and synchronized physiological signals;

[0067] S2, feature extraction step:

[0068] Static feature extraction: extract static morphological features such as density / signal, edge, texture, etc. from CT / MRI;

[0069] Dynamic feature extraction:

[0070] Temporal motion features: Process 4D images, capture the motion trajectory of the liver and lesions, build a motion consistency attention module, and adjust feature weights according to the respiratory stage;

[0071] Body shape change features: align multi-body images, extract multi-angle shape invariants, and build a gated network to enhance the weights of multi-body consistent regions;

[0072] Physiological response characteristics: Correlate physiological signals with dynamic enhanced images, extract "physiological-imaging" features, and design a dynamic weight allocator;

[0073] S3, dynamic fusion step: Based on attention or gating mechanism, static and dynamic feature weights are assigned according to liver movement, posture and breathing matching, and physiological abnormality;

[0074] S4, segmentation output step: output lesion boundaries and dynamic feature interpretations at different angles;

[0075] S5. Result verification step: Generate dynamic feature heatmap and associate it with pathological results to verify the reliability of segmentation.

[0076] As described in steps S1-S5 above, in the diagnosis and treatment of liver cancer, the accuracy of liver cancer lesion segmentation directly affects the clinical assessment of lesion location, size, morphology, and invasiveness, and thus influences clinical decision-making. However, existing liver cancer lesion segmentation technologies have significant limitations in practical applications and are difficult to meet the demand for accurate segmentation.

[0077] The defects of the existing technology are mainly reflected in two aspects. On the one hand, the existing methods are overly dependent on single-frame static CT / MRI images, and fail to fully consider the physiological movement characteristics of the liver due to breathing and heartbeat. This makes the lesion boundaries in static images easily blurred due to motion artifacts, and the static algorithm cannot effectively distinguish between "real lesion boundaries" and "motion artifacts". During the segmentation process, the algorithm often misjudges the grayscale changes caused by motion as the edge of the lesion, or misses tiny lesions due to blurred boundaries, which ultimately leads to low segmentation boundary accuracy. On the other hand, the existing technology is mostly based on single-body images for segmentation, which cannot reflect the morphological changes of lesions in different body postures. Liver cancer lesions may show characteristics such as "grayscale close to normal liver tissue" and "atypical edges" in a single body posture, which directly leads to a missed detection rate of 20% to 30%; at the same time, the difference in "morphological stability" between benign lesions and malignant liver cancer cannot be captured by a single body posture, which further increases the difficulty of distinguishing between benign and malignant.

[0078] By setting up a multi-source data acquisition module to acquire 4D image sequences, liver images in different body postures, and synchronized physiological signals, setting up a static feature and dynamic feature (temporal motion features, body posture change features, physiological response features) extraction module, setting up a dynamic fusion module based on attention or gating mechanisms, and setting up a segmentation output and result verification module, it can accurately distinguish "true lesion boundaries" from "motion artifacts", reduce the missed detection of small lesions, and improve the accuracy of segmentation boundaries. At the same time, it can effectively capture the morphological changes of lesions in different body postures, reducing the missed detection rate. By capturing the differences in "morphological stability" between benign and malignant lesions, it can improve the accuracy of benign and malignant differentiation, ultimately achieving accurate segmentation of liver cancer lesions and providing reliable support for clinical decision-making.

[0079] In one embodiment, the multi-source data acquisition step (S1) includes acquiring a 4D image sequence including respiratory / cardiac cycles, liver images under different body movements, and synchronized physiological signals, including:

[0080] S11. 4D image sequences: Acquired using dynamic MRI or CT equipment, with a temporal resolution of ≤0.5 s / frame and a spatial resolution of ≤1 mm³, and annotated with breath-hold status information (e.g., breath-hold start / end timestamps);

[0081] S12. Different body postures and movements: including supine, right side lying, left side lying, sitting, and deep inhalation / deep exhalation images. At least three respiratory cycles were collected for each body posture.

[0082] S13. Synchronous physiological signals: Respiratory rate (range 0-40 breaths / minute), respiratory depth (chest expansion amplitude ≥ 5 cm), and breath-hold status indicator (breath-hold duration ≥ 10 seconds) are collected synchronously via a respiratory belt and ECG monitor. The signal sampling frequency is ≥ 100 Hz.

[0083] S14. Extension conditions:

[0084] (1) If the breath-holding state accounts for less than 30% of the 4D image sequence, the re-acquisition process needs to be triggered;

[0085] (2) The time stamp deviation between the physiological signal and the image sequence must be ≤50ms, otherwise it will be corrected by the interpolation algorithm;

[0086] As described in the above steps S11-S13, from the perspective of data quality, S11's strict regulations on the temporal and spatial resolution of the 4D image sequence, as well as the annotation of breath-holding state information, can ensure that the image sequence is clear and has a clear reference basis for the respiratory cycle, providing high-quality basic data for the subsequent capture of dynamic features such as the movement trajectory of the liver and lesions; S12 requires the acquisition of images in multiple postures and each posture covers at least 3 respiratory cycles, which can fully capture the morphological changes of the lesions in different postures and avoid information loss due to the atypical characteristics of a single posture; S13's parameter specifications for synchronous physiological signal acquisition can ensure the accuracy of physiological signals such as respiration and electrocardiogram, laying a solid foundation for extracting "physiological-imaging" features; at the same time, the extended conditions of S14 further enhance the validity of the data. When the proportion of breath-holding state is insufficient, re-acquisition is triggered to ensure the adequacy of key state data in the 4D image sequence; the correction of the timestamp deviation between the physiological signal and the image sequence ensures a strong correlation between the two in the time dimension, avoiding the influence of time misalignment on the accuracy of subsequent feature fusion;

[0087] Through this multi-source data acquisition setup, high-quality, multi-dimensional, and time-synchronized imaging and physiological data can be obtained, providing reliable support for the subsequent precise extraction and fusion of dynamic features, thereby helping to improve the accuracy of liver cancer lesion segmentation.

[0088] In one embodiment, the feature extraction step S2 includes:

[0089] S21, static feature extraction sub-step: extracting static morphological features such as density / signal features, lesion edge and texture from CT / MRI images;

[0090] S22, dynamic feature extraction sub-step:

[0091] Including time-series motion feature extraction, body posture change feature extraction, and physiological response feature extraction;

[0092] As described in steps S21-S22 above, in terms of static feature extraction, S21 extracts static morphological features such as density / signal features, lesion edges, and texture from CT / MRI images, which can capture the basic morphological information of the lesion. These static features are the most intuitive representation of the lesion, providing an initial morphological reference for subsequent segmentation and helping to distinguish the basic differences between lesions and normal liver tissue at static conditions.

[0093] Its advantages are more prominent in terms of dynamic feature extraction. Temporal motion feature extraction can capture the motion trajectory of the liver and lesions, and adjust the feature weights in combination with the respiratory stage, which can effectively distinguish between "real lesion boundaries" and "motion artifacts", avoiding misjudgment of grayscale changes caused by motion or missed detection of small lesions due to blurred boundaries; posture change feature extraction can reflect the morphological change rules of lesions in different postures by aligning multi-body images and extracting multi-angle shape invariants, enhance the weights of multi-body consistent regions, reduce missed detections due to atypical lesion features in a single posture, and provide more morphological basis for benign and malignant differentiation; physiological response feature extraction associates physiological signals with dynamic enhanced images, extracts "physiological-imaging" features and designs dynamic weight allocators, which can combine physiological signals with image features to further improve the effectiveness and pertinence of features. Through the extraction of static and dynamic features, it can comprehensively and accurately capture various features of liver cancer lesions, provide rich and reliable feature support for subsequent dynamic fusion and segmentation output, and thus improve the accuracy and reliability of liver cancer lesion segmentation.

[0094] In one embodiment, the static feature extraction sub-step of step S21 includes:

[0095] S211. Density / signal characteristics: HU values ​​(range -1000 to 3000) were extracted from CT images, and T1 / T2 weighted signal intensity was extracted from MRI images. Histogram statistics were used to calculate parameters such as mean, variance, and skewness.

[0096] S212. Lesion edge features: The lesion boundary was extracted using the Canny edge detection algorithm, and the standard deviation of the boundary curvature was calculated (≤0.2 for regular, >0.5 for irregular).

[0097] S213, texture features: Gray-level co-occurrence matrix (GLCM) is used to extract contrast, energy, entropy and other parameters, with the window size set to 5 × 5 pixels;

[0098] As described in steps S211-S213 above, regarding density / signal feature extraction, S211 extracts the HU value and T1 / T2 weighted signal intensity for CT and MRI images, respectively. By using histogram statistics such as mean, variance, and skewness, the grayscale or signal information in the image can be converted into quantifiable feature data. This quantification process not only more accurately captures subtle differences in density or signal between lesions and normal liver tissue, avoiding subjective judgment bias, but also provides a unified reference standard for subsequent feature analysis and comparison, making feature comparisons between different images more scientific.

[0099] For lesion edge feature extraction, the S212 uses the Canny edge detection algorithm to extract lesion boundaries and calculates the standard deviation of boundary curvature to determine whether the boundary is regular, achieving a precise description of the lesion edge morphology. This method can clearly distinguish between regular and irregular lesion boundaries, providing a reliable basis for determining characteristics such as lesion invasiveness. It also provides a clear reference for determining lesion boundaries during subsequent segmentation, reducing segmentation errors caused by blurred edges.

[0100] In texture feature extraction, the S213 uses a gray-level co-occurrence matrix to extract parameters such as contrast, energy, and entropy, and uses a 5×5 pixel window size to effectively capture the grayscale distribution and texture details within the lesion. This precise extraction of texture features can distinguish the internal structural differences between lesions and normal liver tissue. Even when the grayscale of lesions and normal liver tissue are similar, they can be identified based on the difference in texture features, reducing the possibility of missed detection and further enriching the dimensionality of static features.

[0101] The above-mentioned clear extraction methods and quantification standards make static features more accurate, quantitative and comprehensive, laying a solid foundation for subsequent feature fusion and lesion segmentation, and further enhancing the supporting role of static features in lesion identification and segmentation.

[0102] In one embodiment, step S22, the dynamic feature extraction sub-step, includes:

[0103] S221. Temporal Motion Feature Extraction: By processing 4D image sequences, we accurately capture liver displacement, the relative motion trajectory between the lesion and blood vessels, and the lesion motion patterns at different angles. This provides a motion basis for distinguishing motion artifacts from true lesion boundaries. Further features include:

[0104] S2211, using a spatiotemporal convolutional network based on the 3DResNet-50 architecture to process 4D image sequences, with a specific size (128×128×128×T) as input and a 1mm³ resolution liver motion vector field as output, which can effectively distinguish liver motion characteristics under different respiratory rates and breath-holding states.

[0105] S2212. Use the Farneback algorithm to calculate the inter-frame displacement field and combine it with principal component analysis to extract the top three dominant motion modes (accounting for ≥80%). This can accurately capture the motion modes dominated by breathing / heartbeat and breath holding, and reduce non-dominant motion interference.

[0106] S222. Construct a motion consistency attention module to dynamically adjust feature weights based on breathing phases: assign higher weights to static density features in specific breathing phases, verify boundary continuity through motion trajectories in other phases and adjust regional weights to improve feature targeting. Further steps include:

[0107] S2221. Give the static density feature the highest weight at the end of exhalation and in the breath-hold state. Verify the boundary continuity through the motion trajectory during the inspiration period and at different breathing rates. Increase the weight of the area with significant motion differences to achieve a reasonable emphasis on the characteristics of different breathing stages.

[0108] S2222. Dynamically adjust weights based on respiratory cycle, frequency, and breath-holding status: static feature weight during breath-holding is ≥80%, static feature weight at the end of expiration is 60%-70% and supplemented by low respiratory rate motion verification. The weight of areas with significant motion differences between inhalation and high respiratory rate is increased to 50%-60%. Focus on analyzing the overlap of motion trajectories of lesions at special angles such as the diaphragmatic surface of the liver, and further refine the weight adjustment logic.

[0109] S223, Body posture feature extraction: Align multi-body posture images through a deformable registration algorithm to extract multi-angle shape invariants of the lesion; construct a body posture adaptive gating network to enhance the weights of multi-body posture consistent regions and suppress the weights of feature unstable regions to fully capture the impact of body posture on the lesion. Further steps include:

[0110] S2231. Use thin plate spline interpolation to align multi-body image registration, calculate the deformation Jacobian matrix (determinant value > 1.5 indicates significant deformation) to quantify shape changes; preset three body posture and respiratory state combination schemes, enhance the weight of feature stable areas under multiple combination states, and suppress the weight of single body posture feature changing areas to improve the effectiveness of body posture features.

[0111] S224. Physiological response feature extraction: Synchronously analyze physiological signals and dynamic enhanced images to extract physiological-imaging correlation features; design a physiological signal-guided dynamic weight allocator to increase the weight of dynamic enhanced features in physiological abnormal areas, thereby achieving the integration of physiological and imaging features. Further steps include:

[0112] S2241. Process DCE-MRI dynamic enhanced images (one frame is collected every 5 seconds for 5 minutes after contrast agent injection) to extract the correlation between the peak blood perfusion (signal intensity change rate > 30% / second) and the respiratory rate under breath-holding; align the physiological signal with the image enhancement curve time series, and calculate the dynamic correlation through the cross-correlation coefficient; the dynamic weight allocator encodes the physiological signal into a low-dimensional vector, generates the corresponding weight coefficient, introduces a time attenuation factor during breath-holding (decreases by 5% per second), and increases the weight of the abnormal blood flow area by 30%-50%, so that the physiological signal can better assist in feature extraction.

[0113] As described in steps S221-S2241 above, temporal motion feature extraction can effectively distinguish between "real lesion boundaries" and "motion artifacts" by accurately capturing motion trajectories and motion modes, combined with dynamic weight adjustment of the motion consistency attention module, solving the problem of low segmentation boundary accuracy caused by motion artifacts in existing technologies; posture change feature extraction captures the morphological changes of lesions in different postures through multi-posture image alignment and gating networks, reducing missed detections caused by atypical single posture features; physiological response feature extraction associates physiological and imaging features, increasing the feature weights of physiological abnormal areas and providing more basis for benign and malignant differentiation. Overall, the sub-steps of dynamic feature extraction supplement the deficiencies of static features from the three dimensions of motion, posture, and physiology, providing comprehensive and accurate dynamic feature support for subsequent fusion.

[0114] In one embodiment, S3, a dynamic fusion step, employing an attention mechanism or a gating mechanism, dynamically assigns fusion weights of static features and dynamic features based on the liver's motion state, the matching degree of body posture and respiratory state, the degree of physiological signal abnormality, etc., to achieve reasonable fusion of features, further comprising:

[0115] S31, Attention Mechanism: Uses a Squeeze-and-Excitation network, with static features (dimension 512) and dynamic features (dimension 512) as input, and outputs fusion weights (static weight range 0.3-0.7, dynamic weight range 0.3-0.7), making feature fusion more targeted;

[0116] S32. Gating mechanism: A three-layer fully connected network is constructed. The inputs are liver movement speed (>5 mm / s is high speed), the matching degree of body posture and respiratory state (score ≥0.8 when matching), and the abnormality of physiological signals (score ≥0.7 when abnormal). The output is a gating signal (between 0 and 1, with 1 indicating full use of dynamic features) to provide a judgment basis for weight allocation.

[0117] S33. Fusion rule: When the liver motion speed is greater than 10 mm / s or the physiological signal abnormality score is ≥0.9, the dynamic feature weight is increased to 60%-80%, ensuring that the dynamic feature plays a dominant role when the movement is intense or the physiological abnormality occurs;

[0118] As described in steps S31-S34 above, the dynamic fusion step dynamically allocates weights through attention and gating mechanisms, combined with liver movement, posture and breathing matching, and the degree of physiological abnormalities, to achieve complementary advantages between static and dynamic features. Increasing the weight of dynamic features during intense movement or physiological abnormalities can better cope with motion artifacts; reasonably retaining the weight of static features during stable movement ensures the role of basic morphological features. This flexible fusion method solves the limitations of existing technologies that rely solely on static or single posture features, improves the accuracy of feature fusion, and provides high-quality fusion features for subsequent segmentation output.

[0119] In one embodiment, S4, the step of outputting the segmentation results, outputs the segmentation boundaries of the liver cancer lesions at different angles and provides dynamic feature explanations including the influence of body posture and respiratory status to make the segmentation results easier to understand and reference. The step further includes:

[0120] S41, Segmentation boundary: Using the U-Net++ architecture, the input is the fused feature map (size 256×256×32), and the output is the lesion mask (binarized image, 1 indicates the lesion area), ensuring the accurate output of the segmentation boundary;

[0121] S42. Dynamic Feature Interpretation: Generates a heatmap (color mapping: red indicates high contribution, blue indicates low contribution) of body posture (supine / right side lying) and respiratory state (breath holding / deep inhalation) on the segmentation results, visually demonstrating the impact of each factor on segmentation.

[0122] As described in steps S41-S42 above, the segmentation output step outputs precise lesion boundaries through the U-Net++ architecture, solving the problems of low segmentation boundary accuracy and missed detection of tiny lesions in existing technologies. The heat map explained by dynamic features intuitively displays the impact of body posture and respiratory status on the segmentation results, allowing clinical staff to clearly understand the basis for segmentation, enhance their trust in the segmentation results, and provide a clearer reference for clinical decision-making.

[0123] In one embodiment, S5, the result interpretation and verification step, which verifies the reliability of the segmentation by correlating it with the pathological results to ensure the accuracy of the segmentation results, further includes:

[0124] S51, Visualization: Generate dynamic feature heat map, show the contribution ratio of different dynamic features to the segmentation results, clearly present the role of features, further including;

[0125] S511, heat map generation: Use the Grad-CAM algorithm to perform backpropagation on the fused feature map, calculate the gradient weight of each dynamic feature (such as motion trajectory, blood perfusion), and clarify the feature contribution;

[0126] S512. Zoning display: Zoning by lesion angle (left lobe / right lobe / diaphragmatic surface), annotating the feature contribution of each region in specific postures (such as lying on the right side) and breath-holding states (in percentage form, such as "motion feature contribution 65%)", and refining the feature contribution display;

[0127] S52. Clinical Validation: Correlate the invasive range of the lesion in the postoperative pathological section with the dynamic features to verify the reliability of the segmentation results and ensure that the segmentation results are consistent with clinical practice. Further steps include:

[0128] S521. Pathological control: 50 patients undergoing liver cancer surgery were selected to compare the extent of lesion invasion confirmed by pathology (measured under a microscope) with the overlap of dynamic feature segmentation results (Dice coefficient ≥ 0.85 was considered acceptable). Segmentation accuracy was verified by pathology.

[0129] S522. Dynamic feature matching: Focus on verifying the matching degree (accuracy ≥ 90%) between the dynamic features of micro-angle lesions (diameter < 1 cm) in the corresponding posture (such as deep inhalation) and the deep inhalation state (such as motion trajectory difference > 3 mm) and the pathological results to ensure the reliability of micro-lesion segmentation.

[0130] As described in steps S51-S522 above, the result verification step uses a dynamic feature heatmap visualization to clearly demonstrate the contribution of features to segmentation, facilitating understanding of the segmentation logic. Clinical verification ensures the reliability of segmentation results by comparing them with pathological results, particularly verifying the matching of small lesions, addressing the problem of missed detection of small lesions in existing technologies. Furthermore, the regional display of feature contributions allows for targeted analysis of the segmentation basis for lesions at different angles, further enhancing the credibility of segmentation results and providing reliable segmentation evidence for clinical diagnosis.

[0131] Example 2, please refer to Figure 2 As shown, the liver cancer lesion segmentation system based on dynamic feature fusion described in this embodiment includes:

[0132] (1) Multi-source data acquisition module: configured to acquire 4D image sequences including respiratory / cardiac cycles, liver images with different body movements, and synchronized physiological signals;

[0133] The multi-source data acquisition module is further configured to: acquire a 4D image sequence including breath-holding states, liver images of different body movements, and synchronized physiological signals including different respiratory rates, respiratory depths, and breath-holding state indicators;

[0134] Furthermore, the 4D image acquisition unit: integrates dynamic MRI / CT equipment, supports time resolution ≤ 0.5 seconds / frame, spatial resolution ≤ 1mm³, and automatically annotates the breath-holding state timestamp;

[0135] Body image acquisition unit: equipped with an adjustable bed, supporting supine, right side, left side, sitting and other body positions. Each body position automatically triggers image acquisition of three respiratory cycles.

[0136] Physiological signal acquisition unit: integrated respiratory belt, ECG monitor, sampling frequency ≥100Hz, supports synchronization with the timestamp of the image sequence (deviation ≤50ms);

[0137] (2) Feature extraction module:

[0138] (2.1) Static feature extraction submodule: configured to extract static features such as density / signal, edge, texture, etc.

[0139] Furthermore, the density / signal extraction unit: supports HU value statistics (range -1000 to 3000) of CT images and T1 / T2 signal intensity extraction of MRI images, and outputs histogram parameters (mean, variance, skewness);

[0140] Edge feature extraction unit: Integrates the Canny edge detection algorithm to output the standard deviation of the lesion boundary curvature (≤0.2 for regular, >0.5 for irregular);

[0141] Texture feature extraction unit: uses the gray-level co-occurrence matrix (GLCM) algorithm with a window size of 5×5 pixels and outputs parameters such as contrast, energy, and entropy;

[0142] (2.2) Dynamic feature extraction submodule: including temporal motion feature extraction unit, posture change feature extraction unit, and physiological response feature extraction unit;

[0143] (2.2.1) The temporal motion feature extraction unit is configured to: capture the motion trajectory of the liver and lesion and construct a motion consistency attention module;

[0144] Furthermore, the spatiotemporal convolutional network uses the 3DResNet-50 architecture with an input size of 128×128×128×T and outputs the liver motion vector field (resolution 1mm³);

[0145] Optical flow calculation unit: Integrates the Farneback algorithm, outputs the inter-frame displacement field, and extracts the top three dominant motion modes (accounting for ≥80%) through PCA;

[0146] Movement consistency attention module: Built-in breath-hold detection subunit, dynamically adjusts weight (breath-hold weight ≥ 80%, 60%-70% at end-expiration, and increases to 50%-60% at high respiratory rates during inspiration);

[0147] (2.2.2) The posture change feature extraction unit is configured to: analyze liver images in different postures, extract multi-angle shape invariants, and construct a posture adaptive gating network;

[0148] Furthermore, the deformation registration unit uses the TPS algorithm to register multi-body images and outputs the deformation Jacobian matrix (determinant value > 1.5 indicates significant deformation);

[0149] Posture-adaptive gating network: 3 preset combinations of posture and respiratory states are set. The weights of regions with stable features under multiple combinations are enhanced, while the weights of regions with varying features under a single posture are suppressed.

[0150] (2.2.3) The physiological response feature extraction unit is configured to: associate physiological signals with dynamic enhanced image features and design a dynamic weight allocator;

[0151] Furthermore, the dynamic enhancement image processing unit: supports DCE-MRI image processing, acquiring one frame every 5 seconds for 5 minutes after contrast agent injection;

[0152] Physiological signal correlation unit: calculates the correlation between the peak blood perfusion value (signal intensity change rate > 30% / second) and the respiratory rate in the breath-holding state;

[0153] Dynamic weight allocator: generates weight coefficients through a fully connected layer, introduces a time decay factor (5% reduction per second) during breath holding, and increases the weight of abnormal blood flow areas by 30%-50%;

[0154] (3) Dynamic fusion module: configured to dynamically assign static and dynamic feature weights based on the matching degree of liver motion state, body posture and respiratory state;

[0155] Furthermore, it includes an attention mechanism unit, a gating mechanism unit, and a fusion rule unit;

[0156] The attention mechanism unit uses a Squeeze-and-Excitation network, inputs static features (dimension 512) and dynamic features (dimension 512), and outputs fusion weights (static 0.3-0.7, dynamic 0.3-0.7).

[0157] Gating mechanism unit: Build a 3-layer fully connected network, input liver movement speed, body posture matching, and physiological signal abnormality score, and output a gating signal (between 0 and 1);

[0158] Fusion rule unit: When the liver movement speed is greater than 10 mm / s or the physiological signal abnormality score is ≥0.9, the dynamic feature weight is increased to 60%-80%.

[0159] (4) Segmentation result output module: configured to output the segmentation boundaries of liver cancer lesions at different angles and provide dynamic feature interpretation;

[0160] Further, it includes segmentation units and interpretation units;

[0161] The segmentation unit uses the U-Net++ architecture, takes as input the fused feature map (size 256×256×32), and outputs a lesion mask (binarized image).

[0162] Interpretation unit: Generates a dynamic feature heat map that includes the influence of posture and respiratory status (color mapping: red indicates high contribution, blue indicates low contribution).

[0163] (5) Result interpretation and verification module: configured to generate dynamic feature heat maps and associate pathological results with dynamic features for verification;

[0164] Further, it includes a visualization unit and a clinical validation unit;

[0165] Among them, the visualization unit: uses the Grad-CAM algorithm to generate a heat map, and displays the feature contribution (in percentage form) by lesion angle partition;

[0166] Clinical Validation Unit: Compare the overlap between the pathologically confirmed lesion invasion range and the segmentation results (Dice coefficient ≥ 0.85), and verify the dynamic feature matching of subtle angle lesions (accuracy ≥ 90%).

[0167] This system module acquires high-quality, multi-dimensional data through a multi-source data acquisition module, providing a foundation for subsequent processing. The feature extraction module comprehensively extracts static and dynamic features, addressing the limitations of existing technologies that rely on a single feature. The dynamic fusion module achieves reasonable feature fusion, improving feature effectiveness. The segmentation result output module outputs precise boundaries and feature interpretations, providing clear clinical reference. The result interpretation and verification module ensures the reliability of segmentation results through visualization and pathological verification. The entire system forms a complete process from data acquisition to result verification, effectively addressing the existing problems of low segmentation accuracy, high missed detection rate, and difficulty in distinguishing benign and malignant lesions, thereby improving the accuracy and reliability of liver cancer lesion segmentation.

[0168] A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the liver cancer lesion segmentation method based on dynamic feature fusion described in the above-mentioned embodiment 1, including multi-source data acquisition, feature extraction, dynamic fusion, segmentation result output, and result interpretation and verification.

[0169] An electronic device includes a processor and a memory for storing processor-executable instructions, wherein the processor is configured to execute the instructions to implement the liver cancer lesion segmentation method based on dynamic feature fusion as described in any of the above embodiments, and the specific functions include:

[0170] Control the multi-source data acquisition module to acquire 4D images, body images and physiological signals;

[0171] Call the feature extraction module to process static and dynamic features;

[0172] Assign feature weights through dynamic fusion modules;

[0173] Output segmentation results and generate explanation heatmap;

[0174] The reliability of segmentation was verified by correlating it with pathological results.

[0175] Computer-readable storage media provides a carrier for the implementation of the segmentation method, facilitating the storage and application of the method; the electronic device controls the collaborative work of various modules through the processor, automatically completing the entire process of liver cancer lesion segmentation, improving segmentation efficiency, while ensuring the standardization and accuracy of the segmentation process, allowing the segmentation method to be effectively applied in clinical practice and providing reliable technical support for liver cancer diagnosis and treatment.

[0176] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A liver cancer lesion segmentation method based on dynamic feature fusion, characterized in that: The following steps are involved: Multi-source data acquisition steps: Acquire 4D image sequences including respiratory / cardiac cycles, liver images under different body movements, and synchronized physiological signals; Feature extraction includes static feature extraction and dynamic feature extraction; Static feature extraction: extract density / signal, edge, and texture static morphological features from CT / MRI; Dynamic feature extraction includes temporal motion features, body posture change features, and physiological response features; Temporal motion features: Process 4D images, capture the motion trajectory of the liver and lesions, build a motion consistency attention module, and adjust the feature weights according to the respiratory stage. The process of processing 4D images, capturing the motion trajectory of the liver and lesions, building a motion consistency attention module, and adjusting the feature weights according to the respiratory stage is as follows: first, a spatiotemporal convolutional network with a 3DResNet-50 architecture is used to process the 4D image sequence, output a liver motion vector field with a resolution of 1mm³, and then calculate the inter-frame displacement field using the Farneback algorithm. Combined with principal component analysis, the first three dominant motion modes are extracted to distinguish the motion features dominated by breathing / heartbeat and breath holding state; the module has built-in The breath-hold detection subunit dynamically adjusts feature weights based on the respiratory cycle, respiratory rate, and breath-hold state: During breath-hold, the static density and boundary features of the CT values ​​are given the highest weight, with a weight ratio of ≥80%. At the end of expiration, the static density features are given a weight of 60%-70%, supplemented by motion trajectory consistency verification at low respiratory rates. During inspiration and high respiratory rates, if the motion trajectory of a certain area differs from that of the surrounding tissue by more than a preset threshold, the weight of that area is increased to 50%-60%. For lesions at special angles on the diaphragmatic surface of the liver, the overlap of the motion trajectories at the end of inspiration and during breath-hold is analyzed, and the weight of the motion feature is increased when the overlap is less than 50%. Posture change features: Aligning multi-body images, extracting multi-angle shape invariants, and constructing a gating network to enhance the weights of multi-body consistent regions. The process of aligning multi-body images, extracting multi-angle shape invariants, and constructing a gating network to enhance the weights of multi-body consistent regions is as follows: using a thin plate spline interpolation algorithm to align multi-body images, calculating the deformation Jacobian matrix to quantify the magnitude of shape changes under different body postures; presetting three or more body movement and breathing / breath-holding combinations, and constructing a posture-adaptive gating network. The weights of feature-consistent regions that exhibit "low density, irregular shape, and fixed boundaries" in multiple combinations are enhanced, and the weights of feature-unstable regions that only exhibit significant shape changes under a single body posture or breathing state are suppressed. Physiological response features: Associating physiological signals with dynamic enhanced images, extracting "physiological-imaging" features, and designing a dynamic weight allocator. The process of associating physiological signals with dynamic enhanced images, extracting "physiological-imaging" features, and designing a dynamic weight allocator is as follows: using DCE-MRI as the dynamic enhanced image, acquiring one frame every 5 seconds for 5 minutes after contrast agent injection, and extracting the correlation between peak blood perfusion and respiratory rate under breath-holding conditions; aligning respiratory / ECG signals with DCE-MRI enhancement curves according to respiratory state time series, and calculating the dynamic correlation between the two using the cross-correlation coefficient; encoding physiological signals at different respiratory rates and breath-holding states into low-dimensional vectors containing a time dimension, generating weight coefficients corresponding to dynamic enhancement features through a fully connected layer, and introducing a time attenuation factor under breath-holding conditions; and increasing the weight of dynamic enhancement features by 30%-50% in areas where physiological signals indicate abnormal blood flow. Dynamic fusion step: Based on attention or gating mechanism, static and dynamic feature weights are assigned according to liver movement, posture and breathing matching, and physiological abnormality; Segmentation output step: output lesion boundaries and dynamic feature interpretations at different angles; Result verification steps: Generate dynamic feature heat maps and associate them with pathological results to verify segmentation reliability.

2. The method for liver cancer lesion segmentation based on dynamic feature fusion according to claim 1, characterized in that: The 4D images include breath-holding status information; the multi-body movements include various postures; and the physiological signals include breathing rate, depth, and breath-holding markers.

3. The method for liver cancer lesion segmentation based on dynamic feature fusion according to claim 1, characterized in that: In the result verification step, Grad-CAM is used to generate a heat map partitioned by lesion angle, and the characteristic contributions of body posture and breath-holding state are annotated; during clinical verification, the dynamic feature matching degree of small-angle lesions in the corresponding body posture and deep inspiration state is compared.

4. Liver cancer lesion segmentation system based on dynamic feature fusion, characterized by: A method for segmenting liver cancer lesions based on dynamic feature fusion according to any one of claims 1 to 3, comprising: a multi-source data acquisition module for acquiring 4D images, multi-body liver images, and synchronized physiological signals; Feature extraction module: including static feature extraction submodule and dynamic feature extraction submodule; Dynamic fusion module: assigns static and dynamic feature weights based on liver movement, body posture and breathing matching, and the degree of physiological abnormality; Segmentation output module: outputs lesion boundaries and dynamic feature interpretations at different angles; Result verification module: Generates heat map and associates it with pathological result verification.

5. The liver cancer lesion segmentation system based on dynamic feature fusion according to claim 4, characterized in that: The multi-source data acquisition module acquires 4D images including breath-holding status, multiple body posture images, and physiological signals including respiratory rate and breath-holding markers; the dynamic feature extraction submodule constructs a motion consistency attention module, a body posture adaptive gating network, and a physiological signal-guided weight allocator.

6. A computer-readable storage medium, characterized in that A computer program is stored, and when the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images