A medical image enhancement method based on frequency structured state modeling

Abstract

The application provides a medical image enhancement method based on frequency structured state modeling, and belongs to the field of intelligent processing of medical images. The method aims at the problems of blurred boundaries, low contrast and unclear structural details in medical images, and realizes the collaborative enhancement of structural information, boundary details and texture features of the medical images through local context preprocessing, wavelet structured state propagation, low-frequency guided sub-band refinement and frequency domain guided cross-scale feature alignment. Finally, the medical image or medical image feature with clearer structure, more continuous boundary and more sufficient details is obtained, which can provide reliable visual basis support for medical image preprocessing, lesion analysis, organ structure recognition and intelligent auxiliary diagnosis and the like.

Description

Technical Field

[0001] This invention belongs to the field of intelligent medical image processing, and specifically relates to a medical image enhancement method based on frequency-structured state modeling. Background Technology

[0002] Medical images are a crucial data foundation for clinical diagnosis, treatment planning, lesion analysis, and intelligent assisted diagnosis. However, in actual imaging processes, medical images often suffer from problems such as blurred tissue boundaries, insufficient local contrast, unclear structural details, noise interference, and appearance variations across devices and modalities, due to factors such as imaging equipment, acquisition conditions, tissue differences, and modal characteristics. These problems reduce the discernibility of anatomical structures, lesion areas, and local texture information in medical images, thereby affecting the stability of subsequent image analysis and intelligent diagnostic systems.

[0003] In recent years, deep learning methods have been widely used in medical image enhancement and medical image analysis tasks. Convolutional neural networks can effectively extract local texture information, but they still have certain limitations when modeling long-range structural dependencies. Transformer methods can enhance global context modeling capabilities, but their self-attention computation usually brings large computational and memory overhead in high-resolution medical image processing. State-space models provide an efficient long-range modeling approach through directional scanning and recursive state propagation, which is suitable for high-resolution visual feature processing.

[0004] However, existing image processing methods based on state-space models typically require flattening two-dimensional image features into a one-dimensional scan sequence, which easily introduces scan order dependencies and prevents the full utilization of naturally existing two-dimensional neighborhood relationships in the image. Simultaneously, low-frequency structural information and high-frequency boundary details in medical images have different frequency characteristics; directly modeling them uniformly in the spatial domain can easily lead to inconsistencies between the overall structure and local detail representations. Furthermore, during the cross-scale feature fusion process between the encoder and decoder, semantic differences exist between low-level structural features and high-level semantic features; direct addition or splicing may result in insufficient alignment of structural information. Therefore, a medical image enhancement method that can simultaneously consider local context, frequency structure, cross-scale feature alignment, and computational efficiency is needed. Summary of the Invention

[0005] This invention provides a medical image enhancement method based on frequency-structured state modeling, aiming to solve the problems of blurred boundaries, low tissue contrast, unclear structural details, and insufficient cross-scale feature fusion in medical images. The method achieves synergistic enhancement of the structure, boundaries, and texture details of medical images through local context preprocessing, wavelet structured state propagation, low-frequency guided subband thinning, and scale-selective frequency domain guided cross-scale feature alignment.

[0006] The present invention provides a medical image enhancement method based on frequency-structured state modeling, comprising the following steps.

[0007] S1. Acquire the medical image to be enhanced, and perform size normalization, intensity normalization, and channel alignment processing to obtain a standardized input image.

[0008] S2. Use a multi-scale coding network to extract hierarchical medical image features and obtain structural, texture and semantic representations at different resolutions.

[0009] S3. Starting with the lowest resolution encoded features, a step-by-step upsampling decoding process is constructed from low to high resolution. In the decoding stage, the wavelet structured state propagation module WSSP Block is introduced to enhance the structure and boundary details of medical images through local context preprocessing, Haar wavelet subband state propagation and low-frequency guided subband refinement.

[0010] S4. In the low-resolution jump path, a scale-selective frequency domain-guided cross-scale feature alignment module FGSS is introduced to achieve cross-scale alignment of structural information at the encoding end and semantic information at the decoding end through Fourier domain spectral mixing.

[0011] S5. Construct a composite supervised loss function to constrain model training from aspects such as pixel-level structural consistency, regional overlap consistency, and multi-scale auxiliary supervision.

[0012] After the above steps, the output module generates medical images or medical image features with clear structure, continuous boundaries, and enhanced texture details.

[0013] Preferably, the input to the wavelet structured state propagation module is medical image features. First, the medical image features are normalized and channel projected to obtain... ;in This indicates an input projection operation. Indicates feature branches, This represents the gated branch; subsequently, local context preprocessing is performed on the feature branch to obtain context-enhanced features. ;in Represents a local space operator. This indicates that it can learn group gating. This represents the learnable scaling factor. This represents the Sigmoid function. Represents a non-linear activation function. This represents element-wise multiplication; through this local context preprocessing, each spatial location obtains neighborhood spatial information before the orientation state propagation, reducing the impact of scanning order on feature representation.

[0014] Preferably, the wavelet structured state propagation module enhances contextual features. Anti-aliasing treatment was performed to obtain ;in Represents the anti-aliasing operator; if If the height or width of the space is odd, then fill it on the right or bottom boundary to obtain... ; then on Perform Haar wavelet decomposition to obtain ;in Represents Haar wavelet transform, This indicates a low-frequency sub-band. Indicates the horizontal high-frequency sub-band. Indicates the vertical high-frequency sub-band. This indicates a diagonal high-frequency sub-band.

[0015] Preferably, the wavelet structured state propagation module performs differentiated state propagation for different wavelet subbands; let Let the four-directional scan set of the low-frequency structure subband be represented by . Let represent the bidirectional scan set of the horizontal high-frequency subbands, and let Represents the bidirectional scan set of vertical high-frequency subbands; for The subband response after propagation is ;in Indicates the first Individual belt along direction State propagation operation, This represents the operation of restoring the sequence response to its spatial layout; the state propagation parameters corresponding to different wavelet subbands are learned independently to adapt to the structural representation requirements of different frequency components.

[0016] Preferably, for diagonal high-frequency subbands The local detail enhancement path is used for processing to obtain... ;in This represents the depth local enhancement operator. This represents a bounded learnable coefficient; this local detail enhancement path preserves diagonal texture, corner variations, and local residual information in medical images.

[0017] Preferably, the low-frequency guided subband refinement module responds with the propagated low-frequency structure. and enhanced diagonal high-frequency response As input, gating information is generated to adjust the directional high-frequency subband. ;in This represents a low-frequency structure modulation operator. This represents the diagonal high-frequency offset projection operator; subsequently, the horizontal and vertical high-frequency responses are adjusted using the gating information to obtain... and ;in and This represents a bounded, learnable scaling factor; through this low-frequency guiding process, the high-frequency boundary response is constrained by low-frequency structural information, thereby enhancing the stability of boundaries, contours, and fine structures.

[0018] Preferably, after refining the sub-bands guided by low frequency, , , and Input the inverse Haar wavelet transform to obtain the reconstructed features. If boundary filling was performed before Haar wavelet decomposition, then... Trim the dimensions to restore the original space. Subsequently, the recovery operator was used to obtain... Finally, the output of the wavelet structured state propagation module is obtained by projecting the gated residual output. .

[0019] Preferably, the scale-selective frequency-domain guided cross-scale feature alignment module uses encoded end features. and the features of the decoded end after upsampling As input; firstly, the two are aligned in feature space using lightweight channel projection to obtain and Then, a two-dimensional Fourier transform is performed on the projected features to obtain... and According to the normalized frequency radius Constructing radial low-frequency priors ;in By combining the radial low-frequency prior with learnable channel gating, frequency-aware interpolation coefficients are obtained. ;in This indicates learnable channel-level spectrum gating; the encoded spectrum and the decoded spectrum are weighted and mixed according to the frequency-aware interpolation coefficients to obtain... For mixed spectrum Perform an inverse Fourier transform to obtain the frequency-domain aligned spatial features. Subsequently, frequency-domain guided jumper enhancement features are generated through residual correction. ;in This indicates learnable residual gating.

[0020] Preferably, during the model training phase, given an input medical image... and their corresponding pixel-level structural reference labels or region structural reference labels The main output result is generated by the output module. It employs a composite supervision loss consisting of cross-entropy loss and Dice loss. The cross-entropy loss is... Dice loss is ;in Indicates the number of pixels. Indicates the number of categories. Indicates the first The pixel belongs to the first Class annotation value, Indicates the first The pixel belongs to the first Predicted probability of class Represents the numerical stability constant; the main prediction loss is ; Set auxiliary output in at least one decoding stage and through Provide auxiliary supervision; the ultimate training objective is... ,in The auxiliary loss weights are indicated; the above training supervision is used to improve the ability of enhanced features to express the structure, boundaries and local texture of medical images, and no annotation information is required in the inference application stage.

[0021] In one embodiment, the medical image enhancement method can be implemented through a network structure including an image acquisition and preprocessing module, a multi-scale coding module, a local context preprocessing module, a wavelet structured state propagation module, a low-frequency guided subband thinning module, a scale-selective frequency domain guided cross-scale feature alignment module, and an output module. The image acquisition and preprocessing module acquires the medical image to be enhanced and performs normalization processing; the multi-scale coding module extracts multi-scale features of the medical image; the local context preprocessing module injects neighborhood context information before directional state propagation; the wavelet structured state propagation module performs wavelet decomposition and subband state propagation; the low-frequency guided subband thinning module adjusts the high-frequency detail response using low-frequency structures; the scale-selective frequency domain guided cross-scale feature alignment module performs Fourier domain spectral mixing and jumper feature alignment; and the output module generates the enhanced medical image or medical image features.

[0022] Compared with existing technologies, the beneficial effects of this invention are as follows: First, this invention introduces local context preprocessing before directional state propagation, enabling medical image features to obtain neighborhood spatial information before serialization modeling, which helps improve the enhancement stability of boundary and fine structural regions; Second, this invention uses Haar wavelet decomposition to separate low-frequency structural information from high-frequency detail information, and adopts differentiated state propagation strategies for different sub-bands, improving the collaborative expression ability of overall structure and local details; Third, this invention uses low-frequency structural response to refine and adjust directional high-frequency response, making the enhancement results have better continuity in boundary, contour, and local structural change regions; Fourth, this invention introduces frequency domain-guided cross-scale feature alignment in low-resolution jump paths, and calibrates the structural information at the encoding end and the semantic information at the decoding end through radial low-frequency prior and learnable channel gating, improving structural consistency while controlling computational overhead; Fifth, this invention can be applied to medical imaging scenarios such as dermoscopy images, magnetic resonance images, computed tomography images, ultrasound images, or pathological images, providing technical support for medical image preprocessing, structural detail enhancement, lesion analysis, organ structure recognition, and intelligent assisted diagnosis. Attached Figure Description

[0023] Figure 1 This is a flowchart of a medical image enhancement method based on frequency-structured state modeling provided by the present invention.

[0024] Figure 2 This is the overall network structure diagram of WSSP Net provided by the present invention.

[0025] Figure 3 This is a structural diagram of the Visual State Space Module (VSS Block) provided by the present invention.

[0026] Figure 4 This is a network structure diagram of the wavelet structured state propagation module WSSP Block provided by the present invention.

[0027] Figure 5 This is a network structure diagram of the low-frequency guided subband refinement module LFSR provided by the present invention.

[0028] Figure 6 This is a structural diagram of the scale-selective frequency domain guided cross-scale feature alignment FGSS module provided by the present invention. Detailed Implementation

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention.

[0030] Please see Figures 1 to 6 This invention provides a medical image enhancement method based on frequency-structured state modeling. It aims to synergistically enhance structural information, boundary details, and texture responses in medical images through local context preprocessing, wavelet-structured state propagation, low-frequency guided subband thinning, and scale-selective frequency-domain guided cross-scale feature alignment. Figure 1 Demonstrate the overall methodology and process. Figure 2 Demonstrates the overall network structure of WSSP Net. Figure 3 Demonstrates the structure of the Visual State Space Block (VSS Block) at the encoding end. Figure 4 Show the overall structure of the WSSP Block. Figure 5 exhibit Figure 4 The expanded structure of the LFSR module in the middle, Figure 6 Demonstrates the FGSS module structure.

[0031] In this embodiment, the input medical image can be a dermoscopy image, magnetic resonance image, computed tomography image, ultrasound image, or pathological image. First, the medical image is adjusted to a preset input size and intensity is normalized to ensure that medical images from different sources have a relatively consistent numerical distribution. For single-channel medical images, the required number of channels for network input can be converted by copying channels or channel projection. For multi-channel medical images, they can be directly input or aligned through linear mapping.

[0032] In this embodiment, as Figure 2 As shown, WSSP Net employs an encoder-decoder structure; the encoder extracts multi-scale medical image features, and the decoder progressively restores spatial details and enhances structural representation; the encoding end can use a Visual State Space Block (VSS Block) for feature extraction, as shown in the figure. Figure 3 As shown, it includes a normalization layer, a linear projection layer, a deep convolutional layer, a state space propagation layer, a gated branch, and an output projection layer; this module achieves efficient long-range dependency modeling through directional state propagation and enhances the spatial neighborhood response through local convolution.

[0033] In this embodiment, the decoding end uses a wavelet structured state propagation module (WSSP Block) as the core enhancement unit; such as Figure 4 As shown, the WSSP Block as a whole includes processes such as local context preprocessing, Haar wavelet decomposition, subband-specific state propagation, low-frequency guided subband thinning, inverse Haar wavelet reconstruction, and output projection; it should be noted that... Figure 4This section primarily showcases the overall processing flow of the WSSP Block, without delving into all internal details. The specific structure of the low-frequency guided sub-band refinement module (LFSR) is explained below. Figure 5 Further details will be provided.

[0034] Specifically, given input features First, obtain through normalization and channel projection. ; then on Perform local context preprocessing to obtain In this embodiment, Depth can be used Convolution is implemented by dividing the channels into several groups and passing them through... Group gating is performed; this operation enables each spatial location to aggregate surrounding neighborhood information before the state propagation in the direction of entry, thereby improving the context awareness of medical image boundaries and small structural regions.

[0035] Furthermore, anti-aliasing processing is applied to the context-enhanced features to obtain... In this embodiment, Depth can be used Convolution is implemented and can be initialized with an average smoothing kernel; if the height or width of the feature map is odd, padding is applied to the right or bottom boundary, resulting in... Then, Haar wavelet decomposition was performed to obtain... ;in It mainly represents low-frequency structural information. and Primarily represents directional boundary details. It mainly represents diagonal texture, corner points, and local residual information.

[0036] Furthermore, for low-frequency structural subbands Four-directional state propagation is used to model a large range of structural dependencies; for the horizontal high-frequency subband A two-way state propagation in the horizontal direction is employed to enhance the lateral boundary response; for the vertical high-frequency subband... Vertical bidirectional state propagation is employed to enhance the longitudinal boundary response; the propagated subband response is expressed as... ,in For diagonal high-frequency sub-bands, a local enhancement path is adopted. This is to avoid unnecessary long-sequence propagation of highly localized residual information.

[0037] Furthermore, such as Figure 5 As shown, the low-frequency guided subband refinement module LFSR generates adjustment gating using the low-frequency structural response and the diagonal high-frequency response; specifically, it first calculates... ; and then utilize and By adjusting the horizontal and vertical high-frequency responses respectively, the following results were obtained. and This module constrains high-frequency detail responses by using low-frequency structural information, making the enhanced boundaries and contours more continuous and reducing the interference of local noise or pseudo-edges on the high-frequency response.

[0038] Furthermore, , , and Input the inverse Haar wavelet transform to obtain If boundary filling was performed in the aforementioned steps, then... Cut to the original space dimensions to obtain Then, the recovery operator is used to obtain... And obtained by projecting the gated residual output. ;in Enhanced features for WSSP Block output.

[0039] In this embodiment, as Figure 6 As shown, the Scale-Selective Frequency Domain Guided Cross-Scale Feature Alignment Module (FGSS) operates on low-resolution jump paths; this module first performs feature alignment at the encoding end... and decoding end features Perform channel alignment to obtain and Then, through a two-dimensional Fourier transform, we obtain... and Subsequently, a radial low-frequency prior was constructed. And combined with learnable channel gating to obtain Finally, spectral mixing is performed. And obtained through inverse Fourier transform and residual correction ;in This module calibrates the structural information at the encoding end and the semantic information at the decoding end in the frequency domain, avoiding structural mismatch caused by direct spatial fusion.

[0040] In this embodiment, FGSS is only applied to low-resolution jumper paths, such as the third and second stages; for higher-resolution jumper paths, direct addition or no additional frequency domain calculation can be used; this can achieve effective spectrum alignment in the low-resolution stage with large semantic differences, and avoid introducing excessive Fourier calculation overhead in the high-resolution stage.

[0041] S5. Construct a composite supervised loss function to constrain model training from aspects such as pixel-level structural consistency, region overlap consistency, and multi-scale auxiliary supervision; in this embodiment, the main output result is... with annotation The losses between are ,in Meanwhile, auxiliary outputs are set in the last few decoding stages. and calculate The ultimate training objective is In one alternative embodiment, , Auxiliary outputs are used only during the training phase and removed during the inference phase.

[0042] After the model training is completed, the medical image to be enhanced is input into the trained network, and the output module generates a medical image or medical image feature with clear structure, continuous boundaries and enhanced texture details.

[0043] In this embodiment, the method can be implemented using the Python language and the PyTorch deep learning framework; the input image size can be set to... The batch size can be set to 32, the optimizer can be the AdamW optimizer, and the learning rate can be set to... The number of training rounds can be set to 300. The above parameters are only one optional implementation method. Those skilled in the art can adjust the input size, number of network channels, number of training rounds, optimizer and loss weight according to medical image modality, data scale, equipment conditions and application requirements.

[0044] The above are merely preferred embodiments of the present invention. It should be noted that, for those skilled in the art, without departing from the inventive concept of the present invention, various modifications or improvements can be made to the number of network layers, the number of channels, the direction of state propagation, the type of wavelet transform, the form of spectral prior, the combination of loss functions, and the type of applied medical images. All such modifications or improvements fall within the protection scope of the present invention.

Claims

1. A medical image enhancement method based on frequency-structured state modeling, characterized in that, Includes the following steps: S1. Obtain the medical image to be enhanced, and perform size normalization and intensity normalization processing on the medical image to be enhanced to obtain the input image; S2. Construct an encoding network to extract multi-scale features from the input image, obtaining medical image features at multiple scales. S3. Construct a wavelet structured state propagation module. Enhancement of features in the lowest resolution medical images yields In the wavelet structured state propagation module, differentiated state propagation is performed on different wavelet subbands; let Let the four-directional scan set of the low-frequency structure subband be represented by . Let represent the bidirectional scan set of the horizontal high-frequency subbands, and let Represents the bidirectional scan set of vertical high-frequency subbands; for The subband response after propagation is ;in Indicates the first Individual belt along direction State propagation operation, This represents the operation of restoring the sequence response to its spatial layout; for diagonal high-frequency subbands The local detail enhancement path is used for processing to obtain... ;in This represents the depth local augmentation operator. The bounded learnable coefficients are represented; the state propagation parameters corresponding to different wavelet subbands are learned independently to adapt to the structural expression requirements of different frequency components; in the wavelet structured state propagation module, the directional high-frequency response is adjusted using a low-frequency structure-guided subband detail adjustment module; the low-frequency structure-guided subband detail adjustment module uses the propagated low-frequency structural response as the basis for the adjustment. and enhanced diagonal high-frequency response As input, gating information is generated to adjust the directional high-frequency subband. ;in This represents a low-frequency structure modulation operator. This represents the diagonal high-frequency offset projection operator; subsequently, gating information is used to adjust the horizontal and vertical high-frequency responses to obtain... and ;in and This represents a bounded learnable scaling factor; S4. Decoding enhancement is performed in order from low resolution to high resolution. For the first... In each decoding stage, the features decoded in the previous layer are first upsampled to obtain... Then, based on the current scale, frequency-domain guided cross-scale feature alignment is selected. Alternatively, a direct jump-to-fusion can be used to obtain the stage input. The enhanced decoding features are obtained through the wavelet structured state propagation module. ; S5. Input the highest resolution decoded features into the output module to generate medical images or medical image features with enhanced structure, boundary and texture details.

2. The medical image enhancement method based on frequency-structured state modeling according to claim 1, characterized in that, In step S4, the stage input The construction method is as follows: when At that time, the frequency-domain guided cross-scale feature alignment module was used to obtain ;when At that time, direct jump connection fusion was used to obtain ;in This represents a frequency-domain guided cross-scale feature alignment operation. Indicates the first Encoding end features at each scale, Indicates the first Features of the upsampled decoding end at each scale.

3. The medical image enhancement method based on frequency-structured state modeling according to claim 1, characterized in that, The input to the wavelet structured state propagation module is medical image features. First, the medical image features are normalized and channel projected to obtain... ;in This indicates an input projection operation. Indicates feature branches, Indicates a gated branch; Subsequently, local context preprocessing is performed on the feature branches to obtain context-enhanced features. ;in Represents a local space operator. This indicates that it can learn group gating. This represents the learnable scaling factor. This represents the Sigmoid function. Represents a non-linear activation function. This represents element-wise multiplication; through the above local context preprocessing, each spatial location obtains neighborhood spatial information before the direction state propagation.

4. The medical image enhancement method based on frequency-structured state modeling according to claim 1, characterized in that, In the wavelet structured state propagation module, context enhancement features are added. Anti-aliasing treatment was performed to obtain ;in Represents the anti-aliasing operator; if If the height or width of the space is odd, then fill it on the right or bottom boundary to obtain... ; then on Perform Haar wavelet decomposition to obtain ;in Represents Haar wavelet transform, This indicates a low-frequency sub-band. Indicates the horizontal high-frequency sub-band. Indicates the vertical high-frequency sub-band. This indicates a diagonal high-frequency sub-band.

5. A medical image enhancement method based on frequency-structured state modeling according to claim 1, characterized in that... Will , , and Input the inverse Haar wavelet transform to obtain the reconstructed features. ; If boundary filling is performed before Haar wavelet decomposition, then for Trim the dimensions to restore the original space. Subsequently, the recovery operator was used to obtain... Finally, the output of the wavelet structured state propagation module is obtained by projecting the gated residual output. .

6. The medical image enhancement method based on frequency-structured state modeling according to claim 1, characterized in that, Frequency domain-guided cross-scale feature alignment module with encoding end features and the features of the decoded end after upsampling As input; firstly, the two are aligned in feature space using lightweight channel projection to obtain and Then, a two-dimensional Fourier transform is performed on the projected features to obtain... and According to the normalized frequency radius Constructing radial low-frequency priors ;in By combining radial low-frequency priors with learnable channel gating, frequency-aware interpolation coefficients are obtained. ;in This indicates learnable channel-level spectrum gating; the encoded and decoded spectra are weighted and mixed based on frequency-aware interpolation coefficients to obtain... For mixed spectrum Perform an inverse Fourier transform to obtain the frequency-domain aligned spatial features. Subsequently, frequency-domain guided jumper enhancement features are generated through residual correction. ;in This indicates learnable residual gating.

7. The medical image enhancement method based on frequency-structured state modeling according to claim 1, characterized in that, During the model training phase, given an input medical image... and its corresponding pixel-level structural annotations The main prediction result is generated by the output module. It employs a composite supervision loss consisting of cross-entropy loss and Dice loss. The cross-entropy loss is... Dice loss is ;in Indicates the number of pixels. Indicates the number of categories. Indicates the first The pixel belongs to the first Class annotation value, Indicates the first The pixel belongs to the first Predicted probability of class Represents the numerical stability constant; the main prediction loss is ; Set auxiliary output in at least one decoding stage and through Provide auxiliary supervision; the ultimate training objective is... ,in This represents the auxiliary loss weight.

8. The medical image enhancement method based on frequency-structured state modeling according to claim 1, characterized in that, The medical images include at least one of dermoscopy images, magnetic resonance images, computed tomography images, ultrasound images, or pathological images; the enhanced medical images or medical image features are used for medical image preprocessing, structural detail enhancement, lesion area analysis, organ structure recognition, assisted diagnosis, or subsequent intelligent visual analysis tasks.

Images