A medical image segmentation method based on adaptive weight skip connection

Abstract

The application discloses a medical image segmentation method based on adaptive weight skip connection. By introducing the transmission feature attention mechanism of adaptive weight adjustment, the application can dynamically adjust the feature transmission proportion of the skip connection, optimize the utilization efficiency of important features, and reduce the influence of invalid features. The recognition and segmentation accuracy of the model for the edema area in the fundus image are improved, and the overall influence of the features in the model is optimized. The transmission feature attention mechanism of adaptive weight adjustment enhances the adaptability of the network model to different data structures, and improves the compatibility and robustness of the model.

Description

Technical Field

[0001] The present invention belongs to the fields of medical image processing, deep learning, and signal processing technology, and specifically relates to a medical image segmentation method based on adaptive weighted jump connections. Background Art

[0002] In the field of fundus edema segmentation using optical coherence tomography (OCT), the application of traditional image processing techniques and deep learning models has primarily focused on basic methods. Specifically, traditional image processing methods such as thresholding and region growing rely on manually set parameters and thresholds, which somewhat limits their flexibility and adaptability. While these methods can achieve basic image segmentation, they often lack effective handling of complex or inconsistent image features.

[0003] With the rapid advancement of deep learning technology, neural network models have demonstrated outstanding performance in the field of medical image analysis. The DconnNet network model, through its dual decoder architecture and efficient SDE module, effectively captures image features while retaining important contextual information, offering significant advantages in improving segmentation accuracy and adaptability, thereby enhancing the efficiency and accuracy of medical image segmentation.

[0004] In the processing of optical coherence tomography (OCT) fundus images, although the DconnNet model has shown excellent performance, existing technologies still face a series of challenges when processing OCT fundus images. First, since OCT images are affected by light scattering and light absorption, they often show high noise and low contrast, and small target areas. These factors make it difficult for traditional DconnNet models to identify and segment subtle edema areas. Secondly, OCT images obtained from different regions and institutions vary in quality and features. When processing these images, the existing DconnNet model lacks detailed attention and adjustment to local areas, and fails to effectively adapt to the feature levels of different images, which limits its performance in detail segmentation accuracy. Summary of the Invention

[0005] In the field of medical imaging, accurately segmenting edema areas in fundus optical coherence tomography (OCT) images is particularly important for the diagnosis and treatment of ophthalmic diseases. However, existing image processing technologies face many challenges when processing such images, such as uneven image quality, difficulty in capturing details, and complex and variable feature information. These issues significantly affect the accuracy of segmentation and the overall reliability of the system. To overcome these limitations and improve system performance, this paper proposes a medical image segmentation method based on adaptive weighted jump connections, aiming to improve the accuracy and efficiency of edema area segmentation in OCT images.

[0006] The present invention aims to improve the segmentation accuracy and contrast of edema areas in optical coherence tomography (OCT) fundus images and solve key technical problems in the prior art. By introducing a transmission feature attention mechanism with adaptive weight adjustment, the present invention can dynamically adjust the feature transmission ratio of jump connections, optimize the utilization efficiency of important features, and reduce the impact of invalid features. This method improves the model's recognition and segmentation accuracy of edema areas in fundus images, while optimizing the overall impact of features in the model. The transmission feature attention mechanism with adaptive weight adjustment enhances the adaptability of the network model to different data structures and improves the compatibility and robustness of the model. In response to the limitations of the prior art, the present invention provides a new solution that effectively improves segmentation accuracy, compatibility and robustness.

[0007] The core technology of the present invention involves the use of an adaptive weight jump connection module for the jump connection part in the deep learning model DconnNet. The traditional DconnNet model effectively utilizes the high-level features of the encoding layer in different decoding layers through its symmetrical encoding and decoding structure, but the utilization of these features still lacks flexibility and adaptability. The present invention introduces an adaptive weight jump connection module, which greatly enhances the responsiveness to key features by dynamically adjusting the weights of feature channels, thereby improving the efficiency and accuracy of feature transmission. In addition, the module can automatically adjust the weight distribution according to the importance of dimensional features, so that the network can more accurately focus on information that is more critical to the task.

[0008] By applying this technology, the present invention can significantly improve the segmentation accuracy of edema areas in low-contrast and detail-rich OCT images. Experimental results also verify the advantages of the present invention in segmentation accuracy and processing efficiency compared to traditional methods, especially when processing detailed and contrasting areas in the image, showing higher accuracy and robustness.

[0009] A medical image segmentation method based on adaptive weighted skip connection comprises the following steps:

[0010] Step 1: Data acquisition and preprocessing:

[0011] Step 2: Extract image features at different scales through the encoder of the medical image segmentation model:

[0012] Step 3: Skip connection and adaptively adjust attention through adaptive weight skip connection module:

[0013] Step 4: Complete image segmentation through the decoding path decoder combined with the skip connection parameters to obtain the final segmentation output.

[0014] Step 5: Train the medical image segmentation model based on adaptive weight skip connection;

[0015] Step 6: Use the trained model for inference, that is, predict the edema area on new fundus OCT images.

[0016] Furthermore, the specific method of step 1 is as follows:

[0017] Raw OCT images acquired through the OCT system are first normalized and resized to suit subsequent processing requirements. All images are uniformly processed to 512x512 pixels, with three RGB channels. Areas of fundus edema within the images are annotated to generate labeled images. All labeled images are stored in a 512x512 pixel, single-channel format, with each raw OCT image corresponding to a labeled image.

[0018] Furthermore, the specific method of step 2 is as follows:

[0019] After the data is input into the medical image segmentation model based on adaptive weighted skip connections, basic preprocessing, including normalization and denoising, is performed on the image data to ensure the quality and consistency of the input data. The preprocessed image first passes through the encoder structure of a UNet network. This encoder is an encoder contraction path encoder that gradually reduces the spatial dimension through convolutional layers and pooling layers while increasing the depth of features. Each encoder layer extracts corresponding feature data corresponding to image features at different scales. The feature information at the bottom layer of the encoder is transmitted to the decoding path through an SDE module. The SDE module is a neural network module that decouples feature information and directional information.

[0020] Furthermore, the specific method of step 3 is as follows:

[0021] On the encoder path, the extracted feature data of each layer is transmitted to the decoder of the corresponding layer through the jump connection. For the feature data k at the top layer of the encoder, it is input into the adaptive weight jump connection module together with the feature data d at the bottom layer of the decoder for adaptive adjustment, and then transmitted to the decoder of the corresponding layer. The adaptive adjustment is as follows: first, the dimension of d is compressed to 3 dimensions through the view function, and the first two dimensions (B, C) are fixed. d ) dimension, and automatically merge the 3rd and 4th dimensions. Then use the reshape function to change the dimension of k to 3 dimensions, merge the two dimensions through H×W, and transform the three dimensions. Then let the obtained d flat and k flat With the matrix W s Do a dot multiplication. Get the corresponding weight parameter through the softmax function, and k flatAfter multiplication, the view_as function is used to adjust the dimension to the same as k to obtain the final output k of the adaptive weight jump connection module. attend .

[0022] The specific formula is as follows:

[0023] d flat =d.view(B,C d ,-1)

[0024] k flat =k.reshape(C k ,H×W).permute(1,2,0)

[0025] k attend =softmax(d flat W s ·k flat )·k flat .view_as(k)

[0026] The symbols represent the following:

[0027] d: represents the feature data at the bottom layer of the decoder.

[0028] B: The abbreviation for Batchsize, which represents the batch size during training.

[0029] C d : is the number of channels of d.

[0030] view: A function in PyTorch that changes the shape of a tensor without changing its data content. The purpose here is to merge the height and width dimensions of d to form a three-dimensional tensor.

[0031] d flat : Represents the feature vector of the lowest-level feature parameters of the decoder after dimensional transformation, from four dimensions to three dimensions.

[0032] k: represents the top-level feature data of the encoder, that is, the original skip connection parameters.

[0033] C k : is the number of channels of k.

[0034] H: is the height dimension of d and k.

[0035] W: is the width dimension of d and k.

[0036] reshape: A PyTorch function that also changes the shape of a tensor without changing its data content, and can handle non-contiguous tensors. Here, the purpose is to rearrange the dimensions of k, placing the number of channels in the first dimension, the height in the second, and merging the width and batch size into the final dimension.

[0037] permute: A function used to rearrange the dimensions of a tensor, changing the order of the dimensions.

[0038] k flat : Represents the three-dimensional vector of the feature parameters transmitted by the jump connection at the top layer of the encoder after dimension transformation.

[0039] Softmax: It is a normalized exponential function commonly used in deep learning. It is usually used to convert a set of numerical values ​​into a probability distribution, where the ratio of the exponent of each numerical value to the sum of the exponents of all numerical values ​​is used as the output probability.

[0040] view_as: is a function in Pytorch that changes the shape of a tensor to be consistent with the tensor in the brackets without changing its data content.

[0041] W s : An adaptive matrix of size (H / W, H / W*B) that satisfies k flat and d flat The matrix multiplication is used to automatically adjust the transmission ratio of k. The matrix is ​​randomly assigned when the network is initialized and then optimized through backpropagation and gradient descent during network training.

[0042] k attend : The final output of this module, used to replace the original skip connection parameter k.

[0043] The adaptive weighted skip connection module adaptively adjusts the ratio of top-level feature data transmitted via skip connections. This allows the skip connections at this layer to no longer simply transmit features at a constant ratio, but to adjust based on the importance of feature dimensions. More important dimensional features are assigned higher weights, resulting in a larger proportion of data being transmitted to the decoder, optimizing network performance.

[0044] Furthermore, the specific method of step 4 is as follows:

[0045] The decoding path of the medical image segmentation model employs an interactive feature-space decoder (IFD). In the IFD, each decoding layer consists of a set of feature modules (FeatureBlocks) and spatial modules (SpaceBlocks), with feature parameters propagated upward alternately between them. FeatureBlocks focus on feature information, while SpaceBlocks focus on spatial information. These two modules are propagated upward alternately to form an interactive feature-space decoder, which integrates feature information with directional information. The interactive feature-space decoder gradually restores the spatial resolution of the image through upsampling within the module. It then introduces feature data transmitted through skip connections into the feature modules of the corresponding layers. Image features at different scales are concatenated with the output features of the spatial modules of the corresponding decoding layers and then used as input to the feature modules. This effectively integrates high-level semantic information with spatial details, optimizing the resulting feature map. After the output of the feature modules in the last layer of the IFD, the output feature data is aggregated through the region-guided channel of the bilateral voting module to produce the final segmentation output.

[0046] Furthermore, the specific method of step 5 is as follows:

[0047] During the training process, the training data labeled in step 1 is used to train the medical image segmentation model based on adaptive weight jump connection. The medical image segmentation model is used to identify and predict the edema area in the unseen fundus OCT images. The training process uses a two-dimensional cross entropy loss function to evaluate the difference between the model output and the actual annotation, and continuously optimizes the model weights through backpropagation, gradient descent, dynamic learning rate adjustment and regularization techniques. At the same time, the weight matrix W s The parameters in will be adjusted and changed as the training progresses, and finally the model effect will be optimized.

[0048] Furthermore, the specific method of step 6 is as follows:

[0049] The image to be segmented is standardized and resized, processed to 512x512 pixels and RGB three channels, and then input into the trained medical image segmentation model for segmentation. Finally, the segmentation result is generated and the image predicted to be the edema part is output.

[0050] The beneficial effects of the present invention are as follows:

[0051] The core technology of this invention lies in the introduction of an adaptive weighted skip connection module to process the data transmitted in the skip connection. Through this module, feature data of different dimensions are adaptively adjusted in the transmission ratio during the skip connection process, so that more important dimensional features are transmitted to the decoder part with higher weights. This processing step is particularly important for improving the network's sensitivity to image detail features and segmentation accuracy, especially when processing OCT image data with different dimensional features and noise. Compared with the existing technology, this method uses an adaptive matrix to guide feature transmission, which helps to better utilize the dimensional features in the skip connection process, thereby improving segmentation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Schematic diagram of the medical image segmentation model structure.

[0053] Figure 2 Workflow for medical image segmentation models.

[0054] Figure 3 Schematic diagram of the specific working process of the adaptive skip connection module. DETAILED DESCRIPTION

[0055] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0056] Glossary

[0057] Optical Coherence Tomography (OCT) is an imaging technology that uses the principle of light reflection to obtain high-resolution three-dimensional images of tissue interiors by measuring the time delay and intensity of light reflected from tissue. OCT is widely used in fields such as ophthalmology and dermatology, particularly in fundus examinations, enabling detailed imaging of ocular structures such as the retina.

[0058] DconnNet is an innovative neural network architecture specifically designed to improve the accuracy of anatomical consistency in medical image segmentation tasks. Through a unique approach—directional connectivity modeling—DconnNet addresses the problem of traditional connectivity networks ignoring the rich channel direction information. DconnNet's core technology decouples the directional subspace from the shared latent space and tracks and utilizes this directional information, significantly enhancing the feature representation capabilities within the network.

[0059] Skip connections are a technique designed specifically to prevent vanishing gradients in deep learning networks. The skip connection structure allows information to be passed directly from earlier layers of the network to later layers, adding a "bypass" to traditional sequential connections to achieve a direct flow of feature information. This design effectively addresses the training challenges that arise with increasing network depth, enabling the model to better handle information transfer within deep networks without losing contextual detail. The introduction of skip connections has been widely used in various deep learning architectures, demonstrating excellent performance in image recognition and speech processing, particularly in areas that require processing complex or deep structures.

[0060] Specifically, if Figure 1 As shown, the present invention provides an improved OCT network model, specifically a medical image segmentation model, that integrates an adaptive weighted skip connection module within its skip connections to enhance the automatic segmentation of edema areas in fundus OCT images. This technology is particularly suitable for medical image processing tasks requiring high-precision and efficient fundus edema detection.

[0061] The core of the present invention is to introduce an adaptive weighted jump connection module, which is designed to assign adaptive weights to the characteristic parameters transmitted by the input jump connection, so as to strengthen the network's adjustment and absorption of different dimensional features of edema and improve the edema segmentation capability. Figure 2 As shown, the specific implementation process is as follows:

[0062] Preprocessing: The publicly available OIMHS dataset, containing 10,908 image pairs for training and 2,700 pairs for validation, was used. Raw OCT images were first normalized and resized to accommodate subsequent processing. All images were uniformly processed to 512x512 pixels with RGB channels.

[0063] Network Encoding and Decoding Processing: The transformed image data first undergoes basic preprocessing, including normalization and denoising, to ensure input data quality and consistency. The preprocessed image is then fed into the encoder, a four-layer UNet encoder-contraction path encoder. Each encoder layer consists of a max pooling layer and two sets of convolutional layers. The convolutional layers consist of a two-dimensional convolutional layer, a batch normalization layer, and a Reinforced Luminance (ReLU) function. Each encoder layer progressively reduces the spatial dimension while increasing the feature depth. Simultaneously, each encoder layer extracts feature data corresponding to image features at different scales. Feature information at the bottom of the encoder is transferred to the decoder through an SDE module. The SDE module is a neural network module that decouples feature information from directional information. It consists of an embedding compression (ES) module and a manifold projection and enhancement (MPE) module. The ES module converts the input directional enhancement map r' into a directional embedding n', which serves as a high-level directional representation. Then, using n' as the input of MPE, the main feature map d' and the direction embedding n' are projected onto the shared manifold through two 1×1 convolutional projectors. Next, the category-direction relationship is modeled using inter-channel dot products and sigmoid activation by calculating the similarity between the projected direction embedding n'0 and the feature map d'0. Through the dot product operation, the direction information is fused into the re-encoded original feature map d" to generate a new enhanced direction map r'7, which is used as the input of the feature stream and the generation of the direction embedding n'7 of the next layer. The result of the SDE module is that the direction information is effectively fused into the feature map, generating a new enhanced direction map r'7, which is used as the input of the feature stream and the generation of the direction embedding n'7 of the next layer. In this way, the SDE module achieves the effective fusion and transmission of direction information in the feature map, improving the performance of the model in medical image segmentation.

[0064] like Figure 3 As shown in the figure, the adaptive jump connection module: on the encoder path, the extracted feature data of each layer is transmitted to the decoder part of the same layer. At the same time, an adaptive weight jump connection module is added to the top layer, and the feature data is first input into the module before being input into the decoder part. In this module, the top encoder feature data and the bottom decoder feature data are input, denoted as k and d respectively. First, the two are dimensionally compressed, and their dimensions are converted from four dimensions to three dimensions. The dimension of d is first compressed to three dimensions through the view function, and the first two dimensions (B, C) are fixed. d ) dimension, and automatically merge the 3rd and 4th dimensions. Then use the reshape function to change the dimension of k to 3 dimensions, merge the two dimensions through H×W, and transform the three dimensions. Then let the obtained d flat and k flatWith the matrix W s Do a dot multiplication. Get the corresponding weight parameter through the softmax function, and k flat After multiplication, the view_as function is used to adjust the dimension to the same as k to obtain the final output k of the adaptive weight jump connection module. attend .

[0065] The specific formula is as follows:

[0066] d flat =d.view(B,C d ,-1)

[0067] k flat =k.reshape(C k ,H×W).permute(1,2,0)

[0068] k attend =softmax(d flat W s ·k flat )·k flat .view_as(k)

[0069] The symbols represent the following:

[0070] d: represents the feature data at the bottom layer of the decoder.

[0071] B: The abbreviation for Batchsize, which represents the batch size during training.

[0072] C d : is the number of channels of d.

[0073] view: A function in PyTorch that changes the shape of a tensor without changing its data content. The purpose here is to merge the height and width dimensions of d to form a three-dimensional tensor.

[0074] d flat : Represents the feature vector of the lowest-level feature parameters of the decoder after dimensional transformation, from four dimensions to three dimensions.

[0075] k: represents the top-level feature data of the encoder, that is, the original skip connection parameters.

[0076] C k : is the number of channels of k.

[0077] H: is the height dimension of d and k.

[0078] W: is the width dimension of d and k.

[0079] reshape: A PyTorch function that also changes the shape of a tensor without changing its data content, and can handle non-contiguous tensors. Here, the purpose is to rearrange the dimensions of k, placing the number of channels in the first dimension, the height in the second, and merging the width and batch size into the final dimension.

[0080] permute: A function used to rearrange the dimensions of a tensor, changing the order of the dimensions.

[0081] k flat : Represents the three-dimensional vector of the feature parameters transmitted by the jump connection at the top layer of the encoder after dimension transformation.

[0082] Softmax: It is a normalized exponential function commonly used in deep learning. It is usually used to convert a set of numerical values ​​into a probability distribution, where the ratio of the exponent of each numerical value to the sum of the exponents of all numerical values ​​is used as the output probability.

[0083] view_as: is a function in Pytorch that changes the shape of a tensor to be consistent with the tensor in the brackets without changing its data content.

[0084] W s : An adaptive matrix of size (H / W, H / W*B) that satisfies k flat and d flat The matrix multiplication is used to automatically adjust the transmission ratio of k. The matrix is ​​randomly assigned when the network is initialized and then optimized through backpropagation and gradient descent during network training.

[0085] k attend : The final output of this module, used to replace the original skip connection parameter k.

[0086] The final transmitted skip connection characteristic parameter is the final output k attend Replace the top-level feature module FeatureBlock in the input decoder to implement adaptive skip connection feature parameter weight adjustment and optimize network performance.

[0087] The adaptive weighted skip connection module adaptively adjusts the ratio of top-level feature data transmitted via skip connections. This allows the skip connections at this layer to no longer simply transmit features at a constant ratio, but to adjust based on the importance of feature dimensions. More important dimensional features are assigned higher weights, resulting in a larger proportion of data being transmitted to the decoder, optimizing network performance.

[0088] The decoding path of the network is called Interactive Feature-space Decoder (IFD). Within IFD, data is transmitted alternately through the spatial module SpaceBlock and the feature module FeatureBlock, forming two decoder paths for propagation.

[0089] In the spatial module SpaceBlock, the direction-enhanced feature map r' is first taken as input and converted into a direction embedding n', i.e., a high-level direction representation, through the ES module. The ES module uses a global average pooling (GAP) operation to convert the feature map r' into a direction embedding n'. Then, the direction embedding n' is taken as input and the direction representation in the main feature map d' is further enhanced through the MPE module. In the MPE module, two 1×1 convolutional projectors are first used to project the main feature map d' and the direction embedding n' onto a shared manifold at the current resolution. The results after projection are d' and n', respectively. Next, by calculating the similarity between the projected direction embedding n'0 and the feature map d'0, the channel-level dot product and sigmoid activation function are used to obtain the normalized category-direction attention map α. This attention map enhances features related to direction and suppresses features irrelevant to direction. Finally, the dot product operation is used to perform element-wise multiplication of the attention map α with the re-encoded original feature map d', effectively fusing the direction information into the feature map to obtain a new direction-enhanced feature map r'. This new feature map r' will be used as the input of the feature stream and used to generate the direction embedding n' of the next layer.

[0090] In the feature module FeatureBlock, the category-direction relationship is modeled by calculating the similarity between the projected directional embedding n' and the feature map d'. Finally, the normalized category-direction attention is calculated using the inter-channel dot product and sigmoid activation to enhance the directional information in the re-encoded original feature map d". Through this operation, the directional information is effectively fused into the feature map, generating a new enhanced directional feature map, which is used as the input of FeatureFlow and is also used to generate the directional embedding of the next layer.

[0091] The feature data transmitted through the SDE module is first transmitted to the underlying spatial module SpaceBlock. After passing through the underlying spatial module, the feature data forms two propagation paths between the two modules to realize two decoding paths. One path is that the feature data is directly transmitted between the spatial modules SpaceBlock to the upper layer, and the other path is that the feature data in the spatial module SpaceBlock is transmitted to the feature module FeatureBlock and then to the upper spatial module SpaceBlock. The two decoding paths together form an interactive feature space decoder, which can fuse feature information with directional information. The interactive feature space decoder gradually restores the spatial resolution of the image through the upsampling operation within the module. The dimensional features processed from step 3 will be introduced into the feature module FeatureBlock of each layer in different layers. These different dimensional encoder features are directly combined with the output features of the spatial module of the corresponding decoding layer through the Cat function through the splicing operation in the decoding layer as the input of the feature module, effectively fusing high-level semantic information with spatial details and optimizing the feature map results. After the output of the last layer of the interactive feature space decoder, the feature data output by the decoder is aggregated through the regional guidance channel of the bilateral voting module to obtain the final segmentation output. This step first converts the main feature map d n The final connectivity output X is obtained by integration. Each eight channels of X represents the connectivity of a category. The bilateral voting module and the region-oriented channel aggregation module are then used to obtain the final segmentation map. The calculation formula of the bilateral voting module is: j (x,y)=X 9-j (x+a,y+b)=X j (x,y)×X 9-j (x+a,y+b), Among them, j is the jth channel, a,b∈{0,±1} represents the position of adjacent pixels, X n is the Bicon map, and S is the final segmentation prediction output. This yields the probability that each pixel belongs to the edema region. This output is typically a segmentation map of the same size as the input image, where each pixel's value represents the probability that the pixel belongs to the edema region.

[0092] During training, the pre-processed labeled training data, including fundus OCT images and their corresponding edema area annotations, is used. This data is used to train the model to identify and predict edema areas in unseen fundus OCT images. Training typically uses a two-dimensional cross-entropy loss function to evaluate the discrepancy between the model output and the actual annotations. The model weights are continuously optimized through backpropagation, gradient descent, dynamic learning rate adjustment, and regularization. Specifically, background pixels in this dataset have a value of 0, while edema pixels have a value of 192. At the beginning of training, each edema pixel is assigned a probability of 1, and each background pixel is assigned a probability of 0. The forward propagation of the model outputs the corresponding probabilities. The two-dimensional cross-entropy loss function is then applied to obtain a per-pixel loss. The losses calculated for all pixels in the entire image or batch are then accumulated to obtain the final loss value. Backpropagation is used to calculate the gradient of the loss with respect to each model parameter. This gradient represents the sensitivity of the loss function to each parameter under the current model parameters. The optimizer adjusts the value of each model parameter based on the calculated gradients to optimize the model weights. Finally, by continuously iteratively updating the parameters, the minimum value of the loss function is gradually approached to achieve the optimization of the segmentation effect.

[0093] The trained model can be used for inference, predicting edema areas on new fundus OCT images. This process involves subjecting the new image to the same preprocessing, then segmenting it using the trained model, ultimately generating a segmentation result and outputting an image of the predicted edema area. In experiments, the deep learning model DconnNet achieved an 84.20% accuracy for edema segmentation on the OIMHS dataset, while a medical image segmentation model based on adaptive weight skip connections achieved an accuracy of 87.52% on the OIMHS dataset, a 3.32% improvement.

[0094] The core technology of this invention lies in the introduction of an adaptive weighted skip connection module to replace the original top-level skip connection. Through this module, the top-level feature parameters are combined with the underlying decoder features, so that the transmitted feature parameters can be adaptively adjusted and assigned appropriate weights based on the importance of the dimensional features. This processing step is particularly important for improving the network's analysis of image features and segmentation accuracy, especially when processing medical imaging data with complex textures and noise. Compared with existing technologies, this method uses adaptive attention technology to help better capture and utilize structural features in images, thereby improving segmentation performance.

[0095] The above description is a further detailed description of the present invention in conjunction with specific / preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art of the present invention may make various substitutions or modifications to the described embodiments without departing from the scope of the present invention, and such substitutions or modifications should be considered to fall within the scope of protection of the present invention.

[0096] Parts of the present invention that are not described in detail belong to the common knowledge of those skilled in the art.

Claims

1. A medical image segmentation method based on adaptive weighted skip connection, characterized in that: The steps are as follows: Step 1: Data acquisition and preprocessing: Step 2: Extract image features at different scales through the encoder of the medical image segmentation model: Step 3: Skip connection and adaptively adjust attention through adaptive weight skip connection module: On the encoder path, the extracted feature data of each layer is transmitted to the decoder of the corresponding layer through the jump connection; for the feature data k at the top layer of the encoder, it is input into the adaptive weight jump connection module together with the feature data d at the bottom layer of the decoder for adaptive adjustment, and then transmitted to the decoder of the corresponding layer. The adaptive adjustment is specifically as follows: first, the dimension of d is compressed to 3 dimensions through the view function, and the first two dimensions (B, C) are fixed. d ) dimension, and automatically merge the 3rd and 4th dimensions; then use the reshape function to change the dimension of k into 3 dimensions, merge the two dimensions through H×W, and transform the three dimensions; then let the obtained d flat and k flat With the matrix W s Do a dot multiplication; get the corresponding weight parameter through the softmax function, and k flat After multiplication, the view_as function is used to adjust the dimension to the same as k to obtain the final output k of the adaptive weight jump connection module. attend ; The specific formula is as follows: d flat =d.view(B,C d ,-1) k flat =k.reshape(C k ,H×W).permute(1,2,0) k attend =softmax(d flat ·W s ·k flat )·k flat .view_as(k) The symbols represent the following: d: represents the feature data at the bottom layer of the decoder; B: Batchsize abbreviation, representing the batch size during training; C d : is the number of channels of d; view: A function in Pytorch that merges the height and width dimensions of d to form a three-dimensional tensor; d flat : Represents the feature vector of the decoder's lowest-level feature parameters after dimensional transformation, from four dimensions to three dimensions; k: represents the top-level feature data of the encoder, that is, the original skip connection parameters; C k : is the number of channels of k; H: is the height dimension of d and k; W: width dimension of d and k; reshape: A function in Pytorch that rearranges the dimensions of k, placing the number of channels in the first dimension, the height in the second dimension, and merging the width dimension and the batch size together to form the last dimension; permute: A function used to rearrange the dimensions of a tensor, used to change the order of the dimensions; f flat : Represents the three-dimensional vector of the feature parameters transmitted by the jump connection at the top layer of the encoder after dimension transformation; Softmax: is a normalized exponential function; view_as: is a function in Pytorch that changes the shape of a tensor to be consistent with the tensor in the brackets without changing its data content; W s : An adaptive matrix of size (H / W, H / W*B) that satisfies k flat and d flat Matrix multiplication is used to automatically adjust the transmission ratio of k; the matrix is ​​randomly assigned when the network is initialized and then optimized during the network training process; k attend : The final output of this module, used to replace the original skip connection parameter k; Step 4: Complete image segmentation through the decoding path decoder combined with the skip connection parameters to obtain the final segmentation output; Step 5: Train the medical image segmentation model based on adaptive weight skip connection; Step 6: Use the trained model for inference, that is, predict the edema area on new fundus OCT images.

2. The medical image segmentation method based on adaptive weighted skip connection according to claim 1, characterized in that: Step 1: Through the OCT system, the original OCT images obtained are first standardized and resized to adapt to subsequent processing requirements; all images are uniformly processed to 512x512 pixels, RGB three channels; and the fundus edema areas on the images are annotated to generate labeled images; all labeled images are stored in a 512x512 pixel, single-channel format, and each original OCT image corresponds one-to-one with the annotated labeled image.

3. The medical image segmentation method based on adaptive weighted skip connection according to claim 1, characterized in that: Step 2: After the data is input into the medical image segmentation model based on adaptive weight skip connection, basic preprocessing is performed on the image data, including normalization and denoising, to ensure the quality and consistency of the input data; The preprocessed image first passes through an encoder structure of a UNet network; the encoder is an encoder contraction path encoder, which gradually reduces the spatial dimension through convolutional layers and pooling layers while increasing the depth of features; each layer of the encoder will extract corresponding feature data, corresponding to image features of different scales; the feature information at the bottom layer of the encoder is transmitted to the decoding path through an SDE module; the SDE module is a neural network module that decouples feature information and direction information.

4. A medical image segmentation method based on adaptive weighted skip connection according to claim 1 or 3, characterized in that: Step 4: The decoding path decoder of the medical image segmentation model adopts an interactive feature space decoder (IFD). In IFD, each decoding layer contains a set of feature modules (FeatureBlock) and spatial modules (SpaceBlock), and the feature parameters are propagated upward alternately between the two. The interactive feature space decoder gradually restores the spatial resolution of the image through upsampling operations within the module, introduces the feature data transmitted by the jump connection into the feature module of the corresponding layer, and splices the image features of different scales with the output features of the spatial module of the corresponding decoding layer as the input of the feature module. After the feature module of the last layer of IFD is output, the output feature data is aggregated through the regional guided channel of the bilateral voting module to obtain the final segmentation output.

5. The medical image segmentation method based on adaptive weighted skip connection according to claim 1, characterized in that: Step 5: During the training process, the medical image segmentation model based on adaptive weight jump connection is trained by the training data annotated in step 1. The two-dimensional cross entropy loss function is selected in the training process to evaluate the difference between the model output and the actual annotation, and the weight of the model is continuously optimized through back propagation, gradient descent, dynamic learning rate adjustment and regularization technology. At the same time, the weight matrix W s The parameters in will be adjusted and changed as the training progresses, and finally the model effect will be optimized.

6. The medical image segmentation method based on adaptive weighted skip connection according to claim 1, characterized in that: Step 6: The image to be segmented is standardized and resized, processed to 512x512 pixels and RGB three channels, and then input into the trained medical image segmentation model for segmentation. Finally, the segmentation result is generated and the image predicted to be the edema part is output.

Images