Medical image segmentation method based on adaptive weight jump connection

Abstract

The invention discloses a medical image segmentation method based on adaptive weight jump connection. By introducing a transmission feature attention mechanism of adaptive weight adjustment, the feature transmission proportion of jump connection can be dynamically adjusted, the utilization efficiency of important features is optimized, and the influence of invalid features is reduced. The recognition and segmentation precision of the model on the edema region in the eye fundus image is improved, and the overall influence of the features in the model is optimized at the same time. The adaptive weight adjustment transmission feature attention mechanism enhances the adaptability of the network model to different data structures, and improves the compatibility and robustness of the model.

Description

Technical Field

[0001] The present invention belongs to the technical fields of medical image processing, deep learning, and signal processing, and specifically relates to a medical image segmentation method based on adaptive weight jump connection. Background Art

[0002] In the field of fundus edema segmentation in optical coherence tomography (OCT), the application of traditional image processing techniques and deep learning models mainly focuses on basic methods. Specifically, traditional image processing methods such as threshold processing and region growing rely on manually set parameters and thresholds, which to some extent limits their flexibility and adaptability. Although these methods can achieve basic image segmentation, they often lack effective responses to complex or inconsistent image features.

[0003] With the rapid progress of deep learning technology, neural network models have shown their excellent performance in the field of medical image analysis. Among them, the DconnNet network model effectively captures image features while retaining important contextual information through its dual decoder structure and efficient SDE module, thus having significant advantages in improving segmentation accuracy and adaptability, and improving the efficiency and accuracy of medical image segmentation.

[0004] In the processing of optical coherence tomography (OCT) fundus images, although the DconnNet model has shown excellent performance, the existing technology still faces a series of challenges when processing OCT fundus images. First, due to the influence of light scattering and light absorption, OCT images often show high noise and low contrast, and small target areas. These factors make it difficult for the traditional DconnNet model to identify and segment subtle edema areas. Secondly, OCT images acquired from different regions and institutions vary in quality and features. When processing these images, the existing DconnNet model lacks detailed attention and adjustment to local areas and fails to effectively adapt to the feature levels of different images, which limits its performance in detail segmentation accuracy. Summary of the invention

[0005] In the field of medical imaging, accurate segmentation of edema areas in fundus optical coherence tomography (OCT) images is particularly important for the diagnosis and treatment of ophthalmic diseases. However, existing image processing techniques face many challenges in processing such images, such as uneven image quality, difficulty in capturing details, and complex and changeable feature information, which significantly affect the accuracy of segmentation and the overall reliability of the system. In order to overcome these limitations and improve the performance of the system, the present invention proposes a medical image segmentation method based on adaptive weighted jump connections, which aims to improve the accuracy and efficiency of edema area segmentation in OCT images.

[0006] The present invention aims to improve the segmentation accuracy and contrast of edema areas in fundus images of optical coherence tomography (OCT) and solve key technical problems in the prior art. By introducing a transmission feature attention mechanism with adaptive weight adjustment, the present invention can dynamically adjust the feature transmission ratio of jump connections, optimize the utilization efficiency of important features, and reduce the impact of invalid features. This method improves the model's recognition and segmentation accuracy of edema areas in fundus images, while optimizing the overall impact of the features in the model. The transmission feature attention mechanism with adaptive weight adjustment enhances the adaptability of the network model to different data structures and improves the compatibility and robustness of the model. In view of the limitations of the prior art, the present invention provides a new solution that effectively improves segmentation accuracy, compatibility and robustness.

[0007] The core technology of the present invention involves the use of an adaptive weight jump connection module for the jump connection part in the deep learning model DconnNet. The traditional DconnNet model effectively utilizes the high-level features of the encoding layer in different decoding layers through its symmetrical encoding and decoding structure, but the use of these features lacks flexibility and adaptability. The present invention introduces an adaptive weight jump connection module, which greatly enhances the responsiveness to key features by dynamically adjusting the weights of feature channels, thereby improving the efficiency and accuracy of feature transmission. In addition, the module can also automatically adjust the weight distribution according to the importance of dimensional features, so that the network can more accurately focus on information that is more critical to the task.

[0008] Through the application of this technology, the present invention can significantly improve the segmentation accuracy of edema areas in low-contrast and detail-rich OCT images. The experimental results also verify the advantages of the present invention over traditional methods in segmentation accuracy and processing efficiency, especially in processing details and contrast areas in images, showing higher accuracy and robustness.

[0009] A medical image segmentation method based on adaptive weight skip connection comprises the following steps:

[0010] Step 1: Data acquisition and preprocessing:

[0011] Step 2: Extract image features of different scales through the encoder of the medical image segmentation model:

[0012] Step 3: Skip connection and adaptively adjust attention through adaptive weight skip connection module:

[0013] Step 4: Complete image segmentation through the decoding path decoder combined with the skip connection parameters to obtain the final segmentation output.

[0014] Step 5: training a medical image segmentation model based on adaptive weight skip connection;

[0015] Step 6: Use the trained model for inference, that is, predict the edema area on new fundus OCT images.

[0016] Furthermore, the specific method of step 1 is as follows:

[0017] The original OCT images obtained through the OCT system are first standardized and resized to meet the needs of subsequent processing. All images are uniformly processed to 512x512 pixels, RGB three channels. The fundus edema part on the image is annotated to generate a labeled image; all labeled images are stored in a 512x512 pixel, single-channel format, and each original OCT image corresponds to the annotated labeled image one by one.

[0018] Furthermore, the specific method of step 2 is as follows:

[0019] After the data is input into the medical image segmentation model based on adaptive weight jump connection, the image data is preprocessed, including normalization and denoising, to ensure the quality and consistency of the input data. The preprocessed image first passes through the encoder structure of a UNet network. The encoder is an encoder contraction path encoder, which gradually reduces the spatial dimension through convolutional layers and pooling layers while increasing the depth of features. Each layer of the encoder extracts the corresponding feature data, corresponding to image features of different scales. The feature information at the bottom layer of the encoder is transmitted to the decoding path through an SDE module. The SDE module is a neural network module that decouples feature information and direction information.

[0020] Furthermore, the specific method of step 3 is as follows:

[0021] On the encoder path, the extracted feature data of each layer is transmitted to the decoder of the corresponding layer through a skip connection. For the feature data k at the top layer of the encoder, it is input into the adaptive weight skip connection module together with the feature data d at the bottom layer of the decoder for adaptive adjustment, and then transmitted to the decoder of the corresponding layer. The adaptive adjustment is as follows: first, the dimension of d is compressed to 3 dimensions through the view function, and the first two dimensions (B, C) are fixed. d ) and automatically merge the 3rd and 4th dimensions. Then, through the reshape function, the dimension of k is changed to 3 dimensions, the two dimensions are merged through H×W, and the three dimensions are transformed. Then, let the obtained d flat and k flat With the matrix W s Do a dot multiplication. Get the corresponding weight parameter through the softmax function and k flatAfter multiplication, the view_as function is used to adjust the dimension to the same as k to obtain the final output k of the adaptive weight jump connection module. attend .

[0022] The specific formula is as follows:

[0023] d flat =d.view(B,C d ,-1)

[0024] k flat = k.reshape(C k ,H×W).permute(1,2,0)

[0025] k attend =softmax(d flat ·W s ·k flat )·k flat .view_as(k)

[0026] The symbols represent the following:

[0027] d: represents the feature data at the bottom layer of the decoder.

[0028] B: is the abbreviation of Batchsize, which represents the batch size during training.

[0029] C d : is the number of channels of d.

[0030] view: A function in Pytorch that changes the shape of a tensor without changing its data content. The purpose here is to merge the height and width dimensions of d to form a three-dimensional tensor.

[0031] d flat : Represents the feature vector of the lowest-level feature parameters of the decoder after dimensional transformation, from four dimensions to three dimensions.

[0032] k: represents the top-level feature data of the encoder, that is, the original skip connection parameters.

[0033] C k : is the number of channels of k.

[0034] H: is the height dimension of d and k.

[0035] W: is the width dimension of d and k.

[0036] reshape: A function in Pytorch that is also used to change the shape of a tensor without changing the content of its data, and can handle non-continuous tensors. The purpose here is to rearrange the dimensions of k, placing the number of channels in the first dimension, the height in the second dimension, and merging the width dimension and the batch size together to form the last dimension.

[0037] permute: A function used to rearrange the dimensions of a tensor, changing the order of the dimensions.

[0038] k flat : Represents the three-dimensional vector of the feature parameters transmitted by the top layer of the encoder through the skip connection after dimension transformation.

[0039] Softmax: It is a normalized exponential function commonly used in deep learning. It is usually used to convert a set of numerical values ​​into a probability distribution, where the ratio of the exponent of each numerical value to the sum of the exponents of all numerical values ​​is used as the output probability.

[0040] view_as: is a function in Pytorch that changes the shape of a tensor to be consistent with the tensor in the brackets without changing its data content.

[0041] W s : An adaptive matrix of size (H / W, H / W*B) that satisfies k flat and d flat The matrix multiplication of is used to automatically adjust the transmission ratio of k. The matrix is ​​randomly assigned when the network is initialized, and then optimized through back propagation and gradient descent during network training.

[0042] k attend : The final output of this module, used to replace the original skip connection parameter k.

[0043] The adaptive weighted skip connection module can adaptively adjust the ratio of the top-level feature data transmitted through the skip connection, so that the skip connection of this layer is no longer a simple constant-ratio feature transmission, but can be adjusted according to the importance of the feature dimension. More important dimensional features will be given higher weights, so that they occupy a larger proportion in the transmission process to the decoder, optimizing the network effect.

[0044] Furthermore, the specific method of step 4 is as follows:

[0045] The decoder of the decoding path of the medical image segmentation model adopts an interactive feature-space decoder (IFD). In IFD, each decoding layer contains a set of feature modules FeatureBlock and space modules SpaceBlock, and the feature parameters are propagated upward alternately between the two. The feature module FeatureBlock is a module that focuses on feature information, and the space module SpaceBlock is a module that focuses on spatial information. The two modules are propagated upward alternately to form an interactive feature space decoder, which can fuse feature information with directional information. The interactive feature space decoder gradually restores the spatial resolution of the image through upsampling operations within the module, introduces the feature data transmitted by the jump connection into the feature module of the corresponding layer, and splices the image features of different scales with the output features of the space module of the corresponding decoding layer as the input of the feature module, effectively fusing high-level semantic information with spatial details and optimizing the feature map results. After the feature module of the last layer of IFD is output, the output feature data is aggregated through the regional guidance channel of the bilateral voting module to obtain the final segmentation output.

[0046] Furthermore, the specific method of step 5 is as follows:

[0047] During the training process, the medical image segmentation model based on adaptive weight jump connection is trained with the training data annotated in step 1, and the edema area in the unseen fundus OCT images is identified and predicted by the medical image segmentation model. The training process selects the two-dimensional cross entropy loss function to evaluate the difference between the model output and the actual annotation, and continuously optimizes the model weights through back propagation, gradient descent, dynamic learning rate adjustment and regularization techniques. At the same time, the weight matrix W s The parameters in will be adjusted and changed as the training progresses, and finally the model effect will be optimized.

[0048] Furthermore, the specific method of step 6 is as follows:

[0049] The image to be segmented is standardized and resized, processed to 512x512 pixels, RGB three channels, and then input into the trained medical image segmentation model for segmentation. Finally, the segmentation result is generated and the image predicted to be the edema part is output.

[0050] The beneficial effects of the present invention are as follows:

[0051] The core technology of the present invention is to introduce an adaptive weight jump connection module to process the data transmitted in the jump connection. Through this module, feature data of different dimensions are adaptively adjusted in the jump connection process, so that more important dimensional features are transmitted to the decoder part with higher weights. This processing step is particularly important for improving the network's sensitivity to image detail features and segmentation accuracy, especially when processing OCT image data with different dimensional features and noise. Compared with the prior art, this method uses an adaptive matrix to guide feature transmission, which helps to better utilize the dimensional features in the jump connection process, thereby improving segmentation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Schematic diagram of the medical image segmentation model structure.

[0053] Figure 2 Workflow for medical image segmentation models.

[0054] Figure 3 Schematic diagram of the specific working process of the adaptive jump connection module. DETAILED DESCRIPTION

[0055] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0056] Glossary

[0057] OCT (Optical Coherence Tomography) is an imaging technology that uses the principle of light reflection to obtain high-resolution three-dimensional images of the inside of tissues by measuring the time delay and intensity of light reflected from tissues. OCT is widely used in ophthalmology, dermatology and other fields, especially in fundus examination, and can perform fine imaging of eye structures such as the retina.

[0058] DconnNet: is an innovative neural network architecture designed specifically to improve the accuracy of anatomical consistency in medical image segmentation tasks. Through a unique solution - directional connectivity modeling, it solves the problem that traditional connectivity networks ignore rich channel direction information. The core technology of DconnNet is to decouple the directional subspace from the shared latent space, and track and utilize this directional information, thereby significantly enhancing the feature representation capability within the network.

[0059] Skip connection: is a technology designed to prevent gradient vanishing in deep learning networks. The structure of the skip connection allows information to be directly transferred from earlier layers of the network to subsequent layers, and a "bypass" is added to the traditional sequence connection to achieve direct flow of feature information. This design effectively solves the training problem that arises as the depth of the network increases, allowing the model to better handle information transfer in deep networks without losing contextual details. The introduction of skip connections is widely used in various deep learning architectures, especially in the fields of image recognition and speech processing that require processing of complex or deep structures, and has shown excellent performance.

[0060] Specifically, Figure 1 As shown, the present invention provides an improved OCT network model, namely a medical image segmentation model, which integrates an adaptive weight jump connection module in its jump connection part to improve the automatic segmentation effect of edema areas in fundus OCT images. This technology is particularly suitable for medical image processing tasks that require high-precision and high-efficiency fundus edema detection.

[0061] The core of the present invention is to introduce an adaptive weighted jump connection module, which is designed to adaptively weight the characteristic parameters transmitted by the input jump connection, so as to strengthen the network's adjustment and absorption of different dimensional features of edema and improve the edema segmentation capability. Figure 2 As shown, the specific implementation process is as follows:

[0062] Preprocessing stage: The public dataset OIMHS is used, which contains 10,908 pairs of image data as training data and 2,700 pairs of image data as validation test data. The original OCT images are first standardized and resized to adapt to the subsequent processing requirements. All images are uniformly processed to 512x512 pixels and RGB three channels.

[0063] Network encoding and decoding processing: The transformed image data first undergoes basic preprocessing, including normalization and denoising, to ensure the quality and consistency of the input data. The preprocessed image will be input into the encoder, which is a UNet network four-layer encoding contraction path encoder. Each layer of the encoder contains a maximum pooling layer and two sets of convolutional layers. The convolutional layer consists of a two-dimensional convolutional layer, a batch normalization layer, and a ReLu function. Each layer of the encoder will gradually reduce the spatial dimension while increasing the depth of the feature. At the same time, each layer of the encoder will extract the corresponding feature data, corresponding to image features of different scales. The feature information at the bottom of the encoder is transmitted to the decoding path through an SDE module. The SDE module is a neural network module that decouples feature information and directional information. It includes an embedding compression (ES) module and a manifold projection and enhancement (MPE) module. The ES module converts the input directional enhancement map r' into a directional embedding n' as a high-level directional representation. Then, using n' as the input of MPE, the main feature map d' and the directional embedding n' are projected onto the shared manifold through two 1×1 convolutional projectors. Next, the category-direction relationship is modeled using inter-channel dot products and sigmoid activation by calculating the similarity between the projected directional embedding n'0 and the feature map d'0. Through the dot product operation, the directional information is fused into the re-encoded original feature map d" to generate a new enhanced directional map r'7, which is used as the input of the feature stream and the generation of the directional embedding n'7 of the next layer. The result of the SDE module is that the directional information is effectively fused into the feature map, generating a new enhanced directional map r'7, which is used as the input of the feature stream and the generation of the directional embedding n'7 of the next layer. In this way, the SDE module achieves the effective fusion and transmission of directional information in the feature map, improving the performance of the model in medical image segmentation.

[0064] like Figure 3 As shown in the figure, the adaptive jump connection module: on the encoder path, the extracted feature data of each layer is transmitted to the decoder part of the same layer. At the same time, an adaptive weight jump connection module is added to the top layer, and the feature data is first input into the module and then into the decoder part. In this module, the top encoder feature data and the bottom decoder feature data are input, denoted as k and d respectively. First, the dimensions of the two are compressed, and their dimensions are converted from four dimensions to three dimensions. The dimension of d is first compressed to three dimensions through the view function, and the first two dimensions (B, C) are fixed. d ) and automatically merge the 3rd and 4th dimensions. Then, through the reshape function, the dimension of k is changed to 3 dimensions, the two dimensions are merged through H×W, and the three dimensions are transformed. Then, let the obtained d flat and k flatWith the matrix W s Do a dot multiplication. Get the corresponding weight parameter through the softmax function and k flat After multiplication, the view_as function is used to adjust the dimension to the same as k to obtain the final output k of the adaptive weight jump connection module. attend .

[0065] The specific formula is as follows:

[0066] d flat =d.view(B,C d ,-1)

[0067] k flat = k.reshape(C k ,H×W).permute(1,2,0)

[0068] k attend =softmax(d flat ·W s ·k flat )·k flat .view_as(k)

[0069] The symbols represent the following:

[0070] d: represents the feature data at the bottom layer of the decoder.

[0071] B: is the abbreviation of Batchsize, which represents the batch size during training.

[0072] C d : is the number of channels of d.

[0073] view: A function in Pytorch that changes the shape of a tensor without changing its data content. The purpose here is to merge the height and width dimensions of d to form a three-dimensional tensor.

[0074] d flat : Represents the feature vector of the lowest-level feature parameters of the decoder after dimensional transformation, from four dimensions to three dimensions.

[0075] k: represents the top-level feature data of the encoder, that is, the original skip connection parameters.

[0076] C k : is the number of channels of k.

[0077] H: is the height dimension of d and k.

[0078] W: is the width dimension of d and k.

[0079] reshape: A function in Pytorch that is also used to change the shape of a tensor without changing the content of its data, and can handle non-continuous tensors. The purpose here is to rearrange the dimensions of k, placing the number of channels in the first dimension, the height in the second dimension, and merging the width dimension and the batch size together to form the last dimension.

[0080] permute: A function used to rearrange the dimensions of a tensor, changing the order of the dimensions.

[0081] k flat : Represents the three-dimensional vector of the feature parameters transmitted by the top layer of the encoder through the skip connection after dimension transformation.

[0082] Softmax: It is a normalized exponential function commonly used in deep learning. It is usually used to convert a set of numerical values ​​into a probability distribution, where the ratio of the exponent of each numerical value to the sum of the exponents of all numerical values ​​is used as the output probability.

[0083] view_as: is a function in Pytorch that changes the shape of a tensor to be consistent with the tensor in the brackets without changing its data content.

[0084] W s : An adaptive matrix of size (H / W, H / W*B) that satisfies k flat and d flat The matrix multiplication of is used to automatically adjust the transmission ratio of k. The matrix is ​​randomly assigned when the network is initialized, and then optimized through back propagation and gradient descent during network training.

[0085] k attend : The final output of this module, used to replace the original skip connection parameter k.

[0086] The final transmitted skip connection characteristic parameter, i.e. the final output k attend Replace the top-level feature module FeatureBlock in the input decoder to implement adaptive skip connection feature parameter weight adjustment and optimize network performance.

[0087] The adaptive weighted skip connection module can adaptively adjust the ratio of the top-level feature data transmitted through the skip connection, so that the skip connection of this layer is no longer a simple constant-ratio feature transmission, but can be adjusted according to the importance of the feature dimension. More important dimensional features will be given higher weights, so that they occupy a larger proportion in the transmission process to the decoder, optimizing the network effect.

[0088] The decoding path of the network is called Interactive Feature-space Decoder (IFD). In IFD, data is transmitted alternately through the space module SpaceBlock and the feature module FeatureBlock, forming two decoder paths for propagation.

[0089] In the spatial module SpaceBlock, the direction-enhanced feature map r' is first taken as input and converted into a direction embedding n', i.e., a high-level direction representation, through the ES module. The ES module uses a global average pooling (GAP) operation to convert the feature map r' into a direction embedding n'. Then, the direction embedding n' is taken as input and the direction representation in the main feature map d' is further enhanced through the MPE module. In the MPE module, two 1×1 convolutional projectors are first used to project the main feature map d' and the direction embedding n' onto a shared manifold at the current resolution. The results after projection are d' and n', respectively. Next, by calculating the similarity between the projected direction embedding n'0 and the feature map d'0, the normalized category-direction attention map α is obtained using the channel-level dot product and sigmoid activation function. This attention map enhances features related to direction and suppresses features that are not related to direction. Finally, the dot product operation is used to perform element-wise multiplication of the attention map α with the re-encoded original feature map d” to effectively fuse the direction information into the feature map and obtain a new direction-enhanced feature map r’. This new feature map r’ will be used as the input of the feature stream and used to generate the direction embedding n’ of the next layer.

[0090] In the feature module FeatureBlock, the category-direction relationship is modeled by calculating the similarity between the projected directional embedding n' and the feature map d'. Finally, the normalized category-direction attention is calculated using the dot product between channels and sigmoid activation to enhance the directional information in the re-encoded original feature map d". Through such operations, the directional information is effectively fused into the feature map to generate a new enhanced directional feature map, which is used as the input of FeatureFlow and is also used to generate the directional embedding of the next layer.

[0091] The feature data transmitted through the SDE module is first transmitted to the underlying spatial module SpaceBlock. After passing through the underlying spatial module, the feature data forms two propagation paths between the two modules to realize two decoding paths. One path is that the feature data is directly transmitted between the spatial module SpaceBlock to the upper layer, and the other path is that the feature data in the spatial module SpaceBlock is transmitted to the feature module FeatureBlock and then to the upper spatial module SpaceBlock. The two decoding paths together form an interactive feature space decoder, which can fuse feature information with directional information. The interactive feature space decoder gradually restores the spatial resolution of the image through the upsampling operation within the module. The dimensional features processed from step 3 will be introduced into the feature module FeatureBlock of each layer in different layers. These different dimensional encoder features are directly combined with the output features of the spatial module of the corresponding decoding layer through the Cat function in the decoding layer as the input of the feature module, which effectively fuses high-level semantic information with spatial details and optimizes the feature map results. After the last layer of the interactive feature space decoder is output, the feature data output by the decoder is aggregated through the regional guidance channel of the bilateral voting module to obtain the final segmentation output. This step first converts the main feature map d n The final connectivity output X is obtained by integration. Each eight channels of X represent the connectivity of a category. The bilateral voting module and the region-oriented channel aggregation module are then used to obtain the final segmentation map. The calculation formula of the bilateral voting module is: j (x,y)=X 9-j (x+a,y+b)=X j (x,y)×X 9-j (x+a,y+b), Among them, j is the jth channel, a, b∈{0,±1} represents the position of adjacent pixels, X n is the Bicon map, and S is the final output segmentation prediction. The probability of each pixel belonging to the edema area can be obtained. The output is usually a segmentation map of the same size as the input image, and the value of each pixel represents the probability that the pixel belongs to the edema area.

[0092] During the training process, at this stage, the labeled training data obtained in the preprocessing stage is used. These data contain fundus OCT images and their corresponding edema area annotations. With these data, the model can be trained to identify and predict edema areas in unseen fundus OCT images. The training usually selects a two-dimensional cross entropy loss function to evaluate the difference between the model output and the actual annotation, and continuously optimizes the model weights through back propagation, gradient descent, dynamic learning rate adjustment, and regularization techniques. Specifically, the pixel value of the background pixel in the data set is 0, and the pixel value of the edema part is 192. At the beginning of training, each edema pixel is assigned a probability value of 1, and the background part is assigned a probability value of 0. The corresponding probability value is output through the forward propagation of the model, and then the loss for each pixel is obtained through the two-dimensional cross entropy loss function. The loss calculated for all pixels of the entire image or batch is accumulated to obtain the final loss value. Through back propagation, the gradient of the loss relative to each model parameter is calculated. This gradient represents the sensitivity of the loss function to each parameter under the current model parameters. The optimizer adjusts the value of each parameter in the model according to the calculated gradient to optimize the model weight. Finally, by continuously iteratively updating the parameters, the minimum value of the loss function is gradually approached to achieve the optimization of the segmentation effect.

[0093] The trained model can be used for inference, that is, predicting the edema area on the new fundus OCT image. This process includes putting the new image through the same preprocessing, then segmenting it using the trained model, and finally generating the segmentation result, outputting the image predicted to be the edema part. In the experiment, the deep learning model DconnNet achieved an accuracy of 84.20% for edema segmentation on the OIMHS dataset, while the medical image segmentation model based on adaptive weight jump connection achieved an accuracy of 87.52% for edema segmentation on the OIMHS dataset, an increase of 3.32%.

[0094] The core technology of the present invention is to introduce an adaptive weight jump connection module to replace the original top-level jump connection. Through this module, the feature parameters of the top layer will be combined with the underlying decoder features, so that the transmitted feature parameters can be adaptively adjusted and appropriate weights are assigned according to the importance of the dimensional features. This processing step is particularly important for improving the network's analysis of image feature characteristics and segmentation accuracy, especially when processing medical imaging data with complex textures and noise. Compared with the prior art, this method uses adaptive attention technology to help better capture and utilize structural features in the image, thereby improving segmentation performance.

[0095] The above contents are further detailed descriptions of the present invention in combination with specific / preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, they can also make several substitutions or modifications to these described embodiments without departing from the concept of the present invention, and these substitutions or modifications should be regarded as belonging to the protection scope of the present invention.

[0096] Parts of the present invention that are not described in detail belong to the well-known technologies of those skilled in the art.

Claims

1. A medical image segmentation method based on adaptive weighted skip connection, characterized in that: The steps include: Step 1: Data acquisition and preprocessing: Step 2: Extract image features of different scales through the encoder of the medical image segmentation model: Step 3: Skip connection and adaptively adjust attention through adaptive weight skip connection module: Step 4: Complete image segmentation through the decoding path decoder combined with the skip connection parameters to obtain the final segmentation output; Step 5: training a medical image segmentation model based on adaptive weight skip connection; Step 6: Use the trained model for inference, that is, predict the edema area on new fundus OCT images.

2. The medical image segmentation method based on adaptive weight skip connection according to claim 1, characterized in that: Step 1: Through the OCT system, the original OCT images are first standardized and resized to adapt to subsequent processing requirements; all images are uniformly processed to 512x512 pixels, RGB three channels; and the fundus edema part on the image is annotated to generate a labeled image; all labeled images are stored in 512x512 pixels, single-channel format, and each original OCT image corresponds one-to-one to the annotated labeled image.

3. The medical image segmentation method based on adaptive weight skip connection according to claim 1, characterized in that: Step 2: After the data is input into the medical image segmentation model based on adaptive weight skip connection, basic preprocessing is performed on the image data, including normalization and denoising, to ensure the quality and consistency of the input data; The preprocessed image first passes through an encoder structure of a UNet network; the encoder is an encoding contraction path encoder, which gradually reduces the spatial dimension through convolutional layers and pooling layers while increasing the depth of features; each layer of the encoder will extract the corresponding feature data, corresponding to image features of different scales; the feature information at the bottom layer of the encoder is transmitted to the decoding path through an SDE module; the SDE module is a neural network module that decouples feature information and direction information.

4. A medical image segmentation method based on adaptive weight skip connection according to claim 1 or 3, characterized in that: Step 3: On the encoder path, the extracted feature data of each layer is transmitted to the decoder of the corresponding layer through a skip connection. For the feature data k at the top layer of the encoder, it is input into the adaptive weight skip connection module together with the feature data d at the bottom layer of the decoder for adaptive adjustment, and then transmitted to the decoder of the corresponding layer. The adaptive adjustment is as follows: first, the dimension of d is compressed to 3 dimensions through the view function, and the first two dimensions (B, C) are fixed. d ) and automatically merge the 3rd and 4th dimensions; then use the reshape function to change the dimension of k to 3 dimensions, merge the two dimensions through H×W, and transform the positions of the three dimensions; then let the obtained d flat and k flat With the matrix W s Do a dot multiplication; get the corresponding weight parameter through the softmax function, and k flat After multiplication, the view_as function is used to adjust the dimension to the same as k to obtain the final output k of the adaptive weight jump connection module. attend ; The specific formula is as follows: d flat =d.view(B,C d ,-1) k flat =k.reshape(C k ,H×W).permute(1,2,0) k attend =softmax(d flat ·W s ·k flat )·k flat .view_as(k) The symbols are as follows: d: represents the feature data at the bottom layer of the decoder; B: Batchsize abbreviation, representing the batch size during training; C d : is the number of channels of d; view: a function in Pytorch that merges the height and width dimensions of d to form a three-dimensional tensor; d flat : The feature vector representing the feature parameters at the bottom layer of the decoder after dimension transformation, from four dimensions to three dimensions; k: represents the top-level feature data of the encoder, i.e. the original skip connection parameters; C k : is the number of channels of k; H: is the height dimension of d and k; W: is the width dimension of d and k; reshape: A function in Pytorch that rearranges the dimensions of k, placing the number of channels in the first dimension, the height in the second dimension, and merging the width dimension and the batch size together to form the last dimension; permute: a function used to rearrange the dimensions of a tensor, used to change the order of each dimension; k flat : Represents the three-dimensional vector of the feature parameters transmitted by the top layer of the encoder through the skip connection after dimension transformation; Softmax: is a normalized exponential function; view_as: is a function in Pytorch that changes the shape of a tensor to be consistent with the tensor in brackets without changing its data content; W s : An adaptive matrix of size (H / W, H / W*B) that satisfies k flat and d flat , which is used to automatically adjust the transmission ratio of k; the matrix is ​​randomly assigned when the network is initialized and then optimized during the network training process; k attend : The final output of this module, used to replace the original skip connection parameter k.

5. The medical image segmentation method based on adaptive weight skip connection according to claim 4, characterized in that: Step 4: The decoding path decoder of the medical image segmentation model adopts an interactive feature space decoder IFD. In IFD, each decoding layer contains a set of feature modules FeatureBlock and space modules SpaceBlock, and the feature parameters are propagated upward alternately between the two; the interactive feature space decoder gradually restores the spatial resolution of the image through upsampling operations within the module, introduces the feature data transmitted by the jump connection into the feature module of the corresponding layer, and concatenates the image features of different scales with the output features of the space module of the corresponding decoding layer as the input of the feature module; after the feature module of the last layer of IFD is output, the output feature data is aggregated through the regional guided channel of the bilateral voting module to obtain the final segmentation output.

6. The medical image segmentation method based on adaptive weight skip connection according to claim 1, characterized in that: Step 5: In the training process, the medical image segmentation model based on adaptive weight jump connection is trained through the training data annotated in step 1; the two-dimensional cross entropy loss function is selected in the training process to evaluate the difference between the model output and the actual annotation, and the weight of the model is continuously optimized through back propagation, gradient descent, dynamic learning rate adjustment and regularization technology; at the same time, the weight matrix W s The parameters in will be adjusted and changed as the training progresses, and finally the model effect will be optimized.

7. The medical image segmentation method based on adaptive weight skip connection according to claim 1, characterized in that: Step 6: The image to be segmented is standardized and resized, processed to 512x512 pixels and RGB three channels, and then input into the trained medical image segmentation model for segmentation. Finally, the segmentation result is generated and the image predicted to be the edema part is output.

Images