Premature infant retinopathy complex lesion segmentation method based on deep learning

Abstract

The invention discloses a premature infant retinopathy complex lesion segmentation method based on deep learning, and the method comprises the following steps: enhancing an image data set through a data enhancement strategy, inputting the image data set into an encoder to extract multi-scale features, and carrying out the feature fusion through a multi-scale feature fusion module in combination with a strip pooling technology; the decoder carries out up-sampling on the coding features, fusion is carried out again to obtain deep features, and a first prediction map is generated; and in combination with a feature enhancement module, further improving the segmentation capability of the model on a small-size focus, optimizing model parameters through loss calculation, and segmenting the to-be-segmented image by using the trained model to obtain a segmentation result. According to the invention, the identification capability of the model for large-span lesions is enhanced, and the segmentation capability of the model for small-size and non-significant lesions is improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and more specifically, relates to a method for segmenting complex lesions of retinopathy of prematurity based on deep learning. Background Art

[0002] In the field of image processing, the application of automated analysis technology for specific neonatal retinal conditions is crucial for improving processing efficiency and accuracy. These conditions, which are often seen in newborns with low birth weight or short gestation periods, are characterized by abnormal vascular proliferation in the retina, potentially affecting vision. Advances in medical technology have improved the survival rate of these newborns, but this has also increased the demand for automated image analysis technology. Due to the rapid progression of the disease, timely image analysis is crucial.

[0003] In image processing practice, the development of automated tools is crucial for improving processing efficiency and reducing human error. Although researchers have attempted to use clustering algorithms to identify certain lesions, these methods are not yet fully automated and require manual feature determination, limiting their widespread application. Regarding deep learning methods, while there has been extensive research on vascular segmentation, there has been less research on lesion segmentation in this context. Some researchers have attempted to improve segmentation by adding specific modules to the U-Net architecture, but currently no deep learning method can fully address the segmentation of lesions at different stages. Existing binary segmentation methods primarily focus on distinguishing lesions from background and fail to provide more detailed information on lesion classification, limiting their potential for application in image analysis. Therefore, developing image processing techniques that can automatically identify and segment lesions at different stages is crucial for improving the accuracy and efficiency of image analysis.

[0004] Automatic segmentation of ROP lesions faces several challenges. These include the wide range of sizes across different lesion types, requiring the model to be able to learn features for both large and small objects. Some lesions are distributed in a unidirectional, large, irregular, and strip-like pattern, making feature learning challenging. Furthermore, lesions occupy a small area within the fundus image, making them less prominent and difficult to identify. These factors complicate automatic ROP lesion segmentation and require more advanced technologies to overcome.

[0005] The prior art invention patent with publication number CN114998210B proposes a retinopathy of prematurity detection system based on deep learning target detection, including: an image acquisition module, an image preprocessing module, a retinopathy of prematurity analysis module and a case report output module, wherein: the image acquisition module is used to collect multiple retinal fundus images of premature infants; the image preprocessing module is used to preprocess the retinal fundus images of premature infants; the retinopathy of prematurity analysis module is used to identify and classify retinopathy of prematurity and the characteristics and positional relationship of the lesions, and realize the detection of stage 15 and plus lesions in the retinal fundus images of premature infants in zones I / II / III; the case report output module is used to form an auxiliary diagnosis result based on the retinal lesion analysis results of the retinopathy of prematurity analysis module, and generate a diagnosis report through doctor confirmation, modification and / or input of medical advice. Although this solution adopts the CBAM attention mechanism to enhance feature recognition, it does not propose optimization measures specifically for improving the ability to segment small-sized and insignificant lesions. Summary of the Invention

[0006] The present invention aims to solve the problems faced by the existing lesion image segmentation methods, especially the problems of increased difficulty in feature learning caused by the large size span of different categories of lesions, the irregular strip distribution of some lesions with large span in one direction, and the insufficient ability to segment small and insignificant lesions. A deep learning-based method for segmenting complex lesions of retinopathy of prematurity is provided.

[0007] The primary purpose of the present invention is to solve the above technical problems, and the technical solutions of the present invention are as follows:

[0008] The present invention provides a method for segmenting complex lesions of retinopathy of prematurity based on deep learning, comprising the following steps:

[0009] Obtain an image dataset, and use a data enhancement strategy to enhance the dataset to obtain an enhanced dataset;

[0010] Start training and use the preset encoder to encode the enhanced data to obtain n-layer multi-scale encoding features;

[0011] Inputting the n-layer multi-scale coding features into a multi-scale feature fusion module, the multi-scale feature fusion module compresses the feature map dimension using strip pooling, and fuses the high-level features into its own features through matrix multiplication operations to obtain n-1 layers of corrected features;

[0012] The decoder is used to upsample the highest-level coding features step by step to obtain n-1 layers of decoding features, where the number of upsampling layers corresponds to the number of coding layers;

[0013] Inputting the decoded features into a multi-scale feature fusion module for feature fusion to obtain deep features. The deep features are subjected to bilinear interpolation upsampling and convolution to adjust the number of channels to obtain a first prediction map. The model parameters are optimized by using the first prediction map and the label for loss calculation;

[0014] The deep features and the lowest-level correction features are input into the feature enhancement module for feature enhancement to obtain aggregated features, and the aggregated features are subjected to bilinear interpolation upsampling and convolution to adjust the number of channels to obtain a second prediction map;

[0015] Calculate the loss between the first predicted image and the second predicted image and the true label, optimize the model parameters, and iterate the training to obtain a trained model;

[0016] The trained model is used to segment the image to be segmented to obtain a segmentation result.

[0017] Furthermore, the preset encoder is a PVT_V2 pre-trained model, and the specific steps of encoding the image by the model are:

[0018] Input the original image into encoder Enc1 to obtain the encoded feature F1;

[0019] Input the encoding feature F1 into the encoder Enc2 to obtain the encoding feature F2;

[0020] Repeat the above encoding process until the encoding feature F is obtained n The encoding ends, where n is the total number of layers.

[0021] Furthermore, the multi-scale feature fusion is processed in two cases:

[0022] For the top-level multi-scale feature fusion module, the encoded feature F n-1 and encoding feature F n Perform feature fusion to obtain the correction feature F' n-1 ;

[0023] For the non-top-level multi-scale feature fusion module, the encoding feature F i and its two adjacent high-level features F i+1 , F i+2 Perform feature fusion to obtain the correction feature F' i , where n is the total number of layers and i+2 is less than n.

[0024] Furthermore, the specific steps of the non-top-most multi-scale feature fusion module for feature fusion are as follows:

[0025] The encoding feature F i and encoding feature F i+1 Perform feature fusion to obtain enhanced features

[0026] Enhance the features and encoding feature F i+2 Perform feature fusion to obtain the correction feature F' i , where n is the total number of layers and i+2 is less than n.

[0027] Furthermore, the specific steps of the feature fusion are:

[0028] Use convolution operation to encode the feature F i+1 Transformed into a feature with the same encoding feature F i The feature space with the same number of channels C obtains the feature map The expression is as follows:

[0029]

[0030] The high-level features after adjusting the number of channels Strip pooling is used for encoding in the horizontal and vertical directions, and the pooling window sizes are [H i+1 ,1] and [1,W i+1 ], and get two one-dimensional eigenvectors and The stripe pooling process is as follows:

[0031]

[0032] The feature map after strip pooling and Associating, we get an attention matrix A∈R C ×C , where C is the number of channels, the expression is as follows:

[0033]

[0034] in, is a matrix multiplication, M is a learnable matrix with a window size of [H×W], and is initialized to an all-1 matrix at the beginning of training. In particular, when H and W are equal, M is the identity matrix, and the attention matrix can be simplified to:

[0035]

[0036] For the attention matrix A and input feature F i Perform matrix multiplication to obtain enhanced features The expression is as follows:

[0037]

[0038] Where n is the total number of layers, and i+1 is less than n.

[0039] Furthermore, the specific steps of using the decoder to perform step-by-step upsampling on the highest-level coding features are:

[0040] The highest level encoding feature F n Input decoder Dec n , get the decoding feature G n ;

[0041] The decoded feature G n and the correction characteristic F' n-1 Sum the feature matrix and input the summation result into the decoder Dec n-1 , get the decoding feature G n-1 ;

[0042] The above upsampling process is repeated until the decoding feature G2 is obtained, and the upsampling is completed, where n is the total number of layers.

[0043] Furthermore, the feature enhancement module includes two parallel feature enhancement branches, which input shallow features and deep features into the two parallel feature enhancement branches respectively, output two feature enhancement matrices, and connect the two feature enhancement matrices through convolution to obtain aggregated features.

[0044] Furthermore, each of the feature enhancement branches includes a reverse attention module and a boundary extraction module connected in parallel. The output matrices of the reverse attention module and the boundary extraction module connected in parallel are connected by matrix addition to output a feature enhancement matrix. The expression of the reverse attention module is:

[0045] RA(T1,T2)=(1-σ(T1))⊙(T2⊙σ(T2))+T1

[0046] Among them, T1 and T2 represent different feature inputs, σ represents the Sigmod activation function, and ⊙ represents element-by-element multiplication;

[0047] The expression of the boundary extraction module is:

[0048] EA(T1,T2)=T1⊙(C DW (T2))⊙σ(G)

[0049] Among them, T1, T2 represent different feature inputs, C DW represents depth-wise separable convolution, ⊙ represents element-wise multiplication, and G represents the gradient magnitude map, which is expressed as:

[0050]

[0051] Among them, G x and Gy is the convolution kernel parameter with fixed weight, and its expression is:

[0052]

[0053] Furthermore, the shallow feature F'1 and the deep feature G'2 are input into two parallel feature enhancement branches, and two feature enhancement matrices are output. The two feature enhancement matrices are connected by convolution to obtain the aggregated feature F", which is expressed as follows:

[0054] F′ l =RA(F′1,G′2)+EA(F′1,G′2)

[0055] F′ h =RA(G′2,F′1)+EA(G′2,F′1)

[0056] F″=C 3×3 (Concat(F′ l ,F′ h ))

[0057] Among them, RA is the reverse attention module, EA is the boundary extraction module, and F' l is the output matrix of the first branch, F' h is the output matrix of the second branch, C 3×3 (Concat(·)) represents a 3×3 convolution.

[0058] Furthermore, the loss calculation is performed using a cross entropy loss function.

[0059] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0060] To address the challenges of large-scale spans across different lesion categories and the unidirectional, large-span, irregular strip-like distribution of some lesions, which makes feature learning difficult, a multi-scale feature fusion module combined with strip pooling technology is proposed. The skip connection allows the decoder to not only receive features from the encoder at the same level but also preview features from deeper layers. This design combines more detailed spatial features with more abstract contextual information, enhancing the model's ability to recognize large-span lesions. Furthermore, to address the small distribution area of lesions in fundus images, a feature enhancement module is proposed. This module effectively fuses shallow and deep features by combining reverse attention and boundary extraction techniques, thereby improving the model's ability to segment small, inconspicuous lesions. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to make the purpose and technical solution of the present invention clearer, the present invention provides the following drawings and descriptions:

[0062] Figure 1Flowchart of the segmentation model training method provided by an embodiment of the present invention;

[0063] Figure 2 Schematic diagram of feature extraction for segmentation model training provided by an embodiment of the present invention;

[0064] Figure 3 A schematic diagram of a multi-scale feature fusion module provided in an embodiment of the present invention;

[0065] Figure 4 A schematic diagram of a feature enhancement module provided in an embodiment of the present invention;

[0066] Figure 5 A multi-scale coding feature map provided by an embodiment of the present invention;

[0067] Figure 6 The decoding feature map and deep feature map provided by the embodiments of the present invention. DETAILED DESCRIPTION

[0068] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0069] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0070] The present invention provides a deep learning-based complex lesion segmentation method for retinopathy of prematurity. Figure 1 The figure shows a flow chart of a complex lesion segmentation method for retinopathy of prematurity based on deep learning. Figure 2 The figure shows a feature extraction diagram of a complex lesion segmentation method for retinopathy of prematurity based on deep learning provided by the present invention. The specific steps are as follows:

[0071] S1: Obtain ROP lesion segmentation dataset, and use data enhancement strategy to enhance the dataset to obtain the enhanced dataset;

[0072] S2: Start training, use the preset encoder to encode the enhanced data, and obtain n layers of multi-scale coding features. In a specific embodiment, the image is encoded using the PVT_V2 pre-trained model to obtain 4 layers of multi-scale coding features F1, F2, F3, and F4, such as Figure 5 As shown;

[0073] More specifically, the model encodes the image in the following steps:

[0074] Input the original image into encoder Enc1 to obtain the encoded feature F1;

[0075] Input the encoding feature F1 into the encoder Enc2 to obtain the encoding feature F2;

[0076] Repeat the above encoding process until the encoding feature F is obtained n The encoding ends, where n is the total number of layers and the value of n is 4.

[0077] S3: Input the four layers of multi-scale encoded features F1, F2, F3, and F4 into a multi-scale feature fusion module. The multi-scale feature fusion module compresses the feature map dimension using strip pooling and fuses high-level features into its own features through matrix multiplication operations to obtain three layers of corrected features F1', F2', and F3' after multi-scale context correction.

[0078] More specifically, the multi-scale feature fusion is processed in two cases:

[0079] For the top-level multi-scale feature fusion module, the encoded feature F n-1 and encoding feature F n Perform feature fusion to obtain the correction feature F' n-1 ;

[0080] For the non-top-level multi-scale feature fusion module, the encoding feature F i and its two adjacent high-level features F i+1 , F i+2 Perform feature fusion to obtain the correction feature F' i , where n is the total number of layers, i+2 is less than n, and n is 4.

[0081] More specifically, the specific steps of the non-top-most multi-scale feature fusion module for feature fusion are:

[0082] The encoding feature F i and encoding feature F i+1 Perform feature fusion to obtain enhanced features

[0083] Enhance the features and encoding feature F i+2 Perform feature fusion to obtain the correction feature F' i , where n is the total number of layers, i+2 is less than n, and n is 4.

[0084] More specifically, the specific steps of the feature fusion are:

[0085] like Figure 3 As shown, the convolution operation is used to encode the feature Fi+1 Transformed into a feature with the same encoding feature F i The feature space with the same number of channels C obtains the feature map The expression is as follows:

[0086]

[0087] The high-level features after adjusting the number of channels Strip pooling is used for encoding in the horizontal and vertical directions, and the pooling window sizes are [H i+1 ,1] and [1,W i+1 ], and get two one-dimensional eigenvectors and The stripe pooling process is as follows:

[0088]

[0089] The feature map after strip pooling and Associating, we get an attention matrix A∈R C ×C , where C is the number of channels, and the attention matrix A encodes the column and row correlation of the higher-level feature map, as shown below:

[0090]

[0091] in, is a matrix multiplication. To adjust the generalized correlation between the feature maps of rows and columns, M is a learnable matrix with a window size of [H×W]. It is initialized to an all-1 matrix at the beginning of training. In particular, when H and W are equal, M is the identity matrix, and the attention matrix can be simplified to:

[0092]

[0093] For the attention matrix A and input feature F i Perform matrix multiplication to obtain enhanced features The expression is as follows:

[0094]

[0095] Where n is the total number of layers, i+1 is less than n, and n is 4.

[0096] Through this process, the multi-scale feature fusion module can effectively combine features at different levels and improve the expressiveness of features.

[0097] S4: Use the decoder to upsample the highest-level coding feature F4 step by step to obtain n-1 layers of decoding features G4, G3, and G2. The number of upsampling layers corresponds to the number of coding layers, such as Figure 6 As shown;

[0098] More specifically, the specific steps of using the decoder to perform step-by-step upsampling on the highest-level coding feature F4 are:

[0099] The highest level encoding feature F n Input decoder Dec n , get the decoding feature G n ;

[0100] The decoded feature G n and the correction characteristic F' n-1 Sum the feature matrix and input the summation result into the decoder Dec n-1 , get the decoding feature G n-1 ;

[0101] The above upsampling process is repeated until the decoding feature G2 is obtained, and the upsampling is completed, where n is the total number of layers.

[0102] S5: Input the decoded features G4, G3, and G2 into the multi-scale feature fusion module for feature fusion to obtain the deep feature G2', such as Figure 6 As shown, the deep feature G2' is subjected to bilinear interpolation upsampling and convolution to adjust the number of channels to obtain the first prediction image Y1 for auxiliary branch lesion segmentation. The model parameters are optimized by using the first prediction image Y1 and the label for loss calculation;

[0103] S6: The deep features G2' and the lowest-level correction features F1' are input into the feature enhancement module for feature enhancement. The rich spatial information of the shallow features and the rich semantic information of the deep features are effectively integrated to obtain the aggregated features F", which are then subjected to bilinear interpolation upsampling and convolution to adjust the number of channels to obtain the final second prediction map Y2 of the lesion probability;

[0104] More specifically, the feature enhancement module performs feature enhancement in the following manner: shallow features F'1 and deep features G'2 are input into the feature enhancement module, and the shallow features F'1 and deep features G'2 are fused through two parallel feature enhancement branches to compensate for the lack of spatial boundary information of the deep features and the lack of semantic information of the shallow features. Each branch includes a reverse attention module and a boundary extraction module. The outputs of the two parallel feature enhancement branches are connected through a 3x3 convolution to obtain the aggregated feature F", as shown in FIG. Figure 4 As shown, the expression is as follows:

[0105] F′ l=RA(F′1,G′2)+EA(F′1,G′2)

[0106] F′ h =RA(G′2,F′1)+EA(G′2,F′1)

[0107] F″=C 3×3 (Concat(F′ l ,F′ h ))

[0108] Among them, RA is the reverse attention module, EA is the boundary extraction module, and F' l is the output matrix of the first branch, F' h is the output matrix of the second branch, C 3×3 (Concat(·)) represents a 3×3 convolution, and the expression of the reverse attention module is:

[0109] RA(T1,T2)=(1-σ(T1))⊙(T2⊙σ(T2))+T1

[0110] Here, T1 and T2 represent different feature inputs, σ represents the Sigmoid activation function, and ⊙ represents element-by-element multiplication. In this process, the inverse operation of 1-Sigmoid is first applied to the T1 feature map, refining the imprecise and rough estimate into an accurate and complete prediction map. The processed result is then element-by-element multiplied with the T2 feature map, which has been weighted by the activation function, to re-mine and reweight features missed in previous layers.

[0111] The expression of the boundary extraction module is:

[0112] EA(T1,T2)=T1⊙(C DW (T2))⊙σ(G)

[0113] Among them, T1, T2 represent different feature inputs, C DW represents depth-wise separable convolution, ⊙ represents element-wise multiplication, and G represents the gradient magnitude map, which is expressed as:

[0114]

[0115] Among them, G x and G y is the convolution kernel parameter with fixed weight, and its expression is:

[0116]

[0117] S7: Calculate the loss of the first predicted image Y1 and the second predicted image Y2 with the true label, and optimize the model parameters. The model parameters refer to the parameters of the four modules: encoder, decoder, multi-scale feature fusion module and feature enhancement module. Iterate the training to obtain the trained model.

[0118] More specifically, the loss calculation is performed using a cross entropy loss function.

[0119] S8: Segment the image to be segmented using the trained model to obtain a segmentation result.

[0120] The method of the present invention and six other existing advanced medical image segmentation methods are used to conduct comparative experiments on the ROP private dataset. The comparative results of the experiments on the ROP dataset are shown in Table 1.

[0121] Table 1

[0122] Methods line ridge fiber RD mean_auc mean_iou mean_dice CE-Net 19.23 69.08 81.43 88.38 64.54 52.31 65.63 SPNet 14.74 62.62 68.96 92.40 59.68 42.30 55.05 AMSUNet 14.40 63.47 75.15 91.28 61.08 39.29 52.06 MISSformer 16.11 67.79 79.17 90.34 63.35 46.45 59.71 H2former 15.69 66.37 74.19 91.49 61.94 44.22 57.79 SSformer 31.50 70.14 86.40 95.33 70.84 52.63 66.17 Our 33.80 72.39 87.92 93.68 71.95 54.18 67.83

[0123] The comparative experiments above demonstrate that the proposed method can effectively segment the four lesion structures. It significantly outperforms other methods in terms of the three evaluation metrics, mean_auc, mean_iou, and mean_dice. This demonstrates that the proposed method achieves the best performance compared to existing state-of-the-art methods on both datasets.

[0124] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A deep learning-based complex lesion segmentation method for retinopathy of prematurity, characterized by: The steps include: Obtain an image dataset, and use a data enhancement strategy to enhance the dataset to obtain an enhanced dataset; Start training and use the preset encoder to encode the enhanced data to obtain n-layer multi-scale encoding features; Inputting the n-layer multi-scale coding features into a multi-scale feature fusion module, the multi-scale feature fusion module compresses the feature map dimension using strip pooling, and fuses the high-level features into its own features through matrix multiplication operations to obtain n-1 layers of corrected features; The decoder is used to upsample the highest-level coding features step by step to obtain n-1 layers of decoding features, where the number of upsampling layers corresponds to the number of coding layers; Inputting the decoded features into a multi-scale feature fusion module for feature fusion to obtain deep features. The deep features are subjected to bilinear interpolation upsampling and convolution to adjust the number of channels to obtain a first prediction map. The model parameters are optimized by using the first prediction map and the label for loss calculation; The deep features and the lowest-level correction features are input into the feature enhancement module for feature enhancement to obtain aggregated features, and the aggregated features are subjected to bilinear interpolation upsampling and convolution to adjust the number of channels to obtain a second prediction map; Calculate the loss between the first predicted image and the second predicted image and the true label, optimize the model parameters, and iterate the training to obtain a trained model; The trained model is used to segment the image to be segmented to obtain a segmentation result.

2. The deep learning-based complex lesion segmentation method for retinopathy of prematurity according to claim 1, characterized in that: The preset encoder is the PVT_V2 pre-trained model. The specific steps of encoding the image by the model are as follows: Input the original image into encoder Enc1 to obtain the encoded feature F1; Input the encoding feature F1 into the encoder Enc2 to obtain the encoding feature F2; Repeat the above encoding process until the encoding feature F is obtained n The encoding ends, where n is the total number of layers.

3. The deep learning-based complex lesion segmentation method for retinopathy of prematurity according to claim 1, characterized in that: The multi-scale feature fusion is processed in two cases: For the top-level multi-scale feature fusion module, the encoded feature F n-1 and encoding feature F n Perform feature fusion to obtain the correction feature F' n-1 ; For the non-top-most multi-scale feature fusion module, the encoding feature F i and its two adjacent high-level features F i+1 , F i+2 Perform feature fusion to obtain the correction feature F' i , where n is the total number of layers and i+2 is less than n.

4. The deep learning-based complex lesion segmentation method for retinopathy of prematurity according to claim 3, characterized in that: The specific steps of the non-top-most multi-scale feature fusion module for feature fusion are as follows: The encoding feature F i and encoding feature F i+1 Perform feature fusion to obtain enhanced features Enhance the features and encoding feature F i+2 Perform feature fusion to obtain the correction feature F' i , where n is the total number of layers and i+2 is less than n.

5. A deep learning-based complex lesion segmentation method for retinopathy of prematurity according to claim 3 or 4, characterized in that: The specific steps of feature fusion are: Use convolution operation to encode the feature F i+1 Transformed into a feature with the same encoding feature F i The feature space with the same number of channels C obtains the feature map The expression is as follows: The high-level features after adjusting the number of channels Strip pooling is used for encoding in the horizontal and vertical directions, and the pooling window sizes are [H i+1 ,1] and [1,W i+1 ], and get two one-dimensional eigenvectors and The stripe pooling process is as follows: The feature map after strip pooling and Associating, we get an attention matrix A∈R C×C , where C is the number of channels, the expression is as follows: in, is a matrix multiplication, M is a learnable matrix with a window size of [H×W], and is initialized to an all-1 matrix at the beginning of training. In particular, when H and W are equal, M is the identity matrix, and the attention matrix can be simplified to: For the attention matrix A and input feature F i Perform matrix multiplication to obtain enhanced features The expression is as follows: Where n is the total number of layers, and i+1 is less than n.

6. The deep learning-based complex lesion segmentation method for retinopathy of prematurity according to claim 1, characterized in that: The specific steps of using the decoder to perform step-by-step upsampling of the highest-level coding features are as follows: The highest level encoding feature F n Input decoder Dec n , get the decoding feature G n ; The decoded feature G n and the correction characteristic F' n-1 Sum the feature matrix and input the summation result into the decoder Dec n-1 , get the decoding feature G n-1 ; The above upsampling process is repeated until the decoding feature G2 is obtained, and the upsampling is completed, where n is the total number of layers.

7. The deep learning-based complex lesion segmentation method for retinopathy of prematurity according to claim 1, characterized in that: The feature enhancement module includes two parallel feature enhancement branches, which input shallow features and deep features into the two parallel feature enhancement branches respectively, output two feature enhancement matrices, and connect the two feature enhancement matrices through convolution to obtain aggregated features.

8. The deep learning-based complex lesion segmentation method for retinopathy of prematurity according to claim 7, characterized in that: The feature enhancement branch includes a reverse attention module and a boundary extraction module connected in parallel. The output matrices of the reverse attention module and the boundary extraction module connected in parallel are connected by matrix addition to output a feature enhancement matrix. The expression of the reverse attention module is: RA(T1,T20=(1-σ(T1))⊙(T2⊙σ(T2))+T1 Among them, T1 and T2 represent different feature inputs, σ represents the Sigmod activation function, and ⊙ represents element-by-element multiplication; The expression of the boundary extraction module is: EA(T1,T2)=T1⊙(C DW (T2))⊙σ(G) Among them, T1, T2 represent different feature inputs, C DW represents depth-wise separable convolution, ⊙ represents element-wise multiplication, and G represents the gradient magnitude map, which is expressed as: Among them, G x and G y is the convolution kernel parameter with fixed weight, and its expression is:

9. The method for segmenting complex lesions of retinopathy of prematurity based on deep learning according to claim 8, characterized in that: The shallow feature F'1 and the deep feature G'2 are input into two parallel feature enhancement branches, and two feature enhancement matrices are output. The two feature enhancement matrices are connected by convolution to obtain the aggregated feature F". The expression is as follows: F′ l =RA(F′1,G′2)+EA(F′1,G′2) F′ h =RA(G′2,F′1)+EA(G′2,F′1) F″=C 3×3 (Concat(F l ,F h )) Among them, RA is the reverse attention module, EA is the boundary extraction module, and F' l is the output matrix of the first branch, F' h is the output matrix of the second branch, C 3×3 (Concat(·)) represents a 3×3 convolution.

10. The deep learning-based complex lesion segmentation method for retinopathy of prematurity according to claim 1, characterized in that: The loss calculation is performed using a cross entropy loss function.

Images