Fine-grained lesion characteristic diabetic retinopathy grading method and system

Abstract

The application provides a fine-grained lesion feature diabetic retinopathy grading method and system, and belongs to the field of image processing.The method comprises the following steps: S1, inputting a diabetic retinal fundus color image into a high-low frequency feature extraction network, realizing simultaneous extraction of high-low frequency features through a block embedding module and multiple stacking of high-low frequency blocks, and obtaining a processed feature map; S2, inputting the feature map into a sample relationship interaction module, performing information interaction between different samples through a self-attention mechanism of a Transformer, obtaining a new feature map, and obtaining a grading result through a final classifier; and S3, optimizing a student model proposed in S1 and S2 through a momentum distillation learning module, continuously updating a momentum model through an exponential moving average, and then training the student model by using pseudo-labels generated by the momentum model.The method provided by the application can improve the effect of diabetic retinopathy grading and provide a basis for disease grading.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image processing, and particularly relates to a fine-grained lesion feature diabetic retinopathy grading method and system. BACKGROUND

[0002] Diabetic retinopathy (DR) is one of the most common complications of diabetes and one of the leading causes of blindness. With the extension of human life, the non-standardization of lifestyle and other factors, the number of diabetic patients is expected to continue to rise. Under this background, regular retinal screening for diabetic patients has been proven to be a cost-effective and essential care measure. Clinicians can diagnose diabetic retinopathy by observing lesions in fundus images and grading the patient's condition according to the type and number of lesions. Doctors can develop appropriate treatment plans based on the grading results. However, ophthalmologists need to invest a lot of time and effort in diagnosing diabetic retinopathy, which puts a huge pressure on medical resources. Therefore, designing an automatic diabetic retinal diagnosis algorithm not only can reduce the workload of doctors, but also can improve the efficiency and accuracy of diagnosis.

[0003] Although deep learning methods have achieved great success in diabetic retinopathy disease grading, there are still three challenges that need to be addressed.

[0004] First, in clinical practice, ophthalmologists usually identify lesions by observing fine-grained features such as texture and color of lesions, and then grade the disease according to the type and number of lesions. Most current diabetic retinopathy grading methods are based on convolutional neural networks. Although convolutional neural networks can effectively capture fine-grained features through local convolution, even in the case of limited data, they can produce observable results. However, convolutional neural networks have limitations in long-range modeling and cannot effectively capture the relationship between different lesions, which limits the further improvement of diabetic retinopathy grading accuracy. Visual Transformer model captures the dependency between positions in the input sequence through self-attention mechanism, thus modeling long-range dependencies more effectively. However, due to the lack of inductive bias, visual Transformer needs a large amount of data to effectively extract fine-grained features and achieve satisfactory results. Although some researchers have tried to use visual Transformer for diabetic retinopathy disease grading, its performance is difficult to fully meet the needs. Therefore, how to design a network architecture that has the advantages of both convolutional neural networks and visual Transformer to effectively extract fine-grained lesion features in diabetic retinal images has become a challenge in diabetic retinopathy grading.

[0005] In addition, under clinical conditions, the number of patients with different levels of diabetic retinopathy varies, which also leads to serious class imbalance problems in diabetic retinopathy datasets, and the overall distribution is long-tailed. This unbalanced class distribution may cause the model to pay too much attention to the diabetic retinopathy level with more samples, thereby affecting the final grading performance. Although many researchers have proposed a variety of long-tail visual recognition methods, these long-tail visual recognition methods often improve the performance of the tail category at the expense of the performance of the head category. Therefore, how to optimize the performance of the tail category without compromising the performance of the head category has become a challenge in diabetic retinopathy grading.

[0006] Ophthalmologists typically grade lesions based on their type and number. However, due to the morphological variations or subtlety of some lesions, they are sometimes difficult to accurately detect and label. Consequently, some images in a dataset may be incorrectly labeled, potentially causing the model to overfit to noisy labels and thus reducing its generalization performance. In this context, designing a reasonable training method to ensure that the model is not penalized for producing reasonable predictions that differ from the label becomes a challenge in diabetic retinopathy grading. Summary of the Invention

[0007] To solve the above technical problems, the present invention provides a method for grading diabetic retinopathy based on fine-grained lesion characteristics, comprising the following steps:

[0008] Step S1: Input the diabetic retinopathy fundus color image into the high- and low-frequency feature extraction network, and realize the simultaneous extraction of high- and low-frequency features through the block embedding module and multiple stacking of high- and low-frequency blocks to obtain the processed feature map ;

[0009] Step S2: The feature map Input sample relationship interaction module, use Transformer's self-attention mechanism to interact information between different samples, and obtain gradients from different virtual samples so that the tail category can be more fully trained to obtain a new feature map , and obtain the classification results through the final classifier ;

[0010] Step S3: Use the high- and low-frequency feature extraction network and the sample relationship interaction module as a student model, optimize the student model using a representation learning method based on momentum distillation, continuously update the momentum model through exponential moving average, and then use the pseudo labels generated by the momentum model to train the student model.

[0011] Beneficial effects:

[0012] 1. The method disclosed in the application proposes a feature extraction method based on high and low frequency feature combination, which combines attention mechanism and convolution operation by using high and low frequency blocks, can extract fine-grained lesion features, and can extract the relationship between different lesions, and can effectively improve the disease grading ability of the model.

[0013] 2. The method disclosed in the application proposes a long tail visual recognition method based on sample relationship, considers the relationship between samples from the perspective of learning, obtains additional information by feature interaction between the fine-grained lesion features of different category samples, so that the tail category can obtain gradient from different virtual samples, thereby optimizing the performance of the tail category without damaging the performance of the head category.

[0014] 3. The method disclosed in the application proposes a momentum distillation-based representation learning method, which obtains a momentum model by using the moving average of the model, and uses the pseudo-label generated by the momentum model as an additional supervision signal. This method can ensure that the model will not be punished because it produces different but reasonable outputs from the true label. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The flowchart of a fine-grained lesion feature diabetic retinopathy grading method in an embodiment of the application;

[0016] Figure 2A The structural diagram of the high and low frequency feature extraction network in the embodiment of the application;

[0017] Figure 2B The structural diagram of the high and low frequency block;

[0018] Figure 2C The structural diagram of the high and low frequency mixer is shown;

[0019] Figure 3 The calculation process diagram of the sample interaction module in the embodiment of the application;

[0020] Figure 4 The schematic diagram of the momentum distillation framework in the embodiment of the application;

[0021] Figure 5 The module block diagram of a fine-grained lesion feature diabetic retinopathy grading system in the embodiment of the application. DETAILED DESCRIPTION

[0022] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application and not to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0023] Embodiment one

[0024] As Figure 1 shown, the fine-grained lesion feature diabetic retinopathy grading method provided by the embodiment of the present application comprises the following steps:

[0025] Step S1: input the sugar network fundus color image into the high-low frequency feature extraction network, realize the simultaneous extraction of high-low frequency features through the block embedding module and the multiple stacking of high-low frequency blocks, and obtain the processed feature map .

[0026] Step S2: input the feature map into the sample relationship interaction module, use the self-attention mechanism of the Transformer to perform information interaction between different samples, obtain gradients from different virtual samples, enable the tail classes to be trained more fully, obtain a new feature map , and obtain a grading result through the final classifier .

[0027] Step S3: use the momentum distillation-based representation learning method to optimize the student model composed of the high-low frequency feature extraction network and the sample relationship interaction module, continuously update the momentum model through the exponential moving average, and then use the pseudo-label generated by the momentum model to train the student model.

[0028] In one embodiment, the above step S1: input the sugar network fundus color image into the high-low frequency feature extraction network, realize the simultaneous extraction of high-low frequency features through the block embedding module and the multiple stacking of high-low frequency blocks, and obtain the processed feature map , specifically comprises:

[0029] Step S11: construct the high-low frequency feature extraction network, which is composed of multiple block embedding modules and high-low frequency blocks; the sugar network fundus color image or the feature map is first subjected to the block embedding module, subjected to feature processing and down-sampling through convolution operation, and obtains a down-sampled feature map , wherein, , C respectively represent the number of pixel points and the number of channels of the feature map ;

[0030] Step S12: perform the feature map The application layer normalization processing is performed to obtain a normalized feature map , and then is divided into and by channel division, and is input into a high-low frequency block composed of a high frequency mixer and a low frequency mixer, wherein ;

[0031] This step is crucial for accelerating the convergence speed of model training, stabilizing the training process, and effectively improving the generalization ability of the model.

[0032] Step S13: inputting into a high frequency mixer, the high frequency mixer being composed of a max pooling operation and a deep convolution operation, and the high frequency mixer dividing into and , after being processed by a max pooling layer and a linear layer, after being processed by a linear layer and a deep convolution layer, formulas (1) and (2) are as follows:

[0033] (1)

[0034] (2)

[0035] wherein DwConv represents a deep convolution operation, and represent the output of the high frequency mixer;

[0036] The high frequency mixer is composed of a max pooling operation and a deep convolution operation, and is used to extract and combine high frequency features. The max pooling operation reduces the size of the feature map by selecting the maximum value of each local region, so as to retain significant high frequency features. The deep convolution operation further extracts and integrates high frequency features through a series of convolution layers, so as to obtain richer expression ability.

[0037] Step S14: inputting into a low frequency mixer, the low frequency mixer first reducing the size of the low frequency feature map by using a pooling strategy, then processing the features by using a multi-head self-attention mechanism, and finally performing up-sampling to restore the feature map to the original size, as shown in formula (3):

[0038] (3)

[0039] wherein represents an average pooling operation, represents a multi-head self-attention mechanism, represents up-sampling;

[0040] The low-frequency mixer is implemented by using a self-attention operation, and is used to capture global low-frequency features.

[0041] Step S15: The outputs of the high-frequency mixer and the low-frequency mixer are stacked in the channel dimension, and a deep convolution is used to realize information exchange between image blocks, and formulas (4) and (5) are as follows:

[0042] (4)

[0043] (5)

[0044] The high-low frequency feature extraction network designed in the application can effectively capture specific frequency information on the corresponding channel, so as to learn more comprehensive and rich features in a wider frequency range. The structure of the high-low frequency mixer provides strong support for the performance and robustness of the network.

[0045] Step S16: The feature map is obtained by performing a global average pooling in the spatial dimension. After a fully connected layer and a Sigmoid operation, the value is mapped to [0, 1], and finally it is multiplied with the original feature map to obtain a new feature map .

[0046] The high-frequency mixer and the low-frequency mixer respectively perform feature extraction tasks in the spatial dimension, and focus on capturing local details and overall structure information of the image. On this basis, the SE attention (Squeeze-and-Excitation Attention) mechanism further improves the feature expression ability of the model by applying attention operation in the channel dimension. This attention mechanism can enhance the attention of the model to key channels, that is, those channels with greater weight in a specific context, so that the model can focus more on the features most important to the classification and identification task.

[0047] Step S17: The feature map is added to the feature map to obtain a feature map , which is used to promote the rapid convergence of the model.

[0048] Step S18: After the feature map is processed by layer normalization, it is sent to the front feedback layer, and the newly obtained feature map is added to the feature map to obtain a feature map . ​

[0049] Step S19: Repeat steps S11 to S18 to form a structure of multiple block embedding modules and high and low frequency block stacking to ensure that the high and low frequency feature extraction network is fully learned and its performance is optimized.

[0050] The number of repetitions to be performed should be determined according to the complexity of the task and the training requirements. In the embodiment of the present invention, it is set to 4 iterations.

[0051] Figure 2A The schematic diagram of the high- and low-frequency feature extraction network is shown, which adopts the pyramid structure design mode commonly used in computer vision for lesion feature extraction of fundus images. Figure 2A As shown in the figure, during the feature extraction process, as the network deepens, the resolution of the feature map decreases while the number of channels increases. Through the pyramid structure, the network is able to capture image information at different scales. As the network deepens, the resolution of the feature map decreases, which helps extract higher-level abstract features. At the same time, the increase in the number of channels enables the network to better represent rich feature information. Figure 2B It is a structural diagram of high and low frequency blocks. Figure 2C The structural diagram of the high and low frequency mixer is shown.

[0052] In one embodiment, the above step S2: the feature map Input sample relationship interaction module, use Transformer's self-attention mechanism to interact information between different samples, and obtain gradients from different virtual samples so that the tail category can be more fully trained to obtain a new feature map , and obtain the classification results through the final classifier , specifically including:

[0053] Step S21: Enter the sample interaction module, first is transposed, and the self-attention mechanism in the Transformer layer is applied to the sample interaction module to achieve cross-attention between different samples, thereby obtaining the feature map ;

[0054] Step S22: Use the auxiliary classifier and the final classifier to classify Perform detailed classification processing, conduct in-depth analysis of features through the fully connected layer, then apply the Softmax function to calculate the probability distribution, and finally determine the category with the highest probability through the argmax operation to obtain the final diabetic retinopathy grading result. .

[0055] Because the batch statistics of the test data cannot be assumed, there may be a gap between the features before and after the sample interaction module. Therefore, it is not possible to simply remove the sample interaction module to infer new samples. Therefore, in addition to the final classifier, the application designs a new auxiliary classifier, which not only enables the model to learn from the final classifier, but also ensures that the features before and after the sample interaction module are consistent. To achieve this, it is only necessary to share the weights between the auxiliary classifier and the final classifier, and this simple and effective strategy is called a shared classifier. By using the shared classifier strategy, the sample interaction module can be removed at test time while still enjoying the advantages of the sample interaction module in sample relationship learning.

[0056] The relationships between different samples are diverse and complex. Traditional methods usually adopt a direct way to explore the relationships between different samples, such as joint operation on different input images, or knowledge transfer using meta-embedding or loss function. These methods require a clear definition of the relationship between samples to be used for learning, and cannot achieve dynamic representation learning of sample relationships. The sample interaction module proposed by the application establishes a sample relationship modeling mechanism, enabling the network to dynamically learn and adjust the relationship between samples at each iteration, thereby improving the model's ability to handle long-tailed distribution data. Therefore, the application introduces a sample interaction module based on S1 to solve the long-tailed visual recognition problem of DR. By combining the sample interaction module, the model not only better understands and utilizes the relationship between different samples, but also effectively deals with the class imbalance problem in the dataset, thereby significantly improving the recognition performance of the minority classes.

[0057] As shown in Figure 3 , a process schematic diagram of the sample interaction module is shown.

[0058] In one embodiment, the above step S3: taking the high-low frequency feature extraction network and the sample relationship interaction module as the student model, using a momentum distillation-based representation learning method to optimize the student model, continuously updating the momentum model by exponential moving average, and then using the pseudo-label generated by the momentum model to train the student model, specifically including:

[0059] Step S31: obtain a momentum model using an exponential moving average method, take it as a teacher model, and use it to provide supervision for the student model. Formula (6) is as follows:

[0060] (6)

[0061] wherein, represents the momentum model parameter at time t, represents the student model weight at time t, ​Controlling the smoothing degree of the exponential moving average

[0062] Step S32: By minimizing the cross-entropy loss function to make the student model prediction grading results correspond closely to the actual grading labels; by minimizing the momentum matching loss function to reduce the influence of inaccurate labels on the grading results, formulas (7)-(9) are as follows:

[0063] (7)

[0064] (8)

[0065] (9)

[0066] wherein, denotes the total number of samples; M represents the total number of categories; i represents the i-th sample; is the label of category c in the i-th sample, which is 0 or 1; is the prediction probability of category c of the i-th sample; is the pseudo label generated by the momentum model; is the momentum distillation loss weight;

[0067] Step S33: Using a linear dynamic weight adjustment strategy to control the weight of the momentum distillation loss, the weight of the momentum distillation starts from a preset minimum value and is gradually increased linearly in each training round until it reaches a preset maximum weight value, formula (10) is as follows:

[0068] (10)

[0069] wherein, and are the lower limit and the upper limit of the momentum distillation loss weight, denotes the current training round, and E denotes the total number of training rounds.

[0070] Figure 4 A schematic diagram of the momentum distillation learning framework is shown.

[0071] Embodiment two

[0072] As shown in Figure 5 , the embodiment of the present application provides a fine-grained lesion feature diabetic retinopathy grading system, comprising the following modules:

[0073] The high-low frequency feature extraction network module 41 is used for inputting the diabetic fundus color image into the high-low frequency feature extraction network, and through the block embedding module and the multiple stacking of the high-low frequency block, the high-low frequency features are extracted at the same time to obtain the processed feature map ;

[0074] The sample relationship interaction module 42 is configured to perform information interaction between different samples by using a self-attention mechanism of a Transformer, and obtain a new feature map by enabling the tail classes to be trained more fully through gradients obtained from different virtual samples. The input sample relationship interaction module, using the self-attention mechanism of the Transformer, performs information interaction between different samples, enables the tail classes to be trained more fully through gradients obtained from different virtual samples, and obtains a new feature map , and obtains a hierarchical result through a final classifier ;

[0075] The momentum distillation optimization module 43 is configured to use a representation learning method based on momentum distillation to optimize a student model by taking the high-frequency and low-frequency feature extraction network and the sample relationship interaction module as the student model, continuously update a momentum model through an exponential moving average, and then train the student model using pseudo-labels generated by the momentum model.

Claims

1. A method for grading diabetic retinopathy based on fine-grained lesion features, characterized in that: include: Step S1: Input the diabetic retinopathy fundus color image into the high- and low-frequency feature extraction network, and realize the simultaneous extraction of high- and low-frequency features through the block embedding module and multiple stacking of high- and low-frequency blocks to obtain the processed feature map , specifically including: Step S11: Construct a high- and low-frequency feature extraction network, which is composed of multiple block embedding modules and high- and low-frequency block stacks; the retinal color image or feature map first passes through the block embedding module, and is feature processed and downsampled by convolution operation to obtain the downsampled feature map ,in, , C represent feature maps The number of pixels and channels; Step S12: Feature map Apply layer normalization to obtain the normalized feature map , then Divide the channels into and , respectively input the high and low frequency blocks composed of high frequency mixer and low frequency mixer, where, ; Step S13: Input high frequency mixer, which consists of maximum pooling operation and depth convolution operation, Divided into and , After a maximum pooling layer and a linear layer, After processing by a linear layer and a depthwise convolutional layer, formulas (1) and (2) are as follows: (1) (2) Among them, DwConv represents the depth convolution operation, and represents the output of the high-frequency mixer; Step S14: Input the low-frequency mixer, which first uses a pooling strategy to reduce the size of the low-frequency feature map, then uses a multi-head self-attention mechanism for feature processing, and finally performs upsampling to restore the feature map to its original size, as shown in formula (3): (3) in, represents the average pooling operation, represents the multi-head self-attention mechanism, represents upsampling; Step S15: The outputs of the high-frequency mixer and the low-frequency mixer are stacked in the channel dimension, and depthwise convolution is used to implement information exchange between image blocks. Formulas (4) to (5) are as follows: (4) (5) Step S16: The feature map After global average pooling in one spatial dimension, the attention feature map is obtained , After the fully connected layer and Sigmoid operation, the value is mapped to [0, 1] and finally compared with the original feature map Multiply to obtain a new feature map ; Step S17: The feature map With feature map Add to obtain feature map , used to promote rapid convergence of the model; Step S18: Feature Map After layer normalization, it is sent to the forward feedback layer to combine the newly obtained feature map with the feature map Add together to get the feature map ; Step S19: repeating steps S11 to S18 to form a structure in which multiple block embedding modules and high and low frequency blocks are stacked to ensure that the high and low frequency feature extraction network is fully learned and its performance is optimized; Step S2: The feature map Input sample relationship interaction module, use Transformer's self-attention mechanism to interact information between different samples, and obtain gradients from different virtual samples so that the tail category can be more fully trained to obtain a new feature map , and obtain the classification results through the final classifier , specifically including: Step S21: Enter the sample interaction module, first is transposed, and the self-attention mechanism in the Transformer layer is applied to the sample interaction module to achieve cross attention between different samples, thereby obtaining the feature map ; Step S22: Use the auxiliary classifier and the final classifier to classify Perform detailed classification processing, conduct in-depth analysis of features through the fully connected layer, then apply the Softmax function to calculate the probability distribution, and finally determine the category with the highest probability through the argmax operation to obtain the final diabetic retinopathy grading result. ; Step S3: Use the high- and low-frequency feature extraction network and the sample relationship interaction module as a student model, optimize the student model using a representation learning method based on momentum distillation, continuously update the momentum model through exponential moving average, and then use the pseudo labels generated by the momentum model to train the student model.

2. The fine-grained lesion feature diabetic retinopathy grading method according to claim 1, characterized in that: Step S3: using the high- and low-frequency feature extraction network and the sample relationship interaction module as a student model, optimizing the student model using a representation learning method based on momentum distillation, continuously updating the momentum model through exponential moving average, and then training the student model using pseudo labels generated by the momentum model, specifically includes: Step S31: Use the exponential moving average method to obtain the momentum model, which is used as the teacher model to provide supervision for the student model. Formula (6) is as follows: (6) in, express Momentum model parameters at time , represents the student model weight at time t, Used to control the smoothing degree of the exponential moving average; Step S32: By minimizing the cross entropy loss function To make the grading results predicted by the student model closely correspond to the actual grading labels; by minimizing the momentum matching loss function To reduce the impact of inaccurate labels on the classification results, formulas (7) to (9) are as follows: (7) (8) (9) in, Represents the total number of samples; M represents the total number of categories; i represents the i-th sample; is the label of category c in the i-th sample, whose value is 0 or 1; is the predicted probability of the i-th sample category c; Pseudo labels generated for the momentum model; is the momentum distillation loss weight; Step S33: Use a linear dynamic weight adjustment strategy to control the weight of the momentum distillation loss. The weight of the momentum distillation starts from a preset minimum value and gradually increases linearly in each training round until it reaches a preset maximum weight value. Formula (10) is as follows: (10) in, and are the lower and upper limits of the momentum distillation loss weight, represents the current training round, and E represents the total number of training rounds.

3. A fine-grained lesion feature diabetic retinopathy grading system, characterized by: Includes the following modules: The high- and low-frequency feature extraction network module is used to input the diabetic retinopathy fundus color image into the high- and low-frequency feature extraction network. Through the block embedding module and multiple stacking of high- and low-frequency blocks, the high- and low-frequency features are extracted simultaneously to obtain the processed feature map. , specifically including: Step S11: Construct a high- and low-frequency feature extraction network, which is composed of multiple block embedding modules and high- and low-frequency block stacks; the retinal color image or feature map first passes through the block embedding module, and is feature processed and downsampled by convolution operation to obtain the downsampled feature map ,in, , C represent feature maps The number of pixels and channels; Step S12: Feature map Apply layer normalization to obtain the normalized feature map , then Divide the channels into and , respectively input the high and low frequency blocks composed of high frequency mixer and low frequency mixer, where, ; Step S13: Input high frequency mixer, which consists of maximum pooling operation and depth convolution operation, Divided into and , After a maximum pooling layer and a linear layer, After processing by a linear layer and a depthwise convolutional layer, formulas (1) and (2) are as follows: (1) (2) Among them, DwConv represents the depth convolution operation, and represents the output of the high-frequency mixer; Step S14: Input the low-frequency mixer, which first uses a pooling strategy to reduce the size of the low-frequency feature map, then uses a multi-head self-attention mechanism for feature processing, and finally performs upsampling to restore the feature map to its original size, as shown in formula (3): (3) in, represents the average pooling operation, represents the multi-head self-attention mechanism, represents upsampling; Step S15: The outputs of the high-frequency mixer and the low-frequency mixer are stacked in the channel dimension, and depthwise convolution is used to implement information exchange between image blocks. Formulas (4) to (5) are as follows: (4) (5) Step S16: The feature map After global average pooling in one spatial dimension, the attention feature map is obtained , After the fully connected layer and Sigmoid operation, the value is mapped to [0, 1] and finally compared with the original feature map Multiply to obtain a new feature map ; Step S17: The feature map With feature map Add to obtain feature map , used to promote rapid convergence of the model; Step S18: Feature Map After layer normalization, it is sent to the forward feedback layer to combine the newly obtained feature map with the feature map Add together to get the feature map ; Step S19: repeating steps S11 to S18 to form a structure in which multiple block embedding modules and high and low frequency blocks are stacked to ensure that the high and low frequency feature extraction network is fully learned and its performance is optimized; Sample relationship interaction module, used to transform feature maps Input sample relationship interaction module, use Transformer's self-attention mechanism to interact information between different samples, and obtain gradients from different virtual samples so that the tail category can be more fully trained to obtain a new feature map , and obtain the classification results through the final classifier , specifically including: Step S21: Enter the sample interaction module, first is transposed, and the self-attention mechanism in the Transformer layer is applied to the sample interaction module to achieve cross attention between different samples, thereby obtaining the feature map ; Step S22: Use the auxiliary classifier and the final classifier to classify Perform detailed classification processing, conduct in-depth analysis of features through the fully connected layer, then apply the Softmax function to calculate the probability distribution, and finally determine the category with the highest probability through the argmax operation to obtain the final diabetic retinopathy grading result. ; The momentum distillation optimization module is used to use the high- and low-frequency feature extraction network and the sample relationship interaction module as student models, optimize the student model using a representation learning method based on momentum distillation, continuously update the momentum model through exponential moving average, and then use the pseudo labels generated by the momentum model to train the student model.

Images