OCT image classification method based on self-supervised learning
By adding attention mechanisms and designing specific loss functions in self-supervised learning, the problems of limited supervision information and high manual annotation cost in OCT image classification are solved, and the effect of improving model accuracy and learning efficiency is achieved.
Patent Information
- Application Number
- CN202510115326.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-24
- Publication Date
- 2025-03-07
AI Technical Summary
The existing self-supervised learning method in OCT image classification has a low accuracy of the classifier and requires a large number of labeling of sample images, which increases the cost of manual labeling.
The OCT image classification method based on self-supervised learning is adopted to improve the model performance by adding attention mechanisms and designing specific loss functions during pre-training. Specific steps include performing multiple random data augmentation on the original image, using neural networks and attention mechanisms as encoders, mapping the enhanced image to the potential representation space, and adjusting the encoder through the extender to reduce computational losses.
By minimizing the distance between two embedding vectors of the same image, ensuring feature diversity, and preventing the covariance between embedding vectors from zero through the loss function, thereby improving the learning efficiency and classification accuracy of the model and reducing the cost of manual labeling.
Smart Images
Figure CN119580015A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image classification, and in particular to a classification method for OCT images based on self-supervised learning. Background Art
[0002] In traditional image recognition, the training process relies on a large number of well-labeled samples to ensure that the model has good generalization ability. However, in practical applications, obtaining accurate labels for each training sample often takes a lot of time and manpower. Therefore, recognition models based on self-supervised learning have gradually become a hot topic of research. Self-supervised learning automatically generates labels by designing pre-tasks, helping the model learn useful features from unlabeled data without manual labeling.
[0003] In self-supervised learning, the model is trained by self-labeling the intrinsic structure in the data to obtain "pseudo-labels". These pseudo-labels do not rely on manual annotation, but are inferred through the intrinsic relationship of the data itself. For example, by predicting certain transformations of the image (such as rotation, cropping, color change, etc.), the model can learn the high-level semantic features of the image. This approach not only avoids the high cost of manual annotation, but also can train efficient models in the absence of a large amount of labeled data. However, compared with explicit labels, the supervised information of self-supervised learning is usually more limited, which may lead to the trained classifier being less accurate than traditional supervised learning methods. Summary of the invention
[0004] Purpose of the invention: The purpose of the present invention is to provide a classification method for OCT images based on self-supervised learning. During the pre-training process, an attention mechanism is added to the commonly used deep learning, and a loss function is designed to improve the performance of the model and solve the problems existing in the background technology.
[0005] Technical solution: The OCT image classification method based on self-supervised learning described in the present invention comprises the following steps: (1) Perform various random data enhancements on the original image; (2) Using a neural network and an attention mechanism as an encoder to map the enhanced image to a latent representation space; (3) Introducing an expander as an auxiliary tool to adjust the encoder by reducing the computational loss; (4) After training is completed, the features extracted by the encoder are used to classify the OCT images.
[0006] Furthermore, step (1) is as follows: randomly crop and scale the image to a size of 224x224, and use bicubic interpolation to process the scaling; flip the image horizontally, apply color jitter to the image including: adjusting brightness, contrast, saturation and hue, and converting the image to a grayscale image; apply Gaussian blur to the image; convert the image to Tensor format, and normalize the image.
[0007] Furthermore, step (2) is as follows: the neural network is a residual network ResNet-50, and the ECA lightweight channel attention mechanism is introduced. The ECA module is introduced in the residual connection part of each residual module of ResNet-50, that is, after the output feature map of each residual module is added to the input (residual connection), the channel weighting is performed through the ECA module.
[0008] Furthermore, in step (3), the expander is a three-layer fully connected layer.
[0009] Further, in step (3), the details are as follows: Assume that the converted image is obtained by the encoder The view and Encoded as and , then by the expander Map it to an embedded value and .
[0010] Furthermore, in step (3), the loss function is as follows: First, let the dataset The images in are processed in batches according to the set batch size, and the feature vectors from the two branches of the twin architecture are represented as and , and It is composed of n d-dimensional vectors; Then, and The invariance loss function between Defined as the average of the squared Euclidean distances between each pair of vectors: ; definition is the normalized standard deviation, and the formula is: ; in, is a small scalar to prevent numerical instability; Each of the vectors in The vector of dimension values is composed of Representation; Definition is the hinge function of the normalized standard deviation of the embeddings with batch-size dimension: ; in, is a constant target value for the standard deviation; the effect that this loss function aims to achieve is that the regularized standard deviation within each batch is equal to , thus preventing crashes caused by mapping all inputs to the same vector.
[0011] definition The covariance matrix of is: ; The covariance regularization term Defined as , represents the sum of squares of coefficients outside the diagonal, and uses a dimension-dependent scaling factor 1 / d to adjust the impact of the loss function: This term makes the off-diagonal coefficients is close to 0, making different dimensions of the embedding uncorrelated and preventing them from encoding very similarly.
[0012] The present invention provides an OCT image classification system based on self-supervised learning, comprising: Enhancement module: used to perform various random data enhancements on the original image; Encoding module: used to map the enhanced image to the latent representation space using a neural network and an attention mechanism as an encoder; Adjustment module: used to introduce an expander as an auxiliary tool to adjust the encoder by reducing the calculation loss; Classification module: used to classify OCT images using the features extracted by the encoder after training is completed.
[0013] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, it implements a classification method for OCT images based on self-supervised learning according to any one of the items.
[0014] A storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, it implements any one of the OCT image classification methods based on self-supervised learning.
[0015] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: by minimizing the distance between two embedding vectors from the same image, the standard deviation of each embedding vector in the same batch is ensured to be kept above the threshold, thereby ensuring feature diversity; ensuring that the covariance between the same batch of embedding vectors is zero. After the pre-trained model is fine-tuned with parameters, the corresponding eye image classification model is obtained, without the need for a large number of labeled sample images, which greatly saves the cost of manual labeling and improves the efficiency of model learning. Compared with the traditional self-supervised learning method, the method proposed in the present invention improves the classification accuracy of OCT images to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 is a flow chart of the present invention; Figure 2 It is a comparison result diagram of the present invention. DETAILED DESCRIPTION
[0017] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0018] like Figure 1 As shown, an embodiment of the present invention provides a classification method for OCT images based on self-supervised learning, comprising the following steps: Perform various random data augmentations on the original image; specifically: Randomly crop and scale the image to 224x224, using bicubic interpolation (BICUBIC) to handle scaling. Flip the image horizontally with a probability of 50%. Apply color jitter (adjust brightness, contrast, saturation, and hue) to the image with a probability of 80%. Convert the image to grayscale with a probability of 20%. Apply Gaussian blur to the image with a blur level of 100% (always applied). Convert the image to Tensor format for use as input to the PyTorch model. Standardize the image. The mean and standard deviation (std) here are usually based on the statistical values of the ImageNet dataset, with the purpose of adjusting the color distribution of the image to improve training stability and convergence speed; among them, the original image comes from the BOE dataset, which was collected by Srinivasan et al. from 45 subjects. The dataset includes three types of images: 723 AMD images, 1,101 DME images, and 1,407 NORMAL images, totaling 3,231 images.
[0019] (2) A neural network and an attention mechanism are used as encoders to map the enhanced image to a latent representation space. The neural network is a residual network ResNet-50, and an ECA lightweight channel attention mechanism is introduced. The ECA module is introduced in the residual connection part of each residual module of ResNet-50. Specifically, after the output feature map of each residual module is added to the input (residual connection), the channel weighting is performed through the ECA module.
[0020] (3) An expander is introduced as an auxiliary tool to adjust the encoder by reducing the computational loss; the expander is a three-layer fully connected layer. The details are as follows: Assume that the converted image is the encoder. The details are as follows: Assume that the converted image is the encoder The view and Encoded as and , then by the expander Map it to an embedded value and .
[0021] The loss function is as follows: First, let the data set The images in are processed in batches according to the set batch size, and the feature vectors from the two branches of the twin architecture are represented as and , and It is composed of n d-dimensional vectors; Then, and The invariance loss function between Defined as the average of the squared Euclidean distances between each pair of vectors: ; definition is the normalized standard deviation, and the formula is: ; in, is a small scalar to prevent numerical instability; Each of the vectors in The vector of dimension values is composed of Representation; Definition is the hinge function of the normalized standard deviation of the embeddings with batch-size dimension: ; in, is a constant target value for the standard deviation; the effect that this loss function aims to achieve is that the regularized standard deviation within each batch is equal to , thus preventing crashes caused by mapping all inputs to the same vector.
[0022] definition The covariance matrix of is: ; The covariance regularization term Defined as , represents the sum of squares of coefficients outside the diagonal, and uses a dimension-dependent scaling factor 1 / d to adjust the impact of the loss function: This term makes the off-diagonal coefficients is close to 0, making different dimensions of the embedding uncorrelated and preventing them from encoding very similarly.
[0023] (4) After training is completed, the features extracted by the encoder are used to classify the OCT images.
[0024] To evaluate the performance of our model, we compared it with existing self-supervised methods SimCLR, MoCo, SimSiam, and BarlowTwins, all of which were trained under the same settings. We used a variety of evaluation metrics, including the ROC curve and the precision-recall curve (PR). The visualization results of the two curves are shown in Figure 2. Figure 2 As shown, the red line is the method of the present invention.
[0025] The embodiment of the present invention further provides an OCT image classification system based on self-supervised learning, comprising: Enhancement module: used to perform various random data enhancements on the original image; Encoding module: used to map the enhanced image to the latent representation space using a neural network and an attention mechanism as an encoder; Adjustment module: used to introduce an expander as an auxiliary tool to adjust the encoder by reducing the calculation loss; Classification module: used to classify OCT images using the features extracted by the encoder after training is completed.
[0026] An embodiment of the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, the computer program implements a method for classifying OCT images based on self-supervised learning according to any one of the items.
[0027] An embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements any one of the methods for classifying OCT images based on self-supervised learning.
Claims
1. A classification method for OCT images based on self-supervised learning, characterized in that: The following steps are involved: (1) Perform various random data enhancements on the original image; (2) Using a neural network and an attention mechanism as an encoder to map the enhanced image to a latent representation space; (3) Introducing an expander as an auxiliary tool to adjust the encoder by reducing the computational loss; (4) After training is completed, the features extracted by the encoder are used to classify the OCT images.
2. The OCT image classification method based on self-supervised learning according to claim 1, characterized in that: Step (1) is as follows: randomly crop and scale the image to a size of 224x224, and use bicubic interpolation for processing and scaling; Flip the image horizontally and apply color jitter to the image, including adjusting brightness, contrast, saturation, and hue, converting the image to a grayscale image; applying Gaussian blur to the image; converting the image to Tensor format, and standardizing the image.
3. The OCT image classification method based on self-supervised learning according to claim 1, characterized in that: Step (2) is as follows: the neural network is the residual network ResNet-50, and the ECA lightweight channel attention mechanism is introduced; the ECA module is introduced in the residual connection part of each residual module of ResNet-50, that is, after the output feature map of each residual module is added to the input, channel weighting is performed through the ECA module.
4. The OCT image classification method based on self-supervised learning according to claim 1, characterized in that: In step (3), the expander is a three-layer fully connected layer.
5. The OCT image classification method based on self-supervised learning according to claim 4, characterized in that: In step (3), the details are as follows: Assume that the converted image is obtained by the encoder The view and Encoded as and , then by the expander Map it to an embedded value and .
6. The OCT image classification method based on self-supervised learning according to claim 5, characterized in that: In step (3), the loss function is as follows: First, let the dataset The images in are processed in batches according to the set batch size, and the feature vectors from the two branches of the twin architecture are represented as and , and It is composed of n d-dimensional vectors; Then, and The invariance loss function between Defined as the average of the squared Euclidean distances between each pair of vectors: ； definition is the normalized standard deviation, and the formula is: ； in, is a small scalar to prevent numerical instability; Each of the vectors in The vector of dimension values is composed of Representation; Definition is the hinge function of the normalized standard deviation of the embeddings with batch-size dimension: ； in, is a constant target value for the standard deviation; definition The covariance matrix of is: ； The covariance regularization term Defined as , represents the sum of squares of coefficients outside the diagonal, and uses a dimension-dependent scaling factor 1 / d to adjust the impact of the loss function: 。 7. A classification system for OCT images based on self-supervised learning, characterized in that: include: Enhancement module: used to perform various random data enhancements on the original image; Encoding module: used to map the enhanced image to the latent representation space using a neural network and an attention mechanism as an encoder; Adjustment module: used to introduce an expander as an auxiliary tool to adjust the encoder by reducing the calculation loss; Classification module: used to classify OCT images using the features extracted by the encoder after training is completed.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, the computer program implements a classification method for OCT images based on self-supervised learning according to any one of claims 1 to 6.
9. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for classifying OCT images based on self-supervised learning according to any one of claims 1 to 6 is implemented.