Method, device, equipment and medium for constructing ultra-wide-angle fundus image classification model

Abstract

The present application relates to a method, device, equipment and medium for constructing an ultra-wide-angle fundus image classification model, and relates to the field of image recognition technology. Under the condition of a limited number of ultra-wide-angle fundus images, ordinary fundus images can be effectively used to train an application model adapted to ultra-wide-angle fundus images, and a pre-trained basic model is trained based on ordinary fundus images to obtain a first basic model, and the mask autoencoder weights in the first basic model can be further used to further train the first basic model using a limited number of ultra-wide-angle fundus images to gradually adjust the mask autoencoder weights of the first basic model to form a second basic model, and then the decoder of the mask autoencoder in the second basic model is replaced with a classifier for training until convergence to obtain an application model adapted to ultra-wide-angle fundus images, which can accurately set classification labels for ultra-wide-angle fundus images to be classified.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, computer equipment and storage medium for constructing an ultra-wide-angle fundus image classification model. Background Art

[0002] Uveitis is a group of vision-threatening diseases characterized by inflammation of the uvea (iris, ciliary body, and choroid). Diagnosis of uveitis is challenging because symptoms are nonspecific and variable.

[0003] Diagnosis of intermediate and posterior uveitis, which affect the vitreous, retina, and choroid, is particularly difficult. During imaging examinations, when the probe light enters the refractive medium, the turbidity of the medium often causes the Tyndall effect, resulting in blurred imaging and possibly obscuring the fundus, leading to a high rate of misdiagnosis or missed diagnosis.

[0004] The diagnosis of uveitis not only relies on pathogen detection (such as suppurative uveitis, acute retinal necrosis syndrome, tuberculosis-related uveitis, syphilitic uveitis, and toxoplasmic uveitis) and immunological testing (such as Behcet's disease and rheumatoid arthritis-related uveitis), but may also require multidisciplinary consultation with rheumatology and infectious disease experts. Therefore, the time and economic costs required for diagnosis are relatively high. In addition, uveitis, especially posterior uveitis, progresses rapidly. Therefore, misdiagnosis, missed diagnosis, or delayed diagnosis can lead to irreversible visual impairment or blindness. Timely and accurate diagnosis of uveitis is crucial to ensure appropriate management and timely treatment to reduce or avoid blindness.

[0005] Ultra-wide-angle fundus photography technology can capture and record fundus changes in patients with uveitis at a wider angle, which helps clinicians to more comprehensively analyze the fundus characteristics of uveitis in order to make a more accurate diagnosis of uveitis. However, because ultra-wide-angle images are a newer means of ophthalmic examination, the number of publicly available ultra-wide-angle images is far less than that of ordinary fundus images. Generally, training and building basic models requires a large amount of unlabeled data. For example, RETFound uses about 1.6 million ordinary fundus images. However, the number of ultra-wide-angle fundus images is very small, and it is difficult to meet the training set quantity requirements when training a classification model that sets accurate classification labels for ultra-wide-angle fundus images. There are great differences between traditional ordinary fundus images and ultra-wide-angle fundus photos, and it is impossible to effectively use ordinary fundus images to replace ultra-wide-angle fundus photos to quickly train a classification model that sets accurate classification labels for ultra-wide-angle fundus images. Summary of the invention

[0006] Based on this, a method, device, computer equipment and storage medium for constructing an ultra-wide-angle fundus image classification model are provided to solve the technical problem that the number of ultra-wide-angle fundus images is very small and it is difficult to meet the technical problem of meeting the training set quantity requirement when training a classification model that sets accurate classification labels for ultra-wide-angle fundus images.

[0007] On the one hand, a method for constructing an ultra-wide-angle fundus image classification model is provided, the method comprising:

[0008] Constructing a pre-trained basic model, and setting the pre-trained basic model to include a mask autoencoder weight;

[0009] Using common fundus images in a three-channel format of red, green and blue to form a first training set, using the first training set to train the pre-trained basic model, adjusting the mask autoencoder weights of the pre-trained basic model, and forming a first basic model;

[0010] Compressing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, using the second training set to train the first basic model, gradually increasing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusting the mask autoencoder weights of the first basic model to form a second basic model;

[0011] Obtain the encoder of the mask autoencoder in the second basic model, set the classification label of the ultra-wide-angle fundus image in the classifier, add the classifier after the encoder to form an application model, and train the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image.

[0012] In one of the embodiments, when constructing the pre-trained basic model, the input channels of the pre-trained basic model are set to three channels: red, green, and blue;

[0013] When forming the second training set, the ultra-wide-angle fundus image in the red and green two-channel format is processed into an ultra-wide-angle fundus image in the red, green and blue three-channel format, and the blue channel of the ultra-wide-angle fundus image is filled with a zero matrix, and the zero matrix is ​​a matrix of all zeros.

[0014] In one of the embodiments, when constructing the pre-trained basic model, the input channels of the pre-trained basic model are set to red and green channels, and ordinary fundus images in red and green two-channel format are used to form a first training set.

[0015] In one embodiment, the steps of compressing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, using the second training set to train the first basic model, gradually increasing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusting the mask autoencoder weights of the first basic model include:

[0016] Acquire the resolution size of the ordinary fundus image of the first training set as the first resolution size, and set the resolution size of the ultra-wide-angle fundus image of the second training set to the first resolution size;

[0017] Acquiring a resolution size of the ultra-wide-angle fundus image when forming the second basic model as a second resolution size;

[0018] In the process of training the first basic model using the second training set, the resolution size of the ultra-wide-angle fundus image in the second training set is gradually increased from the first resolution size to the second resolution size, so as to gradually adjust the mask autoencoder weights of the first basic model.

[0019] In one embodiment, gradually increasing the resolution size of the ultra-wide-angle fundus images in the second training set from the first resolution size to the second resolution size comprises:

[0020] Obtaining a pixel difference between the first resolution size and the second resolution size;

[0021] dividing the pixel difference into a plurality of incremental pixel quantities;

[0022] The resolution size of the ultra-wide-angle fundus image is increased by an incremental pixel amount from the first resolution size, and it is determined whether the resolution size of the ultra-wide-angle fundus image is changed to the second resolution size. If so, the process ends; if not, the resolution size of the ultra-wide-angle fundus image is further increased by an incremental pixel amount until the resolution size of the ultra-wide-angle fundus image is changed to the second resolution size.

[0023] In one embodiment, setting the classification label of the ultra-wide-angle fundus image in the classifier includes:

[0024] Setting the classification labels of the ultra-wide-angle fundus image in the classifier includes a first layer label, a second layer label, a third layer label and a fourth layer label;

[0025] Setting the first layer of labels to include normal labels and pathological vitreous opacities;

[0026] Setting the second layer of labels to include non-infectious uveitis and infectious uveitis corresponding to the pathological vitreous opacity;

[0027] The third layer of labels are set to include tuberculosis-related uveitis, sarcoidosis uveitis and Behcet's uveitis corresponding to the non-infectious uveitis, and endogenous uveitis and exogenous uveitis corresponding to the non-infectious uveitis;

[0028] The fourth layer of labels includes bacterial uveitis, fungal uveitis and viral uveitis corresponding to the endogenous uveitis, and post-cataract surgery uveitis and post-traumatic uveitis corresponding to the exogenous uveitis.

[0029] In one embodiment, the adding the classifier after the encoder to form an application model, and training the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image comprises:

[0030] In the classifier, a hierarchical loss function is set for the classification labels of the ultra-wide-angle fundus image;

[0031] Obtain corresponding cross entropy losses for the first layer of labels, the second layer of labels, the third layer of labels, and the fourth layer of labels respectively and add them up to obtain a total training loss;

[0032] After adding the classifier after the encoder, an application model is formed, and the parameters of the application model are optimized by a gradient descent method based on the total training loss until the application model converges.

[0033] On the other hand, a device for constructing an ultra-wide-angle fundus image classification model is provided, the device comprising:

[0034] Construct a pre-trained basic model module, which is used to construct a pre-trained basic model and set the pre-trained basic model to include the mask autoencoder weights;

[0035] A first training module is used to form a first training set using common fundus images in a three-channel format of red, green and blue, train the pre-trained basic model using the first training set, adjust the mask autoencoder weights of the pre-trained basic model, and form a first basic model;

[0036] A second training module compresses the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, uses the second training set to train the first basic model, gradually improves the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusts the mask autoencoder weights of the first basic model to form a second basic model;

[0037] The third training module is used to obtain the encoder of the mask autoencoder in the second basic model, set the classification label of the ultra-wide-angle fundus image in the classifier, add the classifier after the encoder to form an application model, and train the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image.

[0038] On the other hand, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for constructing an ultra-wide-angle fundus image classification model described above are implemented.

[0039] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for constructing an ultra-wide-angle fundus image classification model described above are implemented.

[0040] The above-mentioned ultra-wide-angle fundus image classification model construction method, device, computer equipment and storage medium can effectively use ordinary fundus images to train and obtain an application model adapted to ultra-wide-angle fundus images under the condition of a limited number of ultra-wide-angle fundus images, and train a pre-trained basic model based on ordinary fundus images to obtain a first basic model. Using the mask autoencoder weights in the first basic model, the first basic model can be further trained using a limited number of ultra-wide-angle fundus images to gradually adjust the mask autoencoder weights of the first basic model to form a second basic model. During the training process, the resolution size of the ultra-wide-angle fundus images in the red and green two-channel format is gradually increased from the resolution size of the ordinary fundus images, which can achieve reasonable fine-tuning of the mask autoencoder weights, and then the decoder of the mask autoencoder in the second basic model is replaced by a classifier for training until convergence to obtain an application model adapted to ultra-wide-angle fundus images, which can accurately set the classification labels for the ultra-wide-angle fundus images to be classified. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A flowchart of a method for constructing an ultra-wide-angle fundus image classification model in one embodiment of the present application.

[0043] Figure 2 This is a schematic diagram of optimizing the weights of a mask autoencoder in one embodiment of the present application.

[0044] Figure 3 It is a schematic diagram of a method of gradually increasing the resolution size of ordinary fundus images in the training set from the first resolution size to the second resolution size in one embodiment of the present application.

[0045] Figure 4 A schematic diagram of setting classification labels in one embodiment of the present application.

[0046] Figure 5 This is a structural block diagram of a device for building an ultra-wide-angle fundus image classification model in one embodiment of the present application.

[0047] Figure 6 This is a diagram of the internal structure of a computer device in one embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] As mentioned in the background, in the past two years, foundation models have revolutionized the development of deep learning applications. These models are usually trained using large and diverse datasets, and can provide excellent performance features for downstream tasks even with limited data samples. This is even more important in medical scenarios with limited and diverse data samples.

[0050] Zhou et al. proposed a pre-trained foundation model for retinal images, called RETFound[Zhou Y,Chia MA, Wagner SK, et al. A foundation model for generalizable disease detection from retinal images[J]. Nature, 2023, 622(7981): 156-163.], which was trained on 1.6 million fundus images using the Masked auto-encoder (MAE) strategy. During training, the goal of the model is to reconstruct the input image while masking part of the image. This requires the model to learn the relationship between the masked and unmasked parts of the image. In this way, the model can generate better image representations in its latent space, which ultimately benefits downstream tasks. RETFound learns generalizable representations from large-scale fundus images and shows excellent performance in the diagnosis and prognosis of several eye diseases. However, RETFound is mainly trained using traditional fundus images, which are very different from ultra-wide-angle fundus images. Traditional fundus images contain three channels: red, green, and blue, while ultra-wide-angle fundus images only contain two channels: red and green, due to differences in imaging principles. The difference between the modalities is obvious, and the basic model developed on ordinary fundus images has a significant performance degradation when directly applied to ultra-wide-angle fundus images. This is also the reason for constructing the basic model for ultra-wide-angle fundus images. Therefore, directly applying RETFound to ultra-wide-angle fundus image scenarios cannot set accurate classification labels for ultra-wide-angle fundus images.

[0051] RETFound is mainly trained using traditional fundus images, which are very different from ultra-wide-angle fundus images. Usually, training and building a basic model requires a large amount of unlabeled data. For example, RETFound used 1.6 million ordinary fundus images. Because ultra-wide-angle images are a newer means of ophthalmic examination, the number of publicly available images is far less than that of ordinary fundus images. Ordinary fundus photo images contain three channels: red, green, and blue, while ultra-wide-angle fundus images only contain two channels: red and green due to differences in imaging principles. The difference between the modalities is obvious, and the performance of the basic model developed on ordinary fundus images is significantly reduced when directly applied to ultra-wide-angle fundus images. This is also the reason for constructing a basic model for ultra-wide-angle fundus images.

[0052] In addition, a notable feature of ultra-wide-angle images is that they have a larger field of view than ordinary fundus images and contain richer content. The traditional fundus images with an input size of 224x224 used in training RETFound cannot cope well with ultra-wide-angle fundus images that require high-resolution scenes.

[0053] To solve the above problems, the embodiment of the present invention creatively proposes a method for constructing an ultra-wide-angle fundus image classification model, and constructs a basic model specifically for ultra-wide-angle fundus images, called UWFound, which provides better feature extraction capabilities for scenes with fundus photography blur caused by disease conditions, and provides a good model foundation for application tasks based on ultra-wide-angle fundus images.

[0054] like Figure 1 As shown, a method for constructing an ultra-wide-angle fundus image classification model is provided, comprising the following steps:

[0055] Step S11, constructing a pre-trained basic model, and setting the pre-trained basic model to include a mask autoencoder weight;

[0056] Step S12, using common fundus images in a three-channel format of red, green and blue to form a first training set, using the first training set to train the pre-trained basic model, adjusting the mask autoencoder weights of the pre-trained basic model, and forming a first basic model;

[0057] Step S13, compressing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, using the second training set to train the first basic model, gradually increasing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusting the mask autoencoder weights of the first basic model to form a second basic model;

[0058] Step S14, obtain the encoder of the mask autoencoder in the second basic model, set the classification label of the ultra-wide-angle fundus image in the classifier, add the classifier after the encoder to form an application model, and train the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image.

[0059] Among them, under the condition of a limited number of ultra-wide-angle fundus images, ordinary fundus images can be effectively used to train and obtain an application model adapted to ultra-wide-angle fundus images, and a pre-trained basic model can be trained based on ordinary fundus images to obtain a first basic model. Using the mask autoencoder weights in the first basic model, a limited number of ultra-wide-angle fundus images can be used to further train the first basic model to gradually adjust the mask autoencoder weights of the first basic model to form a second basic model. During the training process, the resolution size of the ultra-wide-angle fundus images in the red and green two-channel format is gradually increased from the resolution size of the ordinary fundus images, which can achieve reasonable fine-tuning of the mask autoencoder weights, and then the decoder of the mask autoencoder in the second basic model is replaced by a classifier for training until convergence to obtain an application model adapted to ultra-wide-angle fundus images, which can accurately set classification labels for ultra-wide-angle fundus images to be classified.

[0060] Among them, the mask autoencoder weights in the RETFound model trained with ordinary fundus images can adapt to the resolution size of ordinary fundus images. During the training process, the resolution size of the ultra-wide-angle fundus images in the red and green channel formats is gradually increased from the resolution size of ordinary fundus images, which can achieve reasonable fine-tuning of the mask autoencoder weights and achieve a smooth transition.

[0061] When building the pre-trained basic model, the Masked auto-encoder (MAE) strategy can be used to train it on approximately 1.6 million fundus images.

[0062] like Figure 2 As shown, Figure 2 The principle of optimizing the mask autoencoder weights is demonstrated.

[0063] In one of the embodiments, when constructing the pre-trained basic model, the input channels of the pre-trained basic model are set to three channels: red, green, and blue;

[0064] When forming the second training set, the ultra-wide-angle fundus image in the red and green two-channel format is processed into an ultra-wide-angle fundus image in the red, green and blue three-channel format, and the blue channel of the ultra-wide-angle fundus image is filled with a zero matrix, and the zero matrix is ​​a matrix of all zeros.

[0065] By setting the input channels of the pre-trained basic model to three channels of red, green and blue, and processing the two-channel format of the ultra-wide-angle fundus image according to the three-channel format of the ordinary fundus image to form the second training set, the ultra-wide-angle fundus image can be suitable for model training with the pre-trained basic model, and the application model adapted to the ultra-wide-angle fundus image can be trained, so that the classification labels of the ultra-wide-angle fundus images to be classified can be accurately set.

[0066] In one of the embodiments, when constructing the pre-trained basic model, the input channels of the pre-trained basic model are set to red and green channels, and ordinary fundus images in red and green two-channel format are used to form a first training set.

[0067] The input channel of the pre-trained basic model is set to change from a three-channel color image to a two-channel image, thereby saving the number of model parameters and the amount of calculation, saving computing resources.

[0068] In one embodiment, the steps of compressing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, using the second training set to train the first basic model, gradually increasing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusting the mask autoencoder weights of the first basic model include:

[0069] Acquire the resolution size of the ordinary fundus image of the first training set as the first resolution size, and set the resolution size of the ultra-wide-angle fundus image of the second training set to the first resolution size;

[0070] Acquiring a resolution size of the ultra-wide-angle fundus image when forming the second basic model as a second resolution size;

[0071] In the process of training the first basic model using the second training set, the resolution size of the ultra-wide-angle fundus image in the second training set is gradually increased from the first resolution size to the second resolution size, so as to gradually adjust the mask autoencoder weights of the first basic model.

[0072] In one embodiment, gradually increasing the resolution size of the ultra-wide-angle fundus images in the second training set from the first resolution size to the second resolution size comprises:

[0073] Obtaining a pixel difference between the first resolution size and the second resolution size;

[0074] dividing the pixel difference into a plurality of incremental pixel quantities;

[0075] The resolution size of the ultra-wide-angle fundus image is increased by an incremental pixel amount from the first resolution size, and it is determined whether the resolution size of the ultra-wide-angle fundus image is changed to the second resolution size. If so, the process ends; if not, the resolution size of the ultra-wide-angle fundus image is further increased by an incremental pixel amount until the resolution size of the ultra-wide-angle fundus image is changed to the second resolution size.

[0076] Specifically, Figure 3As shown, the step of gradually increasing the resolution size of the ordinary fundus image from the first resolution size to the second resolution size includes: if the first resolution size obtained is 224x224 pixels, the second resolution size is 800x800 pixels; increasing the resolution size of the ordinary fundus image from 224x224 pixels to 320x320 pixels, increasing 320x320 pixels to 512x512 pixels, and increasing 512x512 pixels to 800x800 pixels.

[0077] The pre-trained model of traditional images usually uses an image resolution size of 224x224, which cannot cope well with ultra-wide-angle fundus images that require high-resolution scenes. To this end, we designed a gradual image resolution improvement fine-tuning training strategy. That is, initially use small-resolution images for fine-tuning, and then gradually increase the resolution for fine-tuning to meet the needs of ultra-wide-angle fundus images. After fine-tuning ordinary fundus images, the pre-trained basic model can be trained until the pre-trained basic model converges to obtain an application model adapted to ultra-wide-angle fundus images.

[0078] like Figure 4 As shown, the setting of the classification label of the ultra-wide-angle fundus image in the classifier includes:

[0079] Setting the classification labels of the ultra-wide-angle fundus image in the classifier includes a first layer label, a second layer label, a third layer label and a fourth layer label;

[0080] Setting the first layer of labels to include normal labels and pathological vitreous opacities;

[0081] Setting the second layer of labels to include non-infectious uveitis and infectious uveitis corresponding to the pathological vitreous opacity;

[0082] The third layer of labels are set to include tuberculosis-related uveitis, sarcoidosis uveitis and Behcet's uveitis corresponding to the non-infectious uveitis, and endogenous uveitis and exogenous uveitis corresponding to the non-infectious uveitis;

[0083] The fourth layer of labels includes bacterial uveitis, fungal uveitis and viral uveitis corresponding to the endogenous uveitis, and post-cataract surgery uveitis and post-traumatic uveitis corresponding to the exogenous uveitis.

[0084] Among them, uveitis is a general term for a class of diseases. Different specific diseases have different characteristics and treatment plans. We have designed a hierarchical classification method for the uveitis screening system, dividing uveitis into different granularities to meet the needs of different diagnostic purposes.

[0085] In this embodiment, the classifier is added after the encoder to form an application model, and the application model is trained until convergence to obtain an application model adapted to the ultra-wide-angle fundus image, including:

[0086] In the classifier, a hierarchical loss function is set for the classification labels of the ultra-wide-angle fundus image;

[0087] Obtain corresponding cross entropy losses for the first layer of labels, the second layer of labels, the third layer of labels, and the fourth layer of labels respectively and add them up to obtain a total training loss;

[0088] After adding the classifier after the encoder, an application model is formed, and the parameters of the application model are optimized by a gradient descent method based on the total training loss until the application model converges.

[0089] During the training process, a hierarchical loss function is set to cope with the hierarchical classification model. Taking a two-layer example, the model classification layer outputs normal y0, pathological vitreous opacity y1, non-infectious y2 and infectious uveitis y3, a total of 4 nodes, to construct a multi-label multi-classification task. If the result is infectious uveitis, the label is recorded as [0, 1, 0, 1], and if it is normal, it is recorded as [1, 0, 0, 0]. Cross entropy loss is used for each layer of the hierarchical structure, and finally the cross entropy losses of all layers are added to get the total training loss. In this example, the first layer label is [y 0 , y 1, ]Calculate the loss L1, and the second layer label is [y 0, y 1, y2] calculates the loss L2, and the final total training loss is L=L1+L2.

[0090] It is understandable that the second basic model may be determined to have converged when the total training loss no longer decreases or when a specified number of training iterations is reached.

[0091] like Figure 5 As shown, a device 10 for constructing an ultra-wide-angle fundus image classification model is provided, and the device comprises:

[0092] Constructing a pre-trained basic model module 1, used to construct a pre-trained basic model, and setting the pre-trained basic model to include mask autoencoder weights;

[0093] A first training module 2 is used to form a first training set using common fundus images in a three-channel format of red, green and blue, train the pre-trained basic model using the first training set, adjust the mask autoencoder weights of the pre-trained basic model, and form a first basic model;

[0094] The second training module 3 compresses the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, uses the second training set to train the first basic model, gradually improves the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusts the mask autoencoder weights of the first basic model to form a second basic model;

[0095] The third training module 4 is used to obtain the encoder of the mask autoencoder in the second basic model, set the classification label of the ultra-wide-angle fundus image in the classifier, add the classifier after the encoder to form an application model, and train the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image.

[0096] For the specific limitations on the ultra-wide-angle fundus image classification model construction device, please refer to the limitations on the ultra-wide-angle fundus image classification model construction method mentioned above, which will not be repeated here.

[0097] Each module in the above-mentioned ultra-wide-angle fundus image classification model construction device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0098] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0099] Constructing a pre-trained basic model, and setting the pre-trained basic model to include a mask autoencoder weight;

[0100] Using common fundus images in a three-channel format of red, green and blue to form a first training set, using the first training set to train the pre-trained basic model, adjusting the mask autoencoder weights of the pre-trained basic model, and forming a first basic model;

[0101] Compressing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, using the second training set to train the first basic model, gradually increasing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusting the mask autoencoder weights of the first basic model to form a second basic model;

[0102] Obtain the encoder of the mask autoencoder in the second basic model, set the classification label of the ultra-wide-angle fundus image in the classifier, add the classifier after the encoder to form an application model, and train the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image.

[0103] For the specific limitations on the steps implemented when the computer program is executed by the processor, please refer to the limitations on the method for constructing the ultra-wide-angle fundus image classification model in the above text, which will not be repeated here.

[0104] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0105] Constructing a pre-trained basic model, and setting the pre-trained basic model to include a mask autoencoder weight;

[0106] Using common fundus images in a three-channel format of red, green and blue to form a first training set, using the first training set to train the pre-trained basic model, adjusting the mask autoencoder weights of the pre-trained basic model, and forming a first basic model;

[0107] Compressing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, using the second training set to train the first basic model, gradually increasing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusting the mask autoencoder weights of the first basic model to form a second basic model;

[0108] Obtain the encoder of the mask autoencoder in the second basic model, set the classification label of the ultra-wide-angle fundus image in the classifier, add the classifier after the encoder to form an application model, and train the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image.

[0109] For the specific limitations on the steps implemented when the processor executes the computer program, please refer to the limitations on the method for constructing the ultra-wide-angle fundus image classification model mentioned above, which will not be repeated here.

[0110] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store ultra-wide-angle fundus image classification model construction data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for constructing an ultra-wide-angle fundus image classification model is implemented.

[0111] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0112] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0113] Constructing a pre-trained basic model, and setting the pre-trained basic model to include a mask autoencoder weight;

[0114] Using common fundus images in a three-channel format of red, green and blue to form a first training set, using the first training set to train the pre-trained basic model, adjusting the mask autoencoder weights of the pre-trained basic model, and forming a first basic model;

[0115] Compressing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, using the second training set to train the first basic model, gradually increasing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusting the mask autoencoder weights of the first basic model to form a second basic model;

[0116] Obtain the encoder of the mask autoencoder in the second basic model, set the classification label of the ultra-wide-angle fundus image in the classifier, add the classifier after the encoder to form an application model, and train the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image.

[0117] For the specific limitations on the steps implemented when the computer program is executed by the processor, please refer to the limitations on the method for constructing the ultra-wide-angle fundus image classification model in the above text, which will not be repeated here.

[0118] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0119] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0120] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for constructing an ultra-wide-angle fundus image classification model, characterized in that: include: Constructing a pre-trained basic model, and setting the pre-trained basic model to include a mask autoencoder weight; Using common fundus images in a three-channel format of red, green and blue to form a first training set, using the first training set to train the pre-trained basic model, adjusting the mask autoencoder weights of the pre-trained basic model, and forming a first basic model; Compressing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, obtaining the resolution size of the ordinary fundus image in the first training set as the first resolution size, setting the resolution size of the ultra-wide-angle fundus image in the second training set to the first resolution size, using the second training set to train the first basic model, gradually improving the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusting the mask autoencoder weights of the first basic model to form a second basic model; Obtain the encoder of the mask autoencoder in the second basic model, set the classification label of the ultra-wide-angle fundus image in the classifier, add the classifier after the encoder to form an application model, and train the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image.

2. The method for constructing an ultra-wide-angle fundus image classification model according to claim 1, characterized in that: When constructing the pre-trained basic model, the input channels of the pre-trained basic model are set to three channels: red, green, and blue; When forming the second training set, the ultra-wide-angle fundus image in the red and green two-channel format is processed into an ultra-wide-angle fundus image in the red, green and blue three-channel format, and the blue channel of the ultra-wide-angle fundus image is filled with a zero matrix, and the zero matrix is ​​a matrix of all zeros.

3. The method for constructing an ultra-wide-angle fundus image classification model according to claim 1, characterized in that: When constructing the pre-trained basic model, the input channels of the pre-trained basic model are set to red and green channels, and ordinary fundus images in the red and green two-channel format are used to form a first training set.

4. The method for constructing an ultra-wide-angle fundus image classification model according to claim 1, characterized in that: The step of training the first basic model using the second training set, gradually increasing the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusting the weight of the mask autoencoder of the first basic model includes: Acquiring a resolution size of the ultra-wide-angle fundus image when forming the second basic model as a second resolution size; In the process of training the first basic model using the second training set, the resolution size of the ultra-wide-angle fundus image in the second training set is gradually increased from the first resolution size to the second resolution size, so as to gradually adjust the mask autoencoder weights of the first basic model.

5. The method for constructing an ultra-wide-angle fundus image classification model according to claim 4, characterized in that: The step of gradually increasing the resolution size of the ultra-wide-angle fundus images in the second training set from the first resolution size to the second resolution size comprises: Obtaining a pixel difference between the first resolution size and the second resolution size; dividing the pixel difference into a plurality of incremental pixel quantities; The resolution size of the ultra-wide-angle fundus image is increased by an incremental pixel amount from the first resolution size, and it is determined whether the resolution size of the ultra-wide-angle fundus image is changed to the second resolution size. If so, the process ends; if not, the resolution size of the ultra-wide-angle fundus image is further increased by an incremental pixel amount until the resolution size of the ultra-wide-angle fundus image is changed to the second resolution size.

6. The method for constructing an ultra-wide-angle fundus image classification model according to claim 1, characterized in that: The step of setting the classification label of the ultra-wide-angle fundus image in the classifier includes: Setting the classification labels of the ultra-wide-angle fundus image in the classifier includes a first layer label, a second layer label, a third layer label and a fourth layer label; Setting the first layer of labels to include normal labels and pathological vitreous opacities; Setting the second layer of labels to include non-infectious uveitis and infectious uveitis corresponding to the pathological vitreous opacity; The third layer of labels are set to include tuberculosis-related uveitis, sarcoidosis uveitis and Behcet's uveitis corresponding to the non-infectious uveitis, and endogenous uveitis and exogenous uveitis corresponding to the non-infectious uveitis; The fourth layer of labels includes bacterial uveitis, fungal uveitis and viral uveitis corresponding to the endogenous uveitis, and post-cataract surgery uveitis and post-traumatic uveitis corresponding to the exogenous uveitis.

7. The method for constructing an ultra-wide-angle fundus image classification model according to claim 6, characterized in that: The application model is formed by adding the classifier after the encoder, and the application model is trained until convergence to obtain an application model adapted to the ultra-wide-angle fundus image, including: In the classifier, a hierarchical loss function is set for the classification labels of the ultra-wide-angle fundus image; Obtain corresponding cross entropy losses for the first layer of labels, the second layer of labels, the third layer of labels, and the fourth layer of labels respectively and add them up to obtain a total training loss; After adding the classifier after the encoder, an application model is formed, and the parameters of the application model are optimized by a gradient descent method based on the total training loss until the application model converges.

8. A device for constructing an ultra-wide-angle fundus image classification model, characterized in that: The device comprises: Construct a pre-trained basic model module, which is used to construct a pre-trained basic model and set the pre-trained basic model to include the mask autoencoder weights; A first training module is used to form a first training set using common fundus images in a three-channel format of red, green and blue, train the pre-trained basic model using the first training set, adjust the mask autoencoder weights of the pre-trained basic model, and form a first basic model; A second training module compresses the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format to form a second training set, obtains the resolution size of the ordinary fundus image in the first training set as the first resolution size, sets the resolution size of the ultra-wide-angle fundus image in the second training set to the first resolution size, trains the first basic model using the second training set, gradually improves the resolution size of the ultra-wide-angle fundus image in the red and green two-channel format during the training process, and gradually adjusts the mask autoencoder weights of the first basic model to form a second basic model; The third training module is used to obtain the encoder of the mask autoencoder in the second basic model, set the classification label of the ultra-wide-angle fundus image in the classifier, add the classifier after the encoder to form an application model, and train the application model until convergence to obtain an application model adapted to the ultra-wide-angle fundus image.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images