Fundus color image classification model training method and fundus color image classification method

Abstract

The embodiment of the present invention provides a fundus color image classification model training method and a fundus color image classification method. The training method of the fundus color image classification model includes: obtaining original fundus color images of different categories; preprocessing the original fundus color images to obtain a target fundus color image; inputting the target fundus color image into a first convolutional neural network, and outputting a fundus vascular image corresponding to the target fundus color image; obtaining the patient age characteristics and patient gender characteristics corresponding to the target fundus color image; using the target fundus color image, the fundus vascular image, the patient age characteristics and the patient gender characteristics, the second convolutional neural network is trained to obtain a fundus color image classification model. According to the embodiment of the present invention, the screening and classification efficiency and accuracy of fundus color images can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of medical imaging and deep learning technology, and in particular relates to a training method and device for a fundus color image classification model, a fundus color image classification method and device, an electronic device, and a computer storage medium. Background Art

[0002] Currently, when classifying fundus color images, manual screening and classification is performed based on doctors' experience, which not only has low screening and classification efficiency but also low screening and classification accuracy.

[0003] Therefore, how to improve the screening and classification efficiency and accuracy of fundus color images is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] The embodiments of the present invention provide a training method and device for a fundus color image classification model, a fundus color image classification method and device, an electronic device and a computer storage medium, which can improve the screening and classification efficiency and accuracy of fundus color images.

[0005] In a first aspect, an embodiment of the present invention provides a method for training a fundus color image classification model, comprising:

[0006] Obtaining original fundus color images of different categories;

[0007] Preprocessing the original fundus color image to obtain a target fundus color image;

[0008] Inputting the target fundus color image into the first convolutional neural network, and outputting the fundus vascular image corresponding to the target fundus color image;

[0009] Acquire the patient age characteristics and patient gender characteristics corresponding to the target fundus color image;

[0010] The second convolutional neural network is trained using the target fundus color image, fundus vascular image, patient age characteristics, and patient gender characteristics to obtain a fundus color image classification model.

[0011] Optionally, preprocessing the original fundus color image to obtain a target fundus color image includes:

[0012] Performing color space transformation and green channel extraction on the original fundus color image to obtain a first fundus color image;

[0013] Adaptive histogram equalization is used to eliminate uneven illumination in the first fundus color image to obtain a target fundus color image.

[0014] Optionally, the backbone network of the first convolutional neural network includes at least one convolutional layer, at least one maximum pooling layer and a sigmoid function layer.

[0015] Optionally, the backbone network of the second convolutional neural network includes at least one convolutional layer, at least one global mean pooling layer, at least one maximum pooling layer, at least one fully connected layer, batch normalization and a rectified linear unit.

[0016] Optionally, preprocessing the original fundus color image to obtain a target fundus color image includes:

[0017] The original fundus color image is augmented to obtain the target fundus color image.

[0018] Optionally, performing image augmentation processing on the original fundus color image to obtain a target fundus color image includes:

[0019] The original fundus color image is processed by denoising and smoothing, pixel normalization, image size unification, random flipping, center cropping and adding Gaussian noise to obtain the target fundus color image.

[0020] In a second aspect, an embodiment of the present invention provides a fundus color image classification method based on a fundus color image classification model, wherein the fundus color image classification model is obtained by using the training method of the fundus color image classification model of the first aspect, comprising:

[0021] Acquire a fundus color image to be classified;

[0022] The fundus color image to be classified is input into the fundus color image classification model, and the classification result of the fundus color image to be classified is output.

[0023] In a third aspect, an embodiment of the present invention provides a training device for a fundus color image classification model, comprising:

[0024] The first acquisition module is used to acquire original fundus color images of different categories;

[0025] A preprocessing module, used for preprocessing the original fundus color image to obtain a target fundus color image;

[0026] An output module, used to input the target fundus color image into the first convolutional neural network, and output a fundus vascular image corresponding to the target fundus color image;

[0027] The second acquisition module is used to acquire the patient age characteristics and patient gender characteristics corresponding to the target fundus color image;

[0028] The model training module is used to perform model training on the second convolutional neural network using the target fundus color image, fundus vascular image, patient age characteristics and patient gender characteristics to obtain a fundus color image classification model.

[0029] Optionally, a preprocessing module is used to perform color space transformation and green channel extraction on the original fundus color image to obtain a first fundus color image; and to use an adaptive histogram equalization method to eliminate uneven illumination in the first fundus color image to obtain a target fundus color image.

[0030] Optionally, the backbone network of the first convolutional neural network includes at least one convolutional layer, at least one maximum pooling layer and a sigmoid function layer.

[0031] Optionally, the backbone network of the second convolutional neural network includes at least one convolutional layer, at least one global mean pooling layer, at least one maximum pooling layer, at least one fully connected layer, batch normalization and a rectified linear unit.

[0032] Optionally, a preprocessing module is used to perform image augmentation processing on the original fundus color image to obtain a target fundus color image.

[0033] Optionally, the preprocessing module is also used to perform denoising and smoothing, pixel normalization, image size unification, random flipping, center cropping and Gaussian noise addition processing on the original fundus color image to obtain a target fundus color image.

[0034] In a fourth aspect, an embodiment of the present invention provides a fundus color image classification device based on a fundus color image classification model, wherein the fundus color image classification model is obtained by using the training method of the fundus color image classification model of the first aspect, and comprises:

[0035] An acquisition module, used for acquiring a fundus color image to be classified;

[0036] The output module is used to input the fundus color image to be classified into the fundus color image classification model and output the classification result of the fundus color image to be classified.

[0037] In a fifth aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising: a processor, and a memory storing computer program instructions;

[0038] The processor reads and executes computer program instructions to implement the training method of the fundus color image classification model of the first aspect; or,

[0039] The processor reads and executes computer program instructions to implement the fundus color image classification method based on the fundus color image classification model of the second aspect.

[0040] In a sixth aspect, an embodiment of the present invention provides a computer storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the training method of the fundus color image classification model of the first aspect is implemented; or,

[0041] The processor reads and executes computer program instructions to implement the fundus color image classification method based on the fundus color image classification model of the second aspect.

[0042] The training method and device of the fundus color image classification model, the fundus color image classification method and device, the electronic device and the computer storage medium of the embodiments of the present invention can improve the screening and classification efficiency and accuracy of fundus color images.

[0043] The training method of the fundus color image classification model obtains original fundus color images of different categories; pre-processes the original fundus color images to obtain target fundus color images; inputs the target fundus color image into the first convolutional neural network, outputs the fundus vascular image corresponding to the target fundus color image; obtains the patient age characteristics and patient gender characteristics corresponding to the target fundus color image; uses the target fundus color image, the fundus vascular image, the patient age characteristics and the patient gender characteristics to train the second convolutional neural network model to obtain the fundus color image classification model. It can be seen that the fundus color image classification model obtained by the training method of the fundus color image classification model can improve the screening and classification efficiency and accuracy of the fundus color images by screening and classifying the fundus color images to be classified. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following is a brief introduction to the drawings required for use in the embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 It is a flowchart of a method for training a fundus color image classification model provided by an embodiment of the present invention;

[0046] Figure 2 It is a flow chart of a method for classifying a fundus color image based on a fundus color image classification model provided by an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of the overall network structure of a fundus color image classification model provided by an embodiment of the present invention;

[0048] Figure 4 is a preprocessed fundus color image provided by an embodiment of the present invention;

[0049] Figure 5 It is a schematic diagram of the structure of a fundus image blood vessel segmentation network provided by an embodiment of the present invention;

[0050] Figure 6 is a schematic diagram of a fundus blood vessel image provided by an embodiment of the present invention;

[0051] Figure 7 It is a structural schematic diagram of a fundus color image classification network provided by an embodiment of the present invention;

[0052] Figure 8 It is a structural schematic diagram of a training device for a fundus color image classification model provided by an embodiment of the present invention;

[0053] Fig. 9 It is a structural schematic diagram of a fundus color image classification device based on a fundus color image classification model provided by an embodiment of the present invention;

[0054] Fig.10 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0056] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0057] Currently, when classifying fundus color images, manual screening and classification is performed based on doctors' experience, which not only has low screening and classification efficiency but also low screening and classification accuracy.

[0058] In order to solve the problems of the prior art, the embodiments of the present invention provide a training method and device for a fundus color image classification model, a fundus color image classification method and device, an electronic device and a computer storage medium. The training method for the fundus color image classification model provided by the embodiments of the present invention is first introduced below.

[0059] Figure 1 FIG. 1 is a flow chart showing a method for training a fundus color image classification model provided by an embodiment of the present invention. Figure 1 As shown, the training method of the fundus color image classification model may include the following steps:

[0060] S101, obtaining original fundus color images of different categories.

[0061] S102: preprocess the original fundus color image to obtain a target fundus color image.

[0062] In one embodiment, preprocessing the original fundus color image to obtain the target fundus color image includes:

[0063] Performing color space transformation and green channel extraction on the original fundus color image to obtain a first fundus color image;

[0064] Adaptive histogram equalization is used to eliminate uneven illumination in the first fundus color image to obtain a target fundus color image.

[0065] In one embodiment, preprocessing the original fundus color image to obtain the target fundus color image includes:

[0066] The original fundus color image is augmented to obtain the target fundus color image.

[0067] In one embodiment, performing image augmentation processing on the original fundus color image to obtain a target fundus color image includes:

[0068] The original fundus color image is processed by denoising and smoothing, pixel normalization, image size unification, random flipping, center cropping and adding Gaussian noise to obtain the target fundus color image.

[0069] S103: Input the target fundus color image into a first convolutional neural network, and output a fundus vascular image corresponding to the target fundus color image.

[0070] In one embodiment, the backbone network of the first convolutional neural network includes at least one convolutional layer, at least one maximum pooling layer and a sigmoid function layer.

[0071] S104, obtaining the patient's age characteristics and patient's gender characteristics corresponding to the target fundus color image.

[0072] S105. Using the target fundus color image, the fundus vascular image, the patient's age characteristics, and the patient's gender characteristics, a second convolutional neural network is trained to obtain a fundus color image classification model.

[0073] In one embodiment, the backbone network of the second convolutional neural network includes at least one convolutional layer, at least one global mean pooling layer, at least one maximum pooling layer, at least one fully connected layer, batch normalization, and a rectified linear unit.

[0074] like Figure 2 As shown, the embodiment of the present application also provides a fundus color image classification method based on a fundus color image classification model. The fundus color image classification model is based on Figure 1 The training method of the fundus color image classification model shown is obtained, including:

[0075] S201, obtaining a fundus color image to be classified.

[0076] S202 , inputting the fundus color image to be classified into a fundus color image classification model, and outputting a classification result of the fundus color image to be classified.

[0077] The training method of the fundus color image classification model mentioned above obtains original fundus color images of different categories; preprocesses the original fundus color images to obtain target fundus color images; inputs the target fundus color images into the first convolutional neural network, outputs the fundus vascular image corresponding to the target fundus color image; obtains the patient age characteristics and patient gender characteristics corresponding to the target fundus color image; uses the target fundus color image, fundus vascular image, patient age characteristics and patient gender characteristics to train the second convolutional neural network model to obtain the fundus color image classification model. It can be seen that the fundus color image classification model obtained by the training method of the fundus color image classification model can improve the screening and classification efficiency and accuracy of the fundus color images by screening and classifying the fundus color images to be classified.

[0078] The above content is explained below with a specific scenario embodiment. The scenario embodiment is first generally described and then described in detail.

[0079] like Figure 3 As shown, data preprocessing is first performed on the fundus color image, which may include: preprocessing the obtained image to remove images containing lens stains and poor exposure quality, then outputting the image G channel of the obtained image, and obtaining the preprocessed image through an adaptive histogram equalization method.

[0080] After obtaining the preprocessed image, the fundus vascular image is extracted, which may include: for the extraction of fundus vascular, an improved convolutional neural network is mainly used, and the backbone network (backbone) includes at least one convolution layer (conv), at least one maximum pooling layer (max pooling) and a sigmoid function. The input image is an RGB color image, and the output is a segmented grayscale image.

[0081] After obtaining the fundus vascular image, the network construction of the fundus color image classification model can include:

[0082] The fundus vascular image and the original fundus color image obtained in the previous step are simultaneously input into the backbone of the convolutional neural network. The backbone contains at least one convolutional layer (conv), at least one global average pooling layer (avgpooling), at least one maximum pooling layer (max pooling), at least one fully connected layer (fc), batch normalization (batch normal) and rectified linear unit (relu), etc. The N-dimensional vector output by the backbone is concatenated (Concat) with the patient's age (age) and gender (sex) to form an N+2-dimensional vector. The new vector is input into the lower fully connected layer, and finally the confidence of each category is obtained through the sigmoid function.

[0083] The above has generally described the embodiment of this scenario, and a specific description will be given below.

[0084] 1. Data preprocessing:

[0085] During the acquisition process of fundus retinal images, they are often affected by external conditions, resulting in uneven grayscale distribution in the acquired images. In addition, the low contrast between the retinal image blood vessels and the background makes it difficult to detect the retinal blood vessels. In order to improve the contrast of the retinal image and improve the quality of the retinal image, preprocessing is required before retinal feature extraction. In the process of image feature extraction, the quality of this step can directly affect the effect of subsequent processing.

[0086] 1.1 Image G channel output:

[0087] Since fundus photographs are easily affected by lighting, equipment, etc., the quality of the images obtained varies greatly, which in turn interferes with the extraction of blood vessels in fundus retinal images. Therefore, in order to make the algorithm well applicable to different images, the color space transformation and color channel extraction are performed before image preprocessing. Commonly used color spaces that are easy to separate brightness channels include RGB, Lab, YCb Cr and Gaussian color spaces.

[0088] By analyzing the retinal color image and its corresponding RGB and Lab channel images, it is found that the green channel retinal image has the best contrast characteristics between the target blood vessels and the image background, while the quality of the retinal images of other channels is poor, and the contrast characteristics between the blood vessels and the image are not obvious. Therefore, in the preprocessing of the color retinal image, the green channel of the retina is extracted.

[0089] 1.2 Eliminate uneven lighting through adaptive histogram equalization:

[0090] In order to further improve the contrast between retinal blood vessels and background, a contrast-limited adaptive histogram equalization method is used. This method controls the enhancement amplitude of local contrast by limiting the height of the local histogram, which not only ensures the local contrast enhancement of the retinal green channel image, but also further limits the amplification of retinal noise. The output image is used as the input image of the neural network. Figure 4 .

[0091] 2. Fundus image blood vessel segmentation network:

[0092] The improved convolutional neural network method is used to extract relevant recognition features from the above grayscale image to achieve the final recognition result. The specific details are as follows Figure 5 shown.

[0093] 2.1 Network Structure

[0094] The framework of the proposed improved convolutional neural network (CNN) architecture is as follows Figure 5 As shown in the figure, this improved structure has richer convolution features than the original CNN network and can better detect the edges of the image.

[0095] Depend on Figure 5 It can be seen that the network is derived from a convolutional neural network and consists of three stages.

[0096] (1) In stage 1, the input image is processed through two layers of 3×3 convolution (Conv3-64) to obtain output 1 and output 2 respectively. Output 1 and output 2 are processed through 1*1 convolution kernel (Conv1-21) again and then the results are aggregated through the Eltwise layer to obtain output 11. On the other hand, output 2 is connected through the maximum pooling layer (Pool 2*2) to obtain output 3 and enter stage 2.

[0097] (2) Unlike stage 1, stage 2 contains three convolutional layers (Conv3-64). After each convolution layer, outputs 4, 5, and 6 are obtained respectively. Outputs 4, 5, and 6 are respectively passed into the 1*1 convolutional layer (Conv1-21). The results are respectively converged through the Eltwise layer to obtain output 12. After output 6 is max-pooled (Pool 2*2) to obtain output 7, it enters the third stage of the network.

[0098] (3) The third stage is similar to the second stage and will not be elaborated here.

[0099] (4) After the results of the three stages (output 11, output 12 and output 13) are converged in the Eltwise layer, they are upsampled (including Upsample_1, Upsample_2 and Upsample_3) and convolved with 1*1 (Conv1-1) respectively. The results are converged again through the Eltwise layer, and then go through the convolution operation of Conv1-1 and the Sigmoid or Loss function layer to obtain the output result (output).

[0100] The advantage of this improved neural network is that, as the network layers increase, it can learn more discriminative features to describe the elongated vascular structure. However, each stage provides useful information, which is essential for our segmentation work, because the shallow layers extract more details and the deep layers provide more abstract information.

[0101] In addition to increasing the depth of the network to obtain discriminatory information, the information of all intermediate layers should also be utilized. The RCF (Richer convolution features) method utilizes multi-scale information and merges the feature maps of each intermediate layer into the output. They fuse these features with a 1x1x1 convolution operation at each stage. Finally, the image (map) of each stage is upsampled to the original size, connected to the corresponding side output, or fused into the feature map (feature map) of the final output. In this way, the network makes full use of the rich information at multiple scales and levels. In addition, with the increase of convolutional layers, these features become too abstract to express meaningful vascular structures. Based on the above, the relevant cut images can be obtained, which can be seen in Figure 6 .

[0102] 3. Fundus color image classification:

[0103] The fundus color image classification model is a fundus color image classification network composed of convolutional layers, global mean pooling layers, maximum pooling layers, fully connected layers, batch normalization and rectified linear units, which is obtained through training.

[0104] The steps to build the fundus color image classification model are as follows:

[0105] (1) Construct training set samples, collect fundus color images of the above eight categories, and perform denoising and smoothing on the fundus color images, pixel normalization, image size unification, and augmentation processing on the fundus color images by random flipping, center cropping, and adding Gaussian noise to obtain the training sample set.

[0106] (2) Construct a fundus color image classification network. The specific network structure is as follows: Figure 7 The fundus vascular image is obtained by the segmentation network in the previous step, such as Figure 7 As shown in FIG. 1 , the left eye fundus vascular image, the right eye fundus vascular image, the left eye fundus color image and the right eye fundus color image are respectively input into the image as shown in FIG. Figure 7 In the network structure in , feature extraction is performed to obtain four image feature maps. Since the fundus vascular image is a single channel and the fundus color image is a three-channel image, they need to be processed separately. The feature map of the fundus vascular image is squeezed to obtain a channel-level feature map, and then the global feature is excited to learn the relationship between each channel, and the weights of different channels are obtained. The channel-level feature map is multiplied, and the resulting fundus vascular feature map is added to obtain the final fundus vascular feature vector. The fundus color image is squeezed to obtain the respective RGB channel feature maps, and the RGB channel feature maps are spatially fused to obtain the final fundus color image feature vector. Finally, the left eye vascular feature vector, right eye fundus vascular feature vector, left eye fundus color feature vector, right eye fundus color feature vector, patient age, and gender feature vectors are concatenated, and after the global mean pooling layer and the fully connected layer, an 8-dimensional vector representing the category is finally obtained.

[0107] (3) The training samples are input into the fundus color image classification network, Focal Loss is used as the loss function, and the Adam optimizer is used to update the network parameters of the fundus color image classification network to determine the network parameters and obtain a fundus multi-disease classification model.

[0108] In the embodiment of the present invention, the existing convolutional neural network is improved in the segmentation of fundus blood vessels, and all the intermediate convolutional features are put into the final CNN classifier to learn richer multi-scale information, so that the blood vessel edge can be accurately located and more tiny blood vessels can be detected. In particular, the characteristics of each layer are combined, and the information of each scale is fully utilized to improve the detection accuracy of blood vessel edges and tiny thin-walled blood vessels.

[0109] like Figure 8 As shown, an embodiment of the present invention further provides a training device for a fundus color image classification model, comprising:

[0110] The first acquisition module 801 is used to acquire original fundus color images of different categories;

[0111] A preprocessing module 802 is used to preprocess the original fundus color image to obtain a target fundus color image;

[0112] An output module 803 is used to input the target fundus color image into the first convolutional neural network, and output a fundus vascular image corresponding to the target fundus color image;

[0113] The second acquisition module 804 is used to acquire the patient age characteristics and patient gender characteristics corresponding to the target fundus color image;

[0114] The model training module 805 is used to perform model training on the second convolutional neural network using the target fundus color image, fundus vascular image, patient age characteristics and patient gender characteristics to obtain a fundus color image classification model.

[0115] Optionally, in one embodiment, the preprocessing module 802 is used to perform color space transformation and green channel extraction on the original fundus color image to obtain a first fundus color image; and use an adaptive histogram equalization method to eliminate uneven illumination in the first fundus color image to obtain a target fundus color image.

[0116] Optionally, in one embodiment, the backbone network of the first convolutional neural network includes at least one convolutional layer, at least one maximum pooling layer and a sigmoid function layer.

[0117] Optionally, in one embodiment, the backbone network of the second convolutional neural network includes at least one convolutional layer, at least one global mean pooling layer, at least one maximum pooling layer, at least one fully connected layer, batch normalization and a rectified linear unit.

[0118] Optionally, in one embodiment, the preprocessing module 802 is used to perform image augmentation processing on the original fundus color image to obtain a target fundus color image.

[0119] Optionally, in one embodiment, the preprocessing module 802 is further used to perform denoising and smoothing, pixel normalization, image size unification, random flipping, center cropping, and Gaussian noise addition processing on the original fundus color image to obtain a target fundus color image.

[0120] Figure 8 Each module in the device shown has the function of realizing Figure 1 The functions of each step in the process can achieve the corresponding technical effects, which will not be described in detail here for the sake of brevity.

[0121] like Fig. 9 As shown, the embodiment of the present invention also provides a fundus color image classification device based on the fundus color image classification model. The fundus color image classification model is based on Figure 1 The training method of the fundus color image classification model shown is obtained, including:

[0122] An acquisition module 901 is used to acquire a fundus color image to be classified;

[0123] The output module 902 is used to input the fundus color image to be classified into the fundus color image classification model, and output the classification result of the fundus color image to be classified.

[0124] Fig. 9 Each module in the device shown has the function of realizing Figure 2 The functions of each step in the process can achieve the corresponding technical effects, which will not be described in detail here for the sake of brevity.

[0125] Fig.10 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown.

[0126] The electronic device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0127] Specifically, the processor 1001 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present invention.

[0128] The memory 1002 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1002 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In one example, the memory 1002 may include a removable or non-removable (or fixed) medium, or the memory 1002 is a non-volatile solid-state memory. The memory 1002 may be inside or outside the electronic device.

[0129] In one example, the memory 1002 may be a read-only memory (ROM). In one example, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0130] The processor 1001 reads and executes the computer program instructions stored in the memory 1002 to implement Figure 1 The method in the embodiment shown in the figure is achieved Figure 1 The corresponding technical effects achieved by executing the methods / steps in the example shown are not repeated here for the sake of brevity.

[0131] In one example, the electronic device may further include a communication interface 1003 and a bus 1010. Fig.10 As shown, the processor 1001, the memory 1002, and the communication interface 1003 are connected via a bus 1010 and communicate with each other.

[0132] The communication interface 1003 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiment of the present invention.

[0133] Bus 1010 includes hardware, software or both, and the components of electronic equipment are coupled to each other. For example, but not limitation, bus may include Accelerated Graphics Port (AGP) or other graphics bus, Enhanced Industry Standard Architecture (EISA) bus, Front Side Bus (FSB), Hyper Transport (HT) interconnection, Industry Standard Architecture (ISA) bus, InfiniBand interconnection, Low Pin Count (LPC) bus, Memory bus, Micro Channel Architecture (MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI-Express (PCI-X) bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association Local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1010 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0134] In addition, the embodiment of the present invention can be implemented by providing a computer storage medium. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, Figure 1 The training method of the fundus color image classification model shown or Figure 2 The fundus color image classification method based on the fundus color image classification model is shown.

[0135] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0136] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), appropriate firmware, plug-in, function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (Radio Frequency, RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0137] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0138] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A training method for a fundus color image classification model, characterized in that: include: Obtaining original fundus color images of different categories; Preprocessing the original fundus color image to obtain a target fundus color image; Inputting the target fundus color image into a first convolutional neural network, and outputting a fundus vascular image corresponding to the target fundus color image; Acquire the patient age characteristics and patient gender characteristics corresponding to the target fundus color image; Using the target fundus color image, the fundus vascular image, the patient age characteristics and the patient gender characteristics, a second convolutional neural network is trained to obtain a fundus color image classification model; The method of performing model training on a second convolutional neural network by using the target fundus color image, the fundus vascular image, the patient age feature, and the patient gender feature includes: Inputting the left eye fundus vascular image, the right eye fundus vascular image, the left eye fundus color image and the right eye fundus color image into the second convolutional neural network for feature extraction, respectively, to obtain image feature maps of the left eye fundus vascular image, the right eye fundus vascular image, the left eye fundus color image and the right eye fundus color image; For the image feature map of the left eye fundus vascular image or the right eye fundus vascular image, the image feature map is squeezed to obtain a channel-level feature map, an excitation operation is performed on the global feature, the relationship between each channel is learned, the weight of each channel is obtained, and the weight of each channel is multiplied by the channel-level feature map to obtain a fundus vascular feature map of each channel, and the fundus vascular feature maps of each channel are added to obtain a target fundus vascular feature vector; For the image feature map of the left eye fundus color image or the right eye fundus color image, the image feature map is squeezed to obtain the RGB channel feature map of each channel, and the RGB channel feature map of each channel is spatially fused to obtain the target fundus color image feature vector; The target fundus blood vessel feature vector, the target fundus color image feature vector, the patient age feature and the gender feature are concatenated, and a global mean pooling layer and a fully connected layer are performed to obtain a target feature vector; Based on the target feature vector, the patient age feature and the patient gender feature, a second convolutional neural network model is trained to obtain a fundus color image classification model.

2. The training method for the fundus color image classification model according to claim 1, characterized in that: The preprocessing of the original fundus color image to obtain a target fundus color image includes: Performing color space transformation and green channel extraction on the original fundus color image to obtain a first fundus color image; Adaptive histogram equalization is used to eliminate uneven illumination in the first fundus color image to obtain the target fundus color image.

3. The training method for the fundus color image classification model according to claim 1, characterized in that: The backbone network of the first convolutional neural network includes at least one convolutional layer, at least one maximum pooling layer and a sigmoid function layer.

4. The method for training a fundus color image classification model according to claim 1, characterized in that: The backbone network of the second convolutional neural network includes at least one convolutional layer, at least one global mean pooling layer, at least one maximum pooling layer, at least one fully connected layer, batch normalization and a rectified linear unit.

5. The method for training a fundus color image classification model according to claim 1, characterized in that: The preprocessing of the original fundus color image to obtain a target fundus color image includes: The original fundus color image is subjected to image augmentation processing to obtain the target fundus color image.

6. The method for training a fundus color image classification model according to claim 5, characterized in that: The performing image augmentation processing on the original fundus color image to obtain the target fundus color image includes: The original fundus color image is subjected to denoising and smoothing, pixel normalization, image size unification, random flipping, center cropping and Gaussian noise addition processing to obtain the target fundus color image.

7. A fundus color image classification method based on a fundus color image classification model, characterized in that: The fundus color image classification model is obtained by using the training method of the fundus color image classification model according to any one of claims 1 to 6, comprising: Acquire a fundus color image to be classified; The fundus color image to be classified is input into the fundus color image classification model, and a classification result of the fundus color image to be classified is output.

8. A training device for a fundus color image classification model, characterized in that: include: The first acquisition module is used to acquire original fundus color images of different categories; A preprocessing module, used for preprocessing the original fundus color image to obtain a target fundus color image; An output module, used for inputting the target fundus color image into a first convolutional neural network, and outputting a fundus vascular image corresponding to the target fundus color image; A second acquisition module is used to acquire the patient age characteristics and patient gender characteristics corresponding to the target fundus color image; A model training module, used to perform model training on a second convolutional neural network using the target fundus color image, the fundus vascular image, the patient age characteristics and the patient gender characteristics to obtain a fundus color image classification model; The model training module is further used to input the left eye fundus vascular image, the right eye fundus vascular image, the left eye fundus color image and the right eye fundus color image into the second convolutional neural network for feature extraction, and obtain image feature maps of the left eye fundus vascular image, the right eye fundus vascular image, the left eye fundus color image and the right eye fundus color image respectively; The model training module is further used to compress the image feature map of the left eye fundus vascular image or the right eye fundus vascular image to obtain a channel-level feature map, perform an excitation operation on the global feature, learn the relationship between each channel, obtain the weight of each channel, multiply the weight of each channel by the channel-level feature map to obtain the fundus vascular feature map of each channel, and add the fundus vascular feature maps of each channel to obtain a target fundus vascular feature vector; The model training module is further used to squeeze the image feature map of the left eye fundus color image or the right eye fundus color image to obtain the RGB channel feature map of each channel, and perform a space-level fusion operation on the RGB channel feature map of each channel to obtain a target fundus color image feature vector; The model training module is further used to concatenate the target fundus blood vessel feature vector, the target fundus color image feature vector, the patient age feature and the gender feature, and obtain the target feature vector through a global mean pooling layer and a fully connected layer; The model training module is also used to perform model training on the second convolutional neural network based on the target feature vector, the patient age characteristics and the patient gender characteristics to obtain a fundus color image classification model.

9. A fundus color image classification device based on a fundus color image classification model, characterized in that: The fundus color image classification model is obtained by using the training method of the fundus color image classification model according to any one of claims 1 to 6, comprising: An acquisition module, used for acquiring a fundus color image to be classified; The output module is used to input the fundus color image to be classified into the fundus color image classification model and output the classification result of the fundus color image to be classified.

10. An electronic device, characterized in that: The electronic device comprises: a processor, and a memory storing computer program instructions; The processor reads and executes the computer program instructions to implement the training method of the fundus color image classification model according to any one of claims 1 to 6; or, The processor reads and executes the computer program instructions to implement the fundus color image classification method based on the fundus color image classification model as described in claim 7.

11. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the training method for the fundus color image classification model according to any one of claims 1 to 6 is implemented; or, The processor reads and executes the computer program instructions to implement the fundus color image classification method based on the fundus color image classification model as described in claim 7.

Images