Method, device and storage medium for glaucoma image detection

Abstract

The present application relates to the field of image recognition technology, and discloses a method for glaucoma image detection, the method comprising: obtaining a retinal fundus image, and intercepting the retinal fundus image to obtain an optic disc image; extracting attention features from the retinal fundus image, and extracting local features from the optic disc image; obtaining explicit factor features and implicit factor features corresponding to the retinal fundus image according to the attention features; obtaining global features corresponding to the retinal fundus image according to the explicit factor features and the implicit factor features; performing feature fusion on the global features and the local features to obtain fused features; inputting the fused features into a preset image detection model to obtain the image type corresponding to the retinal fundus image. This can improve the accuracy of glaucoma image detection. The present application also discloses a device and storage medium for glaucoma image detection.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, for example, to a method and device, and a storage medium for glaucoma image detection. Background Art

[0002] As people are increasingly encouraged to pay attention to their health, they often need to pay attention to the health of various parts of the body, and the eyes are a particularly important part of human health. Retinal fundus images are important images that record the health of the eyes. Currently, most deep learning-based glaucoma image detection methods detect whether the retinal fundus image is a glaucoma type image by detecting local features in the image, such as the ratio of the optic disc / cup (C / D).

[0003] In the process of implementing the embodiments of the present disclosure, it is found that there are at least the following problems in the related art:

[0004] In the prior art, whether a retinal fundus image is a glaucoma type image is identified only by detecting local features such as the ratio of the optic disc / optic cup. However, since each retinal fundus image is different, the detection accuracy of the retinal fundus image is easily poor. Summary of the invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] The embodiments of the present disclosure provide a method, an apparatus, and a storage medium for glaucoma image detection, so as to improve the detection accuracy of retinal fundus images.

[0007] In some embodiments, the method for glaucoma image detection includes: acquiring a retinal fundus image, and intercepting the retinal fundus image to obtain an optic disc image; extracting attention features from the retinal fundus image, and extracting local features from the optic disc image; acquiring explicit factor features and implicit factor features corresponding to the retinal fundus image based on the attention features; the explicit factor features are used to characterize significant features in the retinal fundus image, and the implicit factor features are used to characterize minor features in the retinal fundus image; acquiring global features corresponding to the retinal fundus image based on the explicit factor features and the implicit factor features; performing feature fusion on the global features and the local features to obtain fused features; inputting the fused features into a preset image detection model to obtain an image type corresponding to the retinal fundus image.

[0008] In some embodiments, the device for glaucoma image detection includes a processor and a memory storing program instructions, and the processor is configured to perform the above-mentioned method for glaucoma image detection when executing the program instructions.

[0009] In some embodiments, the storage medium stores program instructions, and when the program instructions are run, the above-mentioned method for glaucoma image detection is executed.

[0010] The method, device and storage medium for glaucoma image detection provided by the embodiments of the present disclosure can achieve the following technical effects: by acquiring a retinal fundus image and intercepting the retinal fundus image, an optic disc image is obtained; attention features are extracted from the retinal fundus image, and local features are extracted from the optic disc image; explicit factor features and implicit factor features corresponding to the retinal fundus image are obtained according to the attention features; global features corresponding to the retinal fundus image are obtained according to the explicit factor features and the implicit factor features; global features and local features are fused to obtain fused features; the fused features are input into a preset image detection model to obtain the image type corresponding to the retinal fundus image. In this way, by acquiring the global features and local features of the retinal fundus image, since the global features include secondary features, a richer fundus image in the retinal fundus image can be obtained, and by fusing the global features and local features to obtain fused features, the retinal fundus image is detected according to the fused features, thereby improving the detection accuracy of the retinal fundus image.

[0011] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0013] Figure 1 is a schematic diagram of a method for glaucoma image detection provided by an embodiment of the present disclosure;

[0014] Figure 2 is a schematic diagram of a network structure for extracting global features provided by an embodiment of the present disclosure;

[0015] Figure 3 is a schematic diagram of another method for glaucoma image detection provided by an embodiment of the present disclosure;

[0016] Figure 4is a schematic diagram of a method for obtaining an image detection model provided by an embodiment of the present disclosure;

[0017] Figure 5 is a schematic diagram of a mutual learning network structure for glaucoma image detection provided by an embodiment of the present disclosure;

[0018] Figure 6 It is a schematic diagram of a device for glaucoma image detection provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0020] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0021] Unless otherwise stated, the term "plurality" means two or more.

[0022] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0023] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0024] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0025] Combination Figure 1 As shown, the present disclosure provides a method for glaucoma image detection, the method comprising:

[0026] Step S101, acquiring a retinal fundus image, and intercepting the retinal fundus image to obtain an optic disc image.

[0027] Step S102, extracting attention features from the retinal fundus image, and extracting local features from the optic disc image.

[0028] Step S103, obtaining explicit factor features and implicit factor features corresponding to the retinal fundus image according to the attention features.

[0029] Step S104, obtaining global features corresponding to the retinal fundus image according to the dominant factor features and the recessive factor features.

[0030] Step S105, performing feature fusion on the global features and the local features to obtain fused features.

[0031] Step S106, inputting the fusion feature into a preset image detection model to obtain the image type corresponding to the retinal fundus image.

[0032] The method for glaucoma image detection provided by the embodiment of the present disclosure is adopted, by acquiring a retinal fundus image and intercepting the retinal fundus image to obtain an optic disc image; extracting attention features from the retinal fundus image and extracting local features from the optic disc image; acquiring dominant factor features and recessive factor features corresponding to the retinal fundus image according to the attention features; acquiring global features corresponding to the retinal fundus image according to the dominant factor features and recessive factor features; performing feature fusion on the global features and local features to obtain fusion features; inputting the fusion features into a preset image detection model to obtain the image type corresponding to the retinal fundus image. In this way, by acquiring the global features and local features of the retinal fundus image, since the global features include secondary features, richer fundus image features in the retinal fundus image can be obtained, by fusing the global features and local features to obtain fusion features, the retinal fundus image is detected according to the fusion features, thereby improving the detection accuracy of the retinal fundus image.

[0033] Optionally, intercepting the retinal fundus image to obtain the optic disc image includes: identifying the optic disc position in the retinal fundus image by a preset detector, and intercepting the retinal fundus image according to the optic disc position to obtain the optic disc image. In this way, by intercepting the optic disc image separately and performing feature analysis, it is easy to extract the features of the optic disc and optic cup, thereby facilitating the detection of whether the fundus retinal image is a glaucoma type image.

[0034] Optionally, the preset detector is Faster R-CNN (Region-Convolutional Neural Networks). Faster R-CNN is used as the target detector due to its flexibility and robustness.

[0035] In some embodiments, the retinal fundus image is input into a preset detector, a shared feature map is generated through the convolution layer in the detector, the position of the sparse rectangular candidate target box is generated according to the shared feature map, and the target score and regression region boundary are predicted through the fully connected layer of the detector, and then the bounding box with the highest confidence is retained as the output of the detector. Optionally, the output of the detector is the bounding box with the highest confidence for each foreground class; in the retinal fundus image, the optic disc and optic cup to be detected are the foreground class, that is, the target to be detected, and the rest are the background class, that is, other factors in the retinal fundus image that are not related to the optic disc and optic cup.

[0036] Optionally, attention features are extracted from the retinal fundus image, and local features are extracted from the optic disc image, including: extracting attention features from the retinal fundus image through a feature extraction backbone network in an image detection model, and extracting local features from the optic disc image. Optionally, the feature extraction backbone network is a Swin-transformer (hierarchical visual transformer). In this way, by extracting features from the retinal fundus image and the optic disc image through the Swin-transformer network structure, it is possible to more significantly focus on the connection between the optic disc and cup area and other areas in the retinal fundus image.

[0037] Optionally, the local features include optic disc / optic cup area ratio, optic disc shape, optic disc color, etc.

[0038] Optionally, the global features include the shape of the optic nerve covering the retina, the color of the optic nerve, the overall color of the fundus image, the morphology of the nerve blood vessels, etc. Optionally, the secondary features are the shape of the optic nerve covering the retina, the color of the optic nerve, the overall color of the fundus image, the morphology of the nerve blood vessels, etc. Optionally, the secondary features are recessive features.

[0039] In some embodiments, the basic idea of ​​the attention mechanism in computer vision is to let the system learn to pay attention, and this attention allows the system to focus on the area it wants to focus on. In an embodiment of the present application, the image detection model can learn to pay attention to the target detection object, that is, the retinal fundus image, and can ignore irrelevant information and focus on key information. In an embodiment of the present application, the attention mechanism can focus on the features of the retinal fundus image, that is, the features of the retinal fundus image are used as attention features.

[0040] Optionally, obtaining explicit factor characteristics and implicit factor characteristics corresponding to the retinal fundus image based on the attention characteristics includes: decomposing the attention characteristics to obtain explicit factor characteristics and implicit factor characteristics corresponding to the retinal fundus image.

[0041] Optionally, the dominant factor features include features of the optic disc and optic cup, such as optic disc / optic cup area ratio, optic disc shape and optic disc color.

[0042] Optionally, the latent factor features include: the shape of the optic nerve covering the retina, the color of the optic nerve, the overall color of the fundus image, the neurovascular morphology, etc.

[0043] Optionally, the global features corresponding to the retinal fundus image are obtained according to the dominant factor features and the recessive factor features, including: fusing the dominant factor features with the attention features to obtain the dominant features, and fusing the recessive factor features with the attention features to obtain the recessive features; fusing the dominant features with the recessive features to obtain the global features. In this way, since the local features are the features of the optic disc and the optic cup, which are the fine-grained features for glaucoma detection, and the global features include richer coarse-grained features, by fusing the global features and the local features, it is possible to pay attention to both the key local features in the fundus image and the global features in the fundus image, thereby paying attention to the intrinsic connection at the feature granularity level and improving the detection accuracy of the retinal fundus image.

[0044] Combination Figure 2 As shown, Figure 2 It is a schematic diagram of a network structure for extracting global features provided by an embodiment of the present disclosure; in some embodiments, attention features are extracted by the stage module in Swin-transformer, and the attention features are decomposed into explicit factor features and implicit factor features, the explicit factor features are fused with the attention features to obtain explicit features, the implicit factor features are fused with the attention features to obtain implicit features, and the global features are obtained by fusing the explicit features with the implicit features.

[0045] Combination Figure 3 As shown, the present disclosure provides a method for glaucoma image detection, the method comprising:

[0046] Step S301, acquiring a retinal fundus image, and intercepting the retinal fundus image to obtain an optic disc image.

[0047] Step S302, extracting attention features from the retinal fundus image, and extracting local features from the optic disc image.

[0048] Step S303, decomposing the attention features to obtain explicit factor features and implicit factor features corresponding to the retinal fundus image.

[0049] Step S304: The explicit factor feature is fused with the attention feature to obtain the explicit feature, and the implicit factor feature is fused with the attention feature to obtain the implicit feature.

[0050] Step S305: fuse the explicit features and the implicit features to obtain the global features.

[0051] Step S306: perform feature fusion on the global features and the local features to obtain fused features.

[0052] Step S307: input the fusion feature into a preset image detection model to obtain the image type corresponding to the retinal fundus image.

[0053] The method for glaucoma image detection provided by the embodiment of the present disclosure is adopted. The attention features of the retinal fundus image are extracted, and the implicit factor features and the explicit factor features are decomposed therefrom. The implicit factor features and the explicit factor features are respectively fused with the attention features to obtain the explicit features and the implicit features. The explicit features and the implicit features are fused to obtain the global features. The global features and local features of the retinal fundus image are obtained, so that richer fundus image features in the retinal fundus image can be obtained. The fused features are obtained by fusing the global features and the local features. The retinal fundus image is detected according to the fused features, thereby improving the detection accuracy of the retinal fundus image.

[0054] Optionally, the image detection model is obtained in the following manner, including: obtaining multiple first training samples and multiple sample labels; the first training sample is a retinal fundus sample image, and the sample label is used to characterize the image type corresponding to the retinal fundus sample image; and training a preset mutual learning network according to the first training sample with the sample label to obtain the image detection model.

[0055] Optionally, the first training sample is a retinal fundus image.

[0056] Optionally, obtaining a plurality of first training samples includes: obtaining a plurality of retinal fundus images; and preprocessing each retinal fundus image to obtain a plurality of first training samples.

[0057] Optionally, each retinal fundus image is preprocessed, including: performing horizontal flipping, vertical flipping, random cropping, etc. on each retinal fundus image. In this way, by preprocessing the retinal fundus image, multiple first training samples can be obtained, thereby achieving data enhancement and increasing the number of training samples.

[0058] Optionally, the mutual learning network includes a first convolutional neural network and a second convolutional neural network, and the second convolutional neural network is a peer network of the first convolutional neural network; the preset mutual learning network is trained according to the first training samples with sample labels to obtain an image detection model, including: extracting features from each first training sample to obtain sample global features corresponding to each first training sample, and intercepting each first training sample to obtain second training samples corresponding to each first training sample; inputting each second training sample into a preset first convolutional neural network and a preset second convolutional neural network for feature extraction to obtain first sample local features and second sample local features; fusing each sample global feature with each first sample local feature and each second sample local feature to obtain first sample fusion features and second sample fusion features; training the mutual learning network according to each first sample fusion feature and each second sample fusion feature to obtain an image detection model.

[0059] Optionally, the mutual learning network includes a fully connected network, and the mutual learning network is trained according to each first sample fusion feature and each second sample fusion feature to obtain an image detection model, including: inputting each first sample fusion feature and each second sample fusion feature into the fully connected network to obtain a first probability value that each training sample is a preset type and a second probability value that each training sample is a preset type; obtaining a first feature distribution similarity corresponding to the first convolutional neural network and a second feature distribution similarity corresponding to the second convolutional neural network according to each first probability value and each second probability value; determining a first loss function corresponding to the first convolutional neural network according to the first feature distribution similarity, and determining a second loss function corresponding to the second convolutional neural network according to the second feature distribution similarity; iteratively training the first convolutional neural network and the second convolutional neural network according to the first loss function and the second loss function, respectively, to obtain an image detection model.

[0060] Optionally, the first feature distribution similarity is used to characterize the similarity of feature distribution between features output by the second convolutional neural network and features output by the first convolutional neural network.

[0061] Optionally, the second feature distribution similarity is used to characterize the similarity of feature distribution between features output by the first convolutional neural network and features output by the second convolutional neural network.

[0062] Optionally, the preset type includes a normal type and an abnormal type.

[0063] Optionally, the first convolutional neural network includes a Swin-transformer network structure; and the second convolutional neural network includes a Swin-transformer network structure.

[0064] Optionally, obtaining the first feature distribution similarity corresponding to the first convolutional neural network according to each first probability value and each second probability value includes: calculating according to a third preset algorithm using each first probability value and each second probability value to obtain the first feature distribution similarity.

[0065] Optionally, performing calculations using each first probability value and each second probability value according to a third preset algorithm to obtain the first feature distribution similarity includes: calculating Obtain the first feature distribution similarity; wherein,

[0066] D KL (P2||P1) is the first feature distribution similarity, P1 is used to characterize the feature distribution corresponding to the first convolutional neural network, and P2 is used to characterize the feature distribution corresponding to the second convolutional neural network; N is the number of first training samples, M=2, x i is the first training sample of the ith order; P1 m (x i ) is the probability value of the first training sample corresponding to the first convolutional neural network being of type m, is the probability value that the first training sample corresponding to the second convolutional neural network is of type m.

[0067] Optionally, obtaining the second feature distribution similarity corresponding to the second convolutional neural network according to each first probability value and each second probability value includes: calculating according to a fourth preset algorithm using each first probability value and each second probability value to obtain the second feature distribution similarity.

[0068] Optionally, performing calculations using each first probability value and each second probability value according to a fourth preset algorithm to obtain a second feature distribution similarity includes: calculating Obtain the second feature distribution similarity; wherein,

[0069] D KL (P1||P2) is the second feature distribution similarity, P1 is used to characterize the feature distribution corresponding to the first convolutional neural network, and P2 is used to characterize the feature distribution corresponding to the second convolutional neural network; N is the number of first training samples, M=2, x i is the first training sample of the ith order; P1 m (x i ) is the probability value of the first training sample corresponding to the first convolutional neural network being of type m, is the probability value that the first training sample corresponding to the second convolutional neural network is of type m.

[0070] Optionally, determining a first loss function corresponding to the first convolutional neural network according to the first feature distribution similarity includes: calculating using each first probability value according to a first preset algorithm to obtain a first prediction result; determining the first loss function according to the first prediction result and the first feature distribution similarity.

[0071] Optionally, performing calculations using each first probability value according to a first preset algorithm to obtain a first prediction result includes: calculating Obtain the first prediction result; where Lc1 is the first prediction result, N is the number of first training samples, M=2, x i is the first training sample of the ith order; P1 m (x i ) is the probability value of type m of the first training sample corresponding to the first convolutional neural network, optionally, m=1 represents “normal”, and m=2 represents “abnormal”; optionally, The predicted type y of the first training sample in the i-th i In the case of m type, I(y i ,m)=1; the predicted category y of the first training sample in the i-th i If it is not of type m, I(y i ,m)=0; Optionally, the probability of the normal type is used to characterize the probability that the test result is not glaucoma, and the probability of the abnormal type is used to characterize the probability that the test result is glaucoma.

[0072] Optionally, the first loss function is L θ1 =Lc1+D KL (P2||P1); where L θ1 is the loss value corresponding to the first loss function, Lc1 is the first preset result, P2 is the feature distribution of the feature output by the second convolutional neural network, P1 is the feature distribution of the feature output by the first convolutional neural network, and D KL (P2||P1) is the first feature distribution similarity, that is, the similarity of the feature distribution between the features output by the second convolutional neural network and the features output by the first convolutional neural network.

[0073] Optionally, determining a second loss function corresponding to the second convolutional neural network according to the similarity of the second feature distribution includes: calculating using each second probability value according to a second preset algorithm to obtain a second prediction result; determining a second loss function according to the second prediction result and the similarity of the second feature distribution.

[0074] Optionally, performing calculations using each second probability value according to a second preset algorithm to obtain a second prediction result includes: calculating Obtain the second prediction result; where Lc2 is the second prediction result, N is the number of first training samples, M=2, xi is the i-th first training sample; is the probability value of type m corresponding to the second convolutional neural network, optionally, m=1 represents “normal”, m=2 represents “abnormal”; optionally, The predicted type y of the first training sample in the i-th i In the case of m type, I(y i ,m)=1; the predicted category y of the first training sample in the i-th i If it is not of type m, I(y i ,m)=0; Optionally, the probability of the normal type is used to characterize the probability that the test result is not glaucoma, and the probability of the abnormal type is used to characterize the probability that the test result is glaucoma.

[0075] Optionally, the second loss function is L θ2 =Lc2+D KL (P1||P2); where L θ2 is the loss value corresponding to the first loss function, Lc2 is the first preset result, P2 is the feature distribution of the feature output by the second convolutional neural network, P1 is the feature distribution of the feature output by the first convolutional neural network, and D KL (P1||P2) is the second feature distribution similarity, that is, the similarity of the feature distribution between the features output by the first convolutional neural network and the features output by the second convolutional neural network.

[0076] Combination Figure 4 As shown, the embodiment of the present disclosure provides a method for obtaining an image detection model, the method comprising:

[0077] Step S401, obtaining a plurality of first training samples and a plurality of sample labels; the sample labels are used to characterize the image types corresponding to the first training samples.

[0078] Step S402 , extracting features from each first training sample to obtain sample global features corresponding to each first training sample, and intercepting each first training sample to obtain second training samples corresponding to each first training sample.

[0079] Step S403: Input each second training sample into a preset first convolutional neural network and a preset second convolutional neural network respectively to perform feature extraction, so as to obtain a first sample local feature and a second sample local feature.

[0080] Step S404: perform feature fusion on each sample global feature with each first sample local feature and each second sample local feature to obtain a first sample fusion feature and a second sample fusion feature.

[0081] Step S405 , training the mutual learning network according to each first sample fusion feature and each second sample fusion feature to obtain an image detection model.

[0082] The method for obtaining an image detection model provided by an embodiment of the present disclosure is adopted. By setting a peer network with the first convolutional neural network, that is, a second convolutional neural network, the first convolutional neural network and the second convolutional neural network are trained through a first training sample, so that the first convolutional neural network and the second convolutional neural network learn from each other, thereby obtaining an image detection model.

[0083] Optionally, the image type includes a normal type or an abnormal type.

[0084] Combination Figure 5 As shown, Figure 5 It is a schematic diagram of a mutual learning network structure for glaucoma image detection provided by an embodiment of the present disclosure; in some embodiments, when training the mutual learning network, global features are extracted from the retinal fundus image, and the retinal fundus image is intercepted to obtain an optic disc image, and the optic disc image is respectively input into the first convolutional neural network and the second convolutional neural network to obtain a first local feature and a second local feature, and the global feature is fused with the first local feature and the second local feature respectively to obtain a first fused feature Z1 and a second fused feature Z2, and the retinal fundus image is predicted to be a first probability value of a preset type according to the first fused feature, and the retinal fundus image is predicted to be a second probability value of a preset type according to the second fused feature; and the first feature distribution similarity and the second feature distribution similarity are obtained so that the two networks learn from each other, so that the feature distribution corresponding to the first convolutional neural network and the feature distribution corresponding to the second convolutional neural network are as similar as possible.

[0085] Optionally, when the retinal fundus image is detected by the image detection model, the prediction result corresponding to the second convolutional neural network is abandoned, and the prediction result corresponding to the first convolutional neural network is determined as the image type corresponding to the retinal fundus image.

[0086] In some embodiments, the first probability value of the retinal fundus image corresponding to the first convolutional neural network is {"the probability of the normal type is: 99%", "the probability of the abnormal type is: 1%"}, then the type with a larger probability value is determined as the prediction result, that is, the prediction result is the normal type, and the "normal type" is determined as the image type corresponding to the retinal fundus image.

[0087] In some embodiments, when the first convolutional neural network and the second convolutional neural network are trained through training samples, the network parameter weights of the second convolutional neural network are fixed unchanged, and the features of the output of the first convolutional neural network, that is, the probability that the training sample is a preset type, are obtained; a first prediction result corresponding to the first convolutional neural network is obtained; the first feature distribution similarity between the features of the output of the second convolutional neural network and the features of the output of the first convolutional neural network is obtained, and a first loss function is determined based on the first prediction result and the first feature distribution similarity, and the first convolutional neural network is gradient back propagated through the first loss function to update the network parameter weights in the first convolutional neural network. At this time, the network parameter weights of the second convolutional neural network remain unchanged. When the first convolutional neural network and the second convolutional neural network are trained through training samples, the network parameter weights of the first convolutional neural network are fixed unchanged, and the features of the output of the second convolutional neural network, that is, the probability that the training sample is of a preset type, are obtained; a second prediction result corresponding to the second convolutional neural network is obtained; the second feature distribution similarity between the features of the output of the first convolutional neural network and the features of the output of the second convolutional neural network is obtained, and a second loss function is determined according to the second prediction result and the second feature distribution similarity, and the second convolutional neural network is gradient back-propagated through the second loss function to update the network parameter weights in the second convolutional neural network, and the network parameter weights of the first convolutional neural network remain unchanged.

[0088] In this way, when the type of training samples is predicted by the first convolutional neural network, the second feature distribution obtained by training the second convolutional neural network is used for reference. When the type of training samples is predicted by the second convolutional neural network, the first feature distribution obtained by training the first convolutional neural network is also used for reference. The similarity between the first feature distribution and the second feature distribution is measured by KL divergence. In each iterative training, the network parameter weights of the first convolutional neural network are adjusted according to the first loss function, and the network parameter weights of the second convolutional neural network are adjusted according to the second loss function. This can make the feature distributions output by the first convolutional neural network and the second convolutional neural network as similar as possible, so that the two networks can learn from each other.

[0089] In some embodiments, the second convolutional neural network can provide experience to the first convolutional neural network in the form of a posteriori probability, that is, the feature distribution similarity between the first feature distribution corresponding to the first convolutional neural network and the second feature distribution corresponding to the second convolutional neural network is measured by KL divergence.

[0090] Most of the prior art methods reduce dimension by adding a downsampling layer, thereby reducing the number of parameters that the network needs to learn and preventing overfitting. In the embodiments disclosed in the present application, no downsampling layer is used. By using an extended convolutional layer instead of a downsampling layer, the clarity of the image can be guaranteed.

[0091] Combination Figure 6 As shown, an embodiment of the present disclosure provides a device for glaucoma image detection, including a processor (processor) 600 and a memory (memory) 601. Optionally, the device may also include a communication interface (Communication Interface) 602 and a bus 603. Among them, the processor 600, the communication interface 602, and the memory 601 can communicate with each other through the bus 603. The communication interface 602 can be used for information transmission. The processor 600 can call the logic instructions in the memory 601 to execute the method for glaucoma image detection of the above embodiment.

[0092] The device for glaucoma image detection provided by the embodiment of the present disclosure is used to obtain a retinal fundus image and intercept the retinal fundus image to obtain an optic disc image; extract attention features from the retinal fundus image and extract local features from the optic disc image; obtain the dominant factor features and recessive factor features corresponding to the retinal fundus image according to the attention features; obtain the global features corresponding to the retinal fundus image according to the dominant factor features and recessive factor features; perform feature fusion on the global features and local features to obtain fusion features; input the fusion features into a preset image detection model to obtain the image type corresponding to the retinal fundus image. In this way, by obtaining the global features and local features of the retinal fundus image, since the global features include secondary features, richer fundus image features in the retinal fundus image can be obtained, and by fusing the global features and local features to obtain fusion features, the retinal fundus image is detected according to the fusion features, thereby improving the detection accuracy of the retinal fundus image.

[0093] In addition, the logic instructions in the memory 601 described above can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0094] The memory 601 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 600 executes the function application and data processing by running the program instructions / modules stored in the memory 601, that is, the method for glaucoma image detection in the above embodiment is implemented.

[0095] The memory 601 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 601 may include a high-speed random access memory and may also include a non-volatile memory.

[0096] An embodiment of the present disclosure provides a storage medium storing program instructions, which, when executed, execute the above-mentioned method for glaucoma image detection.

[0097] An embodiment of the present disclosure provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned method for glaucoma image detection.

[0098] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0099] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0100] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0101] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0102] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0103] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for glaucoma image detection, characterized in that: include: Acquire a retinal fundus image, and intercept the retinal fundus image to obtain an optic disc image; extracting attention features from the retinal fundus image and extracting local features from the optic disc image; Acquire explicit factor features and implicit factor features corresponding to the retinal fundus image according to the attention features; The dominant factor feature is used to characterize the significant features required for detecting glaucoma in the retinal fundus image, and the recessive factor feature is used to characterize the minor features required for detecting glaucoma in the retinal fundus image; Acquire the global features corresponding to the retinal fundus image according to the dominant factor features and the recessive factor features; Performing feature fusion on the global feature and the local feature to obtain a fused feature; The fusion feature is input into a preset image detection model to obtain the image type corresponding to the retinal fundus image.

2. The method according to claim 1, characterized in that Obtaining explicit factor features and implicit factor features corresponding to the retinal fundus image according to the attention features includes: The attention feature is decomposed to obtain explicit factor features and implicit factor features corresponding to the retinal fundus image.

3. The method according to claim 1, characterized in that Acquiring the global features corresponding to the retinal fundus image according to the dominant factor features and the recessive factor features includes: The explicit factor feature is fused with the attention feature to obtain an explicit feature, and the implicit factor feature is fused with the attention feature to obtain an implicit feature; The explicit features and implicit features are fused to obtain the global features.

4. The method according to claim 1, characterized in that The image detection model is obtained by the following method, including: Acquire a plurality of first training samples and a plurality of sample labels; the first training samples are retinal fundus sample images, and the sample labels are used to characterize the image types corresponding to the retinal fundus sample images; The preset mutual learning network is trained according to the first training sample with the sample label to obtain the image detection model.

5. The method according to claim 4, characterized in that The mutual learning network includes a first convolutional neural network and a second convolutional neural network, wherein the second convolutional neural network is a peer network of the first convolutional neural network; Training a preset mutual learning network according to the first training sample with the sample label to obtain the image detection model includes: Extracting features from each of the first training samples to obtain sample global features corresponding to each of the first training samples, and intercepting each of the first training samples to obtain second training samples corresponding to each of the first training samples; Inputting each of the second training samples into a preset first convolutional neural network and a preset second convolutional neural network for feature extraction, respectively, to obtain first sample local features and second sample local features; Fusing the global features of each sample with the local features of each first sample and the local features of each second sample to obtain a first sample fusion feature and a second sample fusion feature; The mutual learning network is trained according to each of the first sample fusion features and each of the second sample fusion features to obtain the image detection model.

6. The method according to claim 5, characterized in that The mutual learning network includes a fully connected network, and the mutual learning network is trained according to each of the first sample fusion features and each of the second sample fusion features to obtain the image detection model, including: Inputting each of the first sample fusion features and each of the second sample fusion features into the fully connected network to obtain a first probability value of each of the training samples being a preset type and a second probability value of each of the training samples being the preset type; Obtaining respectively a first feature distribution similarity corresponding to the first convolutional neural network and a second feature distribution similarity corresponding to the second convolutional neural network according to each of the first probability values ​​and each of the second probability values; Determine a first loss function corresponding to the first convolutional neural network according to the first feature distribution similarity, and determine a second loss function corresponding to the second convolutional neural network according to the second feature distribution similarity; The first convolutional neural network and the second convolutional neural network are iteratively trained according to the first loss function and the second loss function respectively to obtain the image detection model.

7. The method according to claim 6, characterized in that Determining a first loss function corresponding to the first convolutional neural network according to the first feature distribution similarity includes: Perform calculation using each of the first probability values ​​according to a first preset algorithm to obtain a first prediction result; The first loss function is determined according to the first prediction result and the first feature distribution similarity.

8. The method according to claim 6, characterized in that Determining a second loss function corresponding to the second convolutional neural network according to the second feature distribution similarity includes: Perform calculation using each of the second probability values ​​according to a second preset algorithm to obtain a second prediction result; The second loss function is determined according to the second prediction result and the second feature distribution similarity.

9. A device for glaucoma image detection, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the method for glaucoma image detection according to any one of claims 1 to 8 when running the program instructions.

10. A storage medium storing program instructions, characterized in that: When the program instructions are executed, the method for glaucoma image detection according to any one of claims 1 to 8 is executed.

Images