Method, device and storage medium for identifying eye features

Abstract

The present application relates to the field of image recognition technology, and discloses a method for identifying eye features, the method comprising: collecting retinal fundus images; inputting the retinal fundus images into a trained encoder, and decoupling the relevant features and irrelevant features of the retinal fundus; using the relevant features of the retinal fundus to detect the image type of the retinal fundus image; wherein the image type of the retinal fundus image includes a normal type image and an abnormal type image. In this way, the relevant features and irrelevant features of the retinal fundus can be decoupled from the retinal fundus image by the trained encoder, and the type of the retinal fundus image is detected by the relevant features, thereby reducing the interference of irrelevant features on the recognition of the retinal fundus image, thereby improving the recognition accuracy of the retinal fundus image. The present application also discloses a device and a storage medium for identifying eye features.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, for example, to a method and device, and a storage medium for recognizing eye features. Background Art

[0002] As people are increasingly encouraged to pay attention to their health, they often need to pay attention to the health of various parts of the body, and the eyes are a particularly important part of human health. Retinal fundus images are important images that record the health of the eyes. Currently, most of the related technologies based on deep learning glaucoma image detection methods detect whether the retinal fundus image is a normal type or an abnormal type of fundus image by detecting the ratio of the optic disc / cup in the image.

[0003] In the process of implementing the embodiments of the present disclosure, it is found that there are at least the following problems in the related art:

[0004] In the related art, irrelevant features on the retinal fundus image, such as background information and edge information, are usually considered when performing type detection on the retinal fundus image. This can easily interfere with the recognition of eye features, resulting in poor recognition accuracy of the retinal fundus image. Summary of the invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] The embodiments of the present disclosure provide a method and device, and a storage medium for identifying eye features, so as to improve the recognition accuracy of retinal fundus images.

[0007] In some embodiments, the method for identifying eye features includes: acquiring a retinal fundus image; inputting the retinal fundus image into a trained encoder to decouple relevant features and irrelevant features of the retinal fundus; and detecting the image type of the retinal fundus image using the relevant features of the retinal fundus; wherein the image type of the retinal fundus image includes a normal type image and an abnormal type image.

[0008] In some embodiments, the device for identifying eye features includes: an acquisition module configured to acquire a retinal fundus image; a feature extraction module configured to input the retinal fundus image into a preset network structure to obtain detection-related features corresponding to the retinal fundus image; the preset network structure is used to decouple the detection-related features from the retinal fundus image; and a prediction module configured to predict the image type of the retinal fundus image based on the detection-related features.

[0009] In some embodiments, the device for identifying eye features includes: an acquisition module configured to acquire retinal fundus images; a feature extraction module configured to input the retinal fundus images into a trained encoder to decouple relevant features and irrelevant features of the retinal fundus; and a detection module configured to detect the image type of the retinal fundus image using the relevant features of the retinal fundus; wherein the image types of the retinal fundus images include normal type images and abnormal type images.

[0010] In some embodiments, the apparatus for identifying eye features comprises a processor and a memory storing program instructions, and the processor is configured to perform the above-mentioned method for identifying eye features when executing the program instructions.

[0011] In some embodiments, the storage medium stores program instructions, and when the program instructions are run, the above-mentioned method for identifying eye features is executed.

[0012] The method, device and storage medium for identifying eye features provided by the embodiments of the present disclosure can achieve the following technical effects: by collecting retinal fundus images; inputting the retinal fundus images into the trained encoder to decouple the relevant features and irrelevant features of the retinal fundus; using the relevant features of the retinal fundus to detect the image type of the retinal fundus image; wherein the image type of the retinal fundus image includes normal type images and abnormal type images. In this way, the trained encoder can decouple the relevant features and irrelevant features of the retinal fundus from the retinal fundus image, detect the type of the retinal fundus image through the relevant features, reduce the interference of irrelevant features on the recognition of the retinal fundus image, and thus improve the recognition accuracy of the retinal fundus image.

[0013] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0015] Figure 1 is a schematic diagram of a method for identifying eye features provided by an embodiment of the present disclosure;

[0016] Figure 2 is a schematic diagram of a method for training an encoder provided by an embodiment of the present disclosure;

[0017] Figure 3is a schematic diagram of another method for identifying eye features provided by an embodiment of the present disclosure;

[0018] Figure 4 is a schematic diagram of the structure of a cross encoder-decoder glaucoma detection network provided by an embodiment of the present disclosure;

[0019] Figure 5 is a schematic diagram of a device for identifying eye features provided by an embodiment of the present disclosure;

[0020] Figure 6 is a schematic diagram of another device for identifying eye features provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0022] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0023] Unless otherwise stated, the term "plurality" means two or more.

[0024] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0025] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0026] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0027] The technical solution in the embodiment of the present invention can be applied to electronic devices. Optionally, the electronic devices include computers or servers.

[0028] In an embodiment of the present invention, a retinal fundus image is collected by an electronic device, and relevant features and irrelevant features of the retinal fundus are decoupled by a trained encoder, and the image type of the retinal fundus image can be detected using the relevant features.

[0029] The eye feature in the embodiment of the present invention is used to detect whether a retinal fundus image is a glaucoma fundus image.

[0030] Combination Figure 1 As shown, the present disclosure provides a method for identifying eye features, the method comprising:

[0031] Step S101: The electronic device collects retinal fundus images.

[0032] In step S102, the electronic device inputs the retinal fundus image into a trained encoder to decouple relevant features from irrelevant features of the retinal fundus.

[0033] Step S103, the electronic device detects the image type of the retinal fundus image by using relevant features of the retinal fundus; wherein the image type of the retinal fundus image includes a normal type image and an abnormal type image.

[0034] The method for identifying eye features provided by the embodiment of the present disclosure is adopted, by collecting retinal fundus images; inputting the retinal fundus images into a trained encoder, decoupling the relevant features and irrelevant features of the retinal fundus; using the relevant features of the retinal fundus to detect the image type of the retinal fundus image; wherein the image type of the retinal fundus image includes a normal type image and an abnormal type image. In this way, the relevant features and irrelevant features of the retinal fundus can be decoupled from the retinal fundus image by the trained encoder, and the type of the retinal fundus image is detected by the relevant features, which reduces the interference of irrelevant features on the recognition of the retinal fundus image, thereby improving the recognition accuracy of the retinal fundus image.

[0035] Optionally, relevant features of the retinal fundus include optic disc to optic cup ratio, blood vessel thickness, etc.; irrelevant features of the retinal fundus include background information, edge information, etc. of the fundus image.

[0036] Optionally, the encoder is obtained by: collecting normal type images and abnormal type images; inputting the normal type images and abnormal type images into a preset convolutional neural network for training to obtain the encoder.

[0037] Optionally, the normal type image is a normal fundus image, and the abnormal type image is a glaucoma fundus image.

[0038] Optionally, the electronic device inputs the normal type image and the abnormal type image into a preset convolutional neural network for training to obtain an encoder, including: the electronic device inputs the normal type image and the abnormal type image into a preset convolutional neural network for training to obtain a detection network, and determines the encoder in the detection network as a trained encoder. Optionally, the detection network is a cross encoder-decoder glaucoma detection network.

[0039] Optionally, the electronic device inputs the normal type image and the abnormal type image into a preset convolutional neural network for training, including: the electronic device respectively obtains the feature vector corresponding to the normal type image and the feature vector corresponding to the abnormal type image; divides the feature vector corresponding to the normal type image to obtain a first candidate feature vector and a second candidate feature vector, and divides the feature vector corresponding to the abnormal type image to obtain a third candidate feature vector and a fourth candidate feature vector; the first candidate feature vector is used to characterize irrelevant features of the normal type image for image detection, and the second candidate feature vector is used to characterize relevant features of the normal type image for image detection; the third candidate feature vector is used to characterize irrelevant features of the abnormal type image for image detection, and the fourth candidate feature vector is used to characterize relevant features of the abnormal type image for image detection; the first candidate feature vector and the fourth candidate feature vector are vector-concatenated to obtain a fifth candidate feature vector, and the second candidate feature vector and the third candidate feature vector are vector-concatenated to obtain a sixth candidate feature vector; the fifth candidate feature vector is input into a preset decoder to generate a normal type generated image, and the sixth candidate feature vector is input into a preset decoder to generate an abnormal type generated image.

[0040] In this way, a trained encoder can be obtained by training a preset convolutional neural network. The trained encoder can realize the decoupling of relevant features and irrelevant features in the retinal fundus image, and the type of the retinal fundus image can be detected by relevant features, thereby reducing the interference of irrelevant features on the recognition of the retinal fundus image, thereby improving the recognition accuracy of the retinal fundus image.

[0041] Optionally, the feature vector is segmented by a sparse coding method.

[0042] Optionally, the electronic device inputs normal type images and abnormal type images into a preset convolutional neural network for training to obtain an encoder, including: after each round of training, the electronic device adjusts the weight parameters in the convolutional neural network using a preset loss function; and stops training after the convolutional neural network has trained a preset round to obtain an encoder.

[0043] Optionally, the electronic device uses a preset loss function to adjust weight parameters in the convolutional neural network, including: the electronic device uses a preset loss function to perform back propagation.

[0044] Optionally, the preset number of rounds is 15,000 times.

[0045] Optionally, the electronic device stops training after a preset number of rounds of convolutional neural network training to obtain an encoder, including: the electronic device stops training after a preset number of rounds of convolutional neural network training, and determines the encoder in the convolutional neural network as a trained encoder.

[0046] Optionally, the electronic device uses a preset loss function to adjust the weight parameters in the convolutional neural network, including: randomly adjusting the weight parameters in the convolutional neural network so that the value corresponding to the preset loss function becomes smaller and smaller. In this way, after each round of training, back propagation using the preset loss function can make the value corresponding to the loss function smaller and smaller, thereby optimizing the detection network.

[0047] In some embodiments, the convolutional neural network includes an encoder, a decoder, and a classifier; wherein the weight parameters in the convolutional neural network include weight parameters in the encoder, weight parameters in the decoder, and weight parameters in the classifier. In this way, by adjusting the weight parameters in the encoder, decoder, and classifier respectively, the value corresponding to the preset loss function can be converged to a preset threshold range, thereby optimizing the detection network.

[0048] Combination Figure 2 As shown, the present disclosure provides a method for training an encoder, the method comprising:

[0049] Step S201: The electronic device obtains a feature vector corresponding to a normal type image and a feature vector corresponding to an abnormal type image respectively.

[0050] In step S202, the electronic device divides the feature vector corresponding to the normal type image to obtain a first candidate feature vector and a second candidate feature vector, and divides the feature vector corresponding to the abnormal type image to obtain a third candidate feature vector and a fourth candidate feature vector; the first candidate feature vector is used to characterize irrelevant features of the normal type image for image detection, and the second candidate feature vector is used to characterize relevant features of the normal type image for image detection; the third candidate feature vector is used to characterize irrelevant features of the abnormal type image for image detection, and the fourth candidate feature vector is used to characterize relevant features of the abnormal type image for image detection.

[0051] Step S203: the electronic device performs vector concatenation on the first candidate feature vector and the fourth candidate feature vector to obtain a fifth candidate feature vector, and performs vector concatenation on the second candidate feature vector and the third candidate feature vector to obtain a sixth candidate feature vector.

[0052] Step S204: input the fifth candidate feature vector into a preset decoder to generate a normal type generated image, and input the sixth candidate feature vector into a preset decoder to generate an abnormal type generated image.

[0053] The method for training an encoder provided by the embodiment of the present disclosure is adopted, by segmenting the relevant features and irrelevant features in the feature vector corresponding to the normal type image, segmenting the relevant features and irrelevant features in the feature vector corresponding to the abnormal type image, and then vector splicing the segmented relevant features and irrelevant features to obtain the fifth candidate feature vector and the sixth candidate feature vector, and then inputting the fifth candidate feature vector into a preset decoder to generate a normal type generated image, and inputting the sixth candidate feature vector into a preset decoder to generate an abnormal type generated image. The encoder obtained by continuous iterative training in this way can decouple the relevant features and irrelevant features of image detection.

[0054] Optionally, the loss function includes perceptual loss, reconstruction loss and classification loss; the perceptual loss is used to characterize the mean square loss between the input image and the generated image of the convolutional neural network, the reconstruction loss is used to characterize the similarity between the input image and the generated image of the convolutional neural network, and the classification loss is used to characterize the cross entropy error between the detected type and the actual type of the retinal fundus image.

[0055] Optionally, the perceptual loss is in, is the perceived loss, is the tth generated image, that is, a normal type generated image or an abnormal type generated image; X t is the tth input image, i.e., a normal type image or an abnormal type image; i is a VGG (Visual Geometry Group)-16 network, and i is the number of layers in the VGG-16 network. In this way, by applying perceptual loss, we can ensure that the generated image perceptually matches the input image.

[0056] Optionally, the classification loss is Among them, L is the classification loss, N is the total number of retinal fundus images, and L i is the loss value corresponding to the retinal fundus image i, p i is the detection type of the retinal fundus image i, optionally, p i The value range of y is [0,1]; i is the actual type of the retinal fundus image i. Optionally, when the actual type of the retinal fundus image i is a glaucoma fundus image, y i =1; when the actual type of retinal fundus image i is a normal fundus image, y i=0.

[0057] In some embodiments, for glaucoma detection, which is a binary classification task, the detection network is optimized by applying a binary cross entropy loss, where the classification loss is used to characterize the cross entropy error between the detected type and the true type of the retinal fundus image.

[0058] Optionally, the reconstruction loss is in, To reconstruct the loss, I i For normal type images, I j It is an abnormal type of image; For I i The corresponding reconstructed image is the normal type generated image; For I j The corresponding reconstructed image, i.e., the generated image of the non-normal type; is the residual that regularizes the two residuals between the reconstructed image and the original image to be similar, and α is a coefficient that balances the importance of reconstruction loss and residual loss. Optionally, both the normal type image and the abnormal type image are original images. In this way, by setting the reconstruction loss, the reconstructed image can be forced to be consistent with the original image in pixels, ensuring the clarity of the image. At the same time, it can further prevent the cross encoder from encoding the difference between the input original images into the shared features. Optionally, the shared features include irrelevant features of the normal type image for image detection Irrelevant features for image detection of abnormal type images

[0059] Optionally, the preset loss function is in, is the perceived loss, is the reconstruction loss and L is the classification loss.

[0060] In some embodiments, the weight parameters in the decoder and the weight parameters in the encoder are adjusted using perceptual loss and classification loss, and the weight parameters in the classifier are adjusted using classification loss, so that the value corresponding to the preset loss function converges to a preset threshold range, thereby optimizing the detection network.

[0061] Optionally, detecting the image type of the retinal fundus image using relevant features of the retinal fundus includes: inputting the relevant features of the retinal fundus into a preset classifier to obtain the image type of the retinal fundus image.

[0062] Optionally, the classifier is obtained by the following method, including: obtaining a plurality of training sample images and sample labels corresponding to the training sample images; the sample labels are used to characterize the image type of the training sample images; the training sample images with the sample labels are input into an untrained classifier for training to obtain the classifier. Optionally, the image type includes a normal type image and an abnormal type image. In some embodiments, the normal type image is a normal fundus image, and the abnormal type image is a glaucoma fundus image.

[0063] Combination Figure 3 As shown, Figure 3 A method for identifying eye features provided by an embodiment of the present disclosure includes:

[0064] Step S301: The electronic device collects retinal fundus images.

[0065] In step S302, the electronic device inputs the retinal fundus image into a trained encoder to decouple relevant features from irrelevant features of the retinal fundus.

[0066] Step S303, inputting relevant features of the retinal fundus into a preset classifier to obtain the image type of the retinal fundus image; wherein the image type of the retinal fundus image includes a normal type image and an abnormal type image.

[0067] By adopting the method for identifying eye features provided by the embodiments of the present disclosure, the relevant features and irrelevant features of the retinal fundus can be decoupled from the retinal fundus image through a trained encoder, and the relevant features are input into a classifier to detect the type of the retinal fundus image, thereby reducing the interference of irrelevant features in the recognition of the retinal fundus image, thereby improving the recognition accuracy of the retinal fundus image.

[0068] Combination Figure 4 As shown, Figure 4 is a schematic diagram of a cross encoder-decoder glaucoma detection network provided by an embodiment of the present disclosure; the normal type image I i and abnormal type image I j Input encoder 1 for encoding to obtain normal type image I i The corresponding eigenvector and abnormal type image I j The corresponding feature vector, the normal type image I i The corresponding feature vectors are segmented to obtain the irrelevant features of normal type images for image detection Related features of image detection with normal type images The abnormal type image I j The corresponding feature vectors are segmented to obtain irrelevant features of image detection for abnormal type images Related features of image detection with abnormal type images Normal type images are irrelevant features of image detection Related features of image detection with abnormal type images Perform vector concatenation to obtain the fifth candidate feature vector, and convert the irrelevant features of the abnormal type image into image detection Related features of image detection with normal type images Perform vector concatenation to obtain the sixth candidate feature vector; input the fifth candidate feature vector into the preset decoder 2 to generate a normal type generated image The sixth candidate feature vector is input into the preset decoder 2 to generate an abnormal type generated image In some embodiments, after the retinal fundus image is collected, the retinal fundus image is input into a trained encoder to decouple the relevant features and irrelevant features of the retinal fundus; and the relevant features of the retinal fundus are input into a classifier 3 to detect whether the retinal fundus image is a normal type image or an abnormal type image. The two encoders 1 share weights, and the two decoders 2 also share weights. Normal type image I used to shorten the input i and the normal type of output generates the image The distance between them makes the two images highly similar, which can guide the cross encoder-decoder glaucoma detection network to encode the original image into a latent vector, and then generate a new image by vector splicing; Used to characterize the abnormal type image I used to shorten the input j And the output of the non-normal type generates an image The distance between them makes the two images highly similar, which can guide the cross encoder-decoder glaucoma detection network to encode the original image into a latent vector and then generate a new image by vector splicing.

[0069] In this way, by taking a pair of normal type images and abnormal type images each time and sending them into the cross-encoder-decoder glaucoma detection network for cross-training, since the normal type images include some basic features of the retinal layer, such as the overall shape of the fundus image, edge information, etc., these basic features are not necessarily related to the detection of the retinal fundus image, and these irrelevant features are decoupled by cross-training, and the remaining features are the relevant features for detecting the retinal fundus image, which is particularly important for detecting the type of the retinal fundus image. At the same time, the related art usually uses a method based on CNN (Convolutional Neural Network) to identify eye features. In the network structure, a downsampling layer is often added to reduce the clarity of the image, resulting in poor recognition effect. The method for identifying eye features in the embodiment of the present invention does not consider the use of operations such as downsampling layers, fully maintains the clarity and integrity of the fundus image, and further improves the accuracy of type recognition of the retinal fundus image.

[0070] Combination Figure 5 As shown, an embodiment of the present disclosure provides a device for identifying eye features, the device comprising: an acquisition module 501, a feature extraction module 502 and a detection module 503; the acquisition module 501 is configured to acquire retinal fundus images; and send the retinal fundus images to the feature extraction module, the feature extraction module 502 is configured to receive the retinal fundus images sent by the acquisition module, input the retinal fundus images into a trained encoder, decouple relevant features and irrelevant features of the retinal fundus; and send the relevant features of the retinal fundus obtained by decoupling to the detection module, the detection module 503 is configured to receive the relevant features of the retinal fundus sent by the feature extraction module, and detect the image type of the retinal fundus image using the relevant features of the retinal fundus; wherein the image types of the retinal fundus images include normal type images and abnormal type images.

[0071] The device for identifying eye features provided by the embodiment of the present disclosure is used to collect retinal fundus images through the collection module, the feature extraction module inputs the retinal fundus images into the trained encoder, decouples the relevant features and irrelevant features of the retinal fundus, and the detection module detects the image type of the retinal fundus image using the relevant features of the retinal fundus. In this way, the trained encoder can decouple the relevant features and irrelevant features of the retinal fundus from the retinal fundus image, and the type of the retinal fundus image is detected by the relevant features, which reduces the interference of irrelevant features on the recognition of the retinal fundus image, thereby improving the recognition accuracy of the retinal fundus image.

[0072] Optionally, the encoder is obtained by: collecting normal type images and abnormal type images; inputting the normal type images and abnormal type images into a preset convolutional neural network for training to obtain the encoder.

[0073] Optionally, the normal type image and the abnormal type image are input into a preset convolutional neural network for training, including: respectively obtaining feature vectors corresponding to the normal type image and the feature vectors corresponding to the abnormal type image; dividing the feature vector corresponding to the normal type image to obtain a first candidate feature vector and a second candidate feature vector, and dividing the feature vector corresponding to the abnormal type image to obtain a third candidate feature vector and a fourth candidate feature vector; the first candidate feature vector is used to characterize irrelevant features of the normal type image for image detection, and the second candidate feature vector is used to characterize relevant features of the normal type image for image detection; the third candidate feature vector is used to characterize irrelevant features of the abnormal type image for image detection, and the fourth candidate feature vector is used to characterize relevant features of the abnormal type image for image detection; the first candidate feature vector and the fourth candidate feature vector are vector-concatenated to obtain a fifth candidate feature vector, and the second candidate feature vector and the third candidate feature vector are vector-concatenated to obtain a sixth candidate feature vector; the fifth candidate feature vector is input into a preset decoder to generate a normal type generated image, and the sixth candidate feature vector is input into a preset decoder to generate an abnormal type generated image.

[0074] Optionally, normal type images and abnormal type images are input into a preset convolutional neural network for training to obtain an encoder, including: after each round of training, using a preset loss function to adjust the weight parameters in the convolutional neural network; stopping training after the convolutional neural network has trained a preset round to obtain an encoder.

[0075] Optionally, the loss function includes perceptual loss, reconstruction loss and classification loss; the perceptual loss is used to characterize the mean square loss between the input image and the generated image of the convolutional neural network, the reconstruction loss is used to characterize the similarity between the input image and the generated image of the convolutional neural network, and the classification loss is used to characterize the cross entropy error between the detected type and the actual type of the retinal fundus image.

[0076] Optionally, the detection module is configured to detect the image type of the retinal fundus image using relevant features of the retinal fundus in the following manner: inputting the relevant features of the retinal fundus into a preset classifier to obtain the image type of the retinal fundus image.

[0077] Combination Figure 6As shown, the embodiment of the present disclosure provides a device for identifying eye features, including a processor (processor) 600 and a memory (memory) 601. Optionally, the device may also include a communication interface (Communication Interface) 602 and a bus 603. Among them, the processor 600, the communication interface 602, and the memory 601 can communicate with each other through the bus 603. The communication interface 602 can be used for information transmission. The processor 600 can call the logic instructions in the memory 601 to execute the method for identifying eye features of the above embodiment.

[0078] The device for identifying eye features provided by the embodiment of the present disclosure is used to collect retinal fundus images; input the retinal fundus images into a trained encoder to decouple the relevant features and irrelevant features of the retinal fundus; and use the relevant features of the retinal fundus to detect the image type of the retinal fundus image; wherein the image type of the retinal fundus image includes a normal type image and an abnormal type image. In this way, the trained encoder can decouple the relevant features and irrelevant features of the retinal fundus from the retinal fundus image, and detect the type of the retinal fundus image through the relevant features, thereby reducing the interference of irrelevant features on the recognition of the retinal fundus image, thereby improving the recognition accuracy of the retinal fundus image.

[0079] In addition, the logic instructions in the memory 601 described above can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0080] The memory 601 is a computer-readable storage medium that can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 600 executes the function application and data processing by running the program instructions / modules stored in the memory 601, that is, the method for identifying eye features in the above embodiment is implemented.

[0081] The memory 601 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 601 may include a high-speed random access memory and may also include a non-volatile memory.

[0082] An embodiment of the present disclosure provides an electronic device, comprising the above-mentioned device for recognizing eye features.

[0083] Optionally, the electronic device includes a computer or a server.

[0084] The electronic device provided by the embodiment of the present disclosure is used to collect retinal fundus images; input the retinal fundus images into a trained encoder to decouple the relevant features and irrelevant features of the retinal fundus; and use the relevant features of the retinal fundus to detect the image type of the retinal fundus image; wherein the image type of the retinal fundus image includes a normal type image and an abnormal type image. In this way, the trained encoder can decouple the relevant features and irrelevant features of the retinal fundus from the retinal fundus image, and detect the type of the retinal fundus image through the relevant features, thereby reducing the interference of irrelevant features on the recognition of the retinal fundus image, thereby improving the recognition accuracy of the retinal fundus image.

[0085] An embodiment of the present disclosure provides a storage medium storing program instructions, which, when executed, execute the above method for identifying eye features.

[0086] An embodiment of the present disclosure provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned method for identifying eye features.

[0087] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0088] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0089] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0090] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0091] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0092] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A method for identifying eye features, characterized in that: include: Acquiring retinal fundus images; Inputting the retinal fundus image into a trained encoder to decouple relevant features from irrelevant features of the retinal fundus; Detecting the image type of the retinal fundus image using relevant features of the retinal fundus; Wherein, the image types of the retinal fundus image include normal type images and abnormal type images; The encoder is obtained by: collecting normal type images and abnormal type images; inputting the normal type images and the abnormal type images into a preset convolutional neural network for training to obtain the encoder; Inputting the normal type image and the abnormal type image into a preset convolutional neural network for training, including: respectively obtaining a feature vector corresponding to the normal type image and a feature vector corresponding to the abnormal type image; segmenting the feature vector corresponding to the normal type image to obtain a first candidate feature vector and a second candidate feature vector, and segmenting the feature vector corresponding to the abnormal type image to obtain a third candidate feature vector and a fourth candidate feature vector; the first candidate feature vector is used to characterize irrelevant features of the normal type image for image detection, and the second candidate feature vector is used to characterize relevant features of the normal type image for image detection; the third candidate feature vector is used to characterize irrelevant features of the abnormal type image for image detection, and the fourth candidate feature vector is used to characterize relevant features of the abnormal type image for image detection; vector concatenating the first candidate feature vector and the fourth candidate feature vector to obtain a fifth candidate feature vector, and vector concatenating the second candidate feature vector and the third candidate feature vector to obtain a sixth candidate feature vector; inputting the fifth candidate feature vector into a preset decoder to generate a normal type generated image, and inputting the sixth candidate feature vector into a preset decoder to generate an abnormal type generated image.

2. The method according to claim 1, characterized in that Inputting the normal type image and the abnormal type image into a preset convolutional neural network for training to obtain the encoder includes: After each round of training, the weight parameters in the convolutional neural network are adjusted using a preset loss function; After the convolutional neural network has been trained for a preset number of rounds, training is stopped to obtain the encoder.

3. The method according to claim 2, characterized in that The loss function includes perceptual loss, reconstruction loss and classification loss; the perceptual loss is used to characterize the mean square loss between the input image and the generated image of the convolutional neural network, the reconstruction loss is used to characterize the similarity between the input image and the generated image of the convolutional neural network, and the classification loss is used to characterize the cross entropy error between the detected type and the actual type of the retinal fundus image.

4. The method according to any one of claims 1 to 3, characterized in that: The method of detecting the image type of the retinal fundus image by using relevant features of the retinal fundus includes: The relevant features of the retinal fundus are input into a preset classifier to obtain the image type of the retinal fundus image.

5. A device for identifying eye features, characterized in that: include: An acquisition module configured to acquire retinal fundus images; A feature extraction module is configured to input the retinal fundus image into a trained encoder to decouple relevant features from irrelevant features of the retinal fundus; A detection module is configured to detect the image type of the retinal fundus image by using relevant features of the retinal fundus; wherein the image type of the retinal fundus image includes a normal type image and an abnormal type image; The encoder is obtained by: collecting normal type images and abnormal type images; inputting the normal type images and the abnormal type images into a preset convolutional neural network for training to obtain the encoder; Inputting the normal type image and the abnormal type image into a preset convolutional neural network for training, including: respectively obtaining a feature vector corresponding to the normal type image and a feature vector corresponding to the abnormal type image; segmenting the feature vector corresponding to the normal type image to obtain a first candidate feature vector and a second candidate feature vector, and segmenting the feature vector corresponding to the abnormal type image to obtain a third candidate feature vector and a fourth candidate feature vector; the first candidate feature vector is used to characterize irrelevant features of the normal type image for image detection, and the second candidate feature vector is used to characterize relevant features of the normal type image for image detection; the third candidate feature vector is used to characterize irrelevant features of the abnormal type image for image detection, and the fourth candidate feature vector is used to characterize relevant features of the abnormal type image for image detection; vector concatenating the first candidate feature vector and the fourth candidate feature vector to obtain a fifth candidate feature vector, and vector concatenating the second candidate feature vector and the third candidate feature vector to obtain a sixth candidate feature vector; inputting the fifth candidate feature vector into a preset decoder to generate a normal type generated image, and inputting the sixth candidate feature vector into a preset decoder to generate an abnormal type generated image.

6. A device for identifying eye features, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the method for identifying eye features according to any one of claims 1 to 4 when running the program instructions.

7. An electronic device, characterized in that: Comprising the device for identifying eye features as described in claim 6.

8. A storage medium storing program instructions, characterized in that: When the program instructions are executed, the method for identifying eye features according to any one of claims 1 to 4 is executed.

Images