A fundus image labeling method and device and a storage medium
By establishing feature mapping relationships in fundus images and utilizing the correspondence of image features from different angles of the same eyeball, the problem of time-consuming and error-prone labeling of lesion types on multiple fundus images was solved, achieving efficient and accurate labeling.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING ZHIYUAN HUITU TECH CO LTD
- Filing Date
- 2021-12-28
- Publication Date
- 2026-06-02
AI Technical Summary
In the treatment of eye diseases, marking the types of lesions on multiple fundus images is time-consuming and prone to omissions or errors.
By acquiring at least two fundus images, features of the labeled and unlabeled images are extracted, and a feature mapping relationship is established. The unlabeled images are labeled according to the mapping relationship, and the feature correspondence of images from different angles of the same eye is used for labeling.
It reduces labeling time, improves labeling efficiency, and lowers the probability of missed labels and labeling errors.
Smart Images
Figure CN114420265B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, specifically to a method, apparatus, and storage medium for annotating fundus images. Background Technology
[0002] In the treatment of eye diseases, it is necessary to take multiple photos of the eyeball from multiple angles to form multiple fundus images. Then, based on the multiple fundus images, the types of lesions in the eye are analyzed and the types of lesions are marked.
[0003] Because there are many types of lesions, and different lesions have different characteristics, and some lesions are small in size, it takes a lot of time to mark the types of lesions on multiple fundus images, and it is easy to miss marking them or make labeling errors. Summary of the Invention
[0004] To address the aforementioned technical problems in the prior art, this application proposes a method, apparatus, and storage medium for annotating fundus images, thereby solving the problems that it takes a lot of time to mark the types of lesions on multiple fundus images, and that it is easy to miss markings and make annotation errors.
[0005] A first aspect of this application provides a method for annotating fundus images, including:
[0006] Acquire at least two fundus images, which are images of the first eyeball from different angles. The at least two fundus images include an labeled image and at least one unlabeled fundus image. The labeled image includes at least one type of marker, which includes lesions and / or anatomical locations.
[0007] Extract at least one first feature from the labeled image and at least one second feature from the first unlabeled image, wherein the at least one first feature and the at least one second feature are representations of the eyeball structure features from different angles;
[0008] Based on the correspondence between the angles of the labeled image and the angles of the first unlabeled image, a mapping relationship is established between the at least one first feature and the at least one second feature, wherein the mutually mapped first and second features indicate the same structural feature of the eyeball;
[0009] Based on the established mapping relationship between the at least one first feature and the at least one second feature, the corresponding lesions and / or anatomical locations in the first unlabeled image are labeled according to the markers in the labeled image to obtain the first labeled image.
[0010] Preferably, the method further includes:
[0011] Acquire at least two fundus images of the second eyeball, wherein the second eyeball and the first eyeball are the left and right eyes of the same person;
[0012] Extract at least one third feature from a second unlabeled image, wherein the second unlabeled image is one of at least two fundus images of the second eyeball;
[0013] Based on the correspondence between the structures of the first eyeball and the second eyeball, a mapping relationship is established between the at least one first feature and the at least one third feature, wherein the structure of the first eyeball indicated by the mutually mapped first feature corresponds to the structure of the second eyeball indicated by the third feature;
[0014] Based on the mapping relationship between the at least one first feature and the at least one third feature, the corresponding lesions and / or anatomical locations in the second unlabeled image are labeled according to the markers in the labeled image to obtain the second labeled image.
[0015] Preferably, the method further includes:
[0016] Verify whether the type and / or anatomical location of the lesion in the first annotated image are normal;
[0017] If it is determined that the type of lesion in the first labeled image is normal, then the first labeled image is output;
[0018] If it is determined that the type and / or anatomical location of the lesion in the first labeled image is abnormal, then according to the established mapping relationship between the at least one first feature and the at least one second feature, the corresponding lesion and / or anatomical location in the first unlabeled image are manually labeled according to the labels in the labeled image.
[0019] Preferably, establishing the mapping relationship between the at least one first feature and the at least one second feature includes:
[0020] When the labeled image and the first unlabeled image have overlapping areas, a mapping relationship is established between the first feature and the second feature contained in the overlapping area.
[0021] Preferably, after obtaining the second annotated image, the process further includes:
[0022] Establish a disease list, which includes the diseases corresponding to the first ocular lesion contained in the first annotated image;
[0023] A list of diseases with unlabeled images is created, and the diseases in the list of diseases are mapped to the list of diseases with unlabeled images to obtain a list of diseases with labeled images.
[0024] The list of diseases in the labeled images includes the diseases corresponding to the second ocular lesions contained in the second labeled images;
[0025] Determine whether the disease or type corresponding to the second ocular lesion is correct;
[0026] If incorrect, add or delete the disease type corresponding to the second ocular lesion contained in the second labeled image.
[0027] Preferably, before extracting at least one third feature from the second unlabeled image, the method further includes:
[0028] The positions of the optic disc and macula in the fundus of the labeled image and the second unlabeled image are located to obtain the corresponding positions between the optic disc and the macula.
[0029] Based on the corresponding positions between the optic disc and the macula of the optic disc, the labeled image and the second unlabeled image are respectively divided into at least one corresponding region;
[0030] Extract at least one first feature from any region of the at least one region of the labeled image;
[0031] Select a target region from at least one region of the second unlabeled image, the target region corresponding to the region from which at least one first feature is extracted, and extract at least one third feature from the target region.
[0032] Preferably, after obtaining the second annotated image, the method further includes:
[0033] The severity levels of the lesions in the labeled image and the severity levels of the lesions in the second labeled image are set according to the preset severity levels of the lesions.
[0034] For each type of lesion, determine the degree of difference between the severity level of the lesion contained in the second annotated image and the severity level of the lesion contained in the annotated image;
[0035] If the difference between the lesion severity level of the second labeled image and the lesion severity level of the labeled image is greater than a preset level, then the process of setting the lesion severity level of the labeled image is repeated to obtain a first new level, and the lesion severity level of the second labeled image is determined to obtain a second new level. If the difference between the first new level and the second new level is greater than a preset level, then the lesion severity level of the labeled image is determined to be a normal level and the lesion severity level of the second labeled image is determined to be a normal level.
[0036] If the difference between the lesion severity level of the lesion in the second annotated image and the lesion severity level of the annotated image is less than the preset level, then the lesion severity level of the annotated image is determined to be normal and the lesion severity level of the second annotated image is determined to be normal.
[0037] A second aspect of this application provides a fundus image annotation apparatus, the apparatus comprising:
[0038] An acquisition module is used to acquire at least two fundus images, wherein the at least two fundus images are images of a single eyeball from different angles, and the at least two fundus images include an labeled image and at least one unlabeled fundus image. The labeled image includes at least one marker type, and the marker type includes lesions and / or anatomical locations.
[0039] An extraction module is used to extract at least one first feature from the labeled image and at least one second feature from the first unlabeled image, wherein the at least one first feature and the at least one second feature are representations of the eye structure features from different angles;
[0040] The mapping module is used to establish a mapping relationship between the at least one first feature and the at least one second feature based on the correspondence between the angle of the labeled image and the angle of the first unlabeled image, wherein the first and second features of the related mapping indicate the same structural feature of the eyeball;
[0041] The annotation module is used to annotate the corresponding lesions and / or anatomical locations in the first unannotated image according to the markers in the annotated image, based on the established mapping relationship between the at least one first feature and the at least one second feature, to obtain the first annotated image.
[0042] Preferably, the device further includes:
[0043] The verification module is used to verify whether the type and / or anatomical location of the lesion in the first annotated image are normal;
[0044] If it is determined that the type of lesion in the first labeled image is normal, then the first labeled image is output;
[0045] If it is determined that the type and / or anatomical location of the lesion in the first labeled image is abnormal, then according to the established mapping relationship between the at least one first feature and the at least one second feature, the corresponding lesion and / or anatomical location in the first unlabeled image are manually labeled according to the labels in the labeled image.
[0046] A third aspect of this application provides a storage medium storing computer-executable instructions thereon, which, when executed by a computing device, can be used to implement the methods described in the foregoing embodiments.
[0047] In this embodiment, by establishing a mapping relationship between at least one first feature and at least one second feature based on the correspondence between the angles of the labeled image and the angles of the first unlabeled image, the corresponding lesions and / or anatomical locations in the first unlabeled image are labeled according to the markings in the labeled image to obtain a first labeled image. This reduces the time required to label multiple fundus images. Furthermore, by establishing a mapping relationship between at least one first feature and at least one second feature, it is less likely to miss the types and situations of lesions to be labeled, and it is less likely to cause problems with the types and labels of lesions to be incorrect, thereby improving the labeling efficiency and saving a lot of labeling time. Attached Figure Description
[0048] The features and advantages of this application will be more clearly understood by referring to the accompanying drawings, which are illustrative and should not be construed as limiting the application in any way. In the drawings:
[0049] Figure 1 This is a flowchart illustrating a method for annotating fundus images according to some embodiments of this application. Detailed Implementation
[0050] In the following detailed description, numerous specific details of this application are illustrated by example to provide a thorough understanding of the relevant disclosure. However, it will be apparent to those skilled in the art that this application can be practiced without these details. It should be understood that the terms “system,” “apparatus,” “unit,” and / or “module” used in this application are one way of distinguishing different parts, elements, sections, or components at different levels in a sequential arrangement. However, these terms may be replaced with other expressions if other expressions can achieve the same purpose.
[0051] It should be understood that when a device, unit, or module is referred to as being "on," "connected to," or "coupled to" another device, unit, or module, it may be directly connected to or coupled to or communicate with other devices, units, or modules, or there may be intermediate devices, units, or modules present, unless the context explicitly indicates otherwise. For example, the term "and / or" as used herein includes any one and all combinations of one or more of the relevant listed items.
[0052] The terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate that explicitly identified features, integrals, steps, operations, elements, and / or components are included, and such expressions do not constitute an exclusive list, and other features, integrals, steps, operations, elements, and / or components may also be included.
[0053] Referring to the following description and accompanying drawings, these and other features and characteristics, operating methods, functions of related structural elements, combinations of parts, and economics of manufacture of this application can be better understood, wherein the description and drawings form part of the specification. However, it is clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. It is understood that the drawings are not drawn to scale.
[0054] Various structural diagrams are used in this application to illustrate various variations of the embodiments according to this application. It should be understood that the preceding or following structures are not intended to limit this application. The scope of protection of this application is determined by the claims.
[0055] like Figure 1 As shown, this application provides a method for annotating fundus images, including:
[0056] Step S10 acquires at least two fundus images, which are images of the first eye from different angles. The at least two fundus images include an labeled image and at least one unlabeled fundus image. The labeled image includes at least one marker type, which includes lesions and / or anatomical locations. It should be noted that the first eye can be either a left or right eye image. Regardless of whether it is a left or right eye image, it must be from the same patient's eye, and the identification must be determined accordingly. If the data collected in this application includes left and right eye image classification information for each case image, the patient's image can be divided into left and right eye images using the basic information corresponding to the image acquisition or automatic left-right eye classification techniques. If the collected data does not contain left-right eye classification information, algorithms can be used to automatically classify the left and right eyes of a case image. Various automatic classification algorithms are possible, such as traditional machine learning algorithms like the SVM classifier, and various binary classification networks in deep learning algorithms. This application can use any of the above-mentioned eye classification methods. In fact, in order to more accurately and comprehensively label the lesion types on the same eye image, at least two fundus images are needed, and the first eye images obtained from different angles are required. This will result in a more accurate and comprehensive labeled image.
[0057] Step S20 extracts at least one first feature from the labeled image and at least one second feature from the first unlabeled image. The at least one first feature and the at least one second feature represent the eye structure features from different perspectives. Specifically, this application can extract at least one first feature from the labeled image and at least one second feature from the first unlabeled image using a sliding window method. Alternatively, at least one first feature from the labeled image and at least one second feature from the first unlabeled image can be extracted based on key points of the structure on the fundus image. For example, first, the fundus vessels in the labeled image are segmented to find the bifurcation points of the vessels, which are the first features; then, the fundus vessels in the labeled image are segmented again to find the bifurcation points of the vessels, which are the second features.
[0058] Step S30 establishes a mapping relationship between at least one first feature and at least one second feature based on the correspondence between the angles of the labeled image and the angles of the first unlabeled image. The mutually mapped first and second features indicate the same structural features of the eyeball. By establishing this mapping relationship, a one-to-one correspondence is established between at least one first feature and at least one second feature, thus completing the registration between the labeled image and the first unlabeled image. That is, at least one first feature in the labeled image matches the position of at least one second feature in the first unlabeled image. In fact, this process allows the labeled lesion area in the labeled image to be mapped to the corresponding position in the unlabeled image during the next annotation step. Different shooting angles result in different image areas. However, since the same eye is being photographed, the anatomical structure is the same. Corresponding feature points can be obtained from at least one image based on the same anatomical structure. The mapping relationship between the two images can be calculated using the corresponding mapping points on the images.
[0059] Step S40 involves establishing a mapping relationship between the at least one first feature and the at least one second feature, and then labeling the corresponding lesions and / or anatomical locations in the first unlabeled image according to the markers in the labeled image, thereby obtaining a first labeled image. Specifically, based on the established mapping relationship between the at least one first feature and the at least one second feature, i.e., the registration between the labeled image and the first unlabeled image, a registration result is obtained. The registration result is a transformation matrix that initializes the marked lesion region in the labeled image to the corresponding region in the first unlabeled image, thus obtaining the first labeled image.
[0060] The first labeled image is obtained through the above steps, which solves the problem of spending a lot of time labeling multiple images. Furthermore, by establishing a mapping relationship between the at least one first feature and the at least one second feature, it is less likely to miss the types and situations of labeled lesions, and it is less likely to mislabel the types of lesions, thereby improving the efficiency of labeling and saving a lot of labeling time.
[0061] In one embodiment, the method further includes:
[0062] At least two fundus images of the second eyeball are acquired, wherein the second eyeball and the first eyeball are the left and right eyes of the same person. At least one third feature is extracted from a second unlabeled image, which is one of the at least two fundus images of the second eyeball. Based on the correspondence between the structures of the first and second eyeballs, a mapping relationship is established between the at least one first feature and the at least one third feature. The structures of the first eyeball indicated by the mutually mapped first feature and the structures of the second eyeball indicated by the third feature correspond to each other. Based on the mapping relationship between the at least one first feature and the at least one third feature, the corresponding lesions and / or anatomical locations in the second unlabeled image are labeled according to the markers in the labeled image, resulting in a second labeled image. The second eyeball is the left and right eyes of the same person as the first eyeball.
[0063] Specifically, the images of the second and first eyes are images of the same person's two eyes that may have the same condition. In fact, the labeling of lesion types in images of both eyes differs from the labeling of lesion types in images of only one eye. The labeling of lesion types in images of both eyes is based on the correspondence between the structures of the first and second eyes, establishing a mapping relationship between at least one first feature and at least one third feature. Specifically, the correspondence between the structures of the first and second eyes refers to the same physiological structures, such as the optic disc, macula, superior vascular arch, and inferior vascular arch of the left and right eyes. These physiological structures correspond one-to-one. Registration is performed between at least one first feature and at least one third feature. That is, based on the mapping relationship between at least one first feature and at least one third feature, the corresponding lesions in the second unlabeled image are labeled according to the markings in the labeled image, thereby obtaining the second labeled image. Then, it is determined whether the lesion markings on the second labeled image are correct. If not, the second unlabeled image is manually relabeled.
[0064] In another embodiment, it is not necessary to first determine whether the type of lesion contained in the second eyeball is the same as the type of lesion contained in the first eyeball. The above steps can be directly followed. After the second unlabeled image is completed and the second labeled image is obtained, it is then determined whether the type of lesion contained in the second eyeball is the same as the type of lesion contained in the first eyeball. If the types of lesions in the first and second eyes are the same, the second labeled image is successfully labeled. If the types of lesions in the first and second eyes are different, the type of lesion in the other eyeball is confirmed and labeled one by one. At this time, labeling the eyeball type becomes labeling the first and second eyes separately, that is, labeling the first and second unlabeled images separately. The method of labeling a single eyeball has been disclosed above and will not be repeated here. Of course, there is another situation where one of the first or second eyes is normal and the other has a lesion. In this case, only the unlabeled image of the eye with the lesion needs to be labeled.
[0065] In one embodiment, the method further includes:
[0066] The system verifies whether the type and / or anatomical location of the lesion in the first annotated image are normal. If the type of the lesion in the first annotated image is determined to be normal, the first annotated image is output. If the type and / or anatomical location of the lesion in the first annotated image is determined to be abnormal, the system manually marks the corresponding lesion and / or anatomical location in the first unannotated image according to the markings in the annotated image, based on the established mapping relationship between at least one first feature and at least one second feature. This verification allows for timely detection of whether the annotated image is normal, ensuring the accuracy of the annotated image. Simultaneously, it reduces the workload of doctors and improves work efficiency.
[0067] In one embodiment, establishing the mapping relationship between the at least one first feature and the at least one second feature includes:
[0068] When the labeled image and the first unlabeled image have overlapping areas, a mapping relationship is established between the first feature and the second feature contained in the overlapping area.
[0069] Specifically, the at least two fundus images are images of the same eye from different angles, and the at least two fundus images have an overlapping area. A mapping relationship between the at least one first feature and the at least one second feature is established within the overlapping area. Alternatively, the at least two fundus images are images of the same eye from the same angle but at different distances, and the images from the same angle but at different distances have an overlapping area. A mapping relationship between the at least one first feature and the at least one second feature is established within the overlapping area, and the corresponding lesions and / or anatomical locations in the first unlabeled image are labeled according to the markings in the labeled image to obtain the first labeled image. Both methods aim to obtain more accurate labeled images.
[0070] In one embodiment, after obtaining the second annotated image, the method further includes:
[0071] A disease list is established, which includes the diseases corresponding to the first ocular lesion contained in the first labeled image. A disease list for unlabeled images is established, and the diseases in the disease list are mapped to the disease list for unlabeled images to obtain a disease list for labeled images. The disease list for labeled images includes the diseases corresponding to the second ocular lesion contained in the second labeled image. It is determined whether the OR type of the disease corresponding to the second ocular lesion is correct. If it is incorrect, the diseases corresponding to the second ocular lesion contained in the second labeled image are added or deleted.
[0072] Specifically, in order to obtain more accurate lesion types and diseases in the second annotated images, it is necessary to judge the lesion types in the obtained second annotated images and confirm whether the lesion types and corresponding diseases in the list are correct. This requires adding or deleting lesion types and corresponding diseases in the second annotated images, which improves the accuracy of the annotated images obtained after annotating the unannotated images, reduces the likelihood of annotation errors, and facilitates the subsequent use of information on lesion types and corresponding diseases in the second annotated images.
[0073] In one embodiment, before extracting at least one third feature from the second unlabeled image, the method further includes:
[0074] The positions of the optic disc and macula in the labeled image and the second unlabeled image are located to obtain the corresponding positions between the optic disc and the macula. Based on the corresponding positions between the optic disc and the macula, the labeled image and the second unlabeled image are divided into at least one corresponding region. At least one first feature is extracted from any region in the at least one region of the labeled image. A target region is selected from at least one region in the second unlabeled image, and the target region corresponds to the region from which at least one first feature is extracted. At least one third feature is extracted from the target region.
[0075] Specifically, the location of the optic disc and macula in the labeled image and the second unlabeled image can be determined manually by the doctor during the labeling process, or by any algorithm that automatically locates the optic disc and macula. This location requires several physiological structures of the eye, including the optic disc, macula, superior vascular arch, and inferior vascular arch. The positions of the superior and inferior vascular arches are then further determined based on the line connecting the optic disc and macula. After the positions are determined, the fundus image is divided into four regions: the optic disc region, the macula region, the superior vascular arch region, and the inferior vascular arch region. In fact, there can be more than four regions, or a range between one and four regions. This application uses four regions, which encompass the entire eyeball structure. The labeled image and the second unlabeled image are each divided into four corresponding regions. At least one first feature is extracted from any region of the labeled image, and at least one third feature is extracted from the target region. The first and third features are then registered, mapping the lesions in the labeled image onto the second unlabeled image.
[0076] In one embodiment, after obtaining the second annotated image, the method further includes:
[0077] The severity levels of the lesions in the labeled image and the severity levels of the lesions in the second labeled image are set according to the preset severity levels of the lesions.
[0078] For each type of lesion, determine the degree of difference between the severity level of the lesion contained in the second annotated image and the severity level of the lesion contained in the annotated image;
[0079] If the difference between the lesion severity level of the second labeled image and the lesion severity level of the labeled image is greater than a preset level, then the process of setting the lesion severity level of the labeled image is repeated to obtain a first new level, and the lesion severity level of the second labeled image is determined to obtain a second new level. If the difference between the first new level and the second new level is greater than a preset level, then the lesion severity level of the labeled image is determined to be a normal level and the lesion severity level of the second labeled image is determined to be a normal level.
[0080] If the difference between the lesion severity level of the lesion in the second annotated image and the lesion severity level of the annotated image is less than the preset level, then the lesion severity level of the annotated image is determined to be normal and the lesion severity level of the second annotated image is determined to be normal.
[0081] Specifically, different verification rules are set according to the actual situation of the disease. For example, for diabetic retinopathy, the difference in lesions between the first and second eye should preferably not exceed a preset level. The preset level can be divided into multiple levels, and the level values are generally integers. If the difference in lesion severity is too large, a confirmation examination is prompted. For hypertensive retinopathy, the difference in lesions between the first and second eye should also not be too large. If the diagnostic results of the two eyes are different, the person annotating should be prompted to conduct a confirmation examination to see if it is normal.
[0082] This application also provides a fundus image annotation device, the device comprising:
[0083] An acquisition module is used to acquire at least two fundus images, wherein the at least two fundus images are images of a single eyeball from different angles, and the at least two fundus images include an labeled image and at least one unlabeled fundus image. The labeled image includes at least one marker type, and the marker type includes lesions and / or anatomical locations.
[0084] An extraction module is used to extract at least one first feature from the labeled image and at least one second feature from the first unlabeled image, wherein the at least one first feature and the at least one second feature are representations of the eye structure features from different angles.
[0085] The mapping module is used to establish a mapping relationship between the at least one first feature and the at least one second feature based on the correspondence between the angle of the labeled image and the angle of the first unlabeled image, wherein the first and second features of the related mapping indicate the same structural feature of the eyeball.
[0086] The annotation module is used to annotate the corresponding lesions and / or anatomical locations in the first unannotated image according to the markers in the annotated image, based on the established mapping relationship between the at least one first feature and the at least one second feature, to obtain the first annotated image.
[0087] It should be noted that the first eye image can be either the left or right eye image. Of course, regardless of whether it is the left or right eye image, it must be from the same patient's eye, and the left or right eye image must be identified accordingly.
[0088] By employing acquisition, extraction, mapping, and annotation modules, the problem of spending a significant amount of time annotating multiple images is solved. Furthermore, by establishing a mapping relationship between at least one first feature and at least one second feature, it is less likely to miss the types and situations of lesions being labeled, and it is less likely to mislabel the types and types of lesions, thereby improving annotation efficiency and saving a significant amount of annotation time.
[0089] In one embodiment, the device further includes:
[0090] The verification module is used to verify whether the type and / or anatomical location of the lesion in the first labeled image are normal. If it is determined that the type of the lesion in the first labeled image is normal, the first labeled image is output. If it is determined that the type and / or anatomical location of the lesion in the first labeled image is abnormal, the corresponding lesion and / or anatomical location in the first unlabeled image is manually labeled according to the mapping relationship between the at least one first feature and the at least one second feature established according to the labeled image. The verification module prevents unexpected situations from occurring during the labeling process, thereby avoiding labeling omissions or errors. In fact, if a labeling abnormality occurs, an alarm will also be triggered during the labeling process. Thus, after the image is labeled, there is no need to check the labeled image again; it can be directly re-labeled to determine whether the labeling of the corresponding lesion in the first unlabeled image according to the labeling in the labeled image is correct. If the lesion in the labeled image and the corresponding lesion in the first unlabeled image differ by a certain range (this range is preset and known), an alarm will be triggered to determine whether the labeling is incorrect. This greatly improves the accuracy of labeling and prevents labeling errors.
[0091] This application also provides a storage medium storing computer-executable instructions thereon, which, when executed by a computing device, can be used to implement the methods described in the foregoing embodiments.
[0092] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of this application and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of this application should be included within the protection scope of this application. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.
Claims
1. A method for fundus image annotation, characterized in that, include: Acquire at least two fundus images, which are images of the first eyeball from different angles. The at least two fundus images include an labeled image and at least one unlabeled fundus image. The labeled image includes at least one type of marker, which includes lesions and / or anatomical locations. Extract at least one first feature from the labeled image and at least one second feature from the first unlabeled image, wherein the at least one first feature and the at least one second feature are representations of the eyeball structure features from different angles; Based on the correspondence between the angles of the labeled image and the angles of the first unlabeled image, a mapping relationship is established between the at least one first feature and the at least one second feature, wherein the mutually mapped first and second features indicate the same structural feature of the eyeball; Based on the established mapping relationship between the at least one first feature and the at least one second feature, the corresponding lesions and / or anatomical locations in the first unlabeled image are labeled according to the markers in the labeled image to obtain the first labeled image; The established mapping relationship between the at least one first feature and the at least one second feature includes: The at least two fundus images are images of the same eyeball from different angles. When the labeled image and the first unlabeled image have an overlapping area, a mapping relationship between the first feature and the second feature contained in the overlapping area is established. Acquire at least two fundus images of the second eyeball, wherein the second eyeball and the first eyeball are the left and right eyes of the same person; Extract at least one third feature from a second unlabeled image, wherein the second unlabeled image is one of at least two fundus images of the second eyeball; A mapping relationship between the at least one first feature and the at least one third feature is established based on the correspondence between the structures of the first eyeball and the second eyeball, wherein the structure of the first eyeball indicated by the mutually mapped first feature corresponds to the structure of the second eyeball indicated by the third feature; Based on the mapping relationship between the at least one first feature and the at least one third feature, the corresponding lesions and / or anatomical locations in the second unlabeled image are labeled according to the markers in the labeled image to obtain the second labeled image; Verify whether the type and / or anatomical location of the lesion in the first annotated image are normal; If it is determined that the type of lesion in the first labeled image is normal, then the first labeled image is output; If it is determined that the type and / or anatomical location of the lesion in the first labeled image is abnormal, then according to the mapping relationship between the at least one first feature and the at least one second feature, the corresponding lesion and / or anatomical location in the first unlabeled image are manually labeled according to the labels in the labeled image.
2. The method for fundus image annotation according to claim 1, characterized in that, After obtaining the second annotated image, the process further includes: Establish a disease list, which includes the diseases corresponding to the first ocular lesion contained in the first annotated image; Establish a list of diseases for which images are not labeled, and map the diseases in the list of diseases to the list of diseases for which images are not labeled, to obtain a list of diseases for which images are labeled; The list of diseases in the labeled images includes the diseases corresponding to the second ocular lesions contained in the second labeled images; Determine whether the disease or type corresponding to the second ocular lesion is correct; If incorrect, add or delete the disease type corresponding to the second ocular lesion contained in the second labeled image.
3. The method of claim 1, wherein, Before extracting at least one third feature from the second unlabeled image, the process also includes: The positions of the optic disc and macula in the fundus of the labeled image and the second unlabeled image are located to obtain the corresponding positions between the optic disc and the macula. Based on the corresponding positions between the optic disc and the macula of the optic disc, the labeled image and the second unlabeled image are respectively divided into at least one corresponding region; Extract at least one first feature from any region of the at least one region of the labeled image; Select a target region from at least one region of the second unlabeled image, the target region corresponding to the region from which at least one first feature is extracted, and extract at least one third feature from the target region.
4. The method for fundus image annotation according to claim 1, characterized in that, After obtaining the second annotated image, the process also includes: The severity levels of the lesions in the labeled image and the severity levels of the lesions in the second labeled image are set according to the preset severity levels of the lesions. For each type of lesion, determine the degree of difference between the severity level of the lesion contained in the second annotated image and the severity level of the lesion contained in the annotated image; If the difference between the lesion severity level of the second labeled image and the lesion severity level of the labeled image is greater than a preset level, then the process of setting the lesion severity level of the labeled image is repeated to obtain a first new level, and the lesion severity level of the second labeled image is determined to obtain a second new level. If the difference between the first new level and the second new level is greater than a preset level, then the lesion severity level of the labeled image is determined to be a normal level and the lesion severity level of the second labeled image is determined to be a normal level. If the difference between the lesion severity level of the lesion in the second annotated image and the lesion severity level of the annotated image is less than the preset level, then the lesion severity level of the annotated image is determined to be normal and the lesion severity level of the second annotated image is determined to be normal.
5. A fundus image annotation device, characterized in that, The apparatus is used to implement the method for annotating fundus images as described in any one of claims 1-4, the apparatus comprising: An acquisition module is used to acquire at least two fundus images, wherein the at least two fundus images are images of a single eyeball from different angles, and the at least two fundus images include an labeled image and at least one unlabeled fundus image. The labeled image includes at least one marker type, and the marker type includes lesions and / or anatomical locations. An extraction module is used to extract at least one first feature from the labeled image and at least one second feature from the first unlabeled image, wherein the at least one first feature and the at least one second feature are representations of the eye structure features from different angles; The mapping module is used to establish a mapping relationship between the at least one first feature and the at least one second feature based on the correspondence between the angle of the labeled image and the angle of the first unlabeled image, wherein the first and second features of the related mapping indicate the same structural feature of the eyeball; The annotation module is used to annotate the corresponding lesions and / or anatomical locations in the first unannotated image according to the established mapping relationship between the at least one first feature and the at least one second feature, and to obtain the first annotated image.
6. The fundus image annotation device according to claim 5, characterized in that, The device further includes: The verification module is used to verify whether the type and / or anatomical location of the lesion in the first annotated image are normal; If it is determined that the type of lesion in the first labeled image is normal, then the first labeled image is output; If it is determined that the type and / or anatomical location of the lesion in the first labeled image is abnormal, then according to the established mapping relationship between the at least one first feature and the at least one second feature, the corresponding lesion and / or anatomical location in the first unlabeled image are manually labeled according to the labels in the labeled image.
7. A storage medium having stored thereon computer-executable instructions which, when executed by a computing device, can be used to implement the method as described in any one of claims 1-4.