Glass membrane warts lesion extraction method and device

Abstract

The disclosure provides a glass membrane wart lesion extraction method and device, the method comprising: performing feature classification processing on a target eye image to determine whether a glass membrane wart lesion exists in the target eye image; if the glass membrane wart lesion exists in the target eye image, detecting the target eye image by using a pre-trained glass membrane wart lesion detection model to obtain an initial image of a glass membrane wart lesion region; segmenting the initial image by using a pre-trained glass membrane wart lesion segmentation model to obtain a first image of the glass membrane wart lesion; extracting the glass membrane wart lesion region of the target eye image based on a preset image processing algorithm to obtain a second image of the glass membrane wart lesion region; and performing intersection processing on the initial image, the first image of the glass membrane wart lesion and the second image of the glass membrane wart lesion region to obtain a glass membrane wart lesion segmentation result corresponding to the target eye image.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for extracting drusen lesions. Background Technology

[0002] Vision is a vital sense for human survival and progress, and in recent years, the incidence of eye diseases has been increasing. Drupes are gel-like growths that develop on the retina, most commonly seen in the elderly, and generally do not present obvious symptoms in their early stages. If the growths continue to increase, they may affect vision, causing narrowing and decreased visual acuity; therefore, early diagnosis and controlled treatment are crucial. Although drusen are not uncommon, they are sometimes easily misdiagnosed. Currently, the identification and classification of drusen lesions typically rely on manual judgment by doctors and other professionals using ocular imaging. This process is not only labor-intensive and inefficient, but also inherently challenging, making it susceptible to human error that can lead to misidentification or misclassification, thus affecting diagnostic results.

[0003] The information disclosed in the background section of this application is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0004] This disclosure provides a method and apparatus for extracting drusen lesions.

[0005] A first aspect of this disclosure provides a method for extracting drusen lesions, comprising: performing feature classification processing on a target eye image to determine whether a drusen lesion exists in the target eye image; if the drusen lesion exists in the target eye image, detecting the target eye image using a pre-trained drusen lesion detection model to obtain an initial image of the drusen lesion region; segmenting the initial image using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion; extracting the drusen lesion region from the target eye image based on a preset image processing algorithm to obtain a second image of the drusen lesion region; and performing intersection processing on the first image of the drusen lesion and the second image of the drusen lesion region to obtain a drusen lesion segmentation result corresponding to the target image.

[0006] Optionally, before performing feature classification processing on the target eye image, the method further includes: removing the background region of the eye image to be processed to obtain a first eye image; and performing normalization processing on the first eye image to obtain the target eye image.

[0007] Optionally, a target region with a brightness that meets a preset threshold is segmented from the target eye image;

[0008] Remove the disc region contained in the target region to obtain the candidate region;

[0009] Remove regions from the candidate regions whose roundness and / or edge sharpness do not meet the preset range to obtain the initial candidate regions;

[0010] The vascular region is extracted from the target eye image, and a second image of the drusen lesion region is obtained by executing different preset algorithms based on the positional relationship between the initial candidate region and the blood vessels.

[0011] Optionally, based on the positional relationship between the initial candidate region and blood vessels, different preset algorithms are executed to obtain a second image of the drusen lesion region, including:

[0012] Determine the grayscale and color differences between the viewing area and the surrounding background;

[0013] If the positional relationship between the initial candidate region and the blood vessel region conforms to a preset relationship, then the region to be processed is determined based on the first preset size relationship between the grayscale difference and color difference between the initial candidate region and the surrounding background region, and between the grayscale difference and color difference between the optic disc region and the surrounding background.

[0014] If the positional relationship between the initial candidate region and the blood vessel does not conform to the preset relationship, then the region to be processed is determined based on the second preset size relationship between the grayscale difference and color difference between the initial candidate region and the surrounding background region, and between the grayscale difference and color difference between the optic disc region and the surrounding background.

[0015] Calculate the average pixel area of ​​the region to be processed, and select a region from the region to be processed that has a preset relationship with the average pixel area as the target candidate region;

[0016] A second image is used to determine the drusen lesion region based on the distance between target candidate regions.

[0017] Optionally, the initial image is segmented using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion, including: expanding the region selected by the detection box in the initial image to obtain an expanded region; cropping an image block corresponding to the expanded region; determining a label image corresponding to the drusen lesion region within the image block, wherein the label image includes a background region labeled by a first label and the drusen lesion region within it labeled by a second label; and inputting the image block and the label image into the pre-trained drusen lesion segmentation model to obtain the first image of the drusen lesion corresponding to the image block.

[0018] Optionally, before segmenting the initial image using a pre-trained drusen lesion segmentation model, the method further trains the drusen lesion segmentation model, including: acquiring a first sample set, wherein the first sample set contains sample images of drusen lesions; determining sample label images corresponding to the samples in the first sample set through label annotation; and inputting the sample images and the corresponding sample label images into the pre-established drusen lesion segmentation model to train the pre-established drusen lesion segmentation model.

[0019] A second aspect of this disclosure provides a drusen lesion extraction device, comprising: a judgment module configured to perform feature classification processing on a target eye image to determine whether a drusen lesion exists in the target eye image; a detection module configured to, if the drusen lesion exists in the target eye image, detect the target eye image using a pre-trained drusen lesion detection model to obtain an initial image of the drusen lesion region; a first segmentation module configured to segment the initial image using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion; a second segmentation module configured to extract the drusen lesion region from the target eye image based on a preset image processing algorithm to obtain a second image of the drusen lesion region; and a calculation module configured to perform intersection processing on the initial image, the first image of the drusen lesion, and the second image of the drusen lesion region to obtain a drusen lesion segmentation result corresponding to the target eye image.

[0020] Optionally, the first segmentation module is further configured to: expand the region selected by the detection box in the initial image to obtain an expanded region; extract an image block corresponding to the expanded region; and input the image block into the pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion corresponding to the image block.

[0021] A third aspect of this disclosure provides a drusen lesion extraction device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

[0022] A fourth aspect of this disclosure provides a computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0023] The method and apparatus for extracting drusen lesions in this embodiment include: performing feature classification processing on a target eye image to determine whether drusen lesions exist in the target eye image; if drusen lesions are present in the target eye image, detecting the target eye image using a pre-trained drusen lesion detection model to obtain an initial image of the drusen lesion region; segmenting the initial image using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion; extracting the drusen lesion region from the target eye image based on a preset image processing algorithm to obtain a second image of the drusen lesion region; and performing intersection processing on the first image of the drusen lesion and the second image of the drusen lesion region to obtain the segmentation result corresponding to the drusen lesion region. By using a classification-then-segmentation step, the accuracy of segmentation is improved, avoiding the problems of high extraction difficulty and large extraction error caused by direct extraction due to the small size of drusen. Attached Figure Description

[0024] Figure 1 A schematic flowchart of a method for extracting drusen lesions according to an embodiment of the present disclosure is shown as an example;

[0025] Figure 2 A schematic diagram of an eye image, exemplarily illustrating an embodiment of the present disclosure, is shown.

[0026] Figure 3 A block diagram of a drusen extraction device according to an embodiment of the present disclosure is shown as an example;

[0027] Figure 4 This is a block diagram illustrating a drusen lesion extraction device according to an exemplary embodiment. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0029] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such use of data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein.

[0030] It should be understood that in the various embodiments of this disclosure, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0031] It should be understood that in this disclosure, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0032] It should be understood that in this disclosure, "multiple" refers to two or more. "And / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, "and / or B" can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "Contains A, B, and C", "Contains A, B, and C" means that all three A, B, and C are contained; "Contains A, B, or C" means that one of A, B, and C is contained; "Contains A, B, and / or C" means that any one, two, or three of A, B, and C are contained.

[0033] It should be understood that in this disclosure, "B corresponding to A", "B corresponding to A", "A corresponds to B", or "B corresponds to A" means that B is associated with A, and B can be determined based on A. Determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Matching A and B is defined as a similarity between A and B that is greater than or equal to a preset threshold.

[0034] Depending on the context, "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection."

[0035] The technical solutions of this disclosure will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0036] Figure 1 An exemplary flowchart of a method for extracting drusen lesions according to an embodiment of this disclosure is shown, such as... Figure 1 As shown, the method includes:

[0037] Step S101: Perform feature classification processing on the target eye image to determine whether there are drusen lesions in the target eye image.

[0038] In this embodiment, drusen is an ocular lesion. Specifically, drusen are gelatinous or transparent bodies, a degenerative disease occurring in the choroid and retina, caused by the abnormal deposition of abnormal metabolic products of pigment epithelial cells on the retina. It may lead to symptoms such as enlarged physiological blind spot, arcuate scotoma, narrowed visual field, or decreased vision.

[0039] Drupinal lesions can generally be classified into: hard drusen, soft drusen, basal drusen, and calcified drusen. Hard drusen are small and round, usually yellowish-white in color; soft drusen are larger than hard drusen, with indistinct borders, and can merge together; basal drusen are small and round, and uncountable, meaning they often merge together and are difficult to count; calcified drusen have a shimmering appearance in target eye images obtained through methods such as fundus fluorescein angiography.

[0040] For example, during a medical examination of the eye, an image of the eye to be processed can be acquired. For instance, the eye image to be processed can be obtained by means of fundus camera, fundus fluorescein angiography, or optical coherence tomography, and then the target eye image can be classified according to a pre-established classifier.

[0041] Furthermore, the aforementioned classifier can extract features using a deep learning neural network to determine whether drusen lesions exist in the target eye image. Alternatively, feature information of drusen lesions can be obtained through other methods to further determine whether drusen lesions exist in the target eye image. For example, the edge feature information of drusen lesions can be input into a support vector machine model for processing. This support vector machine model can be a model trained with various types of drusen lesions and can be used to determine whether drusen lesions exist in the target eye image.

[0042] Furthermore, in determining whether drusen lesions exist in the target eye image, regression analysis, Bayesian models, and other methods can also be used. This disclosure does not limit the specific judgment method.

[0043] Since drusen are very small, directly extracting them would result in large errors and waste computational resources. Therefore, considering that the accuracy of classification models is higher than that of segmentation models, this embodiment adopts a method of first classifying and determining whether drusen exist in the image, and then segmenting and extracting them to improve segmentation accuracy and reduce the waste of computational resources.

[0044] A classifier can be used to identify and classify lesions to determine whether drusen lesions are present in the target eye image. Pre-classifying lesions can improve the speed and accuracy of subsequent lesion extraction.

[0045] As an optional implementation of this embodiment, before performing feature classification processing on the target eye image, the method further includes: removing the background region of the eye image to be processed to obtain a first eye image; and performing normalization processing on the first eye image to obtain the target eye image.

[0046] In this optional implementation, refer to Figure 2 , Figure 2 The schematic diagram illustrating an eye image according to an embodiment of the present disclosure is shown. Due to factors such as the shape of the lens, a portion of the background area may exist in the eye image to be processed. For example, Figure 2 The black background area in the image lacks information about the eye and may interfere with model calculations. Furthermore, due to the varying resolutions of lenses, the size and resolution of the eye images to be processed may also differ. All of these factors can negatively impact the processing of eye images and the identification and classification of lesions.

[0047] To mitigate the aforementioned adverse effects, the background region of the eye image to be processed can be removed, and the first eye image after background removal can be scaled to a preset size to obtain the target eye image. This processing reduces interference from the background region and ensures that the target eye images processed by the model are of consistent size, thus improving the model's robustness in processing, as well as its robustness in lesion identification and classification.

[0048] In the example, a ROI (region of interest), i.e., the non-background region, can be determined in the eye image to be processed. This non-background region is then cropped to obtain the first eye image. The first eye image can then be normalized to obtain a target eye image of uniform size. Normalization can include, but is not limited to, translation, rotation, and scaling. For example, scaling to a size of 512×512 yields a target eye image of uniform size.

[0049] Step S102: If the drusen lesion exists in the target eye image, the target eye image is detected using a pre-trained drusen lesion detection model to obtain an initial image of the drusen lesion region.

[0050] In this embodiment, the pre-trained drusen lesion detection model is used to analyze the target eye image and detect the location of the fundus drusen lesion. For each fundus drusen lesion, a fundus lesion detection box with a fundus drusen lesion type label and capable of selecting the fundus drusen lesion area is generated.

[0051] Furthermore, the initial image is obtained by overlaying the fundus lesion detection boxes onto the target eye image. Each drusen lesion detection box includes information about the type of drusen lesion it selects. The drusen lesion detection box can be the smallest bounding rectangle of the drusen lesion, or it can be a detection box of other shapes, such as a circle or a triangle. The initial image can include the drusen lesion region selected by the detection boxes. The type of drusen lesion corresponding to the region selected by each detection box in the initial image can include hard drusen, soft drusen, basal drusen, and calcified drusen. This type can be displayed when the mouse pointer moves over the detection box. The drusen lesion detection model can be a YOLOv5 detection network, through which the detection boxes of drusen lesions and the lesion label type corresponding to each detection box can be extracted.

[0052] For example, the structure of a YOLOv5 detection network may include:

[0053] The CBL module consists of Conv+BN+Leaky_relu activation functions; Res-unit: borrows the residual structure from ResNet to build deep networks, and CBM is a sub-module of the residual module; CSP1_X: borrows the CSPNet network structure and consists of CBL, Res-unit, convolutional layers, and Concaten; CSP2_X: borrows the CSPNet network structure and consists of convolutional layers multiplied by X Res-unit modules and then concatenated; FOCUS: concatenates multiple slice results and feeds them into the CBL module; SPP: uses 1x1, 5x5, 9x9, and 13x13 max pooling methods for multi-scale fusion.

[0054] The improvements to the network structure in this embodiment include:

[0055] (1) Input end: During the model training phase, Mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling are used;

[0056] (2) Baseline network: FOCUS and CSP structures were used;

[0057] (3) Neck network: An FPN_PAN structure is inserted between the Backbone and the final Head output layer;

[0058] (4) Head output layer: loss function GIOU_Loss during training, DIOU_nns for predicting the filter box;

[0059] CutMix: Combines two images.

[0060] Mosaic: An improvement on CutMix, it uses four images and stitches them together using random scaling, random cropping, and random arrangement.

[0061] Advantages: Combining several images into one not only enriches the dataset and greatly improves the network training speed, but also reduces model memory usage; furthermore, the input image is a 416*416 three-channel color fundus image, and the label file is in XML format.

[0062] Furthermore, the YOLOv5 detection network in this embodiment can obtain many candidate region bounding boxes (i.e., detection boxes). Non-maximum suppression can be used to remove redundant bounding boxes (detection boxes), and finally the initial image of the drusen lesion region is obtained.

[0063] Step S103: The initial image is segmented using a pre-trained drusen lesion segmentation model to obtain the first image of the drusen lesion.

[0064] In this embodiment, a drusen lesion segmentation model can be pre-established and pre-trained. With the completed training model, when the initial image containing step S102 is input, the completed training drusen lesion segmentation model can segment the drusen lesion area and output the segmented drusen lesion area in each detection box, i.e., the first image. The first image can be the area surrounded by the first segmentation boundary and the first segmentation boundary.

[0065] As an optional implementation of this embodiment, the initial image is segmented using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion. This includes: expanding the region selected by the detection box in the initial image to obtain an expanded region; cropping the image block corresponding to the expanded region; and inputting the image block into the pre-trained drusen lesion segmentation model to obtain the first image of the drusen lesion corresponding to the image block.

[0066] In this optional implementation, the information within the detection box is limited, and the edge of the detection box may overlap with the edge of the drusen lesion, making direct segmentation difficult. Therefore, in order to segment and extract the lesion within the detection box, background information (global information) needs to be obtained to extract the lesion. Thus, the detection box can be expanded, for example, by expanding it upwards, downwards, leftwards, and rightwards to obtain an expanded area, thereby enlarging the detection box, increasing the amount of information, and reducing the possibility of overlap with the edge of the lesion.

[0067] After expanding the area selected by the detection box, image blocks corresponding to the expanded area can be cropped, and each image block can be further segmented using a drusen lesion segmentation model. By pre-cropping image blocks and then accurately segmenting each image block using the drusen lesion segmentation model, the accuracy of drusen lesion segmentation in the fundus can be improved. After segmentation by the drusen lesion segmentation model, the segmentation boundaries in each image block and the areas enclosed by the segmentation boundaries can be obtained. The set of segmentation boundaries in each image block is the first segmentation boundary.

[0068] The above processing can improve the segmentation accuracy of drusen lesions by the lesion segmentation model, thereby obtaining a first segmentation boundary with higher accuracy.

[0069] Step S104: Based on a preset image processing algorithm, the drusen lesion area of ​​the target eye image is extracted to obtain a second image of the drusen lesion area.

[0070] In this embodiment, the segmentation result boundary determined by step 103 is relatively coarse. To make the drusen lesion extraction result more refined, fine segmentation can be achieved as follows: the target eye image is processed by boundary detection using computer vision methods to obtain a second image, which includes a second segmentation boundary and the region enclosed by the second segmentation boundary. In step S105, based on the second image obtained by boundary detection and the first image obtained by the drusen lesion segmentation model, the final drusen lesion segmentation boundary and the region enclosed by the drusen lesion segmentation boundary are determined.

[0071] As an optional implementation of this embodiment, a target region with brightness meeting a preset threshold is segmented from the target eye image; the optic disc region contained in the target region is removed to obtain a candidate region; regions whose roundness does not meet a preset roundness range are removed from the candidate region to obtain an initial candidate region; or, regions whose edge sharpness does not meet a preset range are removed from the candidate region to obtain an initial candidate region; or, regions whose roundness and edge sharpness do not meet both preset ranges are removed from the candidate region to obtain an initial candidate region; a vascular region is extracted from the target eye image, and based on the positional relationship between the initial candidate region and the vascular region, different preset algorithms are executed to obtain a second image of the drusen lesion region.

[0072] In this optional implementation, a Region of Interest (ROI) image can be extracted from the target eye image. Histogram equalization is then applied to the ROI image to expand its grayscale range to 0-255. Dynamic thresholding is performed on the green channel image (or other channels can be used for dynamic thresholding as needed; using the green channel image for dynamic thresholding provides more accurate segmentation compared to other channels), resulting in the brighter areas of the image, i.e., the target region. The dynamic threshold setting can depend on the image's sharpness. Next, the optic disc region is located from the target eye image, and the optic disc region is removed from the target region to obtain candidate regions. The roundness of each candidate region is calculated, and candidate regions with roundness less than a preset value are removed to obtain initial candidate regions. The preset roundness value can be between 0 and 0.5, such as 0.2, 0.3, 0.38, or 0.4. Alternatively, the edge sharpness of each initial candidate region can be calculated, and candidate regions with edge sharpness less than a preset value are removed to obtain initial candidate regions. The preset edge sharpness value can be between 70 and 150, such as 90, 100, or 110.

[0073] Furthermore, the vascular region can be pre-extracted, and after the vascular region is extracted, the vascular arch region can be further obtained. Based on the positional relationship between the initial candidate region and the vascular arch region, different preset algorithms can be executed to obtain the second image of the drusen lesion region.

[0074] As an optional implementation of this embodiment, the second image of the drusen lesion region is obtained by executing different preset algorithms based on the positional relationship between the initial candidate region and the blood vessel. This includes: determining the grayscale difference and color difference between the optic disc region and the surrounding background; if the positional relationship between the initial candidate region and the blood vessel region conforms to a preset relationship, then determining the region to be processed based on a first preset size relationship between the grayscale difference and color difference between the initial candidate region and the surrounding background region, and between the grayscale difference and color difference between the optic disc region and the surrounding background; if the positional relationship between the initial candidate region and the blood vessel does not conform to the preset relationship, then determining the region to be processed based on a second preset size relationship between the grayscale difference and color difference between the initial candidate region and the surrounding background region, and between the grayscale difference and color difference between the optic disc region and the surrounding background; calculating the average pixel area of ​​the region to be processed, and selecting a region from the region to be processed that has a preset relationship with the average pixel area, and using this region as a target candidate region; and determining the second image of the drusen lesion region based on the distance between the target candidate regions.

[0075] Determining whether the initial candidate region's positional relationship with the vascular region conforms to a preset relationship includes determining whether the initial candidate region is within the main vascular arch region.

[0076] In this optional implementation, if the initial candidate region is within the main vascular arch region, then a region in which the grayscale difference and color difference between the initial candidate region and the surrounding background are both greater than a first preset multiple (e.g., 1 / 2) of the grayscale difference and color difference between the optic disc region and the surrounding background is selected, and this region is taken as the region to be processed.

[0077] Furthermore, if the initial candidate region is not within the main vascular arch region, then a region in which both the grayscale difference and color difference between the initial candidate region and the surrounding background are greater than a second preset multiple (e.g., 1 / 3) of the grayscale difference and color difference between the optic disc region and the surrounding background is selected, and this region is taken as the region to be processed. The first preset multiple is greater than the second preset multiple.

[0078] The process involves calculating the pixel area of ​​the region to be processed and the average pixel area based on the pixel areas of all regions to be processed. The difference between the pixel area of ​​the region to be processed and the average pixel area is determined. Regions with a difference between 0.5 and 1.5 times the average pixel area are considered as candidate regions. After obtaining the candidate regions, the distances of each candidate region to other candidate regions are calculated to obtain the minimum distance between each candidate region and other target regions. The average of these minimum distances is then calculated. The minimum distance between each candidate region and other target regions is compared with this average. Target regions with differences between 0.5 and 1.5 times the average are identified as the second image of the drusen region. For example, if the minimum distance is A, the average minimum distance is B, and the difference between A and B is C, and C is between 0.5B and 1.5B, then the target region corresponding to A is identified as the drusen region, resulting in the final second image of the drusen region.

[0079] Step S105: Perform intersection processing on the initial image, the first image of the drusen lesion, and the second image of the drusen lesion region to obtain the segmentation result corresponding to the drusen lesion region.

[0080] In this embodiment, as mentioned above, the first image may have segmentation errors, so the second image can be used to correct the first image. The intersection of the second image (including the second segmentation boundary and the area enclosed by the second segmentation boundary) obtained based on a preset image processing algorithm and the first image (i.e., the image containing the aforementioned first segmentation boundary and the area enclosed by the first segmentation boundary) obtained by the drusen segmentation model is performed. Further, the first image segmented by the drusen segmentation model is determined based on the area selected by the expanded detection box. To ensure that the final segmentation result is the fundus drusen lesion within the rectangular frame selected in the initial image, the initial image and the first image can be intersected to obtain a segmented image of the rectangular frame selected in the initial image. Then, this segmented image is intersected with the second image to obtain the final accurate segmentation result. Alternatively, the first and second images can be intersected before intersecting with the initial image; the order of the intersection operations is not limited here, but the ultimate goal is to achieve the intersection of the three images. The final segmentation result, namely the final drusen lesion segmentation boundary and the image of the area enclosed by the final drusen lesion segmentation boundary, can be obtained through the intersection of these three images.

[0081] By taking the intersection processing step, the error portion in the first segmentation boundary of the drusen lesion segmentation model can be automatically removed, thereby reducing the error of the first segmentation boundary and obtaining a more accurate segmentation boundary for the drusen lesion.

[0082] As an optional implementation of this embodiment, the method further includes: extracting the boundary of the segmentation result of the drusenous lesion to obtain the boundary of the fundus drusenous lesion; and overlaying the boundary onto the target eye image to obtain an overlaid fundus image of the drusenous lesion.

[0083] In this optional implementation, since the segmentation result of drusen lesions is a black and white binary image, in order to facilitate observation, this optional implementation can process the binary image into a line drawing with only the lesion boundary, and then superimpose the lesion boundary onto the target eye image to obtain the final result. Finally, the result can be output to a display device for display.

[0084] As an optional implementation of this embodiment, before segmenting the initial image using a pre-trained drusen lesion segmentation model, the method further trains the drusen lesion segmentation model, including: acquiring a first sample set, wherein the first sample set contains sample images of drusen lesions; determining sample label images corresponding to the samples in the first sample set through label annotation; and inputting the sample images and the corresponding sample label images into the pre-established drusen lesion segmentation model to train the pre-established drusen lesion segmentation model.

[0085] In this optional implementation, the image of the dura mater lesion can be used as a sample image, and its corresponding label image (such as a binary image containing the background area and the dura mater lesion area) can be determined by label annotation; the two are then input into the dura mater lesion segmentation model to complete the training of the model.

[0086] Furthermore, since there is an error between the segmentation result of the lesion segmentation model and the label image, the loss function of the drusen lesion segmentation model is determined by the error between the lesion segmentation result and the label image during training. For example, the loss function can be calculated by the positional error between the segmentation boundaries. The loss function can then be backpropagated to adjust the parameters of the drusen lesion segmentation model based on gradient descent, thereby reducing the loss function. This training process is then iterated multiple times until training conditions are met, such as loss function convergence; reaching a set number of training iterations; or accuracy meeting the precision requirements on the validation set. This disclosure does not impose restrictions on the training conditions. After the training conditions are met, the trained drusen lesion segmentation model is obtained.

[0087] As an optional implementation method in this embodiment, the pre-established lesion segmentation model is a u-net network structure.

[0088] In this embodiment, pre-classifying lesions can improve the segmentation efficiency and accuracy of the lesion segmentation model. Furthermore, correcting the first image with the second image can improve the accuracy of drusen lesion segmentation.

[0089] In this embodiment, the drusen lesion detection model can also be trained. The training samples can be images containing drusen lesions. Before training, the types of drusen lesions can be manually labeled in advance. However, manually labeling the boundaries of drusen lesions in eye image samples is labor-intensive and has a high error rate.

[0090] To avoid the problems of high workload and high error rate caused by directly annotating eye image samples, the following steps can be adopted: perform boundary detection processing on the eye image samples to obtain the third segmentation boundary of the eye image samples; receive the selection processing of the third segmentation boundary to obtain the annotation information corresponding to the third segmentation boundary. After boundary detection, the boundary of the drusen lesion can be selected and annotated manually from the known boundaries, thereby avoiding the problems of high workload and high error rate caused by directly annotating the eye image samples and improving the accuracy of annotation.

[0091] During the training of the drusensis lesion detection model, boundary detection was used to assist the annotators, reducing their workload and improving the annotation quality. This, in turn, improved the accuracy of the drusensis lesion segmentation model and further enhanced the accuracy of the segmentation results.

[0092] Furthermore, the boundary detection method can be similar to the above-mentioned brightness normalization and / or color normalization and threshold segmentation processing, and this disclosure does not limit it.

[0093] Figure 3 A block diagram of a drusen lesion extraction device according to an embodiment of the present disclosure is shown as an example. Figure 3 As shown, the device includes:

[0094] The judgment module is used to perform feature classification processing on the target eye image to determine whether there are drusen lesions in the target eye image;

[0095] The detection module is used to detect the drusen lesion in the target eye image by using a pre-trained drusen lesion detection model when the drusen lesion is present in the eye image, and obtain the drusen lesion detection box.

[0096] The first segmentation module is configured to segment the initial image using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion.

[0097] The second segmentation module is configured to extract the drusen lesion region from the target eye image based on a preset image processing algorithm, and obtain a second image of the drusen lesion region.

[0098] The calculation module is configured to perform intersection processing on the first image of the drusen lesion and the second image of the drusen lesion region to obtain the segmentation result corresponding to the drusen lesion region.

[0099] According to an embodiment of this disclosure, the apparatus further includes: a preprocessing module, configured to remove the background region of the eye image to be processed to obtain a first eye image; and to perform normalization processing on the first eye image to obtain the target eye image.

[0100] According to an embodiment of this disclosure, the second segmentation module is further configured to: sequentially perform multi-scale filtering, brightness normalization, and / or color normalization on the target eye image to obtain an enhanced image; and perform threshold segmentation on the enhanced image to obtain the second image.

[0101] According to an embodiment of this disclosure, the first segmentation module is further configured to include: expanding each region selected by the detection box in the initial image to obtain expanded regions; cropping image blocks corresponding to the expanded regions; and inputting each image block into the pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion corresponding to each image block.

[0102] According to an embodiment of this disclosure, the device further includes a training module for training a drusen lesion segmentation model, comprising: acquiring a first sample set, wherein the first sample set contains sample images of drusen lesions; determining sample label images corresponding to samples in the first sample set by label annotation; and inputting the sample images and the sample label images corresponding to the samples into a pre-established drusen lesion segmentation model to train the pre-established drusen lesion segmentation model.

[0103] According to an embodiment of this disclosure, the apparatus further includes: extracting the boundary of the segmentation result of the drusenous lesion to obtain the boundary of the fundus drusenous lesion; and overlaying the boundary onto the target eye image to obtain an overlaid fundus image of the drusenous lesion.

[0104] Figure 4This is a block diagram illustrating a drusen lesion extraction device according to an exemplary embodiment. For example, the device 1600 may be provided as a terminal or server. The device 1600 includes a processing component 1602 and memory resources represented by a memory 1603 for storing instructions executable by the processing component 1602, such as an application program. The application program stored in the memory 1603 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1602 is configured to execute instructions to perform the described method.

[0105] Device 1600 may also include a power supply component 1606 configured to perform power management of device 1600, a wired or wireless network interface 1605 configured to connect device 1600 to a network, and an input / output (I / O) interface 1608. Device 1600 can operate on an operating system stored in memory 1603, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0106] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0107] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0108] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0109] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0110] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0111] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0112] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0113] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0114] Note that, unless otherwise explicitly stated, all features disclosed in this specification (including any appended claims, abstract, and drawings) may be replaced by alternative features for achieving the same, equivalent, or similar purpose. Therefore, unless explicitly stated otherwise, each disclosed feature is merely one example of a set of equivalent or similar features. Where used, "further," "preferably," "even further," and "more preferably" are simple starting points for describing another embodiment based on the foregoing embodiments, the combination of which with the foregoing embodiments constitutes the complete configuration of another embodiment. Any combination of several "further," "preferably," "even further," or "more preferably" settings following the same embodiment constitutes yet another embodiment.

[0115] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments, and any variations or modifications may be made to the implementation of the present invention without departing from the stated principles.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for extracting drusen lesions, characterized in that, include: The target eye image is subjected to feature classification processing to determine whether there are drusen lesions in the target eye image; If the drusen lesion is present in the target eye image, the target eye image is detected using a pre-trained drusen lesion detection model to obtain an initial image of the drusen lesion region. The initial image is segmented using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion. Based on a preset image processing algorithm, the drusen lesion area is extracted from the target eye image to obtain a second image of the drusen lesion area; The intersection of the initial image, the first image of the drusen lesion, and the second image of the drusen lesion region is performed to obtain the drusen lesion segmentation result corresponding to the target eye image; Based on a preset image processing algorithm, the drusen lesion region is extracted from the target eye image to obtain a second image of the drusen lesion region, including: Segment the target region whose brightness meets the preset threshold from the target eye image; Remove the disc region contained in the target region to obtain the candidate region; Remove regions from the candidate regions whose roundness and / or edge sharpness do not meet the preset range to obtain the initial candidate regions; Extracting vascular regions from the target eye image, and based on the positional relationship between the initial candidate region and the blood vessels, executing different preset algorithms to obtain a second image of the drusen lesion region; including: Determine the grayscale and color differences between the viewing area and the surrounding background; If the initial candidate region is within the main vascular arch region of the vascular region, then select a region where the grayscale difference and color difference between the initial candidate region and the surrounding background are both greater than a first preset multiple of the grayscale difference and color difference between the optic disc region and the surrounding background, and use this region as the region to be processed. If the initial candidate region is not within the main vascular arch region of the vascular region, then a region in which the grayscale difference and color difference between the initial candidate region and the surrounding background are both greater than the grayscale difference and color difference between the optic disc region and the surrounding background by a second preset multiple is selected, and this region is taken as the region to be processed; wherein, the first preset multiple is greater than the second preset multiple; Calculate the average pixel area of ​​the region to be processed, and select a region from the region to be processed that has a preset size relationship with the average pixel area, and use this region as the target candidate region; A second image is used to determine the drusen lesion region based on the distance between target candidate regions.

2. The method for extracting drusen lesions according to claim 1, characterized in that, Before performing feature classification processing on the target eye image, the method further includes: Remove the background area from the eye image to be processed to obtain the first eye image; The first eye image is normalized to obtain the target eye image.

3. The method for extracting drusen lesions according to claim 1, characterized in that, The initial image is segmented using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion, including: Each region selected by the detection box in the initial image is expanded to obtain the expanded region; Extract the image block corresponding to the expanded region; Each of the image blocks is input into the pre-trained drusen segmentation model to obtain the first image of the drusen lesion corresponding to each image block.

4. The method for extracting drusen lesions according to claim 1, characterized in that, Before segmenting the initial image using a pre-trained drusen lesion segmentation model, the method further trains the drusen lesion segmentation model, including: Obtain a first sample set, wherein the first sample set contains sample images of drusen lesions; The sample label images corresponding to the samples in the first sample set are determined by label annotation. The sample image and the corresponding sample label image are input into a pre-established drusen segmentation model to train the pre-established drusen segmentation model.

5. The method for extracting drusen lesions according to claim 1, characterized in that, The method further includes: The boundaries of the drusen lesions in the fundus are obtained by extracting the segmentation results. The boundary is superimposed on the target eye image to obtain an superimposed fundus image of the drusen lesion.

6. A device for extracting drusen lesions, characterized in that, include: The judgment module is configured to perform feature classification processing on the target eye image to determine whether there are drusen lesions in the target eye image; The detection module is configured to detect the drusen lesion in the target eye image by using a pre-trained drusen lesion detection model to obtain an initial image of the drusen lesion region if the drusen lesion is present in the target eye image. The first segmentation module is configured to segment the initial image using a pre-trained drusen lesion segmentation model to obtain a first image of the drusen lesion. The second segmentation module is configured to extract the drusen lesion region from the target eye image based on a preset image processing algorithm, thereby obtaining a second image of the drusen lesion region; wherein, the extraction of the drusen lesion region from the target eye image based on the preset image processing algorithm, thereby obtaining a second image of the drusen lesion region, includes: Segment the target region whose brightness meets the preset threshold from the target eye image; Remove the disc region contained in the target region to obtain the candidate region; Remove regions from the candidate regions whose roundness and / or edge sharpness do not meet the preset range to obtain the initial candidate regions; Extracting vascular regions from the target eye image, and based on the positional relationship between the initial candidate region and the blood vessels, executing different preset algorithms to obtain a second image of the drusen lesion region; including: Determine the grayscale and color differences between the viewing area and the surrounding background; If the initial candidate region is within the main vascular arch region of the vascular region, then select a region where the grayscale difference and color difference between the initial candidate region and the surrounding background are both greater than a first preset multiple of the grayscale difference and color difference between the optic disc region and the surrounding background, and use this region as the region to be processed. If the initial candidate region is not within the main vascular arch region of the vascular region, then a region in which the grayscale difference and color difference between the initial candidate region and the surrounding background are both greater than the grayscale difference and color difference between the optic disc region and the surrounding background by a second preset multiple is selected, and this region is taken as the region to be processed; wherein, the first preset multiple is greater than the second preset multiple; Calculate the average pixel area of ​​the region to be processed, and select a region from the region to be processed that has a preset size relationship with the average pixel area, and use this region as the target candidate region; A second image of the drusen lesion region is determined based on the distance between the target candidate regions; The calculation module is configured to perform intersection processing on the initial image, the first image of the drusen lesion, and the second image of the drusen lesion region to obtain the drusen lesion segmentation result corresponding to the target eye image.

7. A device for extracting drusen lesions, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 5.

Images