An eye fundus image anomaly detection method based on generative counterfactual reconstruction

Abstract

The application discloses an eye fundus image anomaly detection method based on generative counterfactual reconstruction, which comprises the following steps: performing standardization preprocessing on an original eye fundus color image to obtain a standardized eye fundus image; generating a coarse reconstruction image of the standardized eye fundus image by using a coarse reconstruction network, and calculating a reconstruction residual image according to the standardized eye fundus image and the coarse reconstruction image; calculating a perceptual saliency map according to the original eye fundus color image and the coarse reconstruction image, and generating a target anomaly mask according to the reconstruction residual image and the perceptual saliency map; inputting the standardized eye fundus image and the target anomaly mask into a mask repair generation model to generate a pseudo-healthy image after removing a suspected lesion area of the standardized eye fundus image; and calculating the difference between the original eye fundus color image and the pseudo-healthy image to generate an abnormal area image corresponding to the lesion area. The scheme does not need lesion labeling, can accurately locate the lesion area, has high generalization and clinical practicability, and is suitable for large-scale eye fundus disease screening.

Description

Technical Field

[0001] This solution relates to the fields of medical image analysis and computer vision technology, and in particular to a method for detecting fundus image anomalies based on generative counterfactual reconstruction. Background Technology

[0002] Fundus diseases (such as age-related macular degeneration, diabetic retinopathy, and glaucoma-related optic neuropathy) are the main causes of vision loss. Early screening and accurate detection are crucial for disease intervention.

[0003] Existing methods for detecting abnormalities in fundus images mainly include thresholding segmentation methods based on traditional image processing and classification / segmentation methods based on deep learning. However, these methods rely on manually designed features and are difficult to adapt to complex and varied lesion morphologies; deep learning methods require a large amount of labeled data and have limited model generalization ability. In addition, existing methods are unable to generate "counterfactual" comparison images of lesions, and cannot intuitively show the expected appearance of the lesion in a normal state in the image.

[0004] Therefore, there is a need for a method that can detect fundus abnormalities in an unsupervised manner and generate pseudo-healthy control images that conform to a healthy distribution. Summary of the Invention

[0005] This solution aims to at least address the technical problems existing in the prior art. To this end, the first aspect of this invention proposes a method for detecting anomalies in fundus images based on generative counterfactual reconstruction, the method comprising:

[0006] The original fundus color image to be detected is acquired, and the original fundus color image is subjected to standardized preprocessing to obtain a standardized fundus image;

[0007] A coarse reconstruction image of the standardized fundus image is generated using a coarse reconstruction network, and a reconstruction residual map is calculated based on the standardized fundus image and the coarse reconstruction image.

[0008] A perceptual saliency map is calculated based on the original fundus color image and the coarse reconstructed image, and a target anomaly mask is generated based on the reconstructed residual map and the perceptual saliency map.

[0009] The standardized fundus image and the target abnormality mask are input into the mask repair generation model to generate a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image.

[0010] The difference between the original fundus color image and the pseudo-healthy image is calculated to generate an abnormal region map corresponding to the lesion area.

[0011] Optionally, the standardization preprocessing of the original fundus color image includes:

[0012] Locate the target region to be detected from the original fundus color image and crop out the image of the target region;

[0013] The target region image is scaled to a preset resolution, and image enhancement is performed on the target region image at the preset resolution to obtain an enhanced image;

[0014] The image quality level of the enhanced image is determined, and images that meet the preset level standard are selected as standardized fundus images.

[0015] Optionally, calculating the perceptual saliency map based on the original fundus color image and the coarse reconstructed image includes:

[0016] The original fundus color image and the coarse reconstructed image are input into the perceptual difference measurement model to calculate the feature difference vectors of the original fundus color image and the coarse reconstructed image in each layer of the network;

[0017] The feature difference vector is mapped back to the spatial resolution of the original fundus color image by upsampling and aggregation to obtain a perceptual saliency map of the same size as the original fundus color image.

[0018] Optionally, generating the target anomaly mask based on the reconstructed residual map and the perceptual saliency map includes:

[0019] A point-by-point multiplication calculation is performed on the reconstructed residual map and the perceived saliency map to obtain the anomaly response map;

[0020] The abnormal response map is segmented by thresholding, and the region with a gray value greater than or equal to a first preset threshold is selected to obtain the initial abnormal mask;

[0021] The initial anomaly mask is Gaussian blurred, and the Gaussian blurred soft mask is binarized according to a second preset threshold to obtain a target anomaly mask that can clearly distinguish between the area to be repaired and the area to be preserved.

[0022] Optionally, the step of inputting the standardized fundus image and the target abnormality mask into the mask repair generation model to generate a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image includes:

[0023] The standardized fundus image and the target abnormality mask are used as input mask repair to generate a model;

[0024] The model determines the region not covered by the mask in the standardized fundus image based on the target anomaly mask, thus obtaining the masked region and the unmasked region;

[0025] The model reconstructs the texture and structure of the non-masked region according to the context information of the masked region, conforming to a healthy distribution, to obtain a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image.

[0026] Optionally, after generating the abnormal region map corresponding to the lesion area, the method further includes:

[0027] The lesion area is located based on the target anomaly mask.

[0028] Optionally, the mask repair generation model adopts the AOT-GAN model.

[0029] A second aspect of the present invention provides a device for detecting fundus image anomalies based on generative counterfactual reconstruction, the device comprising:

[0030] The preprocessing module is used to acquire the original fundus color image to be detected and to perform standardized preprocessing on the original fundus color image to obtain a standardized fundus image.

[0031] The residual calculation module is used to generate a coarse reconstruction image of the standardized fundus image using a coarse reconstruction network, and to calculate a reconstruction residual map based on the standardized fundus image and the coarse reconstruction image.

[0032] The mask generation module is used to calculate a perceptual saliency map based on the original fundus color image and the coarse reconstructed image, and to generate a target anomaly mask based on the reconstructed residual map and the perceptual saliency map;

[0033] The pseudo-healthy image generation module is used to input the standardized fundus image and the target abnormality mask into the mask repair generation model to generate a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image.

[0034] The lesion area calculation module is used to calculate the difference between the original fundus color image and the pseudo-healthy image, and generate an abnormal area map corresponding to the lesion area.

[0035] A third aspect of the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the fundus image anomaly detection method based on generative counterfactual reconstruction as described in the first aspect.

[0036] A fourth aspect of the present invention provides a computer-readable storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the fundus image anomaly detection method based on generative counterfactual reconstruction as described in the first aspect.

[0037] The embodiments of the present invention have the following beneficial effects:

[0038] This invention provides a method for detecting fundus image anomalies based on generative counterfactual reconstruction. The method includes: acquiring an original fundus color image to be detected, and performing standardized preprocessing on the original fundus color image to obtain a standardized fundus image; generating a coarse reconstruction image of the standardized fundus image using a coarse reconstruction network, and calculating a reconstruction residual map based on the standardized fundus image and the coarse reconstruction image; calculating a perceptual saliency map based on the original fundus color image and the coarse reconstruction image, and generating a target anomaly mask based on the reconstruction residual map and the perceptual saliency map; inputting the standardized fundus image and the target anomaly mask into a mask repair generation model to generate a pseudo-healthy image after removing suspected lesion areas from the standardized fundus image; calculating the difference between the original fundus color image and the pseudo-healthy image to generate an anomaly region map corresponding to the lesion area. This solution first generates a target anomaly mask by fusing coarse reconstruction and perceptual saliency to suppress false positives of normal structures; then, it generates a pseudo-healthy image that conforms to a healthy distribution based on mask repair, which can intuitively display the expected appearance of the image under normal conditions. This approach requires no lesion labeling, only training with healthy data, and can accurately locate lesion areas. It has high generalizability, interpretability, and clinical applicability, making it suitable for large-scale retinal disease screening. Attached Figure Description

[0039] Figure 1 A flowchart illustrating the steps of a fundus image anomaly detection method based on generative counterfactual reconstruction provided in an embodiment of the present invention;

[0040] Figure 2 This is a structural block diagram of a fundus image anomaly detection device based on generative counterfactual reconstruction, provided in an embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present solution, and not all embodiments. Based on the embodiments of the present solution, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present solution.

[0042] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, the use of "based on" or "according to" implies openness and inclusiveness, because processes, steps, calculations, or other actions "based on" or "according to" one or more of the stated conditions or values ​​may in practice be based on additional conditions or beyond the stated values.

[0043] Figure 1 This is a flowchart illustrating the steps of a fundus image anomaly detection method based on generative counterfactual reconstruction, as provided in an embodiment of the present invention.

[0044] like Figure 1 As shown, the method includes the following steps:

[0045] Step 101: Obtain the original fundus color image to be detected, and perform standardized preprocessing on the original fundus color image to obtain a standardized fundus image.

[0046] Standardized preprocessing can include operations such as size normalization, color normalization, noise reduction, and enhancement. The purpose is to eliminate differences caused by different devices and shooting conditions, so that all images have uniform size, brightness, and contrast, which facilitates subsequent processing.

[0047] As an optional embodiment, the standardization preprocessing of the original fundus color image includes:

[0048] Step 101: Locate the target region to be detected from the original fundus color image and crop out the target region image;

[0049] Step 102: Scale the target region image to a preset resolution, and perform image enhancement on the target region image at the preset resolution to obtain an enhanced image;

[0050] Step 103: Determine the image quality level of the enhanced image and select images that meet the preset level standards as standardized fundus images.

[0051] In steps 101-103, the target region to be detected can be the optic disc region, the macula region, or the effective visual field region. Scaling the target region image to a preset resolution is equivalent to size normalization.

[0052] Image enhancement is performed, specifically including brightness enhancement and contrast enhancement, to improve image quality. The image quality level of the enhanced image is then determined. Based on image quality from low to high, the image quality levels are divided into three categories: Reject, Useful, and Good. Images at the Reject level are discarded because their image quality is too poor for subsequent lesion detection. Images at the Useful and Good levels are retained and used as standardized fundus images for subsequent detection and processing.

[0053] In addition, before step 101, there is a model training phase, which mainly trains the coarse reconstruction network, the perceptual difference measurement model, and the mask restoration generation model. After the model is trained, the fundus image anomaly detection method of steps 101-105 is executed.

[0054] During the model training phase, fundus images are first obtained from publicly available datasets, including but not limited to IDRID, Messidor, FGADR, EyePacs, and AIROGS. Based on the labels in the publicly available datasets, the samples are divided into a normal sample set and an abnormal sample set. Standardization preprocessing is then performed on each fundus image.

[0055] The standardized preprocessing during the model training phase specifically includes:

[0056] Visual field localization or cropping: Locating the optic disc / macula region or cropping the effective visual field area;

[0057] Size normalization: scales the image to a preset resolution;

[0058] Image enhancement: Applying at least one of the following to the samples during the training phase: brightness, contrast, color jitter, rotation, flipping, scaling, and noise perturbation;

[0059] Quality control: Determine whether an image is classifiable; remove or downgrade images that are not classifiable. Image levels can be divided into three categories: Reject, Useful, and Good. We can use images of the Useful and Good categories as training samples.

[0060] Step 102: Generate a coarse reconstruction image of the standardized fundus image using a coarse reconstruction network, and calculate the reconstruction residual map based on the standardized fundus image and the coarse reconstruction image.

[0061] Standardized fundus images Inputting the coarse reconstruction network yields a coarsely reconstructed image. ,in Indicates encoder, This indicates the decoder.

[0062] Based on standardized fundus images With coarse reconstructed image Calculate the reconstructed residual map .

[0063] The coarse reconstruction network is an autoencoder (AE) or U-Net, which has only been trained on healthy samples and cannot perfectly reconstruct regions containing lesions. Since lesion regions usually have large reconstruction errors, the reconstruction residual map can initially indicate the location of abnormalities, but may contain noise or false positives at the edges.

[0064] Step 103: Calculate the perceptual saliency map based on the original fundus color image and the coarse reconstructed image, and generate a target anomaly mask based on the reconstructed residual map and the perceptual saliency map.

[0065] Further calculation of the perceptual saliency map Perceptual saliency maps can be obtained based on LPIPS or other perceptual difference measurement methods. Perceptual saliency maps can measure the regions of difference that the human eye or the network "attention" and can suppress unimportant texture differences.

[0066] To further filter out noise or false positives at the edges, a target anomaly mask is generated based on the reconstructed residual map and the perceived saliency map. The target anomaly mask indicates the possible lesion area, which can preliminarily locate the lesion, while filtering out false positives generated by normal structures (such as blood vessels).

[0067] As an optional embodiment, step 103, which calculates a perceptual saliency map based on the original fundus color image and the coarse reconstructed image, includes:

[0068] Step 1031: Input the original fundus color image and the coarse reconstructed image into the perceptual difference measurement model, and calculate the feature difference vectors of the original fundus color image and the coarse reconstructed image in each layer of the network;

[0069] Step 1032: Map the feature difference vector back to the spatial resolution of the original fundus color image by upsampling and aggregation to obtain a perceptual saliency map of the same size as the original fundus color image.

[0070] In steps 1031-1032, feature maps of the original fundus color image and the coarsely reconstructed image are extracted at different layers of the perceptual difference model. At each layer, the difference vector between the two feature maps is calculated. The method for calculating the difference vector may include the L1 or L2 norm of the pixel-wise difference, or the cosine distance.

[0071] Since low-level network differences mainly detect fine-grained changes such as texture and noise, while high-level network differences mainly detect global anomalies such as shape and structure, lesions may show obvious differences in one layer but not in another. Therefore, this scheme calculates the feature difference vectors of each layer network and fuses the differences of multiple layers to make the detection results more robust.

[0072] Since the spatial resolution of each layer's feature map is smaller than that of the original image, it is necessary to restore the difference information at these different scales to the original image size. Therefore, upsampling and aggregation methods are used.

[0073] Upsampling uses bilinear interpolation, transposed convolution, or simple nearest neighbor interpolation to upscale the feature difference map of each layer to the resolution of the original image. Aggregation merges the difference maps from multiple upsampled layers to output a single-channel perceptual saliency map the same size as the original fundus color image, where the value of each pixel represents the perceptual difference intensity at that location.

[0074] As an optional embodiment, step 103, generating a target anomaly mask based on the reconstructed residual map and the perceived saliency map, includes:

[0075] Step 1034: Perform point-by-point multiplication on the reconstructed residual map and the perceived saliency map to obtain the anomaly response map;

[0076] Step 1035: Perform threshold segmentation on the abnormal response map, and select the region with a gray value greater than or equal to the first preset threshold to obtain the initial abnormal mask;

[0077] Step 1036: Gaussian blur the initial anomaly mask, and binarize the Gaussian blurred soft mask according to the second preset threshold to obtain a target anomaly mask that can clearly distinguish the area to be repaired and the area to be preserved.

[0078] In steps 1034-1036, based on the reconstructed residual map Perceptual saliency map Calculate the abnormal response graph The method is as follows:

[0079]

[0080] in, This indicates point-by-point multiplication.

[0081] Anomaly response diagram Threshold segmentation is performed to obtain the initial anomaly mask. :

[0082]

[0083] in, Indicates the position in the initial anomaly mask The normalized gray value at that location, Indicates the location in the abnormal response diagram The normalized gray value refers to mapping the gray values ​​in the image to the interval [0, 1]. This represents the first preset threshold. You can choose a value based on experience, such as 0.3, 0.5, etc.

[0084] In addition, threshold segmentation can be performed by taking quantile values. For example, the high response region with the highest gray value (q%) in the abnormal response map can be taken as the initial abnormal mask.

[0085] Then, in order to make the initial anomalous mask smoother, more continuous, and more suitable for subsequent generative repair, the initial anomalous mask was modified. Gaussian blurring is applied, followed by hard thresholding to obtain the final anomaly mask. :

[0086]

[0087] in, The standard deviation is expressed as Gaussian blur operation, This represents the hard threshold function.

[0088] A hard thresholding function can forcibly convert a grayscale image or a continuous numerical image into a binary image according to a set threshold. If the pixel value at a certain location is greater than or equal to the set threshold, the image will be converted to a binary image. If the pixel value is less than a set threshold, the pixel value at that location is set to 1, indicating that the location belongs to an abnormal region; if the pixel value is less than a set threshold, the pixel value at that location is set to 1. If the pixel value is 0, then the pixel value is set to 0, indicating that the location belongs to the normal area.

[0089] In this patent, the hard threshold function serves to convert the soft mask after Gaussian blurring back into a clear binary anomaly mask, enabling the model to clearly distinguish which areas need repair and which areas should be preserved.

[0090] Step 104: Input the standardized fundus image and the target abnormality mask into the mask repair generation model to generate a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image.

[0091] Mask inpainting generation models can employ conditional GANs or image inpainting networks.

[0092] Specifically, a standardized fundus image and a target abnormality mask are input into a mask repair generation model. The model fills the area inside the mask with healthy textures outside the mask area to generate a healthy image without lesions.

[0093] This process can be represented as:

[0094]

[0095] in, denoted as a pseudo-healthy image, x represents a standardized fundus image, and M represents a target anomalous mask.

[0096] As an optional embodiment, the mask repair generation model adopts the AOT-GAN model.

[0097] The AOT-GAN model is trained on normal fundus images to learn the distribution of healthy fundus images. When inferring from an input image containing abnormal regions, the model reconstructs the texture and structure of the masked region according to the context information not covered by the mask, conforming to the healthy distribution, thus obtaining a pseudo-healthy image, which serves as an approximate counterfactual comparison of the input image in a lesion-free state.

[0098] As an optional embodiment, step 104 includes:

[0099] Step 1041: Combine the standardized fundus image with the target abnormality mask to input the mask and generate a repair model;

[0100] Step 1042: The model determines the region not covered by the mask in the standardized fundus image based on the target anomaly mask, thus obtaining the masked region and the non-masked region;

[0101] Step 1043: The model reconstructs the texture and structure of the non-masked region according to the context information of the masked region, which conforms to the healthy distribution, to obtain a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image.

[0102] In steps 1041-1043, the non-masked areas represent healthy, normal fundus tissue (such as the normal retina, blood vessels, and optic disc). The model retains the original pixels of these areas as a reference for generating new content.

[0103] The masked area is initially identified as a lesion area (such as hemorrhage points, exudates, drusen, etc.). The mask repair generation model discards the original pixels of this area and generates new, seemingly healthy tissue to replace it.

[0104] The mask restoration generation model uses convolutional networks or attention mechanisms to extract texture, color, and structural features of healthy tissue from the unmasked region. Based on the features of the surrounding healthy tissue, the model reasonably infers what the masked region should originally look like, reconstructs the texture and structure of the unmasked region in accordance with the healthy distribution, and outputs a pseudo-healthy image.

[0105] Step 105: Calculate the difference between the original fundus color image and the pseudo-healthy image to generate an abnormal area map corresponding to the lesion area.

[0106] Original image With pseudo-health images The images are compared, their differences are calculated, and an anomaly map is generated. The anomaly map can be a grayscale or color heatmap, highlighting the differences between the original image and the pseudo-healthy image; these differences represent the lesion areas.

[0107] As an optional embodiment, after generating the abnormal area map corresponding to the lesion area, the method further includes:

[0108] The lesion area is located based on the target anomaly mask.

[0109] In this embodiment of the invention, the target anomaly mask can be further used. This yields a more regular lesion area marker, i.e., the lesion area localization result.

[0110] In summary, this invention provides a method for detecting fundus image anomalies based on generative counterfactual reconstruction. The method includes: acquiring an original fundus color image to be detected, and performing standardized preprocessing on the original fundus color image to obtain a standardized fundus image; generating a coarse reconstruction image of the standardized fundus image using a coarse reconstruction network, and calculating a reconstruction residual map based on the standardized fundus image and the coarse reconstruction image; calculating a perceptual saliency map based on the original fundus color image and the coarse reconstruction image, and generating a target anomaly mask based on the reconstruction residual map and the perceptual saliency map; inputting the standardized fundus image and the target anomaly mask into a mask repair generation model to generate a pseudo-healthy image after removing suspected lesion areas from the standardized fundus image; calculating the difference between the original fundus color image and the pseudo-healthy image to generate an anomaly region map corresponding to the lesion area. This solution first generates a target anomaly mask by fusing coarse reconstruction and perceptual saliency to suppress false positives of normal structures; then, it generates a pseudo-healthy image that conforms to a healthy distribution based on mask repair, which can intuitively display the expected appearance of the image under normal conditions. This approach requires no lesion labeling, only training with healthy data, and can accurately locate lesion areas. It has high generalizability, interpretability, and clinical applicability, making it suitable for large-scale retinal disease screening.

[0111] Figure 2 This is a structural block diagram of a fundus image anomaly detection device based on generative counterfactual reconstruction, provided in an embodiment of the present invention.

[0112] like Figure 2 As shown, the device 200 includes:

[0113] The preprocessing module 201 is used to acquire the original fundus color image to be detected, and to perform standardized preprocessing on the original fundus color image to obtain a standardized fundus image.

[0114] The residual calculation module 202 is used to generate a coarse reconstruction image of the standardized fundus image using a coarse reconstruction network, and to calculate a reconstruction residual map based on the standardized fundus image and the coarse reconstruction image.

[0115] The mask generation module 203 is used to calculate a perceptual saliency map based on the original fundus color image and the coarse reconstructed image, and to generate a target anomaly mask based on the reconstructed residual map and the perceptual saliency map;

[0116] The pseudo-healthy image generation module 204 is used to input the standardized fundus image and the target abnormality mask into the mask repair generation model to generate a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image.

[0117] The lesion area calculation module 205 is used to calculate the difference between the original fundus color image and the pseudo-healthy image, and generate an abnormal area map corresponding to the lesion area.

[0118] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0119] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0120] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0121] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0122] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for detecting anomalies in fundus images based on generative counterfactual reconstruction, characterized in that, The method includes: The original fundus color image to be detected is acquired, and the original fundus color image is subjected to standardized preprocessing to obtain a standardized fundus image; A coarse reconstruction image of the standardized fundus image is generated using a coarse reconstruction network, and a reconstruction residual map is calculated based on the standardized fundus image and the coarse reconstruction image. A perceptual saliency map is calculated based on the original fundus color image and the coarse reconstructed image, and a target anomaly mask is generated based on the reconstructed residual map and the perceptual saliency map. The standardized fundus image and the target abnormality mask are input into the mask repair generation model to generate a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image. The difference between the original fundus color image and the pseudo-healthy image is calculated to generate an abnormal region map corresponding to the lesion area.

2. The method according to claim 1, characterized in that, The standardization preprocessing of the original fundus color image includes: Locate the target region to be detected from the original fundus color image and crop out the image of the target region; The target region image is scaled to a preset resolution, and image enhancement is performed on the target region image at the preset resolution to obtain an enhanced image; The image quality level of the enhanced image is determined, and images that meet the preset level standard are selected as standardized fundus images.

3. The method according to claim 1, characterized in that, The calculation of the perceptual saliency map based on the original fundus color image and the coarse reconstructed image includes: The original fundus color image and the coarse reconstructed image are input into the perceptual difference measurement model to calculate the feature difference vectors of the original fundus color image and the coarse reconstructed image in each layer of the network; The feature difference vector is mapped back to the spatial resolution of the original fundus color image by upsampling and aggregation to obtain a perceptual saliency map of the same size as the original fundus color image.

4. The method according to claim 1, characterized in that, The step of generating a target anomaly mask based on the reconstructed residual map and the perceptual saliency map includes: A point-by-point multiplication calculation is performed on the reconstructed residual map and the perceived saliency map to obtain the anomaly response map; The abnormal response map is segmented by thresholding, and the region with a gray value greater than or equal to a first preset threshold is selected to obtain the initial abnormal mask; The initial anomaly mask is Gaussian blurred, and the Gaussian blurred soft mask is binarized according to a second preset threshold to obtain a target anomaly mask that can clearly distinguish between the area to be repaired and the area to be preserved.

5. The method according to claim 1, characterized in that, The step of inputting the standardized fundus image and the target abnormality mask into the mask repair generation model to generate a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image includes: The standardized fundus image and the target abnormality mask are used as input mask repair to generate a model; The model determines the region not covered by the mask in the standardized fundus image based on the target anomaly mask, thus obtaining the masked region and the unmasked region; The model reconstructs the texture and structure of the non-masked region according to the context information of the masked region, conforming to a healthy distribution, to obtain a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image.

6. The method according to claim 1, characterized in that, After generating the abnormal region map corresponding to the lesion area, the following is also included: The lesion area is located based on the target anomaly mask.

7. The method according to claim 1, characterized in that, The mask repair generation model adopts the AOT-GAN model.

8. A device for detecting fundus image anomalies based on generative counterfactual reconstruction, characterized in that, The device includes: The preprocessing module is used to acquire the original fundus color image to be detected and to perform standardized preprocessing on the original fundus color image to obtain a standardized fundus image. The residual calculation module is used to generate a coarse reconstruction image of the standardized fundus image using a coarse reconstruction network, and to calculate a reconstruction residual map based on the standardized fundus image and the coarse reconstruction image. The mask generation module is used to calculate a perceptual saliency map based on the original fundus color image and the coarse reconstructed image, and to generate a target anomaly mask based on the reconstructed residual map and the perceptual saliency map; The pseudo-healthy image generation module is used to input the standardized fundus image and the target abnormality mask into the mask repair generation model to generate a pseudo-healthy image after removing the suspected lesion area from the standardized fundus image. The lesion area calculation module is used to calculate the difference between the original fundus color image and the pseudo-healthy image, and generate an abnormal area map corresponding to the lesion area.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the fundus image anomaly detection method based on generative counterfactual reconstruction as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or instruction set is loaded and executed by a processor to implement the fundus image anomaly detection method based on generative counterfactual reconstruction as described in any one of claims 1-7.

Images