Diabetes Complications Risk Assessment Method and System

Abstract

The present invention relates to the technical field of diabetes medical informatics, and discloses a method and system for assessing the risk of diabetes complications, including analyzing blood glucose indicators to identify diabetic positive patients; obtaining fundus images of the diabetic positive patients and identifying missing regions in the fundus images; taking the missing regions and the context regions of the missing regions as conditions, generating local restoration images based on a conditional generative adversarial network, and using the local restoration images to fill the missing regions to obtain effective fundus images; and assessing the risk of diabetes complications according to the effective fundus images. In the method and system of the present invention, a precise local restoration image is generated based on the missing regions and their context regions by using a conditional generative adversarial network, so as to fill the missing regions and generate a complete fundus image that matches the real image, ensuring the reliability and accuracy of assessing the complication risk according to the fundus image.

Description

Technical Field

[0001] The present invention relates to the technical field of diabetes medical informatics, and in particular to a method and system for assessing the risk of diabetes complications. Background Art

[0002] With the continuous increase in the number of diabetes patients, the effective management and risk assessment of diabetes and its complications have become particularly important; among them, retinopathy is a common and serious complication of diabetes, which may lead to blindness. Timely and accurate assessment of the retinal condition of diabetes patients is crucial for early intervention and treatment.

[0003] In recent years, the rapid development of artificial intelligence (AI) technology has provided a new approach for the risk assessment of diabetes complications. By analyzing fundus images, especially using machine learning and deep learning algorithms, doctors can quickly identify and evaluate the risk of retinopathy in diabetes patients. The information contained in fundus images is of great significance for judging the vascular health status and identifying lesion areas. Although existing technologies can already accurately identify various fundus lesions, in actual applications, due to shadows, reflections, or occlusion of the retinal structure, missing areas often appear in fundus images, thus affecting the accuracy of the overall assessment.

[0004] When facing the problem of missing areas in fundus images, traditional methods usually use interpolation techniques to fill in the missing parts of the images. The interpolation method calculates the relationship between the pixel values of the missing area and the surrounding areas, and uses mathematical formulas (such as linear interpolation, bilinear interpolation, etc.) to infer the pixel values of the missing area. This interpolation method can alleviate the impact of the missing area on the risk assessment to a certain extent. However, the interpolation method has certain limitations. Especially when dealing with fundus images with complex structures, the missing areas filled by interpolation cannot restore the fine structural features, and may lead to unnatural edges of the image or the generated image lacking a sense of reality, resulting in the distortion of the feature generation in the missing area, thus affecting the subsequent risk assessment of diabetes complications and causing deviations in the assessment results. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem of deviation in the risk assessment of complications caused by partial missing of fundus images in the prior art, and provide a method and system for assessing the risk of diabetes complications, which use a conditional generative adversarial network to generate accurate local restoration images based on the missing area and its context area, so as to fill in the missing area and generate a complete fundus image that matches the real image, ensuring the reliability and accuracy of assessing the risk of complications based on the fundus image.

[0006] In a first aspect, to solve the above technical problem, the present invention provides a method for assessing the risk of diabetes complications, including,

[0007] Analyze blood glucose indicators to identify diabetic positive patients;

[0008] Obtain the fundus images of the diabetic positive patients and identify the missing regions of the fundus images;

[0009] Using the missing regions and the context regions of the missing regions as conditions, generate local restoration images based on a conditional generative adversarial network, and use the local restoration images to fill the missing regions to obtain effective fundus images;

[0010] Evaluate the risk of diabetic complications according to the effective fundus images.

[0011] In one embodiment of the present invention, the conditional generative adversarial network includes a generator; generating local restoration images based on a conditional generative adversarial network includes performing mask marking on the missing regions to obtain mask information; inputting the fundus images with the mask information into the generator, performing a first-scale deconvolution operation on the missing regions according to the mask information to generate a main blood vessel feature map; performing a second-scale deconvolution operation on the main blood vessel feature map to generate a capillary feature map; performing a third-scale deconvolution operation on the capillary feature map to generate a microvessel feature map; fusing the main blood vessel feature map, the capillary feature map, and the microvessel feature map to generate the local restoration image.

[0012] In one embodiment of the present invention, any two of the main blood vessel feature map, the capillary feature map, and the microvessel feature map are skip-connected, and the feature maps after skip-connection are fused to generate the local restoration image.

[0013] In one embodiment of the present invention, fusing the main blood vessel features, capillary features, and microvessel features to generate the local restoration image includes respectively assigning weights to the main blood vessel feature map, the capillary feature map, and the microvessel feature map; performing a weighted operation according to their respective weights, and performing a splicing operation on the weighted main blood vessel feature map, capillary feature map, and microvessel feature map along the dimension of the feature channel to obtain a spliced feature map; skip-connecting the main blood vessel feature map and the microvessel feature map, and skip-connecting the microvessel feature map and the main blood vessel feature map on the spliced feature map; connecting the spliced feature map to a Laplacian convolutional layer to generate the local restoration image.

[0014] In one embodiment of the present invention, obtaining the spliced feature map further includes skip-connecting the capillary feature map and the microvessel feature map, and skip-connecting the microvessel feature map and the capillary feature map.

[0015] In one embodiment of the present invention, an effective fundus image is obtained by using the locally restored image to fill the missing area, including generating a gradient mask according to the boundary of the missing area; wherein, the mask value of the gradient mask gradually changes from 0 to 1; performing a pixel-by-pixel weighted average calculation on the locally restored image and the missing area context region image by using the gradient mask to obtain boundary connection information; and filling the missing area with the locally restored image according to the boundary connection information.

[0016] In one embodiment of the present invention, a gradient mask is generated according to the following formula:

[0017] ;

[0018] wherein, M(x) represents the value of the gradient mask; represents the center position of the boundary of the missing area; represents the standard deviation, which is used to control the width of the gradient; x represents the spatial position of the mask.

[0019] In one embodiment of the present invention, analyzing blood glucose indicators to identify diabetic positive patients includes obtaining the blood glucose value and glycated hemoglobin test value of a user; if the blood glucose value is greater than the blood glucose value threshold, then comparing the glycated hemoglobin test value with the test value threshold; if the glycated hemoglobin test value is greater than or equal to the test value threshold, then determining that the user is a diabetic positive patient.

[0020] In one embodiment of the present invention, evaluating the risk of diabetic complications based on the effective fundus image includes training a neural network model with a complete fundus image sample set as the input and a diabetic complication probability sample set as the output until the convergence condition is met to obtain a complication risk assessment model; inputting the effective fundus image into the complication risk assessment model to obtain a complication probability.

[0021] In a second aspect, based on the same inventive concept, to solve the above technical problems, the present invention further provides a diabetic complication risk assessment system for performing the diabetic complication risk assessment method, including,

[0022] A blood glucose index analysis module for analyzing blood glucose indicators to identify diabetic positive patients;

[0023] A fundus image processing module for obtaining the fundus image of the diabetic positive patient and identifying the missing area of the fundus image;

[0024] An effective fundus image generation module for generating a locally restored image according to conditions by using a conditional generative adversarial network with the missing area and the context region of the missing area as conditions, and obtaining an effective fundus image by using the locally restored image to fill the missing area;

[0025] The complication risk assessment module is used to assess the risk of diabetic complications based on the effective fundus image.

[0026] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0027] For the method and system for assessing the risk of diabetic complications of the present invention, a conditional generative adversarial network is used to generate a precise local restored image based on the missing area and its context area, thereby filling the missing area and generating a complete fundus image that matches the real image, so as to assess the complication risk according to the complete fundus image, achieving the purpose of ensuring the reliability and accuracy of assessing the complication risk based on the fundus image. Description of the Drawings

[0028] In order to make the content of the present invention easier to be clearly understood, the following further details the present invention according to the specific embodiments of the present invention and in conjunction with the drawings, wherein,

[0029] Figure 1 is a schematic flowchart of the method for assessing the risk of diabetic complications in the preferred embodiment of the present invention;

[0030] Figure 2 is a schematic flowchart of generating a local restored image in the preferred embodiment of the present invention;

[0031] Figure 3 is a schematic flowchart of fusing and generating the local restored image in the preferred embodiment of the present invention;

[0032] Figure 4 is a schematic flowchart of using the local restored image to fill the missing area to obtain an effective fundus image in the preferred embodiment of the present invention;

[0033] Figure 5 is a structural block diagram of the system for assessing the risk of diabetic complications in the preferred embodiment of the present invention. Detailed Embodiments

[0034] The following further illustrates the present invention in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the examples given are not intended to limit the present invention. Embodiment 1

[0035] Refer to Figure 1As shown in the figure, an embodiment of the present invention discloses a method for assessing the risk of diabetic complications, including: S100, analyzing blood glucose indicators to identify diabetic positive patients; S200, obtaining fundus images of the diabetic positive patients and identifying the missing regions in the fundus images; S300, taking the missing regions and the context regions of the missing regions as conditions, generating local restoration images based on a conditional generative adversarial network, and using the local restoration images to fill the missing regions to obtain effective fundus images; S400, evaluating the risk of diabetic complications according to the effective fundus images.

[0036] The method for assessing the risk of diabetic complications according to the present invention utilizes a conditional generative adversarial network to generate accurate local restoration images based on the missing regions and their context regions, thereby filling the missing regions and generating a complete fundus image (or effective fundus image) that matches the real image. Thus, the risk of complications is evaluated according to the complete fundus image, achieving the purpose of ensuring the reliability and accuracy of assessing the risk of complications based on the fundus image.

[0037] In a specific application scenario, S100 analyzes blood glucose indicators to identify diabetic positive patients, including obtaining the fasting blood glucose value, postprandial blood glucose value, and hemoglobin test value (HbA1c) of the user through blood glucose detection and glycated hemoglobin detection. The fasting blood glucose value, postprandial blood glucose value, and hemoglobin test value constitute blood glucose indicators. The fasting blood glucose value and postprandial blood glucose value reflect the blood glucose level and are basic indicators for diabetes management. Long-term hyperglycemia can lead to diabetes. By monitoring the fasting blood glucose and postprandial blood glucose levels, it is possible to evaluate whether the patient's blood glucose control is stable and detect in advance the risk of potential complications. The hemoglobin test value reflects the average blood glucose level in the past 2-3 months. A high hemoglobin test value means long-term hyperglycemia. The goal of diabetic patients is usually to control HbA1c below 7%. An excessive HbA1c may indicate an increased risk of complications such as fundus lesions and nephropathy. Monitoring the HbA1c level helps determine whether the patient has problems in controlling blood glucose, thus providing early warnings.

[0038] Specifically, analyzing blood glucose indicators to identify diabetic positive patients includes comparing the blood glucose value with the blood glucose value threshold. If the blood glucose value is greater than the blood glucose value threshold, then compare the glycated hemoglobin test value with the test value threshold; if the glycated hemoglobin test value is greater than or equal to the test value threshold, then determine that the user is a diabetic positive patient. In specific application scenarios, both the blood glucose value threshold and the glycated hemoglobin test value threshold are set according to medical standards. For example, the fasting blood glucose threshold is set at 7.0 mmol / L (126 mg / dL), and the 2-hour postprandial blood glucose threshold is set at 11.1 mmol / L (200 mg / dL); the glycated hemoglobin test value threshold is set at 6.5%; by comparing the blood glucose value with the blood glucose value threshold and the glycated hemoglobin test value with the test value threshold, diabetic positive patients can be accurately identified.

[0039] S200 Obtain the fundus image of the diabetic positive patient and identify the missing area of the fundus image; in specific applications, patients identified as diabetic positive need to regularly check their eyes according to clinical reactions to prevent diabetic retinopathy. The fundus images of diabetic positive patients can be obtained by means of a fundus camera, a fluorescence fundus angiography camera FFA, an optical coherence tomography OCT, etc. The shooting angle and shooting environment (such as appropriate lighting conditions) directly affect the quality of the fundus image, because it involves the coverage of the retina, the focal length, color, contrast, lighting uniformity of the image, as well as the display effect of blood vessels and lesions. Improper shooting may not only lead to blurred images and loss of details, but also block key lesion areas, increase image noise, and reduce the analysis and recognition accuracy of doctors or machine learning models for fundus images.

[0040] Among them, the loss of details will seriously affect the reliability and accuracy of subsequent complication risk assessment. Especially when the lost details involve the lesion area, it will directly cause misjudgment of risk assessment.

[0041] After obtaining the fundus image of the diabetic positive patient, the missing area can be identified based on a pre-constructed and trained neural network model, or the missing area can be obtained according to the manual marking of doctors. After locating the missing area, it provides basic data for subsequent local image restoration.

[0042] Based on the missing region and the context region of the missing region, the S300 generates a local restored image using a conditional generative adversarial network, and uses the local restored image to fill the missing region to obtain a valid fundus image; the S400 evaluates the risk of diabetic complications based on the valid fundus image. In a specific application scenario, the valid fundus image is analyzed by means of a machine learning algorithm to finally obtain the risk of diabetic complications; specifically, a large number of sample data are obtained, including a complete fundus image sample set and a corresponding diabetic complication probability sample set, and a neural network model is trained with the complete fundus image sample set as the input and the diabetic complication probability sample set as the output until the convergence condition is met to obtain a complication risk assessment model; the valid fundus image is input into the complication risk assessment model to obtain the complication probability.

[0043] In the implementation scheme of the present invention, a conditional generative adversarial network is used to generate a local restored image to fill the missing region. The conditional generative adversarial network includes a generator and a discriminator. A random noise and a fundus image with a missing region are input into the generator, and the generator generates a local restored image with the missing region and the context region of the missing region as conditions; a real image and the local restored image generated by the generator are input into the discriminator, and the discriminator outputs a probability value indicating whether the local restored image is a real image, and this result is fed back and affects the generator. Through adversarial training, the generator is continuously optimized to generate a more real local restored image to fill the missing region.

[0044] In the implementation scheme of the present invention, a deep convolutional conditional generative adversarial network is preferably used; referring to Figure 2 As shown, generating a local restored image based on a conditional generative adversarial network includes performing a mask marking on the missing region to obtain mask information; inputting the fundus image with the mask information into the generator, and performing a first-scale transposed convolution operation on the missing region according to the mask information to generate a main blood vessel feature map; performing a second-scale transposed convolution operation on the main blood vessel feature map to generate a capillary feature map; performing a third-scale transposed convolution operation on the capillary feature map to generate a microvessel feature map; fusing the main blood vessel feature map, the capillary feature map and the microvessel feature map to generate the local restored image.

[0045] In a specific application scenario, the missing region in the fundus image is first marked. Usually, the mask of the missing region is a binary image, the missing region is marked as 1, and the known region is marked as 0; the fundus image and the mask information are input into the generator. Through the mask information, the generator can clearly know which regions are missing and which regions are known. The mask information is crucial for the subsequent generation of the local restored image. The mask provides a guide for the generator to restore the image, ensuring that the generated missing region is coherent with the context region.

[0046] In the generator, first, a first-scale transposed convolution operation is performed. The transposed convolution operation is carried out using a relatively large convolution kernel (7×7) to obtain a main blood vessel feature map. This feature map captures the structural features of larger blood vessels. The generated main blood vessel feature map provides the large-scale structural information in the locally restored image. The restoration of the main blood vessels provides the global context for the subsequent restoration of finer blood vessels.

[0047] Based on the first scale, a second-scale transposed convolution operation is continued. Upsampling is performed using a smaller convolution kernel (5×5). At this stage, the generator restores the capillary feature map, which is responsible for restoring finer blood vessel structures, especially tiny capillary branches. The generated capillary feature map captures the features of medium-sized blood vessels. The restoration of capillaries can ensure the natural transition of finer blood vessels in the locally restored image and help prevent the loss of important information when generating the microvascular feature map. At the same time, the generation of the capillary feature map helps to refine the blood vessel branch structure, making the locally restored image have more local details.

[0048] Based on the second scale, a third-scale transposed convolution operation is performed. Refinement is carried out using an even smaller convolution kernel (such as 3×3 or 1x1). This stage is mainly used to restore the microvascular feature map, that is, the smallest blood vessel branches and their details in the image. The generated microvascular feature map contains the details of the smallest blood vessel branches in the image. Especially for microvascular lesions caused by diabetes, the generator can capture subtle features such as the curvature and crossing of blood vessels. The restoration of the microvascular feature map is crucial for the fine repair of fundus images, especially for retaining the real blood vessel details when locally restoring the image.

[0049] Considering the fine and complex characteristics of the fundus image structure, the generator sets up a multi-scale transposed convolution operation. The generation of the main blood vessel feature map provides the infrastructure for the structure of the entire image, ensuring that the general direction and overall layout of the blood vessels are consistent. The generation of the capillary feature map refines the blood vessel branch structure, enabling the image to be restored in terms of medium-scale details. The generation of the microvascular feature map restores the fine features of microvessels, ensuring that the details of the smallest blood vessels in the image are not lost. Through the multi-scale transposed convolution operation and feature map fusion, the finally generated locally restored image not only has global consistency but also has its local details accurately restored. Especially in the restoration of complex structures such as diabetic retinopathy, it shows remarkable effects.

[0050] As a further improvement of the embodiment of the present invention, any two of the main blood vessel feature map, the capillary feature map, and the microvascular feature map are skip-connected, and the feature maps after the skip connection are fused to generate the locally restored image.

[0051] It should be noted that a skip connection is a structure that enables the output of a network at a certain layer to be directly transmitted to deeper layers. That is to say, skip connections do not transmit information layer by layer, but directly transmit the feature map (or activation value) of a certain layer to subsequent layers of the network through "skipping"; skip connections enable the network to better preserve detailed information, especially those local details that may be lost in deep networks. In the implementation scheme of the present invention, skip connections are used in the generator and discriminator to ensure that the generated images not only have a consistent global structure, but also the details and local features are restored.

[0052] In the implementation scheme of the present invention, the task of generating locally restored images is heavy. The main blood vessel feature map, capillary feature map, and microvessel feature map represent different levels of information about the blood vessel structure in fundus images: the main blood vessel feature map mainly represents thick blood vessel structures and is responsible for providing the general structure and orientation in the image; the capillary feature map represents thinner blood vessel branches, refining the blood vessel structure and usually involving larger local areas; the microvessel feature map represents the tiny blood vessel branches and details in the image, mainly involving the blood vessel structure at the microscopic scale. Through skip connections, these feature maps at different levels can be fused, thus producing a more refined and consistent image restoration effect.

[0053] In one implementation scheme, the main blood vessel feature map is skip-connected to the capillary feature map. The main blood vessel feature map provides the rough shape and global structure of the blood vessels, while the capillary feature map refines the branch structure of the blood vessels; skip-connecting the two allows the network to not only consider the local small blood vessel features when restoring details, but also maintain the global blood vessel layout. By skip-connecting and considering the information of both at the same time, the blood vessel structure of the generated image is more coherent, avoiding unnatural seams caused by the loss of local details or the neglect of large structures.

[0054] In another implementation scheme, the main blood vessel feature map is skip-connected to the microvessel feature map. The main blood vessel feature map focuses on the shape of thicker blood vessels, while the microvessel feature map is responsible for restoring very tiny blood vessel branches; skip-connecting the two helps to maintain the overall structural consistency of the main blood vessels when restoring microvessels, so that the generation of microvessels does not deviate from the entire blood vessel network. At the same time, skip connections help prevent local detail distortion when restoring fine blood vessels, ensuring a natural and smooth transition between fine blood vessels and main blood vessels.

[0055] In another embodiment, the capillary feature map and the microvessel feature map are skip-connected. The capillary feature map refines the branches of larger blood vessels, while the microvessel feature map focuses on smaller blood vessels and details. Skipping the connection between the two can ensure a smoother transition from the branches of larger blood vessels to microvessels. The skip connection ensures a natural and continuous transition from capillaries to microvessels, so that there are no abrupt seams in the branches and fine structures of blood vessels. The combination of the microvessel feature map and the capillary feature map can carefully restore the tiny blood vessels in the image while maintaining the continuity of the blood vessel structure.

[0056] In summary, skip-connecting feature maps at different levels can integrate their respective advantages and enhance the complementarity of features; at the same time, when generating a local restored image, the network pays attention to both local details and the global structure, improves the consistency between the local and global structures, and avoids problems such as detail loss and incoherence.

[0057] Further, as shown in Figure 3 generating the local restored image by fusing the main blood vessel features, capillary features, and microvessel features includes respectively assigning weights to the main blood vessel feature map, capillary feature map, and microvessel feature map; performing a weighted operation according to their respective weights, and performing a splicing operation on the weighted main blood vessel feature map, capillary feature map, and microvessel feature map along the dimension of the feature channel to obtain a spliced feature map; skip-connecting the main blood vessel feature map and the microvessel feature map on the spliced feature map, and skip-connecting the microvessel feature map and the main blood vessel feature map; and connecting the spliced feature map to a Laplacian convolutional layer to generate the local restored image.

[0058] In a specific application scenario, different weights are respectively assigned to the three generated feature maps (the main blood vessel feature map, capillary feature map, and microvessel feature map); the weights can be determined through experiments. Usually, the weight of the main blood vessel feature map is larger because it provides the global structure of the image, while the weights of the capillary and microvessel feature maps are smaller because they refine local details; the setting of these weights can be determined through an optimization algorithm or manual adjustment to achieve the optimal image restoration effect.

[0059] The weighted operation ensures that the contribution degree of each feature map is adjusted according to its importance; the splicing operation enables the generator to simultaneously utilize blood vessel information at different scales (thick main blood vessels, thin capillaries, and microvessels) to generate a more complete and detailed image.

[0060] Perform skip connection operations on the spliced feature map to connect the main blood vessel feature map and the micro blood vessel feature map, and further connect the micro blood vessel feature map and the main blood vessel feature map, so that the generated image maintains a good balance between local details and overall structure; better transition the edge between the missing area and the known area, making the image smoother and more natural, and avoiding abrupt seams.

[0061] Input the spliced feature map into the Laplacian convolutional layer to enhance the edges of the image; the Laplacian convolutional layer can detect high-frequency details in the image (such as the edges of blood vessels), and improve the sharpness and contrast of the image through convolutional operations, making the blood vessels, micro blood vessels, and detailed parts in the image clearer, and avoiding blurring or distortion in the detailed parts of the generated image. The Laplacian convolution also helps to smooth the image edges, making the transition between the restored area and the context area more natural, and avoiding discontinuous seams.

[0062] Furthermore, obtaining the spliced feature map also includes skip connecting the capillary feature map and the micro blood vessel feature map, and skip connecting the micro blood vessel feature map and the capillary feature map.

[0063] In a specific application scenario, through skip connection on the spliced feature map, connect the capillary feature map and the micro blood vessel feature map together, and further reversely skip connect the micro blood vessel feature map and the capillary feature map; skip connection can combine the finer micro blood vessel details with the local information of capillaries, combine the finer micro blood vessel details with the local information of capillaries, and the restoration of details is more accurate at this stage; the details of the micro blood vessels will be reversely fed back to the capillary feature map through skip connection to ensure that no details are lost in the generated image. These two skip connections will jointly provide multi-level detail information for the generated image. Especially when restoring the branches of small blood vessels, the features of capillaries and micro blood vessels will be well preserved and fused.

[0064] In a further improvement of the implementation of the present invention, refer to Figure 4 As shown, use the local restored image to fill the missing area to obtain an effective fundus image, including generating a gradient mask according to the boundary of the missing area; wherein, the mask value of the gradient mask gradually changes from 0 to 1; use the gradient mask to perform pixel-by-pixel weighted average calculation on the local restored image and the image of the context area of the missing area to obtain boundary connection information; the local restored image fills the missing area according to the boundary connection information. The method of filling the local restored image based on the gradient mask and pixel-by-pixel weighted average calculation can significantly improve the restoration quality of the image, ensuring the accurate restoration of the details and structure of the image.

[0065] In a specific application scenario, the solution of this embodiment uses a gradient mask and pixel-by-pixel weighted averaging calculation to fuse the locally restored image with the image in the context area, gradually filling the missing area to generate an effective fundus image. First, a gradient mask is generated according to the boundary of the missing area. The mask value of the gradient mask will gradually change from 0 to 1, where 0 represents completely transparent and 1 represents completely filled, and the values in the transition area are between 0 and 1. The gradual change of the mask value from 0 to 1 reduces the artificial traces when filling the missing area through a smooth transition. The mask value in the edge area gradually increases from 0 to 1 to ensure a smooth transition between the filled area and the surrounding context area.

[0066] For the generated locally restored image and the image of the context area of the missing area, pixel-by-pixel weighted averaging calculation is performed using the gradient mask. The value of each pixel is weighted and calculated by the information of the context area and the restored image through the mask value: for each pixel in the missing area, the locally restored image and the image of the context area are weighted and averaged using the gradient mask, so that the boundary area gradually transitions to the complete area. At the center of the missing area, the weight of the locally restored image is 1, while the weight of the context area is close to 0. At the boundary area, the weights of the locally restored image and the context area are gradually balanced. The weighted averaging operation can ensure a smooth transition between the filled area and the surrounding area. Under the guidance of the mask, the boundary part in the image is naturally filled, avoiding artificial traces or abrupt seams.

[0067] During the filling process, the locally restored image is seamlessly combined with the context area through boundary connection information, so that there is no obvious dividing line between the restored area and the known area. The generated effective fundus image can seamlessly fill the missing area, and the filled part is completely consistent with the structure and texture of the known area. The boundary of the filled area has a natural transition without abrupt seams. The details and structure of the filled image are restored. Especially when restoring tiny blood vessels and complex microvessels, the details are retained and highly consistent with the surrounding area, while maintaining the coherence of complex details such as blood vessels and retinal layers.

[0068] Specifically, the gradient mask is generated according to the following formula:

[0069] ;

[0070] where M(x) represents the value of the gradient mask; represents the central position of the boundary of the missing area; represents the standard deviation, which is used to control the width of the gradient; x represents the spatial position of the mask.

[0071] The closer x is to , the closer the mask value M(x) is to 1, indicating that the pixel at this position needs to be completely filled; the farther x is from When it is farther away, the mask value M(x) is closer to 0, indicating that the pixels at this position are closer to the missing area and the filling intensity is lower. A Gaussian function is used to generate a smooth gradient mask, making the gradient more natural and avoiding overly abrupt boundaries, better simulating natural transitions. Embodiment 2

[0072] Based on the same inventive concept, the present invention discloses a diabetes complication risk assessment system for performing a diabetes complication risk assessment method, including

[0073] A blood glucose index analysis module for analyzing blood glucose indices to identify diabetic positive patients;

[0074] An fundus image processing module for obtaining the fundus image of the diabetic positive patient and identifying the missing area of the fundus image;

[0075] An effective fundus image generation module for generating a local restoration image by a conditional generative adversarial network based on the missing area and the context area of the missing area, and using the local restoration image to fill the missing area to obtain an effective fundus image;

[0076] A complication risk assessment module for assessing the diabetes complication risk according to the effective fundus image.

[0077] The diabetes complication risk assessment system of the embodiment of the present invention for performing the diabetes complication risk assessment method has the same technical effects as above and will not be elaborated here.

[0078] In summary, the diabetes complication risk assessment method and system of the present invention utilize a conditional generative adversarial network to generate accurate local restoration images based on the missing area and its context area, thereby filling the missing area and generating a complete fundus image that matches the real image, so as to assess the complication risk according to the complete fundus image, achieving the purpose of ensuring the reliability and accuracy of assessing the complication risk according to the fundus image.

[0079] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in one or more flows and / or one or more blocks in the flowcharts and / or block diagrams. Figure 1 in one or more flows and / or one or more blocks Figure 1 in one or more blocks

[0081] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction means that implements the functions specified in one or more flows and / or one or more blocks Figure 1 in one or more flows and / or one or more blocks Figure 1 in one or more blocks

[0082] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows and / or one or more blocks Figure 1 in one or more flows and / or one or more blocks Figure 1 in one or more blocks

[0083] Obviously, the above embodiments are merely examples for clear illustration and are not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all implementation manners here. And the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A method for assessing the risk of diabetic complications, characterized in that: include, Analyze blood sugar indexes to identify diabetes-positive patients; Acquire a fundus image of the diabetes-positive patient, and identify missing areas of the fundus image; Taking the missing area and the context area of ​​the missing area as conditions, generating a local restored image based on a conditional generative adversarial network, and using the local restored image to fill the missing area to obtain a valid fundus image; assessing the risk of diabetic complications based on the effective fundus images; Among them, the local restoration image is generated based on the conditional generative adversarial network, including: Performing mask marking on the missing area to obtain mask information; Inputting the fundus image with the mask information into a generator, performing a first-scale deconvolution operation on the missing area according to the mask information, and generating a main vessel feature map; Performing a second-scale deconvolution operation on the trunk blood vessel characteristic map to generate a capillary blood vessel characteristic map; Performing a third-scale deconvolution operation on the capillary feature map to generate a microvessel feature map; The local restored image is generated by fusing the trunk blood vessel characteristic map, the capillary blood vessel characteristic map and the microvascular characteristic map.

2. The method for assessing the risk of diabetes complications according to claim 1, characterized in that: Any two of the trunk blood vessel feature map, the capillary blood vessel feature map and the microvascular feature map are jump-connected, and the feature maps after the jump connection are fused to generate the local restored image.

3. The method for assessing the risk of diabetes complications according to claim 1 or 2, characterized in that: The local restoration image is generated by fusing the trunk blood vessel characteristic map, the capillary blood vessel characteristic map and the microvascular characteristic map, including: Assigning weights to the trunk blood vessel characteristic map, the capillary blood vessel characteristic map and the microvascular characteristic map respectively; A weighted operation is performed according to the respective weights, and a splicing operation is performed on the trunk blood vessel feature map, the capillary blood vessel feature map, and the microvascular feature map after the weighted operation along the dimension of the feature channel to obtain a spliced ​​feature map; Jump-connecting the trunk blood vessel feature map and the microvascular feature map on the spliced ​​feature map, and jump-connecting the microvascular feature map and the trunk blood vessel feature map; The spliced ​​feature map is connected to a Laplacian convolution device to generate the local restored image.

4. The method for assessing the risk of diabetes complications according to claim 3, characterized in that: Obtaining the splicing feature map also includes skipping connecting the capillary feature map and the microvascular feature map, and skipping connecting the microvascular feature map and the capillary feature map.

5. The method for assessing the risk of diabetes complications according to claim 1, characterized in that: Filling the missing area with the local restored image to obtain a valid fundus image includes: Generate a gradient mask according to the boundary of the missing area; wherein the mask value of the gradient mask gradually changes from 0 to 1; Using the gradient mask, performing pixel-by-pixel weighted average calculation on the local restoration image and the missing region context region image to obtain boundary connection information; The local restored image fills the missing area according to the boundary connection information.

6. The method for assessing the risk of diabetes complications according to claim 5, characterized in that: The gradient mask is generated according to the following formula: ； Where M(x) represents the gradient mask value; Indicates the center position of the boundary of the missing area; represents the standard deviation, which is used to control the width of the gradient; x represents the spatial position of the mask.

7. The method for assessing the risk of diabetes complications according to claim 1, characterized in that: Analyze blood sugar indicators to identify diabetes-positive patients, including, Obtain the user's blood sugar and glycosylated hemoglobin test values; If the blood glucose value is greater than the blood glucose value threshold, comparing the glycated hemoglobin detection value with the detection value threshold; If the glycated hemoglobin test value is greater than or equal to the test value threshold, the user is determined to be a diabetes-positive patient.

8. The method for assessing the risk of diabetes complications according to claim 1, characterized in that: Assessing the risk of diabetic complications based on the effective fundus image includes: The neural network model is trained with the complete fundus image sample set as input and the diabetic complication probability sample set as output until the convergence condition is met to obtain the complication risk assessment model; The effective fundus image is input into the complication risk assessment model to obtain the complication probability.

9. A diabetes complication risk assessment system, used to implement the diabetes complication risk assessment method according to any one of claims 1 to 8, characterized in that: include, Blood sugar index analysis module, which analyzes blood sugar index to identify diabetes-positive patients; A fundus image processing module, which obtains the fundus image of the diabetes-positive patient and identifies the missing area of ​​the fundus image; An effective fundus image generation module, taking the missing area and the context area of ​​the missing area as conditions, generates a local restoration image according to a conditional generative adversarial network, and uses the local restoration image to fill the missing area to obtain an effective fundus image; A complication risk assessment module is used to assess the risk of diabetic complications based on the effective fundus image.

Images