Diabetic complication risk assessment method and system

Abstract

The invention relates to the technical field of diabetes mellitus medical informatics, and discloses a diabetes mellitus complication risk assessment method and system, and the method comprises the steps: analyzing a blood glucose index to recognize a diabetes mellitus positive patient; acquiring an eye fundus image of the diabetes positive patient, and identifying a missing region of the eye fundus image; taking the missing region and the context region of the missing region as conditions, generating a local recovery image based on a conditional generative adversarial network, and filling the missing region with the local recovery image to obtain an effective fundus image; and evaluating the risk of diabetic complications according to the effective fundus image. According to the method and the system, the accurate local recovery image is generated on the basis of the missing region and the context region of the missing region by using the conditional generative adversarial network, so that the missing region is filled, the complete fundus image matched with the real image is generated, and the reliability and the accuracy of evaluating the complication risk according to the fundus image are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of diabetes medical informatics, and in particular to a method and system for assessing the risk of diabetes complications. Background Art

[0002] As the number of patients with diabetes continues to increase, effective management and risk assessment of diabetes and its complications have become particularly important. Among them, retinopathy is a common and serious complication of diabetes that may lead to blindness. Timely and accurate assessment of the retinal condition of diabetic patients is crucial for early intervention and treatment.

[0003] In recent years, the rapid development of artificial intelligence (AI) technology has provided a new way to assess the risk of diabetic complications. By analyzing fundus images, especially using machine learning and deep learning algorithms, doctors can quickly identify and assess the risk of retinopathy in diabetic patients. The information contained in fundus images is important for judging the health of blood vessels and identifying lesion areas. Although existing technologies can accurately identify a variety of fundus lesions, in actual applications, fundus images often have missing areas due to shadows, reflections or occlusion of retinal structures, thus affecting the accuracy of the overall assessment.

[0004] When faced with the problem of missing areas in fundus images, traditional methods usually use interpolation technology to fill in missing images. The interpolation method calculates the relationship between the pixel values ​​of the missing area and the surrounding area, and uses mathematical formulas (such as linear interpolation, bilinear interpolation, etc.) to infer the pixel values ​​of the missing area. This interpolation method can alleviate the impact of missing areas on risk assessment to a certain extent, but the interpolation method has certain limitations, especially when dealing with fundus images with complex structures. The missing areas filled by interpolation cannot restore fine structural features, and may cause the edges of the image to be unnatural or the generated image to lack realism, resulting in distortion of the features of the missing area, thereby affecting the subsequent risk assessment of diabetes complications and causing deviations in the assessment results. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the problem of biased complication risk assessment caused by partial missing fundus images in the prior art, and to provide a method and system for assessing the risk of diabetes complications, which utilizes a conditional generative adversarial network to generate an accurate local restored image based on the missing area and its context area, thereby filling the missing area and generating a complete fundus image that matches the real image, thereby ensuring the reliability and accuracy of assessing the risk of complications based on the fundus image.

[0006] In a first aspect, in order to solve the above technical problems, the present invention provides a method for assessing the risk of diabetic complications, comprising: Analyze blood sugar indexes to identify diabetes-positive patients; Acquire a fundus image of the diabetes-positive patient, and identify missing areas of the fundus image; Taking the missing area and the context area of ​​the missing area as conditions, generating a local restored image based on a conditional generative adversarial network, and using the local restored image to fill the missing area to obtain a valid fundus image; The risk of diabetic complications is assessed based on the effective fundus image.

[0007] In one embodiment of the present invention, the conditional generative adversarial network includes a generator; generating a local restored image based on the conditional generative adversarial network, including performing mask marking on the missing area to obtain mask information; inputting the fundus image with the mask information into the generator, performing a first scale deconvolution operation on the missing area according to the mask information to generate a trunk vessel feature map; performing a second scale deconvolution operation on the trunk vessel feature map to generate a capillary feature map; performing a third scale deconvolution operation on the capillary feature map to generate a microvessel feature map; and fusing the trunk vessel feature map, the capillary feature map and the microvessel feature map to generate the local restored image.

[0008] In one embodiment of the present invention, any two of the trunk blood vessel feature map, the capillary blood vessel feature map and the microvascular feature map are jump-connected, and the feature maps after the jump connection are fused to generate the local restored image.

[0009] In one embodiment of the present invention, the main vessel features, capillary features and microvascular features are fused to generate the local restored image, including assigning weights to the main vessel feature map, capillary feature map and microvascular feature map respectively; performing weighted operations according to their respective weights, and performing splicing operations on the weighted main vessel feature map, capillary feature map and microvascular feature map along the dimension of the feature channel to obtain a spliced ​​feature map; jump-connecting the main vessel feature map and the microvascular feature map on the spliced ​​feature map, and jump-connecting the microvascular feature map and the main vessel feature map on the spliced ​​feature map; connecting the spliced ​​feature map to a Laplacian convolution device to generate the local restored image.

[0010] In one embodiment of the present invention, obtaining the splicing feature map further includes skipping the capillary feature map and the microvessel feature map, and skipping the microvessel feature map and the capillary feature map.

[0011] In one embodiment of the present invention, the missing area is filled with the local restored image to obtain a valid fundus image, including generating a gradient mask according to the boundary of the missing area; wherein the mask value of the gradient mask gradually changes from 0 to 1; using the gradient mask to perform pixel-by-pixel weighted averaging calculation on the local restored image and the context area image of the missing area to obtain boundary connection information; the local restored image fills the missing area according to the boundary connection information.

[0012] In one embodiment of the present invention, the gradient mask is generated according to the following formula: ; Where M(x) represents the value of the gradient mask; Indicates the center position of the boundary of the missing area; represents the standard deviation, which is used to control the width of the gradient; x represents the spatial position of the mask.

[0013] In one embodiment of the present invention, blood sugar indicators are analyzed to identify diabetes-positive patients, including obtaining the user's blood sugar value and glycosylated hemoglobin test value; if the blood sugar value is greater than the blood sugar value threshold, comparing the glycosylated hemoglobin test value with the test value threshold; if the glycosylated hemoglobin test value is greater than or equal to the test value threshold, determining that the user is a diabetes-positive patient.

[0014] In one embodiment of the present invention, the risk of diabetic complications is assessed based on the effective fundus image, including training a neural network model with a complete fundus image sample set as input and a diabetic complication probability sample set as output until a convergence condition is met to obtain a complication risk assessment model; and inputting the effective fundus image into the complication risk assessment model to obtain the complication probability.

[0015] In the second aspect, based on the same inventive concept, in order to solve the above technical problems, the present invention also provides a diabetes complication risk assessment system for executing a diabetes complication risk assessment method, comprising: Blood sugar index analysis module, used to analyze blood sugar index to identify diabetes-positive patients; A fundus image processing module, used for acquiring the fundus image of the diabetes-positive patient and identifying the missing area of ​​the fundus image; An effective fundus image generation module is used to generate a local restoration image based on a conditional generative adversarial network with the missing area and the context area of ​​the missing area as conditions, and use the local restoration image to fill the missing area to obtain an effective fundus image; A complication risk assessment module is used to assess the risk of diabetic complications based on the effective fundus image.

[0016] The above technical solution of the present invention has the following beneficial effects compared with the prior art: The diabetes complication risk assessment method and system described in the present invention utilize a conditional generative adversarial network to generate an accurate local restoration image based on the missing area and its context area, thereby filling the missing area and generating a complete fundus image that matches the real image, thereby assessing the complication risk based on the complete fundus image, thereby achieving the purpose of ensuring the reliability and accuracy of assessing the complication risk based on the fundus image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein: Figure 1 It is a flow chart of a method for assessing the risk of diabetic complications in a preferred embodiment of the present invention; Figure 2 A schematic diagram of a process for generating a local restored image in a preferred embodiment of the present invention; Figure 3 A schematic diagram of a process of fusing and generating the local restored image in a preferred embodiment of the present invention; Figure 4 A schematic diagram of a process of using a local restored image to fill in a missing area to obtain a valid fundus image in a preferred embodiment of the present invention; Figure 5 It is a structural block diagram of a diabetes complication risk assessment system in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0018] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention. Embodiment 1

[0019] Reference Figure 1 As shown, an embodiment of the present invention discloses a method for assessing the risk of diabetic complications, including, S100, analyzing blood sugar indicators to identify diabetes-positive patients; S200, acquiring a fundus image of the diabetes-positive patient, and identifying a missing area of ​​the fundus image; S300, taking the missing area and the context area of ​​the missing area as conditions, generating a local restored image based on a conditional generative adversarial network, and using the local restored image to fill the missing area to obtain a valid fundus image; S400, assessing the risk of diabetic complications based on the valid fundus image.

[0020] The diabetic complication risk assessment method described in the present invention utilizes a conditional generative adversarial network to generate an accurate local restoration image based on the missing area and its context area, thereby filling the missing area and generating a complete fundus image (or effective fundus image) that matches the real image, thereby assessing the complication risk based on the complete fundus image, thereby achieving the purpose of ensuring the reliability and accuracy of assessing the complication risk based on the fundus image.

[0021] In specific application scenarios, S100 analyzes blood sugar indexes to identify patients with diabetes, including obtaining the user's fasting blood sugar value, postprandial blood sugar value and hemoglobin test value (HbA1c) through blood sugar testing and glycosylated hemoglobin testing. Fasting blood sugar value, postprandial blood sugar value and hemoglobin test value constitute blood sugar index. Fasting blood sugar value and postprandial blood sugar value reflect blood sugar level and are basic indicators for diabetes management. Long-term high blood sugar can lead to diabetes. By monitoring fasting blood sugar and postprandial blood sugar levels, it is possible to assess whether the patient's blood sugar control is stable and discover the risk of complications in advance. The hemoglobin test value reflects the average blood sugar level in the past 2-3 months. A high hemoglobin test value means long-term high blood sugar. The goal of diabetic patients is usually to control HbA1c below 7%. Excessive HbA1c may indicate an increased risk of complications such as fundus lesions and kidney disease. Monitoring HbA1c levels helps to determine whether patients have problems in controlling blood sugar, thereby providing early warning.

[0022] Specifically, blood sugar index is analyzed to identify patients with diabetes, including comparing blood sugar value with blood sugar value threshold. If the blood sugar value is greater than the blood sugar value threshold, the glycated hemoglobin test value is compared with the test value threshold; if the glycated hemoglobin test value is greater than or equal to the test value threshold, the user is determined to be a patient with diabetes. In specific application scenarios, the blood sugar value threshold and the glycated hemoglobin test value threshold are set according to medical standards, such as the fasting blood sugar threshold is set to 7.0mmol / L (126 mg / dL), and the 2-hour postprandial blood sugar threshold is set to 11.1 mmol / L (200 mg / dL); the glycated hemoglobin test value threshold is set to 6.5%; by comparing the blood sugar value with the blood sugar value threshold, and the glycated hemoglobin test value with the test value threshold, diabetes patients can be accurately identified.

[0023] S200 obtains the fundus image of the diabetes-positive patient and identifies the missing area of ​​the fundus image; in specific applications, patients identified as diabetes-positive need to have their eyes checked regularly according to clinical responses to prevent diabetes-induced retinopathy, and fundus images of diabetes-positive patients are obtained by means of fundus cameras, fluorescent fundus cameras FFA, optical coherence tomography OCT, etc. The shooting angle and shooting environment (such as suitable light conditions) directly affect the quality of the fundus image, because it involves the coverage of the retina, the focal length, color, contrast, uniformity of illumination, and the display effect of blood vessels and lesions. Improper shooting may not only cause image blur and loss of details, but also block key lesion areas, increase image noise, and reduce the analysis and recognition accuracy of fundus images by doctors or machine learning models.

[0024] Among them, the loss of details will seriously affect the reliability and accuracy of subsequent complication risk assessment, especially when the lost details involve the lesion area, it will directly lead to misjudgment of risk assessment.

[0025] After obtaining the fundus images of diabetes-positive patients, the missing areas can be identified based on a pre-built and trained neural network model, or they can be obtained based on manual markings by doctors. After locating the missing areas, basic data can be provided for subsequent local image restoration.

[0026] S300 generates a local restored image based on a conditional generative adversarial network with the missing area and the context area of ​​the missing area as conditions, and uses the local restored image to fill the missing area to obtain a valid fundus image; S400 evaluates the risk of diabetic complications based on the valid fundus image. In a specific application scenario, the effective fundus image is analyzed with the help of a machine learning algorithm to finally obtain the risk of diabetic complications; specifically, a large amount of sample data is obtained, including a complete fundus image sample set and a corresponding diabetic complication probability sample set, and a neural network model is trained with the complete fundus image sample set as input and the diabetic complication probability sample set as output until the convergence condition is met to obtain a complication risk assessment model; the effective fundus image is input into the complication risk assessment model to obtain the complication probability.

[0027] In an embodiment of the present invention, a conditional generative adversarial network is used to generate a local restored image to fill the missing area. The conditional generative adversarial network includes a generator and a discriminator. Random noise and a fundus image with a missing area are input to the generator. The generator generates a local restored image with the missing area and the context area of ​​the missing area as conditions; the real image and the local restored image generated by the generator are input to the discriminator. The discriminator outputs a probability value indicating whether the local restored image is a real image. This result is fed back to and affects the generator. Through adversarial training, the generator is continuously optimized to generate a more realistic local restored image to fill the missing area.

[0028] In the embodiment of the present invention, it is preferred to use a deep convolution conditional generative adversarial network; Figure 2 As shown, a local restored image is generated based on a conditional generative adversarial network, including performing mask marking on the missing area to obtain mask information; inputting the fundus image with the mask information into a generator, performing a first-scale deconvolution operation on the missing area according to the mask information to generate a trunk vessel feature map; performing a second-scale deconvolution operation on the trunk vessel feature map to generate a capillary feature map; performing a third-scale deconvolution operation on the capillary feature map to generate a microvessel feature map; and fusing the trunk vessel feature map, the capillary feature map and the microvessel feature map to generate the local restored image.

[0029] In specific application scenarios, the missing areas in the fundus image are first marked. Usually, the mask of the missing area is a binary image, with the missing area marked as 1 and the known area marked as 0. The fundus image and mask information are input into the generator. Through the mask information, the generator can clearly know which areas are missing and which areas are known. The mask information is crucial for the subsequent local restoration image generation. The mask provides guidance for the generator when restoring the image, ensuring that the generated missing area is coherent with the context area.

[0030] In the generator, the first-scale deconvolution operation is first performed through a larger convolution kernel (7×7) to obtain a main vessel feature map, which captures the structural features of larger blood vessels. The generated main vessel feature map provides large structural information in the locally restored image. The restoration of the main vessels provides a global context for the subsequent restoration of small blood vessels.

[0031] Based on the first scale, the second scale deconvolution operation is continued, and upsampling is performed through a smaller convolution kernel (5×5). At this stage, the generator restores the capillary feature map, which is responsible for restoring the finer vascular structure, especially the tiny capillary branches; the generated capillary feature map captures the characteristics of medium-sized blood vessels; the restoration of capillaries can ensure the natural transition of finer blood vessels in the local restoration image, and help generate the microvascular feature map without losing important information; at the same time, the generation of the capillary feature map helps to refine the vascular branch structure, so that the local restoration image has more local details.

[0032] Based on the second scale, the third scale deconvolution operation is performed, and the image is refined through a smaller convolution kernel (such as 3×3 or 1x1). This stage is mainly used to restore the microvascular feature map, that is, the smallest vascular branches and their details in the image; the generated microvascular feature map contains the details of the smallest vascular branches in the image, especially for microvascular lesions caused by diabetes, the generator can capture subtle features such as the bending and crossing of blood vessels; the restoration of the microvascular feature map is crucial for the fine restoration of fundus images, especially when the image is locally restored, the real vascular details can be retained.

[0033] The generator takes into account the fine and complex structure of the fundus image and sets up a multi-scale deconvolution operation. The generation of the trunk blood vessel feature map provides the basic framework for the structure of the entire image, ensuring that the general direction of the blood vessels is consistent with the overall layout; the generation of the capillary feature map refines the branching structure of the blood vessels, so that the image is restored in medium-scale details; the generation of the microvascular feature map restores the fine features of the microvessels, ensuring that the details of the smallest blood vessels in the image are not lost. Through the multi-scale deconvolution operation and feature map fusion, the final generated local restoration image is not only globally consistent, but also accurately restored in local details, especially in the restoration of complex structures such as diabetic retinopathy, showing a significant effect.

[0034] As a further improvement of the embodiment of the present invention, any two of the trunk blood vessel feature map, the capillary feature map and the microvascular feature map are jump-connected, and the feature maps after the jump connection are fused to generate the local restored image.

[0035] It should be noted that a skip connection is a structure that allows the output of a certain layer of the network to be directly transmitted to a deeper layer. In other words, a skip connection does not transmit information layer by layer, but directly transmits the feature map (or activation value) of a certain layer to the subsequent layer of the network through "jumping". The skip connection enables the network to better retain detail information, especially those local details that may be lost in a deep network. In the embodiment of the present invention, the skip connection is used in the generator and the discriminator to ensure that the generated image not only has a consistent global structure, but also restores details and local features.

[0036] In the implementation scheme of the present invention, the task of generating a local restored image is heavy. The trunk vascular feature map, capillary feature map and microvascular feature map represent different levels of information on the vascular structure in the fundus image: the trunk vascular feature map mainly represents the thick vascular structure, which is responsible for providing the general structure and direction in the image; the capillary feature map represents the thinner vascular branches, refines the vascular structure, and usually involves a larger local area; the microvascular feature map represents the tiny vascular branches and details in the image, mainly involving the microscopic scale vascular structure. Through jump connections, these different levels of feature maps can be fused to produce a more refined and consistent image restoration effect.

[0037] In one implementation, the trunk vessel feature map is jump-connected to the capillary feature map. The trunk vessel feature map provides the rough morphology and global structure of the blood vessels, while the capillary feature map refines the branching structure of the blood vessels. Jump-connecting the two allows the network to not only consider the local small blood vessel features when restoring details, but also maintain the global blood vessel layout. By considering the information of both at the same time through jump connection, the blood vessel structure of the generated image is more coherent, avoiding unnatural seams caused by the loss of local details or the neglect of large structures.

[0038] In another embodiment, the trunk vessel feature map and the microvascular feature map are jump-connected. The trunk vessel feature map focuses on the morphology of thicker blood vessels, while the microvascular feature map is responsible for restoring very small blood vessel branches. The jump connection between the two helps to maintain the overall structural consistency of the trunk vessels when restoring microvessels, so that the generation of microvessels does not deviate from the entire vascular network. At the same time, the jump connection helps to prevent local detail distortion when restoring microvessels, ensuring a natural and smooth transition between microvessels and trunk vessels.

[0039] In another embodiment, the capillary feature map is jump-connected with the microvascular feature map, and the capillary feature map refines the branches of larger blood vessels, while the microvascular feature map focuses on smaller blood vessels and details. Jump-connecting the two can ensure a smoother transition from larger blood vessel branches to microvessels. Jump connection ensures that the transition from capillaries to microvessels is natural and continuous, so that the branches and fine structures of blood vessels do not produce abrupt seams. The combination of the microvascular feature map and the capillary feature map can carefully restore the tiny blood vessels in the image while maintaining the continuity of the blood vessel structure.

[0040] In summary, jump connections between feature maps at different levels can integrate their respective advantages and enhance the complementarity of features. At the same time, the network pays attention to both local details and global structures when generating local restored images, improves the consistency of local and global structures, and avoids the problems of detail loss and incoherence.

[0041] Further, refer to Figure 3As shown, the main vessel features, capillary features and microvascular features are fused to generate the local restored image, including assigning weights to the main vessel feature map, capillary feature map and microvascular feature map respectively; performing weighted operations according to their respective weights, and performing splicing operations on the weighted main vessel feature map, capillary feature map and microvascular feature map along the dimension of the feature channel to obtain a spliced ​​feature map; jump-connecting the main vessel feature map and the microvascular feature map on the spliced ​​feature map, and jump-connecting the microvascular feature map and the main vessel feature map on the spliced ​​feature map; connecting the spliced ​​feature map to the Laplacian convolution device to generate the local restored image.

[0042] In specific application scenarios, different weights are assigned to the three generated feature maps (trunk blood vessel feature map, capillary feature map and microvascular feature map); the weights can be determined through experiments. Usually, the trunk blood vessel feature map has a larger weight because it provides the global structure of the image, while the capillary and microvascular feature maps have smaller weights because they refine local details; the setting of these weights can be determined through optimization algorithms or manual adjustments to achieve the optimal image restoration effect.

[0043] The weighted operation ensures that the contribution of each feature map is adjusted according to its importance; the splicing operation enables the generator to simultaneously utilize vascular information of different scales (thick trunk blood vessels, tiny capillaries and microvessels) to generate more complete and detailed images.

[0044] A jump connection operation is performed on the spliced ​​feature map to connect the trunk vessel feature map and the microvascular feature map, and then the microvascular feature map and the trunk vessel feature map are further connected, so that the generated image maintains a good balance between local details and overall structure; the edges of the missing area and the known area are better transitioned, making the image smoother and more natural, avoiding abrupt seams.

[0045] The concatenated feature map is input into the Laplacian convolution device to enhance the edges of the image; the Laplacian convolution device can detect high-frequency details in the image (such as the edges of blood vessels) and improve the sharpness and contrast of the image through convolution operations, making the blood vessels, microvessels and details in the image clearer, avoiding blur or distortion in the details of the generated image. Laplacian convolution also helps smooth the edges of the image, making the transition between the restored area and the context area more natural and avoiding incoherent seams.

[0046] Furthermore, obtaining the spliced ​​feature map also includes skipping connecting the capillary feature map and the microvascular feature map, and skipping connecting the microvascular feature map and the capillary feature map.

[0047] In specific application scenarios, the capillary feature map and the microvascular feature map are connected together through jump connections on the spliced ​​feature map, and the microvascular feature map and the capillary feature map are further reversely jump-connected; the jump connection can combine the finer microvascular details with the local information of the capillaries, and the finer microvascular details with the local information of the capillaries. The recovery of details at this stage is more accurate; the details of the microvessels will be fed back to the capillary feature map through the jump connection to ensure that the generated image is not lost in the detail level. These two jump connections will jointly provide multi-level detail information for the generated image, especially when restoring the branches of small blood vessels, the features of capillaries and microvessels will be well preserved and integrated.

[0048] In further improvements of the embodiments of the present invention, reference is made to Figure 4 As shown, the missing area is filled with the local restored image to obtain a valid fundus image, including generating a gradient mask according to the boundary of the missing area; wherein the mask value of the gradient mask gradually changes from 0 to 1; using the gradient mask to perform pixel-by-pixel weighted average calculation on the local restored image and the context area image of the missing area to obtain boundary connection information; the local restored image fills the missing area according to the boundary connection information. The local restored image filling method based on the gradient mask and pixel-by-pixel weighted average calculation can significantly improve the image restoration quality and ensure the accurate restoration of the details and structure of the image.

[0049] In specific application scenarios, this embodiment scheme uses a gradient mask and pixel-by-pixel weighted average calculation to fuse the local restored image with the image of the context area, gradually fill in the missing area, and generate a valid fundus image. First, a gradient mask is generated according to the boundary of the missing area. The mask value of the gradient mask will gradually change from 0 to 1, 0 means completely transparent, 1 means completely filled, and the value of the transition zone is between 0 and 1; the mask value gradually changes from 0 to 1 to reduce the artificial traces when filling the missing area through a smooth transition. The mask value of the edge area gradually increases from 0 to 1 to ensure a smooth transition between the filled area and the surrounding context area.

[0050] For the generated local restored image and the missing area context area image, a gradient mask is used to perform pixel-by-pixel weighted averaging. The value of each pixel is weighted by the information of the context area and the restored image through the mask value: for each pixel in the missing area, a gradient mask is used to perform weighted averaging of the local restored image and the context area image, so that the boundary area gradually transitions to the complete area. In the center of the missing area, the weight of the local restored image is 1, while the weight of the context area is close to 0; in the boundary area, the weights of the local restored image and the context area are gradually balanced. The weighted average operation can ensure a smooth transition between the filled area and the surrounding area. Guided by the mask, the boundary part of the image is naturally filled to avoid artificial traces or abrupt seams.

[0051] During the filling process, the local restored image is seamlessly combined with the context area through the boundary connection information, so that there is no obvious boundary between the restored area and the known area; the generated effective fundus image can seamlessly fill the missing area, and the structure and texture of the filled part are completely consistent with the known area, and the boundary transition of the filled area is natural without abrupt seams. The details and structure of the filled image are restored, especially when restoring tiny blood vessels and complex microvessels, the details are retained and highly consistent with the surrounding area, while maintaining the coherence of complex details such as blood vessels and retinal layers.

[0052] Specifically, the gradient mask is generated according to the following formula: ; Where M(x) represents the value of the gradient mask; Indicates the center position of the boundary of the missing area; represents the standard deviation, which is used to control the width of the gradient; x represents the spatial position of the mask.

[0053] The closer x is to The closer the mask value M(x) is to 1, the more pixels at that position need to be filled completely; The farther away, the closer the mask value M(x) is to 0, which means that the pixel at this position is closer to the missing area and the filling intensity is lower. Using the Gaussian function to generate a smooth gradient mask makes the gradient more natural and avoids too abrupt boundaries, better simulating natural transitions. Embodiment 2

[0054] Based on the same inventive concept, the present invention discloses a diabetes complication risk assessment system for executing a diabetes complication risk assessment method, including: Blood sugar index analysis module, used to analyze blood sugar index to identify diabetes-positive patients; A fundus image processing module, used for acquiring the fundus image of the diabetes-positive patient and identifying the missing area of ​​the fundus image; An effective fundus image generation module is used to generate a local restoration image based on a conditional generative adversarial network with the missing area and the context area of ​​the missing area as conditions, and use the local restoration image to fill the missing area to obtain an effective fundus image; A complication risk assessment module is used to assess the risk of diabetic complications based on the effective fundus image.

[0055] The diabetes complication risk assessment system of the embodiment of the present invention is used to execute the diabetes complication risk assessment method, and has the same technical effects as described above, which will not be described in detail here.

[0056] In summary, the diabetes complication risk assessment method and system described in the present invention utilize a conditional generative adversarial network to generate an accurate local restoration image based on the missing area and its context area, thereby filling the missing area and generating a complete fundus image that matches the real image, thereby assessing the complication risk based on the complete fundus image, thereby achieving the purpose of ensuring the reliability and accuracy of assessing the complication risk based on the fundus image.

[0057] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0058] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0059] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0060] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0061] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. A method for assessing the risk of diabetic complications, characterized in that: include, Analyze blood sugar indexes to identify diabetes-positive patients; Acquire a fundus image of the diabetes-positive patient, and identify missing areas of the fundus image; Taking the missing area and the context area of ​​the missing area as conditions, generating a local restored image based on a conditional generative adversarial network, and using the local restored image to fill the missing area to obtain a valid fundus image; The risk of diabetic complications is assessed based on the effective fundus image.

2. The method for assessing the risk of diabetes complications according to claim 1, characterized in that: The conditional generative adversarial network includes a generator; Generate local restored images based on conditional generative adversarial networks, including: Performing mask marking on the missing area to obtain mask information; Inputting the fundus image with the mask information into a generator, performing a first-scale deconvolution operation on the missing area according to the mask information, and generating a main vessel feature map; Performing a second-scale deconvolution operation on the trunk blood vessel characteristic map to generate a capillary blood vessel characteristic map; Performing a third-scale deconvolution operation on the capillary feature map to generate a microvessel feature map; The local restored image is generated by fusing the trunk blood vessel characteristic map, the capillary blood vessel characteristic map and the microvascular characteristic map.

3. The method for assessing the risk of diabetes complications according to claim 2, characterized in that: Any two of the trunk blood vessel feature map, the capillary blood vessel feature map and the microvascular feature map are jump-connected, and the feature maps after the jump connection are fused to generate the local restored image.

4. The method for assessing the risk of diabetes complications according to claim 2 or 3, characterized in that: The main vessel features, capillary features and microvessel features are integrated to generate the local restored image, including: Assigning weights to the trunk blood vessel characteristic map, the capillary blood vessel characteristic map and the microvascular characteristic map respectively; A weighted operation is performed according to the respective weights, and a splicing operation is performed on the trunk blood vessel feature map, the capillary blood vessel feature map, and the microvascular feature map after the weighted operation along the dimension of the feature channel to obtain a spliced ​​feature map; Jump-connecting the trunk blood vessel feature map and the microvascular feature map on the spliced ​​feature map, and jump-connecting the microvascular feature map and the trunk blood vessel feature map; The spliced ​​feature map is connected to a Laplacian convolution device to generate the local restored image.

5. The method for assessing the risk of diabetes complications according to claim 4, characterized in that: Obtaining the splicing feature map also includes skipping connecting the capillary feature map and the microvascular feature map, and skipping connecting the microvascular feature map and the capillary feature map.

6. The method for assessing the risk of diabetes complications according to claim 2, characterized in that: Filling the missing area with the local restored image to obtain a valid fundus image includes: Generate a gradient mask according to the boundary of the missing area; wherein the mask value of the gradient mask gradually changes from 0 to 1; Using the gradient mask, performing pixel-by-pixel weighted average calculation on the local restoration image and the missing region context region image to obtain boundary connection information; The local restored image fills the missing area according to the boundary connection information.

7. The method for assessing the risk of diabetes complications according to claim 6, characterized in that: The gradient mask is generated according to the following formula: ； Where M(x) represents the gradient mask value; Indicates the center position of the boundary of the missing area; represents the standard deviation, which is used to control the width of the gradient; x represents the spatial position of the mask.

8. The method for assessing the risk of diabetes complications according to claim 1, characterized in that: Analyze blood sugar indicators to identify diabetes-positive patients, including, Obtain the user's blood sugar and glycosylated hemoglobin test values; If the blood glucose value is greater than the blood glucose value threshold, comparing the glycated hemoglobin detection value with the detection value threshold; If the glycated hemoglobin test value is greater than or equal to the test value threshold, the user is determined to be a diabetes-positive patient.

9. The method for assessing the risk of diabetes complications according to claim 1, characterized in that: Assessing the risk of diabetic complications based on the effective fundus image includes: The neural network model is trained with the complete fundus image sample set as input and the diabetic complication probability sample set as output until the convergence condition is met to obtain the complication risk assessment model; The effective fundus image is input into the complication risk assessment model to obtain the complication probability.

10. A diabetes complication risk assessment system, used to implement the diabetes complication risk assessment method according to any one of claims 1 to 9, characterized in that: include, Blood sugar index analysis module, which analyzes blood sugar index to identify diabetes-positive patients; A fundus image processing module, which obtains the fundus image of the diabetes-positive patient and identifies the missing area of ​​the fundus image; An effective fundus image generation module, taking the missing area and the context area of ​​the missing area as conditions, generates a local restoration image according to a conditional generative adversarial network, and uses the local restoration image to fill the missing area to obtain an effective fundus image; A complication risk assessment module is used to assess the risk of diabetic complications based on the effective fundus image.

Images