Diabetic patient risk assessment method and system

Abstract

The invention relates to the technical field of risk assessment of diabetic patients, in particular to a risk assessment method and system for diabetic patients. The eye fundus image containing local defects is directly processed through the trained lesion risk assessment model, lesion risk assessment is carried out on the decoupled lesion feature vector, and the preprocessing step of carrying out manual or algorithm restoration on the defect image is omitted, so that distortion errors possibly introduced in the restoration process are avoided, the assessment efficiency is improved, and the method is suitable for popularization and application. By constructing a feature decoupling network and constructing a total loss function including comparison loss, independence constraint loss and classification loss, the model is forced to learn pure lesion feature vectors, lesion information is ensured to be effectively separated from imaging quality information, and the defect that a traditional model excessively depends on image integrity is overcome; the technical problem that the risk assessment effect of most existing DR risk assessment models based on deep learning is poor when a repaired complete eye fundus image is input is solved.

Description

Technical Field

[0001] This invention relates to the field of risk assessment technology for diabetic patients, and more particularly to a method and system for risk assessment of diabetic patients. Background Technology

[0002] Using AI to analyze patients' fundus images to assess the risk of retinal diseases (DR) has become an important method. However, clinically acquired fundus images often have defects such as missing information in local areas, occlusion, or quality degradation due to factors such as shooting angle, eyelid and eyelash obstruction, refractive media opacity (such as cataracts), or limitations of the equipment itself. Since the number of defective images currently stored is relatively small, directly training the model with defective images will result in poor evaluation performance. Therefore, for such defective images, traditional methods generally use image restoration techniques such as interpolation or generative inpainting to preprocess the defective image to obtain a complete image, which is then input into the model for risk level determination.

[0003] Most existing deep learning-based DR risk assessment models heavily rely on the integrity and consistency of the input images for their training and inference paradigms. Therefore, the model learns a mapping from the pixel space of a complete image to disease risk labels. When a damaged image is input, even if it has been repaired, the high-level features extracted by the model will be distorted or biased because the pixel distribution differs from the distribution of the complete image seen during training. This is because the complete image on which it relies has been destroyed. Therefore, the root of the problem lies in the excessive reliance of feature representation on image integrity. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a DR risk assessment model that is superior to most existing deep learning-based models. This solves the technical problem that most existing deep learning-based DR risk assessment models have poor risk assessment performance when given a restored complete fundus image as input.

[0005] To address the aforementioned technical problems, the present invention provides the following technical solution: a method for risk assessment of diabetic patients, which specifically includes the following steps: S1. Obtain fundus images of diabetic patients with localized ocular defects; S2. The fundus image is input into a lesion risk assessment model pre-trained using a fundus image with localized missing regions. The lesion risk assessment model is constructed based on a feature decoupling network, which encodes and decouples the input image into mutually independent lesion feature vectors and imaging feature vectors. The lesion feature vectors are used to characterize the morphological texture related to diabetic retinopathy, and the imaging feature vectors are used to characterize the imaging attributes related to image acquisition quality. The fundus image with localized missing regions is obtained by adding a mask constructed based on the fundus image with localized missing regions to a complete fundus image of a diabetic patient. S3. Determine the risk level of retinopathy in the diabetic patient to be evaluated based on the lesion feature vector.

[0006] Preferably, the training steps for the lesion risk assessment model are as follows: A risk identification model and a total loss function are constructed. The risk identification model includes a shared encoder, parallel lesion decoders and imaging decoders, and a risk level classifier. The shared encoder is used to encode the input image, the lesion decoder is used to extract the lesion feature vector from the encoded image, the imaging decoder is used to extract the imaging feature vector from the encoded image, and the risk level classifier is used to determine the risk level of retinopathy in the diabetic patient to be evaluated based on the lesion feature vector. To obtain a sample set, a complete fundus image containing several images of diabetic patients labeled with disease risk levels was acquired. Each complete fundus image in the sample set is labeled as the first sample, and a mask to simulate fundus image defects is added to each complete fundus image to generate at least one corresponding mask image as the second sample; Construct training samples containing positive sample pairs and negative sample pairs. Each training sample contains a first sample and a second sample. The first sample and the second sample in the positive sample pair correspond to the same complete fundus image, while the first sample and the second sample in the negative sample pair correspond to different complete fundus images. The training samples are input into the lesion risk assessment model and trained based on contrastive learning until the model converges to obtain a well-trained lesion risk assessment model.

[0007] Preferably, the total loss function is a weighted sum of the contrast loss function, the independence constraint loss, and the lesion feature classification loss; The contrastive loss function is used to maximize the distance between the two lesion feature vectors extracted from the positive sample pair in the feature space, and minimize the distance between the two lesion feature vectors extracted from the negative sample pair in the feature space. The independence constraint loss is used to minimize the statistical correlation between the lesion feature vector and the imaging feature vector obtained by decoupling; The lesion feature classification loss is used to minimize the difference between the lesion risk level predicted by the lesion risk assessment model based on the lesion feature vector and the actual lesion risk level label.

[0008] In the above formula, Represents the total loss function. This represents the contrastive loss function. express This indicates a loss of lesion feature classification. This represents the weighting coefficient of the comparative loss. This represents the classification weight coefficient for lesion features, and N represents the number of complete fundus images in the sample set corresponding to the training samples. This represents the set of mask images generated based on the i-th complete fundus image in the sample set. Represents a set The number of masked images, This represents the lesion feature vector extracted from the i-th complete fundus image. This represents the lesion feature vector extracted from the m-th masked image generated based on the i-th complete fundus image. This represents the function for calculating cosine similarity. This represents the similarity adjustment factor. Let represent the lesion feature vector of any image in the j-th complete fundus image and the mask image generated based on the j-th complete fundus image, and let C represent the covariance matrix between the lesion feature vector and the imaging feature vector. Denotes the square of the Frobenius norm of a matrix. This represents the total number of the first and second samples. Let represent the lesion feature vector of the k-th image in the set consisting of the first and second samples. Let represent the imaging feature vector of the k-th image in the set consisting of the first and second samples. Let represent the mean vector of the lesion feature vectors of all images in the set consisting of the first sample and the second sample. Let represent the mean vectors of the imaging feature vectors of all images in the set consisting of the first sample and the second sample, respectively, and let A represent the number of risk levels of diabetic retinopathy. This is a uniquely encoded value, representing the true lesion risk level label corresponding to the i-th complete fundus image. Its value is 1 when it belongs to risk level a, and 0 otherwise. This represents the probability that the i-th complete fundus image predicted by the lesion risk assessment model belongs to risk level a.

[0009] Preferably, the specific steps for generating the mask image are as follows: S41. Obtain fundus images of the user with defects; S42. Extract the contour of the defect area in the user's fundus image and normalize it to obtain the normalized defect area contour. Then, classify the normalized defect area contour into several groups of defect areas and calculate the corresponding positions. S43. Based on the coordinates of each pixel on the normalized contour of each defect region in each group of defect regions, calculate a reference contour to characterize the shape distribution of the contour of the group of defect regions, and derive several similar contours. S44. Mark the missing region contour and similar contours as the missing contour; S45. Randomly select the defect contour as a mask and randomly add it to the complete fundus image to obtain a mask image. The position of the defect contour added in the complete fundus image is within the position of its corresponding defect area.

[0010] Preferably, step S42 specifically includes the following steps: S421. Extract the contour of the defect area, the optic disc contour, and the center coordinates of the optic disc contour from the fundus images of each user. S422. Construct a standard spectral disk containing the center coordinates and diameter, and establish a two-dimensional standard coordinate system with the standard spectral disk as the origin. S423. Calculate the ratio of the standard optic disc diameter to the diameter of the optic disc outline in each user's fundus image to obtain the scaling ratio. S424. Translate each user's fundus image until the center of their optic disc contour coincides with the center of the standard optic disc. Then, scale the image according to the scaling ratio to obtain a normalized fundus image. The formula for calculating the coordinates of each pixel in the normalized fundus image is as follows: In the above formula, Let represent the coordinates of any pixel in the normalized fundus image. This represents the pixel coordinates in the user's fundus image before scaling. Indicates the scaling ratio. Indicates the coordinates of the center of the standard viewing disk; S425. Extract all the contours of the defect areas in each normalized fundus image and mark them as normalized defect area contours. S426. In the standard coordinate system, calculate the ratio of the intersection of the smallest bounding rectangles of any two normalized defect region contours to the area enclosed by the smallest bounding rectangle, to obtain the intersection ratio. The calculation formula is: In the above formula, This represents the ratio of the intersection surfaces of the normalized defect region contour m and the normalized defect region contour n. and Let m and n represent the regions enclosed by the smallest bounding rectangles of the normalized defect region contours, respectively. S426. Set the intersection ratio threshold, and mark the normalized defect region contours with an intersection ratio greater than the intersection ratio threshold as the same group to obtain several groups of defect regions. S427. Obtain the location of the group of defective regions by the union of several normalized defective region contours in each group of defective regions.

[0011] Preferably, step S43 specifically includes the following steps: S431. Obtain the coordinates of each pixel on the normalized contour of each defect region in each group of defect regions; S432. Sequentially perform similarity transformation and fitting on the contours of each normalized defect region in each group of defect regions to output the reference contour fitted by each group of defect regions, and the transformed defect contour after similarity transformation of the contours of each normalized defect region. S433. Calculate the covariance matrix S of each transformed defect contour in each group of defect regions. The calculation formula is as follows: In the above formula, This indicates the number of transformed defect contours in the defect region. This represents the shape vector formed by the pixel coordinates on the i-th transformed defect contour in the group of defect regions. Represents the shape vector formed by the coordinates of pixels on the reference contour; S434. Perform eigenvalue decomposition on the covariance matrix based on principal component analysis to obtain eigenvalues ​​and corresponding eigenvectors; S435. Extract the eigenvectors corresponding to the t largest eigenvalues ​​to construct the shape transformation matrix; S436. Derive several similar contours based on the reference contour and the shape transformation matrix.

[0012] Preferably, step S432 specifically includes the following steps: S4321. Select any normalized defect region contour from each group of defect regions as a reference contour. S4322. Perform a similarity transformation on the normalized defect region contours in each group of defect regions to minimize the Protodyakonov distance between the normalized defect region contour and the reference contour of the group of defect regions, so as to obtain the transformed defect contour. S4323. Calculate the arithmetic mean of each pixel on all transformed defect contours in each group of defect regions to obtain the average shape of the group of defect regions, normalize the average shape and mark it as the reference contour. S4324. Repeat steps S4322-S4323 until the reference contour converges, and output the reference contour and the transformed defect contour when each group of defect areas converges.

[0013] Preferably, in step S436, the expression for similar contours is: In the above formula, This represents the vector composed of the coordinates of points on the derived similar contour. Indicates a reference profile. Let represent the shape transformation matrix constructed from the eigenvectors corresponding to the t largest eigenvalues. Indicates the bias term. This represents the bias limiting factor. This represents the j-th element in the shape transformation matrix. Indicates in The value of the j-th element in the bias term selected within the range.

[0014] Preferably, the bias limiting coefficient b is 2 or 3.

[0015] The present invention also provides a risk assessment system for diabetic patients, including a processor and a memory, wherein the memory is used to store a computer program, and the computer program, when executed by the processor, implements the aforementioned risk assessment method for diabetic patients.

[0016] By employing the above technical solution, the present invention provides a method and system for risk assessment of diabetic patients, which has at least the following beneficial effects: 1. This invention directly processes fundus images containing local defects using a trained lesion risk assessment model and performs lesion risk assessment on the decoupled lesion feature vectors. This eliminates the need for preprocessing steps of manually or algorithmically repairing the defective images, thereby avoiding distortion errors that may be introduced during the repair process and improving assessment efficiency.

[0017] 2. This invention constructs a feature decoupling network and a total loss function that includes contrast loss, independence constraint loss and classification loss, so that the model is forced to learn pure lesion feature vectors and ensures that lesion information and imaging quality information are effectively separated, thus overcoming the defect of traditional models that rely too much on image integrity.

[0018] 3. This invention synthesizes a reasonable mask that conforms to reality by statistically analyzing the shape and distribution characteristics of the missing areas in fundus images and analyzing the patterns of the real defect contours. Compared with completely random masks, this invention significantly improves the quality and effectiveness of training data, enabling the model to learn defect patterns that are closer to clinical reality, thereby enhancing the model's generalization performance in dealing with complex defects in the real world.

[0019] 4. The risk assessment method provided by this invention automates the entire process from image input to risk level output, eliminating the need for manual intervention in image repair or feature analysis. This significantly improves screening and assessment efficiency, reduces operational barriers and time costs, and provides efficient and stable technical support for large-scale diabetic retinopathy screening.

[0020] 5. This invention designs a novel method for calculating the cross-sectional ratio to achieve a more accurate classification of the region containing the normalized defect contour, thereby facilitating the subsequent determination of the actual location of each defect region in the fundus image. Attached Figure Description

[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of a user's fundus image with defects, as presented in this invention. Figure 2 This is a reference schematic diagram for calculating the intersection ratio in this invention; Figure 3 This is a flowchart of the risk assessment method for diabetic patients according to the present invention. Detailed Implementation

[0022] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. This will allow for a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects, and to facilitate its implementation.

[0023] To address the issue that most existing deep learning-based DR risk assessment models perform poorly when given a restored complete fundus image as input, this invention provides a risk assessment method for diabetic patients. Instead of attempting to restore fundus images with missing regions using traditional methods, this method uses a small number of complete fundus images from patients and combines the shape and distribution characteristics of missing regions in the fundus images to construct a large number of mask images. This allows for model training, and the trained model can directly assess the risk of diverse fundus images with missing regions. The method specifically includes the following steps: S1. Obtain fundus images of diabetic patients with localized missing areas. Fundus images are usually obtained by medical devices such as fundus cameras. Fundus images contain features such as the morphology of the patient's retina, vascular distribution, and possible lesions. In actual clinical imaging, due to factors such as shooting angle, patient cooperation (e.g., eyelid occlusion and eye movements), equipment limitations, or the influence of ocular media (e.g., cataracts), localized areas in the acquired fundus images may be blurred, shadowed, or completely missing. These types of fundus images are complex and varied. Existing models can directly assess the risk of diabetic patients with complete fundus images, which will not be discussed further in this invention. This invention mainly focuses on assessing the risk of fundus images with localized missing areas.

[0024] S2. The fundus image is input into a pre-trained lesion risk assessment model based on fundus images with localized missing regions. The lesion risk assessment model is constructed based on a feature decoupling network, which encodes and decouples the input image into independent lesion feature vectors and imaging feature vectors. The lesion feature vector is used to characterize the morphological texture related to diabetic retinopathy, striving to purely express semantic information related to diabetic retinopathy, such as microaneurysms, hemorrhages, exudates, and neovascularization. The imaging feature vector is used to characterize imaging attributes related to image acquisition quality, such as brightness, contrast, and occlusion shadows. The goal is to make the lesion feature vector and imaging feature vector as statistically independent as possible, so that the lesion feature vector contains as little information related to imaging quality as possible, and the imaging feature vector contains as little semantic information related to diabetic retinopathy as possible, thereby making the lesion feature vector extracted by the lesion risk assessment model as accurate as possible. Fundus images with localized missing regions are obtained by adding masks based on the missing regions to complete fundus images of diabetic patients. In other words, the shape and location features of the missing regions are analyzed and masks are constructed based on the missing regions in the fundus images. The rationality of the masks is ensured, and different masks are randomly added to the complete fundus images of diabetic patients. This allows a large number of fundus images with localized missing regions to be obtained from a small sample, which can be used to fully train the disease risk assessment model.

[0025] S3. Finally, the lesion risk assessment model determines the risk level of retinopathy in the diabetic patient to be assessed based on the lesion feature vector, realizing the mapping from the lesion feature vector to the lesion risk level. For example, the lesion risk level includes "no DR", "mild non-proliferative DR", "moderate non-proliferative DR", "severe non-proliferative DR" and "proliferative DR", etc.

[0026] Therefore, the risk assessment model can directly determine the risk level of diabetic retinopathy in patients with localized missing areas in their fundus images. The training steps for the risk assessment model are as follows: First, a risk identification model and a total loss function need to be constructed. The risk identification model includes a shared encoder, parallel lesion decoders and imaging decoders, and a risk level classifier. The shared encoder encodes the input image, the lesion decoder extracts lesion feature vectors from the encoded image, the imaging decoder extracts imaging feature vectors from the encoded image, and the risk level classifier determines the risk level of retinopathy in the diabetic patient being assessed based on the lesion feature vectors. The shared encoder is typically a convolutional neural network such as ResNet or VGG, which encodes features from the input fundus image to obtain a deep feature map that integrates all information. The parallel lesion decoder and imaging decoder are key to feature decoupling. They receive the encoded features output from the shared encoder but reconstruct different information through different decoding paths (usually composed of deconvolutional layers or upsampling layers), i.e., constructing lesion feature vectors and imaging feature vectors respectively. The risk level classifier typically consists of fully connected layers and softmax layers. Its input receives the lesion feature vectors and constructs a mapping relationship between the lesion feature vectors and discrete risk level labels.

[0027] To obtain a sample set, it is necessary to acquire a number of complete fundus images of diabetic patients with risk levels of diabetic retinopathy. The performance limit of a deep learning model largely depends on the quantity and quality of the training data. Acquiring a large number of complete fundus images with accurate labels of diabetic retinopathy risk levels by professional doctors is the foundation for the model to achieve correct classification.

[0028] To make the model robust to unpredictable image defects in clinical settings, the most direct approach is to expose it to various possible defect scenarios during the training phase. Therefore, each complete fundus image in the sample set is labeled as the first sample, and a mask simulating fundus image defects is added to each complete fundus image to generate at least one corresponding mask image as the second sample. This second sample is created manually based on a small number of complete images, summarizing patterns from existing fundus images with locally missing regions, and then added to the complete fundus images. This process constructs a large number of masks simulating fundus image defects with localized defects. The fundus image of the region, i.e., the second sample, is compared with the first sample. Therefore, the mask generation is not simply adding a mask of a certain shape and size randomly onto the complete fundus image. Instead, it summarizes the shape and location characteristics of the missing areas in the fundus images of existing locally missing regions, and adds corresponding masks based on these characteristics. For example, there are edge bar masks simulating eyelid occlusion, ring masks simulating camera lens barrel shadows, and block masks simulating pathological occlusion. This regular mask generation makes the training more realistic and efficient. The specific steps of mask image generation are further described in detail below: S41. To generate a reasonable mask, it is first necessary to obtain a defective image of the user's fundus, such as... Figure 1 As shown, by collecting fundus images of users with defects caused by eyelid occlusion, eyelash shadows, lens halos, pathological occlusion, and imaging artifacts in clinical practice, we can analyze the statistical patterns of the actual shape, size, and location distribution of the defect area to ensure that the subsequently generated mask can simulate the real clinical scenario, rather than a purely imaginary one.

[0029] S42. Because the size, resolution, and optic disc size of fundus images vary among different patients, direct analysis may result in the same region in one fundus image corresponding to different locations in another. Therefore, normalization is necessary. This involves extracting the contour of the defective region from the user's fundus image and normalizing it to obtain a normalized defective region contour. The normalized defective region contour is then classified into several groups of defective regions, and their corresponding locations are calculated. The specific steps include: S421. Image segmentation techniques such as threshold segmentation or edge detection can be used to extract the contour of the defective region, the optic disc contour, and the center coordinates of the optic disc contour in the fundus images of each user. S422. Construct a standard visual disk containing the center coordinates and diameter, and establish a two-dimensional standard coordinate system with the standard visual disk as the origin. The reason for choosing the visual disk instead of the fundus image boundary as the coordinate system and image scaling reference is that the visual disk is a stable structure in the fundus image, and its size is relatively more stable. Normalization based on it can eliminate scale changes caused by shooting distance and equipment differences.

[0030] S423. Calculate the ratio of the standard optic disc diameter to the diameter of the optic disc outline in each user's fundus image to obtain the scaling ratio.

[0031] S424. Translate each user's fundus image until the center of their optic disc contour coincides with the center of the standard optic disc. Then, scale the image according to the scaling ratio to obtain a normalized fundus image. The formula for calculating the coordinates of each pixel in the normalized fundus image is as follows: In the above formula, Let represent the coordinates of any pixel in the normalized fundus image. This represents the pixel coordinates in the user's fundus image before scaling. Indicates the scaling ratio. Indicates the center coordinates of the standard viewing disk.

[0032] S425. Extract all the contours of the defect areas in each normalized fundus image and mark them as normalized defect area contours. S426. In the standard coordinate system, calculate the ratio of the intersection of the smallest bounding rectangles of any two normalized defect region contours to the area enclosed by the smallest bounding rectangle, to obtain the intersection ratio. The calculation formula is: In the above formula, This represents the ratio of the intersection surfaces of the normalized defect region contour m and the normalized defect region contour n. and Let m and n represent the regions enclosed by the minimum bounding rectangles of the normalized defect region contours, respectively. Here, the intersection-to-union ratio (IOU) has been redesigned; please refer to [reference needed]. Figure 2 When the normalized defect region contour m is located within the normalized defect region contour n, and the normalized defect region contour m is much smaller than the normalized defect region contour n, their intersection-union ratio (IOU) is very small, and they may not be able to be classified as the same region in the future. In fact, they are in the same region. However, by redesigning the intersection ratio, it can be found that the intersection ratio of the two is 1. Therefore, it is possible to classify the normalized defect region contour m and the normalized defect region contour n into the same defect region in the subsequent classification.

[0033] S426. Set the intersection ratio threshold, which is generally set to 0.4. Mark the normalized defect region contours with an intersection ratio greater than the intersection ratio threshold as the same group to obtain several groups of defect regions, such as "upper eyelid occlusion group" and "central cataract occlusion group".

[0034] S427. Obtain the location of the group of missing regions by taking the union of several normalized missing region contours in each group of missing regions. For example, remove the largest normalized missing region contours in each group of missing regions and take the union of the remaining normalized missing region contours as the location of the group of missing regions. Alternatively, other methods can be used to process the normalized missing region contours in each group of missing regions to obtain the location of the missing regions.

[0035] S43. Based on the coordinates of each pixel on the normalized defect region contour of each group of defect regions, calculate the reference contour used to characterize the shape distribution of the contour of the group of defect regions, and derive several similar contours. This step analyzes the shape characteristics of all normalized defect region contours in each group of defect regions and summarizes their characteristics, so as to facilitate the subsequent derivation of more similar contours that conform to the actual situation as masks. Specifically, it includes the following steps: S431. Obtain the coordinates of each pixel on the normalized defect region contour of each defect region in each group of defect regions.

[0036] S432. Sequentially perform similarity transformation and fitting on the contours of each normalized defect region in each group of defect regions to output the reference contour fitted to each group of defect regions, as well as the transformed defect contours after similarity transformation of the contours of each normalized defect region. In order to obtain more accurate reference contours and transformed defect contours and improve the rationality of the subsequently obtained similar contours, the specific steps include the following: S4321. Select any normalized defect region contour from each group of defect regions as a reference contour.

[0037] S4322. Perform a similarity transformation on the normalized defect region contours in each group of defect regions to minimize the Protodyakonov distance between the normalized defect region contour and the reference contour of the group of defect regions, so as to obtain the transformed defect contour. S4323. Calculate the arithmetic mean of all pixels on the transformed defect contours in each group of defective regions to obtain the average shape of the group of defective regions. Normalize the average shape and mark it as the reference contour.

[0038] S4324. Repeat steps S4322-S4323 until the reference contour converges, and output the reference contour and the transformed defect contour when each group of defect areas converges.

[0039] S433. Calculate the covariance matrix S of each transformed defect contour in each group of defect regions. The calculation formula is as follows: In the above formula, This indicates the number of times the defect contour has been transformed within the defect area. This represents the shape vector formed by the pixel coordinates on the i-th transformed defect contour in the group of defect regions. This represents the shape vector formed by the coordinates of the pixels on the reference contour.

[0040] S434. Based on principal component analysis, perform eigenvalue decomposition on the covariance matrix to obtain eigenvalues ​​and corresponding eigenvectors.

[0041] S435. Extract the eigenvectors corresponding to the t largest eigenvalues ​​to construct the shape transformation matrix, which can be expressed as: , This represents the shape transformation matrix constructed from the eigenvectors corresponding to the t largest eigenvalues. This represents the t-th eigenvector arranged in descending order.

[0042] S436. Several similar contours are derived based on the reference contour and the shape transformation matrix. Their expressions are as follows: In the above formula, This represents the vector composed of the coordinates of points on the derived similar contour. Indicates a reference profile. Let represent the shape transformation matrix constructed from the eigenvectors corresponding to the t largest eigenvalues. Indicates the bias term. This represents the bias limiting factor. This represents the j-th element in the shape transformation matrix. Indicates in The value of the j-th element in the bias term selected within the range is used to obtain similar contours. The vector formed by the coordinates of all points on the vector, It is a t-dimensional parameter vector, which controls each element in the bias term. The numerical value can generate a large number of new vectors composed of the coordinates of points on similar contours, that is... The bias constraint coefficient b is best set to 2 or 3. When b is 2, it means that the new shape is allowed to deviate from the average shape by a maximum of twice the standard deviation of that direction in each principal component direction. According to the properties of the normal distribution, about 95.4% of the sample data will fall within this range. This ensures that the generated contour is highly consistent with the shape pattern of the original defect area contour. The generated mask is very reasonable and almost no unreasonable strange shapes are produced. When b is 3, the allowed deviation range is expanded to 3 times the standard deviation, covering about 99.7% of the data distribution range. This provides greater diversity and exploration space for shape generation, while still strictly ensuring the reasonableness of the generated similar contours. Only a very small number of shapes that are on the edge of the training data distribution but still have statistical basis are allowed to appear.

[0043] S44. Mark the missing region contour and similar contours as missing contours.

[0044] S45. Randomly select the defect contour as a mask and randomly add it to the complete fundus image to obtain a mask image. The position of the defect contour added in the complete fundus image is within the position of its corresponding defect area. For example, for a complete fundus image, at least one set of positions corresponding to the defect area will be selected, and then a defect contour corresponding to the set of defect areas will be randomly added as a mask at each selected position. In this way, a large number of mask images will be continuously generated, producing more reasonable training samples. Compared with adding masks of random shape and size at random positions in the complete fundus image to generate mask images, it can reduce invalid or counterintuitive mask images, thereby accelerating model convergence and improving the generalization performance of the model in real complex defect scenarios.

[0045] After obtaining the masked image, to improve the model's classification ability, contrastive learning is used for model training. This involves constructing training samples containing several positive and negative sample pairs. Each training sample includes a first sample and a second sample. In a positive sample pair, the first and second samples correspond to the same complete fundus image. Regardless of the mask of the second sample, both the first and second samples in the positive sample pair originate from the same pathological content. The model should try to bring the lesion feature vectors extracted from this pair of samples as close as possible in the feature space, ensuring that the lesion feature vectors remain consistent despite differences in appearance (one with defects and the other without). In a negative sample pair, the first and second samples correspond to different complete fundus images, meaning they originate from different complete fundus images with different pathological contents. The model should learn to push the lesion feature vectors of this pair as far apart as possible to distinguish features from different patients or different degrees of lesion, thus enhancing the model's discriminative power regarding lesion features.

[0046] The training samples are input into the lesion risk assessment model and trained based on contrastive learning until the model converges to obtain a well-trained lesion risk assessment model. This is an iterative training process. In each training iteration, a batch of sample pairs containing positive and negative sample pairs is input into the lesion risk assessment model for forward propagation. For each sample, a lesion feature vector and an imaging feature vector are output. Subsequently, the total loss function is calculated. Through the backpropagation algorithm, the gradient of the total loss function with respect to all trainable parameters of the network (such as the weights of the encoder, decoder, classifier, etc.) is calculated, and an optimizer (such as Adam) is used to update these parameters according to the gradient. Multiple parameter updates are achieved through multiple iterations of training. Finally, when the total loss function no longer decreases significantly on the validation dataset, or when the model's classification accuracy on the validation set reaches a stable level and meets the requirements, the model is considered to have converged. At this point, the final parameters are saved, and the well-trained lesion risk assessment model is obtained.

[0047] The total loss function is a weighted sum of the contrast loss function, independence constraint loss, and lesion feature classification loss. It aims to solve three sub-problems: the contrast loss function maximizes the distance between two lesion feature vectors extracted from a positive sample pair in the feature space, and minimizes the distance between two lesion feature vectors extracted from a negative sample pair; the independence constraint loss minimizes the statistical correlation between the decoupled lesion feature vectors and the imaging feature vectors; and the lesion feature classification loss minimizes the difference between the lesion risk level predicted by the lesion risk assessment model based on the lesion feature vectors and the actual lesion risk level label. Therefore, the expressions for the total loss function, contrast loss function, independence constraint loss, and lesion feature classification loss can be represented as follows: In the above formula, Represents the total loss function. This represents the contrastive loss function. express This indicates a loss of lesion feature classification. This represents the weighting coefficient of the comparative loss. This represents the classification weight coefficient for lesion features, and N represents the number of complete fundus images in the sample set corresponding to the training samples. This represents the set of mask images generated based on the i-th complete fundus image in the sample set. Represents a set The number of masked images, This represents the lesion feature vector extracted from the i-th complete fundus image. This represents the lesion feature vector extracted from the m-th masked image generated based on the i-th complete fundus image. This represents the function for calculating cosine similarity. This represents the similarity adjustment factor. Let represent the lesion feature vector of any image in the j-th complete fundus image and the mask image generated based on the j-th complete fundus image, and let C represent the covariance matrix between the lesion feature vector and the imaging feature vector. Denotes the square of the Frobenius norm of a matrix. This represents the total number of the first and second samples. Let represent the lesion feature vector of the k-th image in the set consisting of the first and second samples. Let represent the imaging feature vector of the k-th image in the set consisting of the first and second samples. Let represent the mean vector of the lesion feature vectors of all images in the set consisting of the first sample and the second sample. Let represent the mean vectors of the imaging feature vectors of all images in the set consisting of the first sample and the second sample, respectively, and let A represent the number of risk levels of diabetic retinopathy. This is a uniquely encoded value, representing the true lesion risk level label corresponding to the i-th complete fundus image. Its value is 1 when it belongs to risk level a, and 0 otherwise. This represents the probability that the i-th complete fundus image predicted by the lesion risk assessment model belongs to risk level a.

[0048] The present invention also provides a risk assessment system for diabetic patients, including a processor and a memory, wherein the memory is used to store a computer program, and the computer program is executed by the processor to implement a method for risk assessment of diabetic patients.

[0049] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0050] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Since the above embodiments are substantially similar to the method embodiments, their descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0051] The above embodiments provide a detailed description of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for risk assessment in diabetic patients, characterized in that, The method specifically includes the following steps: S1. Obtain fundus images of diabetic patients with localized ocular defects; S2. Input the fundus image into a lesion risk assessment model pre-trained using fundus images with localized missing areas; The lesion risk assessment model is constructed based on a feature decoupling network, which is used to encode the input image and decouple it into mutually independent lesion feature vectors and imaging feature vectors; The lesion feature vector is used to characterize the morphological texture associated with diabetic retinopathy. The imaging feature vector is used to characterize imaging attributes related to image acquisition quality; The fundus image with missing local regions was obtained by adding a mask based on the fundus image with missing local regions to a complete fundus image of a diabetic patient; S3. Determine the risk level of retinopathy in the diabetic patient to be evaluated based on the lesion feature vector.

2. The method for risk assessment of diabetic patients according to claim 1, characterized in that, The training steps for the disease risk assessment model are as follows: A risk identification model and a total loss function are constructed. The risk identification model includes a shared encoder, parallel lesion decoders and imaging decoders, and a risk level classifier. The shared encoder is used to encode the input image, the lesion decoder is used to extract the lesion feature vector from the encoded image, the imaging decoder is used to extract the imaging feature vector from the encoded image, and the risk level classifier is used to determine the risk level of retinopathy in the diabetic patient to be evaluated based on the lesion feature vector. To obtain a sample set, a complete fundus image containing several images of diabetic patients labeled with disease risk levels was acquired. Each complete fundus image in the sample set is labeled as the first sample, and a mask to simulate fundus image defects is added to each complete fundus image to generate at least one corresponding mask image as the second sample; Construct training samples containing positive sample pairs and negative sample pairs. Each training sample contains a first sample and a second sample. The first sample and the second sample in the positive sample pair correspond to the same complete fundus image, while the first sample and the second sample in the negative sample pair correspond to different complete fundus images. The training samples are input into the lesion risk assessment model and trained based on contrastive learning until the model converges to obtain a well-trained lesion risk assessment model.

3. The method for risk assessment of diabetic patients according to claim 1, characterized in that, The total loss function is a weighted sum of the contrast loss function, the independence constraint loss, and the lesion feature classification loss; The contrastive loss function is used to maximize the distance between the two lesion feature vectors extracted from the positive sample pair in the feature space, and minimize the distance between the two lesion feature vectors extracted from the negative sample pair in the feature space. The independence constraint loss is used to minimize the statistical correlation between the lesion feature vector and the imaging feature vector obtained by decoupling; The lesion feature classification loss is used to minimize the difference between the lesion risk level predicted by the lesion risk assessment model based on the lesion feature vector and the actual lesion risk level label; In the above formula, Represents the total loss function. This represents the contrastive loss function. express This indicates a loss of lesion feature classification. This represents the weighting coefficient of the comparative loss. This represents the classification weight coefficient for lesion features, and N represents the number of complete fundus images in the sample set corresponding to the training samples. This represents the set of mask images generated based on the i-th complete fundus image in the sample set. Represents a set The number of masked images, This represents the lesion feature vector extracted from the i-th complete fundus image. This represents the lesion feature vector extracted from the m-th masked image generated based on the i-th complete fundus image. This represents the function for calculating cosine similarity. This represents the similarity adjustment factor. Let represent the lesion feature vector of any image in the j-th complete fundus image and the mask image generated based on the j-th complete fundus image, and let C represent the covariance matrix between the lesion feature vector and the imaging feature vector. Denotes the square of the Frobenius norm of a matrix. This represents the total number of the first and second samples. Let represent the lesion feature vector of the k-th image in the set consisting of the first and second samples. Let represent the imaging feature vector of the k-th image in the set consisting of the first and second samples. Let represent the mean vector of the lesion feature vectors of all images in the set consisting of the first sample and the second sample. Let represent the mean vectors of the imaging feature vectors of all images in the set consisting of the first sample and the second sample, respectively, and let A represent the number of risk levels of diabetic retinopathy. This is a uniquely encoded value, representing the true lesion risk level label corresponding to the i-th complete fundus image. Its value is 1 when it belongs to risk level a, and 0 otherwise. This represents the probability that the i-th complete fundus image predicted by the lesion risk assessment model belongs to risk level a.

4. The method for risk assessment of diabetic patients according to claim 1, characterized in that, The specific steps for generating the mask image are as follows: S41. Obtain fundus images of the user with defects; S42. Extract the contour of the defect area in the user's fundus image and normalize it to obtain the normalized defect area contour. Then, classify the normalized defect area contour into several groups of defect areas and calculate the corresponding positions. S43. Based on the coordinates of each pixel on the normalized contour of each defect region in each group of defect regions, calculate a reference contour to characterize the shape distribution of the contour of the group of defect regions, and derive several similar contours. S44. Mark the missing region contour and similar contours as the missing contour; S45. Randomly select the defect contour as a mask and randomly add it to the complete fundus image to obtain a mask image. The position of the defect contour added in the complete fundus image is within the position of its corresponding defect area.

5. The method for risk assessment of diabetic patients according to claim 4, characterized in that, Step S42 specifically includes the following steps: S421. Extract the contour of the defect area, the optic disc contour, and the center coordinates of the optic disc contour from the fundus images of each user. S422. Construct a standard spectral disk containing the center coordinates and diameter, and establish a two-dimensional standard coordinate system with the standard spectral disk as the origin. S423. Calculate the ratio of the standard optic disc diameter to the diameter of the optic disc outline in each user's fundus image to obtain the scaling ratio. S424. Translate each user's fundus image until the center of their optic disc contour coincides with the center of the standard optic disc. Then, scale the image according to the scaling ratio to obtain a normalized fundus image. The formula for calculating the coordinates of each pixel in the normalized fundus image is as follows: In the above formula, Let represent the coordinates of any pixel in the normalized fundus image. This represents the pixel coordinates in the user's fundus image before scaling. Indicates the scaling ratio. Indicates the coordinates of the center of the standard viewing disk; S425. Extract all the contours of the defect areas in each normalized fundus image and mark them as normalized defect area contours. S426. In the standard coordinate system, calculate the ratio of the intersection of the smallest bounding rectangles of any two normalized defect region contours to the area enclosed by the smallest bounding rectangle, to obtain the intersection ratio. The calculation formula is: In the above formula, This represents the ratio of the intersection surfaces of the normalized defect region contour m and the normalized defect region contour n. and Let m and n represent the regions enclosed by the smallest bounding rectangles of the normalized defect region contours, respectively. S426. Set the intersection ratio threshold, and mark the normalized defect region contours with an intersection ratio greater than the intersection ratio threshold as the same group to obtain several groups of defect regions. S427. Obtain the location of the group of defective regions by the union of several normalized defective region contours in each group of defective regions.

6. The method for risk assessment of diabetic patients according to claim 4, characterized in that, Step S43 specifically includes the following steps: S431. Obtain the coordinates of each pixel on the normalized contour of each defect region in each group of defect regions; S432. Sequentially perform similarity transformation and fitting on the contours of each normalized defect region in each group of defect regions to output the reference contour fitted by each group of defect regions, and the transformed defect contour after similarity transformation of the contours of each normalized defect region. S433. Calculate the covariance matrix S of each transformed defect contour in each group of defect regions. The calculation formula is as follows: In the above formula, This indicates the number of transformed defect contours in the defect region. This represents the shape vector formed by the pixel coordinates on the i-th transformed defect contour in the group of defect regions. Represents the shape vector formed by the coordinates of pixels on the reference contour; S434. Perform eigenvalue decomposition on the covariance matrix based on principal component analysis to obtain eigenvalues ​​and corresponding eigenvectors; S435. Extract the eigenvectors corresponding to the t largest eigenvalues ​​to construct the shape transformation matrix; S436. Derive several similar contours based on the reference contour and the shape transformation matrix.

7. The method for risk assessment of diabetic patients according to claim 6, characterized in that, Step S432 specifically includes the following steps: S4321. Select any normalized defect region contour from each group of defect regions as a reference contour. S4322. Perform a similarity transformation on the normalized defect region contours in each group of defect regions to minimize the Protodyakonov distance between the normalized defect region contour and the reference contour of the group of defect regions, so as to obtain the transformed defect contour. S4323. Calculate the arithmetic mean of each pixel on all transformed defect contours in each group of defect regions to obtain the average shape of the group of defect regions, normalize the average shape and mark it as the reference contour. S4324. Repeat steps S4322-S4323 until the reference contour converges, and output the reference contour and transformed defect contour when each group of defect areas converges.

8. A method for risk assessment of diabetic patients according to claim 6, characterized in that, In step S436, the expression for similar contours is: In the above formula, This represents the vector composed of the coordinates of points on the derived similar contour. Indicates a reference profile. Let represent the shape transformation matrix constructed from the eigenvectors corresponding to the t largest eigenvalues. Indicates the bias term. This represents the bias limiting factor. This represents the j-th element in the shape transformation matrix. Indicates in The value of the j-th element in the bias term selected within the range.

9. A method for risk assessment of diabetic patients according to claim 8, characterized in that, The bias limiting coefficient b can be either 2 or 3.

10. A system for implementing the risk assessment method for diabetic patients according to any one of claims 1-9, characterized in that, It includes a processor and a memory, the memory being used to store a computer program that, when executed by the processor, implements the risk assessment method for diabetic patients as described in any one of claims 1-9.

Images