Retinal blood vessel parameter acquisition method and apparatus, and retinal analysis device
By constructing a retinal vascular parameter acquisition model through a multi-task joint pre-training model, the problem of low accuracy in retinal vascular parameter acquisition is solved, achieving efficient and accurate retinal vascular parameter acquisition, applicable to different types of retinal vascular images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2025-07-25
- Publication Date
- 2026-06-23
AI Technical Summary
Existing technologies have low accuracy in acquiring retinal vascular parameters and require complex step-by-step processing procedures and manual intervention.
A multi-task joint pre-trained model is used to construct a retinal vessel parameter acquisition model, including an encoder and multiple predictors. By directly inputting retinal vessel images, the encoder learns deep retinal vessel knowledge, extracts robust features, solves the feature loss problem caused by viewpoint differences, and improves the model's generalization ability.
This method improves the accuracy and efficiency of acquiring retinal vascular parameters, reduces processing complexity and human error, enhances the broad applicability of the method, and shortens the acquisition time.
Smart Images
Figure CN121190378B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of medical technology, and in particular to a method, device, and retinal analysis equipment for acquiring retinal vascular parameters. Background Technology
[0002] In the medical field, retinal vascular features (such as diameter and fractal dimension) are important biomarkers for assessing ophthalmic and cardiovascular diseases, making it particularly important to obtain the corresponding retinal vascular parameters.
[0003] In related technologies, semi-automated software is mainly used to obtain retinal vascular parameters in retinal vascular images by manually intervening to correct vascular segmentation and arteriovenous identification. Alternatively, a fully automatic model is used to perform step-by-step calculations, first segmenting the retinal vascular image and then obtaining the retinal vascular parameters in the retinal vascular image.
[0004] However, the relevant technologies suffer from the problem of low accuracy in obtaining retinal vascular parameters. Summary of the Invention
[0005] Therefore, it is necessary to provide a method, device, and retinal analysis equipment for acquiring retinal vascular parameters to address the aforementioned technical problems.
[0006] In a first aspect, this application provides a method for obtaining retinal vascular parameters, including:
[0007] Acquire retinal vessel images to be analyzed; retinal vessel images can be central images of the retina or images of the retina outside the retina.
[0008] The retinal vessel image is input into a pre-trained retinal vessel parameter acquisition model to obtain the retinal vessel parameters in the retinal vessel image.
[0009] The retinal vascular parameter acquisition model includes an encoder and multiple predictors. The retinal vascular parameter acquisition model is constructed by training a multi-task joint pre-trained model, which includes an original encoder, a decoding module, and multiple original predictors. During the training process, the original encoder learns deep retinal vascular knowledge based on the decoding module.
[0010] In one embodiment, a retinal vessel image is input into a pre-trained retinal vessel parameter acquisition model to acquire retinal vessel parameters from the retinal vessel image, including:
[0011] The retinal vessel image is input into the encoder to obtain the corresponding multidimensional feature vector;
[0012] The multidimensional feature vectors are input into each predictor to obtain the retinal vascular parameters in the retinal vascular image.
[0013] In one embodiment, the process of constructing the retinal vascular parameter acquisition model includes:
[0014] Acquire a set of retinal vessel images; the set of retinal vessel images includes sample retinal vessel images acquired from multiple different acquisition angles within a standard field of view;
[0015] Based on a retinal vessel image set, a multi-task joint pre-training model is trained to generate a trained model.
[0016] Based on the trained model, a model for obtaining retinal vascular parameters is constructed.
[0017] In one embodiment, a multi-task joint pre-trained model is trained based on a retinal vessel image set to generate a trained model, including:
[0018] The original encoder and decoder module are trained based on a set of retinal blood vessel images to obtain the encoder and the trained decoder module.
[0019] Based on the retinal blood vessel image set and the encoder, each original predictor is trained to obtain each predictor;
[0020] The encoder, the trained decoding module, and each predictor are defined as the trained model.
[0021] In one embodiment, the original encoder and decoding module are trained based on a retinal vessel image set to obtain the encoder and the trained decoding module, including:
[0022] The retinal vessel image set is input into the original encoder to obtain the predicted feature vector;
[0023] The predicted feature vector is input into the decoding module to obtain the corresponding intermediate prediction result;
[0024] Based on the intermediate prediction results, the model parameters of the original encoder and decoding module are optimized to obtain the encoder and the trained decoding module.
[0025] In one embodiment, the decoding module includes a first decoder, a second decoder, and a decoding unit. The intermediate prediction results include a binary segmentation image of blood vessels, a binary segmentation image of the optic disc, and a true / false probability vector of the predicted optic disc center image. The predicted feature vector is input into the decoding module to obtain the corresponding intermediate prediction results, including:
[0026] The predicted feature vector is input into the first decoder to obtain a binary segmentation image of blood vessels; and the predicted feature vector is input into the second decoder to obtain a binary segmentation image of the optic disc; and the predicted feature vector is input into the decoding unit to obtain a predicted true / false probability vector.
[0027] In one embodiment, the model parameters of each original predictor are optimized based on the predicted blood vessel parameters to obtain each predictor, including:
[0028] Model parameters of the original encoder and the first decoder are optimized based on the binary segmentation image of the blood vessel; and model parameters of the original encoder and the second decoder are optimized based on the binary segmentation image of the optic disc; and model parameters of the original encoder and the decoding unit are optimized based on the predicted true and false probability vectors.
[0029] Until the current first decoder satisfies the preset first parameter optimization termination condition, the current second decoder satisfies the preset second parameter optimization termination condition, and the current decoding unit satisfies the preset third parameter optimization termination condition, the current original encoder is determined as the encoder, and the current first decoder, the current second decoder, and the current decoding unit are determined as the trained decoding module.
[0030] In one embodiment, based on a set of retinal vessel images and an encoder, each original predictor is trained to obtain predictors, including:
[0031] The retinal vessel image set is input into the encoder to obtain the trained feature vector;
[0032] The trained feature vectors are input into each original predictor to obtain the predicted blood vessel parameters;
[0033] The model parameters of each original predictor are optimized based on the predicted blood vessel parameters to obtain each predictor.
[0034] Secondly, this application also provides a device for acquiring retinal vascular parameters, comprising:
[0035] The image acquisition module is used to acquire retinal vessel images to be analyzed; the retinal vessel images are either central images of the retina or images of the retina outside the retina.
[0036] The parameter acquisition module is used to input retinal vessel images into a pre-trained retinal vessel parameter acquisition model to obtain retinal vessel parameters in the retinal vessel images.
[0037] The retinal vascular parameter acquisition model includes an encoder and multiple predictors. The retinal vascular parameter acquisition model is constructed by training a multi-task joint pre-trained model, which includes an original encoder, a decoding module, and multiple original predictors. During the training process, the original encoder learns deep retinal vascular knowledge based on the decoding module.
[0038] Thirdly, this application also provides a retinal analysis device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the embodiments of the first aspect described above.
[0039] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.
[0040] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.
[0041] The aforementioned method, apparatus, and retinal analysis device for acquiring retinal vascular parameters include: acquiring a retinal vascular image to be analyzed; inputting the retinal vascular image into a pre-trained retinal vascular parameter acquisition model to obtain retinal vascular parameters from the retinal vascular image; wherein the retinal vascular image is a retinal disc center image or a retinal image other than the retinal disc center image; the retinal vascular parameter acquisition model includes an encoder and multiple predictors; the retinal vascular parameter acquisition model is constructed by training a multi-task joint pre-training model, which includes an original encoder, a decoding module, and multiple original predictors; during training, the original encoder learns deep retinal vascular knowledge based on the decoding module; the encoder in the retinal vascular parameter acquisition model in the above method is trained based on the decoding module, allowing the encoder to learn more or deeper retinal vascular knowledge (i.e., the original encoder can extract robust features unrelated to the retinal disc center image during training). This approach addresses the issue of retinal vessel feature loss due to acquisition angle in subsequent applications, removes limitations on the type of input retinal vessel image, and increases the generalization ability of the retinal vessel parameter acquisition model. This enhances the applicability of the retinal vessel parameter acquisition method and improves the success rate of acquiring retinal vessel parameters for different types of retinal vessel images. Furthermore, the method eliminates the need for complex step-by-step processing; retinal vessel images can be directly input into the model for parameter acquisition. This simplifies the process, reduces complexity and errors, and improves the accuracy and efficiency of the acquired parameters. Finally, the method eliminates the need for manual intervention, reducing human error and shortening the time required to acquire retinal vessel parameters. Attached Figure Description
[0042] Figure 1 This is a flowchart illustrating a method for obtaining retinal vascular parameters in one embodiment;
[0043] Figure 2 This is a flowchart illustrating a method for acquiring retinal vascular parameters in another embodiment;
[0044] Figure 3 This is a flowchart illustrating a method for acquiring retinal vascular parameters in another embodiment;
[0045] Figure 4 This is a flowchart illustrating a method for acquiring retinal vascular parameters in another embodiment;
[0046] Figure 5 This is a flowchart illustrating a method for acquiring retinal vascular parameters in another embodiment;
[0047] Figure 6 This is a flowchart illustrating a method for acquiring retinal vascular parameters in another embodiment;
[0048] Figure 7 This is a flowchart illustrating a method for acquiring retinal vascular parameters in another embodiment;
[0049] Figure 8 This is a structural block diagram of a retinal vascular parameter acquisition device in one embodiment;
[0050] Figure 9 This is an internal structural diagram of a retinal analysis device in one embodiment. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0052] In the medical field, it is common to use medical equipment to assist in the analysis of results to determine related diseases. For example, retinal vascular features (such as vessel diameter and fractal dimension) are important biomarkers for assessing ophthalmic and cardiovascular diseases. Therefore, obtaining retinal vascular parameters corresponding to these features is particularly important. Related technologies mainly employ semi-automated software, requiring manual intervention to correct vessel segmentation and arteriovenous identification to obtain retinal vascular parameters from retinal vascular images. Alternatively, fully automated models are used with step-by-step calculations, first segmenting the retinal vascular image and then obtaining the retinal vascular parameters. However, these technologies suffer from low accuracy in obtaining retinal vascular parameters. Therefore, this application provides a method for obtaining retinal vascular parameters that improves the accuracy of the obtained parameters.
[0053] The retinal vascular parameter acquisition method provided in this application can be applied to a retinal analysis device. This retinal analysis device can have the functions of retinal lesion detection and retinal vascular parameter acquisition. This application mainly describes the retinal vascular parameter acquisition function of the retinal analysis device. Specifically, the retinal analysis device can acquire retinal vascular images of a user or animal and process the retinal vascular images based on a pre-trained retinal vascular parameter acquisition model to obtain the retinal vascular parameters in the retinal vascular images. In this application embodiment, the aforementioned retinal analysis device is a medical device. The following embodiments use the retinal analysis device as the execution subject to describe the retinal vascular parameter acquisition method.
[0054] In one exemplary embodiment, such as Figure 1 As shown, a method for obtaining retinal vascular parameters is provided. Taking the application of this method in a retinal analysis device as an example, the method can be implemented through the following steps:
[0055] S100. Obtain the retinal vessel image to be analyzed. The retinal vessel image is either a central image of the retina or a non-central image of the retina.
[0056] In practical applications, retinal analysis devices can directly acquire retinal vessel images to be analyzed in real time, or obtain pre-acquired retinal vessel images to be analyzed from local storage, disks, hard drives, or other locations. In the embodiments of this application, the size of the retinal vessel images is not limited, and the retinal vessel images are typically three-dimensional tensors.
[0057] Optionally, the aforementioned retinal-optic disc center image can be understood as an ocular imaging image taken with the retinal-optic disc as the core; the aforementioned retinal-non-optic disc center image can be understood as an ocular imaging image taken with other areas of the retina (non-optic disc) as the core.
[0058] It should be noted that different acquisition angles of retinal vascular images will result in different types of retinal vascular images in the acquired samples, namely, images of the retinal disc center or images of the retinal non-disc center.
[0059] S200. Input the retinal vessel image into a pre-trained retinal vessel parameter acquisition model to obtain the retinal vessel parameters in the retinal vessel image. The retinal vessel parameter acquisition model includes an encoder and multiple predictors. The model is constructed by training a multi-task joint pre-trained model, which includes an original encoder, a decoding module, and multiple original predictors. During training, the original encoder learns deep retinal vessel knowledge based on the decoding module.
[0060] In this embodiment, the retinal vessel parameter acquisition model can consist of an encoder and multiple predictors. Optionally, the retinal vessel parameters in the retinal vessel image can be understood as the specific numerical values corresponding to the retinal vessel features in the retinal vessel image.
[0061] Specifically, the retinal analysis device can input retinal vessel images into a pre-trained retinal vessel parameter acquisition model, which then acquires and outputs retinal vessel parameters from the retinal vessel images.
[0062] In addition, the retinal analysis device can preprocess the retinal vessel image to obtain a preprocessed image, and then input the preprocessed image into a pre-trained retinal vessel parameter acquisition model. This model extracts retinal vessel parameters from the preprocessed image and outputs them. Optionally, the above preprocessing can be normalization, cropping, data augmentation, etc., to improve image quality.
[0063] In this embodiment, the aforementioned retinal vessel parameter acquisition model is constructed by training a multi-task joint pre-training model, which includes an original encoder, a decoding module, and multiple original predictors. During training, the original encoder can learn more or deeper retinal vessel knowledge based on the decoding module (i.e., the original encoder can extract robust features unrelated to the optic disc center image during training). This enables the retinal vessel parameter acquisition model to solve the problem of retinal vessel feature loss due to the acquisition angle in subsequent applications, remove the limitation on the type of input retinal vessel image, improve the generalization ability of the retinal vessel parameter acquisition model, and thus improve the accuracy of the parameters acquired by the retinal vessel parameter acquisition model.
[0064] It should be noted that the above-mentioned multi-task joint pre-trained model may include an original encoder, a decoding module, and multiple original predictors.
[0065] The technical solution in this application embodiment acquires a retinal vessel image to be analyzed, inputs the retinal vessel image into a pre-trained retinal vessel parameter acquisition model, and obtains retinal vessel parameters from the retinal vessel image. The retinal vessel image is either a retinal image at the center of the optic disc or a retinal image outside the center of the optic disc. The retinal vessel parameter acquisition model includes an encoder and multiple predictors. The retinal vessel parameter acquisition model is constructed by training a multi-task joint pre-training model, which includes an original encoder, a decoding module, and multiple original predictors. During training, the original encoder learns deep retinal vessel knowledge based on the decoding module. In the above method, the encoder in the retinal vessel parameter acquisition model is trained based on the decoding module, allowing the encoder to learn more or deeper retinal vessel knowledge (i.e., the original encoder can extract robust features unrelated to the optic disc center image during training) for subsequent... This application enables the retinal vessel parameter acquisition model to address the problem of retinal vessel feature loss due to the acquisition perspective, removes the limitation on the type of input retinal vessel image, and increases the generalization ability of the retinal vessel parameter acquisition model. This improves the broad applicability of the retinal vessel parameter acquisition method and increases the success rate of acquiring retinal vessel parameters for different types of retinal vessel images. Furthermore, this method does not require complex step-by-step processing; retinal vessel images can be directly input into the retinal vessel parameter acquisition model to achieve retinal vessel parameter acquisition. The processing is relatively simple, reducing the complexity of retinal vessel parameter acquisition, minimizing processing errors, and thus improving the accuracy and efficiency of the acquired retinal vessel parameters. In addition, this method does not require manual intervention, thereby reducing human error and shortening the time required to acquire retinal vessel parameters.
[0066] The process of inputting a retinal vessel image into a pre-trained retinal vessel parameter acquisition model to obtain retinal vessel parameters from the image is described below. In one embodiment, as... Figure 2 As shown, the steps in S200 above can be implemented in the following ways:
[0067] S210. Input the retinal blood vessel image into the encoder to obtain the corresponding multidimensional feature vector.
[0068] In the embodiments of this application, in the retinal vascular parameter acquisition model, the output of the encoder is connected to the input of each predictor, that is, the encoder and each predictor are in a serial architecture, and the predictors are in a parallel architecture.
[0069] Specifically, the retinal analysis device can directly input retinal vessel images into an encoder, which outputs a multidimensional feature vector corresponding to the retinal vessel image. This encoder can perform dimensionality reduction processing on the retinal vessel image.
[0070] For example, if the retinal vessel image is a 512*512*3 image, the encoder can output a 1024-dimensional feature vector; wherein, the dimension of the multidimensional feature vector finally output by the encoder corresponds to the size of the retinal vessel image received by the encoder.
[0071] S220. Input the multidimensional feature vectors into each predictor to obtain the retinal vascular parameters in the retinal vascular image.
[0072] Furthermore, the multidimensional feature vectors output by the encoder can be input into each predictor, so that each predictor can process the multidimensional feature vectors and obtain the retinal vascular parameters in the retinal vascular image.
[0073] In this embodiment, there can be multiple retinal vascular parameters, and each predictor can output a retinal vascular parameter corresponding to a retinal vascular feature. These retinal vascular features can include characteristics such as the tortuosity, bifurcation angle, and diameter of the retinal vessels.
[0074] The technical solution in this application embodiment inputs the retinal vessel image into an encoder to obtain the corresponding multidimensional feature vector, and then inputs the multidimensional feature vector into each predictor to obtain the retinal vessel parameters in the retinal vessel image. The architecture of the retinal vessel parameter acquisition model in the above method is simple, which can reduce the complexity of the retinal vessel parameter acquisition method, speed up the acquisition of retinal vessel parameters, reduce processing errors, and improve the accuracy of the acquired retinal vessel parameters.
[0075] The construction process of the above-mentioned retinal vascular parameter acquisition model is described below. In one embodiment, as follows... Figure 3 As shown, the construction process of the retinal vessel parameter acquisition model can be achieved in the following way:
[0076] S300. Acquire a set of retinal vessel images. The set of retinal vessel images includes sample retinal vessel images acquired from multiple different acquisition angles within a standard viewing angle range.
[0077] In the embodiments of this application, the acquisition angles of the retinal vessel images of each sample in the retinal vessel image set may be the same or different. Optionally, the above-mentioned standard viewing angle range may be 1° to 200°; the acquisition angle of any sample retinal vessel image may be any acquisition angle within the standard viewing angle range.
[0078] Meanwhile, the aforementioned retinal vascular image set may include retinal vascular images of different users or animals from multiple acquisition perspectives, i.e., multiple sample retinal vascular images.
[0079] Specifically, retinal analysis devices can acquire multiple pre-collected retinal vascular images from local storage, disks, hard drives, the cloud, etc., forming a retinal vascular image set.
[0080] S400. Based on the retinal vessel image set, train the multi-task joint pre-training model to generate the trained model.
[0081] In practical applications, retinal analysis devices can input retinal vessel image sets into a multi-task joint pre-training model according to preset training rules, and generate a corresponding trained model after training the multi-task joint pre-training model. In the embodiments of this application, the multi-task joint pre-training model can be understood as the model to be trained (including the original encoder, decoding module and multiple original predictors) including the original predictors of multiple tasks.
[0082] The trained model may include a trained original encoder (i.e., encoder), a trained decoding module, and multiple trained original predictors (i.e., multiple predictors). In this embodiment, the output of the original encoder in the multi-task joint pre-training model is connected to the input of the decoding module, and the output of the original encoder is connected to the input of each original predictor in the multi-task joint pre-training model, that is, the original predictors are in parallel architecture.
[0083] S500: Based on the trained model, a model for obtaining retinal vascular parameters is constructed.
[0084] In this embodiment, the retinal analysis device can remove the trained decoding module from the trained model and determine the encoder and multiple predictors connected to it as the retinal vascular parameter acquisition model.
[0085] In one embodiment, such as Figure 4 As shown, the step in S400 above, which trains the multi-task joint pre-trained model based on the retinal vessel image set to generate the trained model, can be implemented in the following way:
[0086] S410. Train the original encoder and decoding module based on the retinal blood vessel image set to obtain the encoder and the trained decoding module.
[0087] Specifically, the retinal analysis device can train the original encoder and decoding module based on a set of retinal blood vessel images according to a preset training method to obtain the encoder and the trained decoding module.
[0088] The following describes the process of training the original encoder and decoder module based on the retinal blood vessel image set to obtain the encoder and trained decoder module. In one embodiment, as... Figure 5 As shown, the steps in S410 above can be implemented in the following ways:
[0089] S411. Input the retinal vessel image set into the original encoder to obtain the predicted feature vector.
[0090] Specifically, the retinal analysis device can input a set of retinal vessel images into the original encoder to obtain a predicted feature vector.
[0091] S412. Input the predicted feature vector into the decoding module to obtain the corresponding intermediate prediction result.
[0092] Furthermore, the predicted feature vector can be input into the decoding module to obtain the corresponding intermediate prediction result.
[0093] S413. Optimize the model parameters of the original encoder and decoding module based on the intermediate prediction results to obtain the encoder and the trained decoding module.
[0094] Then, the loss function can be used to calculate the loss value based on the intermediate prediction results and the standard results. If the loss value is greater than the preset threshold or the number of iterations is less than the preset number of iterations, the model parameters of the original encoder and the decoding module are optimized or updated. Based on the updated original encoder and the updated decoding module, the steps in S411-S413 above are continued until the current loss value is less than the preset threshold or the number of iterations is greater than the preset number of iterations. The current original encoder is then determined as the encoder, and the current decoding module is determined as the trained decoding module.
[0095] The aforementioned decoding module can consist of multiple decoders, with the original encoder connected to each decoder. Specifically, the retinal analysis device can input a set of retinal vessel images into the original encoder, and then input the predicted feature vectors output by the original encoder into each decoder. A loss function is used to calculate the difference between the input results of each decoder and the corresponding standard results. If all differences are greater than a preset threshold or the number of iterations is less than a preset number of iterations, the model parameters in the original encoder and each decoder are updated. The above steps are then repeated based on the updated original encoder and each updated decoder until all corresponding differences are less than the preset threshold or the number of iterations is greater than the preset number of iterations. At this point, the current original encoder is determined as the encoder, and the current decoders are determined as the trained decoding modules.
[0096] The loss function mentioned above can be the cross-entropy loss function, the focus loss function, the contrastive loss function, the adversarial loss function, etc.; at the same time, the loss function corresponding to each branch architecture composed of the original encoder and different decoders can be the same or different.
[0097] It should be noted that the above multi-task joint pre-trained model may include an original encoder, a decoding module, and multiple original predictors; wherein, the output of the original encoder is connected to the input of the decoding module.
[0098] S420: Based on the retinal blood vessel image set and encoder, each original predictor is trained to obtain each predictor.
[0099] It should be noted that after the encoder is trained, the output of the trained encoder can be connected to the input of each original predictor to further train each original predictor.
[0100] In this process, each predictor can be trained using a preset training method based on a set of retinal blood vessel images and an encoder.
[0101] In the embodiments of this application, the number of original predictors in the multi-task joint pre-training model can be set according to actual needs. That is, the original predictors in the multi-task joint pre-training model can be arbitrarily expanded, so that the final retinal vascular parameter acquisition model can acquire more types of retinal vascular parameters.
[0102] Optionally, each of the above-mentioned original predictors may be composed of a two-layer lightweight convolutional network model, but in the embodiments of this application, each of the above-mentioned original predictors may be composed of a two-layer multilayer perceptron (MLP).
[0103] In practical applications, when increasing the number of original predictors in a multi-task joint pre-trained model, the newly added original predictors can be trained directly using the pre-trained encoder in combination with the newly added original predictors, without the need to retrain the encoder.
[0104] In one embodiment, such as Figure 6 As shown, the step in S420 above, which trains each original predictor based on the retinal vessel image set and the encoder to obtain each predictor, may include:
[0105] S421. Input the retinal blood vessel image set into the encoder to obtain the trained feature vector.
[0106] S422. Input the trained feature vectors into each original predictor to obtain the predicted blood vessel parameters.
[0107] S423. Optimize the model parameters of each original predictor based on the predicted blood vessel parameters to obtain each predictor.
[0108] Furthermore, the model parameters in the encoder can be maintained during the training of each original predictor, that is, the model parameters of the encoder remain unchanged during the training of each original predictor.
[0109] Specifically, a set of retinal vessel images can be input into an encoder to obtain trained feature vectors. These trained feature vectors are then input into each original predictor. For any original predictor, the original predictor outputs the corresponding predicted vessel parameters. The predicted vessel parameters and standard vessel parameters are then substituted into the corresponding loss function to obtain the loss value. If the loss value is greater than a preset loss value or the number of iterations is less than a preset number of iterations, the model parameters of the original predictor are optimized until the final loss value is less than the preset loss value or the number of iterations is greater than the preset number of iterations. The original predictor at this point is then determined as the corresponding predictor.
[0110] During training, the loss functions corresponding to different original predictors can be the same or different; the loss functions corresponding to each original predictor can be, but are not limited to, cross-entropy loss function, focus loss function, contrastive loss function, and adversarial loss function. In the embodiments of this application, the loss function corresponding to each original predictor is mean squared error.
[0111] In fact, in the embodiments of this application, the original encoder and decoder modules in the multi-task joint pre-training model are first trained. When the two reach their optimal state, the trained encoder is then used to train each original predictor in the multi-task joint pre-training model so that each original predictor reaches its optimal state.
[0112] S430. The encoder, the trained decoding module, and each predictor are identified as the trained model.
[0113] Among them, the encoder, the trained decoding module, and each predictor obtained based on the previous steps can be determined as the trained model.
[0114] The technical solution in this application involves acquiring a set of retinal vessel images, training a multi-task joint pre-training model based on the retinal vessel image set to generate a trained model, and constructing a retinal vessel parameter acquisition model based on the trained model. The retinal vessel image set includes sample retinal vessel images acquired from multiple different acquisition angles within a standard viewing angle range. This method allows for training of the multi-task joint pre-training model based on sample retinal vessel images acquired from different acquisition angles, thus preparing for the subsequent acquisition of a retinal vessel parameter acquisition model with higher generalization ability and improving the applicability of the final acquired retinal vessel parameter acquisition model to images outside the optic disc center.
[0115] In one embodiment, such as Figure 7 As shown, the step in S413 above, which optimizes the model parameters of the original encoder and decoding module based on the intermediate prediction results to obtain the encoder and the trained decoding module, may include:
[0116] S4131, Optimize the model parameters of the original encoder and the first decoder based on the binary segmentation image of blood vessels; and optimize the model parameters of the original encoder and the second decoder based on the binary segmentation image of the optic disc; and optimize the model parameters of the original encoder and the decoding unit based on the predicted true and false probability vectors.
[0117] Specifically, the retinal analysis device can optimize the model parameters of the original encoder and the first decoder based on the binary segmented blood vessel image and the standard binary segmented blood vessel image output by the first decoder using a first loss function; optimize the model parameters of the original encoder and the second decoder based on the binary segmented optic disc image and the standard binary segmented optic disc image output by the second decoder using a second loss function; and optimize the model parameters of the original encoder and the decoding unit based on the predicted true / false probability vector and the standard probability vector output by the discriminator in the decoding unit using a third loss function.
[0118] Optionally, the predicted true / false probability vector may include the probability that the retinal optic disc center image output by the third decoder in the decoding unit is a standard optic disc center image (i.e., a true optic disc center image) and the probability that the retinal optic disc center image is not a standard optic disc center image.
[0119] In the embodiments of this application, the model parameters in the first decoder, the second decoder and the decoding unit can be optimized or updated synchronously, and when optimizing the model parameters of the first decoder, the second decoder and the decoding unit, the model parameters of the original encoder can be updated synchronously.
[0120] It should be noted that if the model parameters in the first decoder are the first to reach the optimal value, then only the model parameters of the second decoder and the decoding unit need to be updated. At the same time as updating the model parameters of the second decoder and the decoding unit, the model parameters of the original encoder are also updated. Furthermore, if the model parameters in the second decoder also reach the optimal value, then only the model parameters of the decoding unit need to be updated. At the same time as updating the model parameters of the decoding unit, the model parameters of the original encoder are also updated.
[0121] S4132. Until the current first decoder satisfies the preset first parameter optimization termination condition, the current second decoder satisfies the preset second parameter optimization termination condition, and the current decoding unit satisfies the preset third parameter optimization termination condition, the current original encoder is determined as the encoder, and the current first decoder, the current second decoder, and the current decoding unit are determined as the trained decoding module.
[0122] It should be noted that the current first decoder satisfying the first parameter optimization termination condition can be understood as follows: the corresponding first loss value determined by the first loss function based on the blood vessel binary segmentation image output by the current first decoder and the standard blood vessel binary segmentation image is less than the first preset loss value, or the first iteration number is greater than the first preset iteration number. Optionally, the first iteration number can represent the actual number of iterations for optimizing the model parameters of the first decoder.
[0123] Meanwhile, the current second decoder satisfying the second parameter optimization termination condition can be understood as follows: the corresponding second loss value determined by the second loss function based on the current second decoder's output binary segmented image and the standard binary segmented image is less than the second preset loss value, or the second iteration number is greater than the second preset iteration number. Optionally, the second iteration number can represent the actual number of iterations for optimizing the model parameters of the second decoder.
[0124] Furthermore, the current decoding unit satisfying the preset third parameter optimization termination condition can be understood as follows: the corresponding third loss value determined by the third loss function based on the predicted true / false probability vector output by the discriminator in the current decoding unit and the standard probability vector is less than the third preset loss value, or the third iteration number is greater than the third preset iteration number. Optionally, the third iteration number can represent the actual number of iterations for optimizing the model parameters of the decoding unit.
[0125] The first, second, and third loss functions mentioned above can be, but are not limited to, at least one of the following: cross-entropy loss function, focus loss function, contrastive loss function, and adversarial loss function. Optionally, the first, second, and third loss functions can be the same or different. In the embodiments of this application, the first and second loss functions can both be the sum of cross-entropy (L_CE) and Dice loss (L_Dice); the third loss function can be the sum of GAN loss (L_G / L_D) and structured loss (L1 norm).
[0126] Optionally, the first preset loss value, the second preset loss value, and the third preset loss value may be equal or unequal; the first iteration number, the second iteration number, and the third iteration number may be equal or unequal; and the first preset iteration number, the second preset iteration number, and the third preset iteration number may be equal or unequal.
[0127] It should be noted that during the training process, the current first decoder can satisfy the preset first parameter optimization termination condition, the current second decoder can satisfy the preset second parameter optimization termination condition, and the current decoding unit can satisfy the preset third parameter optimization termination condition, either synchronously or asynchronously.
[0128] In one embodiment, the decoding module includes a first decoder, a second decoder, and a decoding unit. The intermediate prediction results include a binary segmentation image of blood vessels, a binary segmentation image of the optic disc, and a true / false probability vector of the predicted optic disc center image. The step in S410 of inputting the predicted feature vector into the decoding module to obtain the corresponding intermediate prediction results may include: inputting the predicted feature vector into the first decoder to obtain a binary segmentation image of blood vessels; inputting the predicted feature vector into the second decoder to obtain a binary segmentation image of the optic disc; and inputting the predicted feature vector into the decoding unit to obtain a predicted true / false probability vector.
[0129] In this embodiment, the decoding module includes a first decoder, a second decoder, and a decoding unit. The decoding unit includes a third decoder and a discriminator. The output of the original encoder is connected to the inputs of the first decoder, the second decoder, and the third decoder, respectively. The output of the third decoder is connected to the input of the discriminator.
[0130] It should be noted that in a multi-task joint pre-trained model, the original encoder and decoder modules can be referred to as the upstream pre-trained model architecture, and each original predictor can be referred to as the downstream multi-task fine-tuning architecture. The original encoder, discriminator, first decoder, second decoder, and third decoder can all be composed of multi-layer U-Net architectures, but the number of layers in the U-Net architecture of the original encoder, discriminator, first decoder, second decoder, and third decoder can be equal or unequal.
[0131] In the embodiments of this application, the original encoder and discriminator can each be composed of a 24-layer visual Transformer model (i.e., ViT); the first decoder, the second decoder and the third decoder can each be composed of an 8-layer visual Transformer model.
[0132] In practical applications, the retinal analysis device can synchronously input the predicted feature vector into the first decoder, the second decoder, and the decoding unit. Correspondingly, the first decoder outputs the binary segmented image of the retinal vessels corresponding to the sample retinal vessel image (i.e., the binary segmented image of the vessels), the second decoder outputs the binary segmented image of the optic disc corresponding to the sample retinal vessel image, and the discriminator in the decoding unit outputs the predicted true / false probability vector.
[0133] It should be noted that the third decoder in the decoding unit can output the retinal disc center image corresponding to the sample retinal blood vessel image and input the retinal disc center image to the discriminator. The discriminator can compare the retinal disc center image with the standard retinal disc center image (i.e. the real retinal disc center image) and output the vector corresponding to the probability that the retinal disc center image is the standard retinal disc center image and is not the standard retinal disc center image, that is, the prediction true and false probability vector.
[0134] In the embodiments of this application, the training process for the original encoder, decoding module and each original predictor is similar to the traditional training process, and will not be described in detail in the embodiments of this application.
[0135] The technical solution in this application embodiment inputs the predicted feature vector into a first decoder to obtain a binary segmentation image of blood vessels, inputs the predicted feature vector into a second decoder to obtain a binary segmentation image of the optic disc, and inputs the predicted feature vector into a decoding unit to obtain a predicted true / false probability vector. The above method trains the decoding module to enable the encoder to learn more retinal blood vessel knowledge, so that the model parameters of the encoder obtained in the end are optimal, which prepares for the subsequent construction of a simple retinal blood vessel parameter acquisition model with higher generalization ability, reduces the steps in the subsequent retinal blood vessel parameter acquisition process, thereby reducing processing errors and improving the accuracy of the finally obtained retinal blood vessel parameters.
[0136] In one embodiment, this application also provides a method for acquiring retinal vascular parameters, applied to a retinal analysis device, the method comprising the following steps:
[0137] (1) Obtain the retinal vessel image to be analyzed; the retinal vessel image is either a central image of the retinal disc or a non-central image of the retina.
[0138] (2) Input the retinal vessel image into the encoder in the retinal vessel parameter acquisition model to obtain the corresponding multidimensional feature vector; wherein, the retinal vessel parameter acquisition model includes an encoder and multiple predictors; the retinal vessel parameter acquisition model is constructed by training a multi-task joint pre-training model, which includes an original encoder, a decoding module and multiple original predictors; during the training process, the original encoder learns deep retinal vessel knowledge based on the decoding module;
[0139] (3) Input the multidimensional feature vector into each predictor in the retinal vessel parameter acquisition model to obtain the retinal vessel parameters in the retinal vessel image;
[0140] The construction process of the retinal vascular parameter acquisition model includes:
[0141] (4) Acquire retinal vessel image set; The retinal vessel image set includes sample retinal vessel images acquired from multiple different acquisition angles within the standard viewing angle range;
[0142] (5) Input the retinal vessel image set into the original encoder to obtain the predicted feature vector;
[0143] (6) Input the predicted feature vector into the first decoder to obtain a binary segmentation image of blood vessels; and input the predicted feature vector into the second decoder to obtain a binary segmentation image of the optic disc; and input the predicted feature vector into the decoding unit to obtain a predicted true / false probability vector;
[0144] (7) Optimize the model parameters of the original encoder and the first decoder based on the binary segmentation image of blood vessels; and optimize the model parameters of the original encoder and the second decoder based on the binary segmentation image of the optic disc; and optimize the model parameters of the original encoder and the decoding unit based on the predicted true and false probability vectors;
[0145] (8) Until the current first decoder satisfies the preset first parameter optimization termination condition, the current second decoder satisfies the preset second parameter optimization termination condition, and the current decoding unit satisfies the preset third parameter optimization termination condition, the current original encoder is determined as the encoder, and the current first decoder, the current second decoder, and the current decoding unit are determined as the trained decoding module.
[0146] (9) Input the retinal vessel image set into the encoder to obtain the trained feature vector;
[0147] (10) Input the trained feature vectors into each original predictor to obtain the predicted blood vessel parameters;
[0148] (11) Optimize the model parameters of each original predictor based on the predicted blood vessel parameters to obtain each predictor;
[0149] (12) The encoder, the trained decoding module and each predictor are determined as the trained model;
[0150] (13) Based on the trained model, a model for obtaining retinal vascular parameters is constructed.
[0151] The specific execution process of (1) to (10) above can be found in the description of the above embodiments. The implementation principle and technical effect are similar, and will not be repeated here.
[0152] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0153] Based on the same inventive concept, this application also provides a retinal vascular parameter acquisition device for implementing the aforementioned retinal vascular parameter acquisition method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the retinal vascular parameter acquisition device provided below can be found in the limitations of the retinal vascular parameter acquisition method described above, and will not be repeated here.
[0154] In one embodiment, Figure 8 This is a schematic diagram of a retinal vascular parameter acquisition device in one embodiment of this application. The retinal vascular parameter acquisition device provided in this embodiment can be applied to retinal analysis equipment. Figure 8 As shown, the retinal vascular parameter acquisition device of this application embodiment may include: an image acquisition module 11 and a parameter acquisition module 12, wherein:
[0155] Image acquisition module 11 is used to acquire retinal vessel images to be analyzed; the retinal vessel images are either central images of the retina or non-central images of the retina.
[0156] The parameter acquisition module 12 is used to input the retinal vessel image into the pre-trained retinal vessel parameter acquisition model to obtain the retinal vessel parameters in the retinal vessel image.
[0157] The retinal vascular parameter acquisition model includes an encoder and multiple predictors. The retinal vascular parameter acquisition model is constructed by training a multi-task joint pre-trained model, which includes an original encoder, a decoding module, and multiple original predictors. During the training process, the original encoder learns deep retinal vascular knowledge based on the decoding module.
[0158] The retinal vascular parameter acquisition device provided in this application embodiment can be used to execute the technical solutions in the above-described retinal vascular parameter acquisition method embodiments of this application. Its implementation principle and technical effect are similar, and will not be repeated here.
[0159] In one embodiment, the parameter acquisition module 12 includes: a first acquisition unit and a second acquisition unit, wherein:
[0160] The first acquisition unit is used to input the retinal blood vessel image into the encoder to obtain the corresponding multidimensional feature vector;
[0161] The second acquisition unit is used to input the multidimensional feature vector into each predictor to obtain the retinal vascular parameters in the retinal vascular image.
[0162] The retinal vascular parameter acquisition device provided in this application embodiment can be used to execute the technical solutions in the above-described retinal vascular parameter acquisition method embodiments of this application. Its implementation principle and technical effect are similar, and will not be repeated here.
[0163] In one embodiment, the retinal vascular parameter acquisition device further includes:
[0164] The image set acquisition module is used to acquire a retinal vessel image set; the retinal vessel image set includes sample retinal vessel images acquired from multiple different acquisition angles within a standard viewing angle range;
[0165] The model training module is used to train the multi-task joint pre-trained model based on the retinal blood vessel image set and generate the trained model.
[0166] The model building module is used to build a model for obtaining retinal blood vessel parameters based on the trained model.
[0167] The retinal vascular parameter acquisition device provided in this application embodiment can be used to execute the technical solutions in the above-described retinal vascular parameter acquisition method embodiments of this application. Its implementation principle and technical effect are similar, and will not be repeated here.
[0168] In one embodiment, the model training module includes: a first training unit, a second training unit, and a determination unit, wherein:
[0169] The first training unit is used to train the original encoder and decoder module based on the retinal blood vessel image set to obtain the encoder and the trained decoder module.
[0170] The second training unit is used to train each original predictor based on the retinal blood vessel image set and the encoder to obtain each predictor;
[0171] The determination unit is used to determine the encoder, the trained decoding module, and each predictor as the trained model.
[0172] The retinal vascular parameter acquisition device provided in this application embodiment can be used to execute the technical solutions in the above-described retinal vascular parameter acquisition method embodiments of this application. Its implementation principle and technical effect are similar, and will not be repeated here.
[0173] In one embodiment, the first training unit includes: a first acquisition subunit, a second acquisition subunit, and a parameter optimization unit, wherein:
[0174] The first acquisition subunit is used to input the retinal vessel image set into the original encoder to obtain the predicted feature vector;
[0175] The second acquisition subunit is used to input the predicted feature vector into the decoding module to obtain the corresponding intermediate prediction result;
[0176] The parameter optimization unit is used to optimize the model parameters of the original encoder and decoder module based on the intermediate prediction results, so as to obtain the encoder and the trained decoder module.
[0177] The retinal vascular parameter acquisition device provided in this application embodiment can be used to execute the technical solutions in the above-described retinal vascular parameter acquisition method embodiments of this application. Its implementation principle and technical effect are similar, and will not be repeated here.
[0178] In one embodiment, the decoding module includes a first decoder, a second decoder, and a decoding unit. The decoding unit includes a third decoder and a discriminator. The intermediate prediction results include a binary segmentation image of blood vessels, a binary segmentation image of the optic disc, and a true / false probability vector of the predicted optic disc center image. The second acquisition subunit is specifically used for:
[0179] The predicted feature vector is input into the first decoder to obtain a binary segmentation image of blood vessels; and the predicted feature vector is input into the second decoder to obtain a binary segmentation image of the optic disc; and the predicted feature vector is input into the decoding unit to obtain a predicted true / false probability vector.
[0180] The retinal vascular parameter acquisition device provided in this application embodiment can be used to execute the technical solutions in the above-described retinal vascular parameter acquisition method embodiments of this application. Its implementation principle and technical effect are similar, and will not be repeated here.
[0181] In one embodiment, the parameter optimization unit is specifically used for:
[0182] Model parameters of the original encoder and the first decoder are optimized based on the binary segmentation image of the blood vessel; and model parameters of the original encoder and the second decoder are optimized based on the binary segmentation image of the optic disc; and model parameters of the original encoder and the decoding unit are optimized based on the predicted true and false probability vectors.
[0183] Until the current first decoder satisfies the preset first parameter optimization termination condition, the current second decoder satisfies the preset second parameter optimization termination condition, and the current decoding unit satisfies the preset third parameter optimization termination condition, the current original encoder is determined as the encoder, and the current first decoder, the current second decoder, and the current decoding unit are determined as the trained decoding module.
[0184] The retinal vascular parameter acquisition device provided in this application embodiment can be used to execute the technical solutions in the above-described retinal vascular parameter acquisition method embodiments of this application. Its implementation principle and technical effect are similar, and will not be repeated here.
[0185] In one embodiment, the second training unit is specifically used for:
[0186] The retinal vessel image set is input into the encoder to obtain the trained feature vector;
[0187] The trained feature vectors are input into each original predictor to obtain the predicted blood vessel parameters;
[0188] The model parameters of each original predictor are optimized based on the predicted blood vessel parameters to obtain each predictor.
[0189] The retinal vascular parameter acquisition device provided in this application embodiment can be used to execute the technical solutions in the above-described retinal vascular parameter acquisition method embodiments of this application. Its implementation principle and technical effect are similar, and will not be repeated here.
[0190] Specific limitations regarding the retinal vascular parameter acquisition device can be found in the limitations of the retinal vascular parameter acquisition method described above, and will not be repeated here. Each module in the aforementioned retinal vascular parameter acquisition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the retinal analysis device in hardware form or independently of it, or stored in the memory of the retinal analysis device in software form, so that the processor can call and execute the corresponding operations of each module.
[0191] In one exemplary embodiment, a retinal analysis device is provided. This retinal analysis device can be a terminal, and its internal structure diagram can be as follows: Figure 9 As shown, the retinal analysis device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for acquiring retinal vascular parameters. The display unit of the retinal analysis device is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the retinal analysis device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the retinal analysis device, or external keyboards, touchpads, or mice, etc.
[0192] Those skilled in the art will understand that Figure 9The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the retinal analysis device to which the present application is applied. A specific retinal analysis device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0193] In one exemplary embodiment, a retinal analysis device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0194] Acquire retinal vessel images to be analyzed; retinal vessel images can be central images of the retina or images of the retina outside the retina.
[0195] The retinal vessel image is input into a pre-trained retinal vessel parameter acquisition model to obtain the retinal vessel parameters in the retinal vessel image.
[0196] The retinal vascular parameter acquisition model includes an encoder and multiple predictors. The retinal vascular parameter acquisition model is constructed by training a multi-task joint pre-trained model, which includes an original encoder, a decoding module, and multiple original predictors. During the training process, the original encoder learns deep retinal vascular knowledge based on the decoding module.
[0197] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:
[0198] Acquire retinal vessel images to be analyzed; retinal vessel images can be central images of the retina or images of the retina outside the retina.
[0199] The retinal vessel image is input into a pre-trained retinal vessel parameter acquisition model to obtain the retinal vessel parameters in the retinal vessel image.
[0200] The retinal vascular parameter acquisition model includes an encoder and multiple predictors. The retinal vascular parameter acquisition model is constructed by training a multi-task joint pre-trained model, which includes an original encoder, a decoding module, and multiple original predictors. During the training process, the original encoder learns deep retinal vascular knowledge based on the decoding module.
[0201] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:
[0202] Acquire retinal vessel images to be analyzed; retinal vessel images can be central images of the retina or images of the retina outside the retina.
[0203] The retinal vessel image is input into a pre-trained retinal vessel parameter acquisition model to obtain the retinal vessel parameters in the retinal vessel image.
[0204] The retinal vascular parameter acquisition model includes an encoder and multiple predictors. The retinal vascular parameter acquisition model is constructed by training a multi-task joint pre-trained model, which includes an original encoder, a decoding module, and multiple original predictors. During the training process, the original encoder learns deep retinal vascular knowledge based on the decoding module.
[0205] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one of relational databases and non-relational databases. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0206] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0207] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for obtaining retinal vascular parameters, characterized in that, The method includes: Acquire retinal vessel images to be analyzed; the retinal vessel images are either central images of the retina or non-central images of the retina. The retinal vessel image is input into a pre-trained retinal vessel parameter acquisition model to obtain the retinal vessel parameters in the retinal vessel image; The retinal vascular parameter acquisition model includes an encoder and multiple predictors. The model is constructed by training a multi-task joint pre-trained model, which includes an original encoder, a decoding module, and multiple original predictors. During training, the original encoder learns deep retinal vascular knowledge based on the decoding module. The decoding module includes a first decoder, a second decoder, and a decoding unit; the training of the multi-task joint pre-trained model includes: The retinal vessel image set is input into the original encoder to obtain the predicted feature vector; The predicted feature vector is input into the decoding module to obtain the true and false probability vectors of the corresponding binary segmented blood vessel image, binary segmented optic disc image, and predicted optic disc center image; The model parameters of the original encoder and the first decoder are optimized based on the binary segmented blood vessel image; the model parameters of the original encoder and the second decoder are optimized based on the binary segmented optic disc image; the model parameters of the original encoder and the decoding unit are optimized based on the predicted true / false probability vector; and the encoder and the trained decoding module are obtained after the model parameter optimization. The retinal vessel parameter acquisition model is obtained based on the retinal vessel image set, the encoder, and the trained decoding module.
2. The method according to claim 1, characterized in that, The step of inputting the retinal vessel image into a pre-trained retinal vessel parameter acquisition model to acquire retinal vessel parameters from the retinal vessel image includes: The retinal vessel image is input into the encoder to obtain the corresponding multidimensional feature vector; The multidimensional feature vectors are input into each of the predictors to obtain the retinal vascular parameters in the retinal vascular image.
3. The method according to claim 1 or 2, characterized in that, The construction process of the retinal vascular parameter acquisition model includes: Acquire a set of retinal vascular images; the set of retinal vascular images includes sample retinal vascular images acquired from multiple different acquisition angles within a standard viewing angle range; The multi-task joint pre-training model is trained based on the retinal vessel image set to generate a trained model; Based on the trained model, the retinal vascular parameter acquisition model is constructed.
4. The method according to claim 1 or 2, characterized in that, The step of obtaining the retinal vessel parameter acquisition model based on the retinal vessel image set, the encoder, and the trained decoding module includes: Based on the retinal vessel image set and the encoder, each of the original predictors is trained to obtain each of the predictors; The encoder, the trained decoding module, and each of the predictors are determined as the trained model.
5. The method according to claim 1 or 2, characterized in that, The step of inputting the predicted feature vector into the decoding module to obtain the corresponding true / false probability vectors of the binary segmented blood vessel image, the binary segmented optic disc image, and the predicted optic disc center image includes: The predicted feature vector is input into the first decoder to obtain the binary segmented image of the blood vessel; and the predicted feature vector is input into the second decoder to obtain the binary segmented image of the optic disc; and the predicted feature vector is input into the decoding unit to obtain the predicted true / false probability vector.
6. The method according to claim 1 or 2, characterized in that, The process of obtaining the encoder and trained decoding module after model parameter optimization includes: Until the current first decoder obtained after the model parameters are optimized satisfies the preset first parameter optimization termination condition, the current second decoder satisfies the preset second parameter optimization termination condition, and the current decoding unit satisfies the preset third parameter optimization termination condition, the obtained current original encoder is determined as the encoder, and the obtained current first decoder, the current second decoder, and the current decoding unit are determined as the trained decoding module.
7. The method according to claim 4, characterized in that, The process of training each of the original predictors based on the retinal vessel image set and the encoder to obtain each of the predictors includes: The retinal vessel image set is input into the encoder to obtain the trained feature vector; The trained feature vectors are input into each of the original predictors to obtain the predicted blood vessel parameters; The model parameters of each of the original predictors are optimized based on the predicted blood vessel parameters to obtain each of the predicted predictors.
8. A device for acquiring retinal vascular parameters, characterized in that, The device includes: The image acquisition module is used to acquire retinal vessel images to be analyzed; the retinal vessel images are either central images of the retina or non-central images of the retina. The parameter acquisition module is used to input the retinal vessel image into a pre-trained retinal vessel parameter acquisition model to obtain the retinal vessel parameters in the retinal vessel image. The retinal vascular parameter acquisition model includes an encoder and multiple predictors. The model is constructed by training a multi-task joint pre-trained model, which includes an original encoder, a decoding module, and multiple original predictors. During training, the original encoder learns deep retinal vascular knowledge based on the decoding module. The decoding module includes a first decoder, a second decoder, and a decoding unit; the training of the multi-task joint pre-trained model includes: The retinal vessel image set is input into the original encoder to obtain the predicted feature vector; The predicted feature vector is input into the decoding module to obtain the true and false probability vectors of the corresponding binary segmented blood vessel image, binary segmented optic disc image, and predicted optic disc center image; The model parameters of the original encoder and the first decoder are optimized based on the binary segmented blood vessel image; the model parameters of the original encoder and the second decoder are optimized based on the binary segmented optic disc image; the model parameters of the original encoder and the decoding unit are optimized based on the predicted true / false probability vector; and the encoder and the trained decoding module are obtained after the model parameter optimization. The retinal vessel parameter acquisition model is obtained based on the retinal vessel image set, the encoder, and the trained decoding module.
9. A retinal analysis device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.