Choroid blood vessel image generation method and device and terminal equipment

Abstract

The invention discloses a choroid blood vessel image generation method and device and terminal equipment, and the method comprises the steps: extracting a choroid blood vessel structure on indocyanine green angiography as a label, training a segmentation model, achieving the effect of generating a high-precision choroid blood vessel image through employing a relatively fuzzy non-invasive fundus image, and improving the detection precision of the choroid blood vessel image. The accuracy of recognizing the blood vessel structure of the choroid based on the noninvasive fundus image is improved, the influence of human errors caused by manual annotation is reduced, and the accuracy of recognizing the blood vessel structure of the choroid based on the noninvasive fundus image is further improved.

Description

Technical Field

[0001] This application belongs to the field of image processing, and in particular relates to a method, apparatus and terminal device for generating choroidal blood vessel images. Background Technology

[0002] The choroid is the most densely packed tissue with blood vessels in the eye. Many ophthalmic diseases, such as myopia, retinal diseases, and macular diseases, can manifest as changes in the choroidal vascular structure. Furthermore, changes in the choroidal vascular structure are also a contributing factor to some ophthalmic diseases. For example, thinning of the choroid is a characteristic of myopia development and one of the factors that promotes its progression.

[0003] Traditional assessment methods for choroidal vessels typically involve acquiring non-invasive fundus images using color fundus photography (CFP) and manually labeling the choroidal vascular structures within these images. However, due to limitations in the resolution of color fundus photography and the inaccuracies of manual labeling, current methods for identifying choroidal vessels based on non-invasive fundus images suffer from low accuracy. Summary of the Invention

[0004] This application provides a method, apparatus, and terminal device for generating choroidal vessel images, aiming to solve the problem of low recognition accuracy of choroidal vessels based on non-invasive fundus image recognition.

[0005] In a first aspect, embodiments of this application provide a method for generating choroidal vessel images, comprising:

[0006] Acquire sample non-invasive fundus images and sample indocyanine green angiography images, and match the sample non-invasive fundus images and sample indocyanine green angiography images;

[0007] The choroidal vessels on the sample indocyanine green angiography images are extracted and used as training labels for the matching sample non-invasive fundus images. The segmentation model is trained and the trained segmentation model is determined as the target segmentation model.

[0008] Non-invasive fundus images are input into the target segmentation model to generate choroidal vessel segmentation images with the same accuracy as indocyanine green angiography images.

[0009] In one possible implementation of the first aspect above, after inputting the non-invasive fundus image into the target segmentation model to generate a choroidal vessel segmentation image with the same accuracy as the indocyanine green angiography image, the method further includes:

[0010] The choroidal vessel segmentation image is quantized based on a preset quantization dimension.

[0011] In one possible implementation of the first aspect described above, matching the sample non-invasive fundus image with the sample indocyanine green angiography image includes:

[0012] Feature point matching was performed on the vascular maps corresponding to the non-invasive fundus images and indocyanine green angiography images of the same test subject, and after alignment, CFP-ICGA sample pairs were formed.

[0013] In one possible implementation of the first aspect described above, the segmentation model includes a generator and a discriminator, extracting choroidal vessels from sample indocyanine green angiography images as training labels for matching sample non-invasive fundus images, and training the segmentation model, including:

[0014] The sample non-invasive fundus image is input into the generator to extract the vascular features of the sample non-invasive fundus image, and the extracted vascular features are merged to output the corresponding pseudo choroidal vascular image.

[0015] The pseudo choroidal vessel image output by the generator is input into the discriminator, which adjusts the network parameters of the generator based on the training labels and the pseudo choroidal vessel image.

[0016] In one possible implementation of the first aspect above, quantizing the choroidal vessel segmentation image based on a preset quantization dimension includes:

[0017] The segmented image of the choroidal vessels is input into the quantization model, which then extracts the features of the vascular structure in the segmented image of the choroidal vessels.

[0018] The quantization model outputs numerical values ​​for vessel diameter, branch angle, tortuosity, density, and complexity based on the characteristics of the vascular structure in the segmented choroidal vessel image.

[0019] In one possible implementation of the first aspect above, after extracting the vascular structure from the color fundus image using a target segmentation model and generating the target vascular image, the method further includes:

[0020] Obtain user information of the test subjects and correct the quantification results based on the user information of the test subjects.

[0021] In one possible implementation of the first aspect above, obtaining user information of the detection object and correcting the quantization result based on the user information of the detection object includes:

[0022] Obtain the corresponding quantization model based on the user information of the detection object;

[0023] The choroidal vessel segmentation image is quantized based on a quantization model corresponding to user information.

[0024] Secondly, embodiments of this application provide an apparatus for generating choroidal vessel images, the apparatus comprising:

[0025] The matching module is used to acquire sample non-invasive fundus images and sample indocyanine green angiography images, and to match the sample non-invasive fundus images and sample indocyanine green angiography images.

[0026] The training module is used to extract choroidal vessels from sample indocyanine green angiography images, which are then used as training labels for matching sample non-invasive fundus images. This is used to train the segmentation model, and the trained segmentation model is then identified as the target segmentation model.

[0027] The segmentation module is used to input non-invasive fundus images into the target segmentation model and generate choroidal vessel segmentation images with the same accuracy as indocyanine green angiography images.

[0028] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for generating choroidal vessel images as described in the first aspect above.

[0029] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method for generating choroidal vessel images as described in the first aspect above.

[0030] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0031] The beneficial effects of the embodiments in this application compared with the prior art are:

[0032] In this embodiment, the segmentation model is trained using cross-modal sample data, specifically by using the choroidal vessel images extracted from indocyanine green angiography images as training labels. This enables the trained target segmentation model to generate high-precision choroidal vessel images based on non-invasive fundus images with poor visibility, thereby improving the accuracy of choroidal vessel image recognition based on non-invasive fundus images. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the implementation of a method for generating choroidal blood vessel images according to an embodiment of this application;

[0034] Figure 2 This is a comparative schematic diagram of a CFP image and an ICGA image provided in an embodiment of this application;

[0035] Figure 3This is a schematic diagram illustrating the application of a method for generating choroidal blood vessel images according to an embodiment of this application;

[0036] Figure 4 This is a schematic diagram of an image of the choroidal vessels extracted by a segmentation model provided in an embodiment of this application;

[0037] Figure 5 This is a flowchart illustrating the implementation of a method for generating choroidal blood vessel images according to another embodiment of this application;

[0038] Figure 6 This is a schematic diagram of a blood vessel diameter image output by a quantization model provided in an embodiment of this application;

[0039] Figure 7 This is a schematic diagram of a blood vessel tortuosity image output by a quantization model provided in an embodiment of this application;

[0040] Figure 8 This is a schematic diagram of the structure of a device for generating choroidal blood vessel images according to an embodiment of this application;

[0041] Figure 9 This is a structural block diagram of a terminal device provided in one embodiment of this application. Detailed Implementation

[0042] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.

[0043] Myopia is a lifelong vision problem that can significantly impact a person's life. Any degree of myopia increases the risk of adverse complications, and high myopia can lead to serious vision-threatening diseases such as glaucoma, myopic macular degeneration, and retinal detachment.

[0044] The choroid is a soft, smooth, elastic brown membrane located between the retina and the sclera. Studies have shown that choroidal thinning is a common characteristic and contributing factor to myopia, and improving choroidal blood flow is considered an effective measure for controlling myopia. Therefore, accurately quantifying choroidal vessels and using the results as biomarkers in myopia management plays a crucial role in myopia analysis and management.

[0045] Currently, the quantification of choroidal vessels typically uses the tessellation density (TD) index, a method that does not directly reflect choroidal vessels. Specifically, fundus images are acquired using a fundus camera, and leopard-like structures (indicating exposed large choroidal vessels) are manually labeled in the images. TD is then measured based on the density of these labeled structures. However, existing quantification methods rely on manual labeling and the image quality of the fundus. Due to the low visibility of choroidal vessels in fundus images and the inherent subjectivity of the labeling process, the accuracy of choroidal vessel identification is low. On the other hand, indocyanine green angiography (ICGA) can clearly visualize large and small choroidal vessels using contrast agents, but it requires intravenous injection of the contrast agent.

[0046] Based on this, this application provides a method for generating high-precision, non-invasive choroidal vessel images using multimodal sample information. By extracting choroidal vessel structures from indocyanine green angiography images as labels, the segmentation model is trained, achieving the effect of generating high-precision choroidal vessel images even using relatively blurry non-invasive fundus images. This improves the accuracy of identifying choroidal vessel structures based on non-invasive fundus images, reduces the impact of human error caused by manual annotation, and further enhances the accuracy of identifying choroidal vessel structures based on non-invasive fundus images.

[0047] The method for generating choroidal vessel images provided in the embodiments of this application will be described below with reference to the accompanying drawings:

[0048] See Figure 1 , Figure 1 This application provides a flowchart illustrating the steps of a method for generating choroidal vessel images according to an embodiment of the present application. Figure 1 As shown in the embodiments of this application, a method for generating choroidal vessel images may specifically include the following steps:

[0049] S101, acquire the sample non-invasive fundus image and the sample indocyanine green angiography image, and match the sample non-invasive fundus image and the sample indocyanine green angiography image.

[0050] Fundus images refer to images that can record fundus structures such as the retina, optic nerve, and retinal blood vessels. Fundus structures can include the optic disc (optic nerve), retinal tissue, and choroid. The non-invasive fundus images and indocyanine green angiography images of the above samples are both images that can record fundus structures.

[0051] The aforementioned non-invasive fundus images and indocyanine green angiography images can be obtained from clinical databases. Specifically, multiple non-invasive fundus images and indocyanine green angiography images of the subjects can be obtained from existing clinical databases as sample data, and the segmentation model can be trained based on this sample data. Of course, the aforementioned non-invasive fundus images and indocyanine green angiography images can also be obtained from other data; this application does not restrict the method of obtaining sample data.

[0052] For example, the aforementioned non-invasive fundus images can be obtained using a fundus camera. Specifically, the aforementioned non-invasive fundus images can be CFP images acquired using color fundus photography (CFP). The aforementioned indocyanine green angiography images can be ICGA images acquired using indocyanine green angiography (ICGA).

[0053] See Figure 2 , Figure 2 A comparative diagram of CFP and ICGA images is shown. In CFP images, due to artifacts and low resolution in the peripheral fundus region, some blood vessels are poorly visible, while indocyanine green angiography can provide a high-penetration view of the choroidal circulation. Figure 2 A comparative diagram of CFP and ICGA images is shown, such as... Figure 2 As shown, the vascular clarity of indocyanine green angiography images is significantly better than that of color fundus images.

[0054] In one embodiment of this application, the matching of the sample non-invasive fundus image and the sample indocyanine green angiography image can specifically involve pixel-level matching. Specifically, this involves: first, extracting the retinal vessels from the sample non-invasive fundus image and the sample indocyanine green angiography image of the same detection object, performing feature detection, and then aligning them based on the feature points to form a CFP-ICGA retinal vessel pair. Then, aligning the CFP-ICGA sample pairs. Alternatively, image alignment of the CFP-ICGA sample pairs can also involve: marking feature points in the sample CFP image and marking feature points at corresponding positions in the sample ICGA image; or marking feature points in the ICGA image and then marking feature points at corresponding positions in the sample CFP image, and then performing image alignment based on the feature points, thereby achieving pixel-level matching between the two.

[0055] It should be noted that when marking feature points, you can choose to mark them at easily identifiable locations such as the corners or branches of blood vessels, or at locations such as the optic nerve head.

[0056] In practical applications, a pre-trained ICGA vessel segmentation model can be used to segment the choroidal vessels in the sample indocyanine green angiography image, extract the choroidal vessels on the sample indocyanine green angiography image, and then select one or more key locations from them, marking the key locations as feature points (i.e., the second feature points mentioned above).

[0057] For sample color fundus images, the first feature point can be obtained by marking the position corresponding to the second feature point.

[0058] By selecting multiple key locations for marking, each first feature point can be aligned with its corresponding second feature point, thereby achieving the matching of the above sample non-invasive fundus image with the sample indocyanine green angiography image.

[0059] It is important to understand that non-invasive fundus images and indocyanine green angiography images of the same target may have discrepancies due to differences in acquisition order and angle. By performing operations such as deflection and translation on the indocyanine green angiography images to align them with the non-invasive fundus images, the angular deviation between the vascular structures in the non-invasive fundus images and the vascular structures in the indocyanine green angiography images can be effectively reduced, thus improving the training efficiency and accuracy of the initial segmentation model.

[0060] In practical applications, to improve the quality of sample images, the obtained non-invasive fundus images and indocyanine green angiography images can be screened. For example, professionals, such as ophthalmologists, can manually review the obtained CFA-ICGA sample pairs to remove samples with incorrect choroidal vessel segmentation or unclear choroidal vessels in the ICGA.

[0061] S102, extract the choroidal vessels from the matched indocyanine green angiography images and use them as training labels for the matched non-invasive fundus images to train the segmentation model. The trained segmentation model is then determined as the target segmentation model.

[0062] Here, the purpose of training the segmentation model is to enable the segmentation model (i.e., the target segmentation model) to generate choroidal vessel images based on the input non-invasive fundus images, and the accuracy of the generated choroidal vessel images is close to that of the choroidal vessel images extracted from indocyanine green angiography images, that is, to achieve the effect of generating high-precision choroidal vessel images using relatively blurry non-invasive fundus images.

[0063] Understandably, since the visibility of non-invasive fundus images is lower than that of indocyanine green angiography images, this embodiment of the application extracts the choroidal vessels from the sample indocyanine green angiography images and uses them as training labels. The segmentation model continuously adjusts the model parameters by utilizing the error between the choroidal vessels extracted from the sample non-invasive fundus images and the labels. Therefore, the segmentation model after training can extract choroidal vessels similar to those extracted from indocyanine green angiography images from non-invasive fundus images, thereby improving the accuracy of choroidal vessel segmentation in non-invasive fundus images and improving the accuracy of choroidal vessel recognition.

[0064] In this embodiment, cross-modal paired images (i.e., the CFP-ICGA sample pairs mentioned above) are used to train the segmentation model, so that the segmentation model (i.e. the target segmentation model) after training can also generate choroidal vessel images similar to ICGA images based on CFP images, which can provide a more accurate basis for subsequent quantization.

[0065] In practical applications, the above segmentation model can be used, but is not limited to, Unet neural networks, generative adversarial networks (GANs), etc.

[0066] It should be noted that the extraction of choroidal vessels from sample indocyanine green angiography images can be achieved by using an existing pre-trained ICGA vessel segmentation model to segment the choroidal vessels in the sample indocyanine green angiography images, thereby extracting the choroidal vessel images from the sample indocyanine green angiography images and using them as training labels for the segmentation model.

[0067] S103: Input the non-invasive fundus image into the target segmentation model to generate a choroidal vessel segmentation image with the same accuracy as the indocyanine green angiography image.

[0068] It is understood that the method for generating choroidal vessel images in this application includes a segmentation model training stage and a stage of obtaining choroidal vessel images using a target segmentation model. That is, the method in this application can include a training stage and an application stage. S101 to S102 belong to the training stage, and S103 belongs to the application stage. The aforementioned non-invasive fundus image is an image obtained by photographing the fundus of the object requiring fundus analysis during the application process. After obtaining the non-invasive fundus image, it can be input into the target segmentation model, which can then identify and extract choroidal vessel segmentation images similar to ICGA images.

[0069] Here, the correspondence accuracy in the choroidal vessel segmentation image corresponding to the indocyanine green angiography image refers to the similarity between the generated choroidal vessel segmentation image and the choroidal segmentation image generated from the indocyanine green angiography image, for example, the similarity between the two exceeds the similarity threshold.

[0070] It should be noted that the above similarity threshold can be set according to the accuracy required for actual application, such as 98%, etc. This application does not impose specific restrictions on it.

[0071] It should be noted that the non-invasive fundus images mentioned in the embodiments of this application can be, in addition to color fundus images, ultra-wide-angle fundus photography, multispectral fundus photography, infrared or near-infrared fundus photography. The aforementioned indocyanine green angiography images may also include ultra-wide-angle ICGA.

[0072] As can be seen from this, the method for generating choroidal vessel images provided in this application can train a segmentation model using cross-modal sample data. Specifically, the choroidal vessel images extracted from indocyanine green angiography images are used as training labels, enabling the trained target segmentation model to generate high-precision choroidal vessel images based on non-invasive fundus images with poor visibility. This improves the accuracy of choroidal vessel image recognition based on non-invasive fundus images.

[0073] In one embodiment of this application, the above segmentation model can be built based on a generative adversarial network.

[0074] Please see Figure 3 , Figure 3 This illustration shows an application diagram of the method for generating choroidal vessel images provided in an embodiment of this application. For example... Figure 3 As shown, the choroidal vessel image can be extracted from the ICGA image in the matched CFP-ICGA sample pair. Then, the extracted choroidal vessel image is used as the label of the matched CFP image, and a generative adversarial network is trained based on this to segment the choroidal vessel image from the CFP image.

[0075] In practical applications, the aforementioned segmentation model can include a generator and a discriminator. The generator can be used to identify and extract vascular structures in sample non-invasive fundus images to obtain choroidal vessel images. The discriminator can be used to determine the error between the choroidal vessel image extracted by the generator and the label, and adjust the model parameters of the segmentation model based on this error.

[0076] In specific applications, the generator mentioned above may include a convolutional neural network. When training the segmentation model, sample non-invasive fundus images can be input into the generator. The generator can extract shape features related to blood vessels in the sample non-invasive fundus images, merge the extracted features, and output the corresponding choroidal vascular images.

[0077] The discriminator updates the generator's model coefficients based on the choroidal vessel image, label (i.e., the choroidal vessel image extracted from the indocyanine green angiography image), and loss function output by the generator.

[0078] In practical applications, the sample data can be divided into training, validation, and test sets. To facilitate model processing, the size of the sample non-invasive fundus images can be adjusted to 512×512 pixels. To avoid overfitting, data augmentation techniques can be used, such as random brightness and contrast, random gamma, random resizing and cropping, random horizontal or vertical flipping, random channel shuffling, and random rotation, to preprocess the sample non-invasive fundus images. The batch size for model training can be set to 4, the learning rate to 0.0002, and each training session to 50 epochs.

[0079] In practical applications, the hyperparameters of the segmentation model can be adjusted using a validation set. The hyperparameters of the segmentation model can be adjusted based on the area under the ROC curve and the coordinate axis (AUC). Specifically, the model with the highest AUC can be selected for the testing phase. During the testing phase, the performance of the trained target segmentation model can be evaluated using a test set.

[0080] In practical applications, the performance of a target segmentation model can be evaluated based on AUC, the accuracy of the target segmentation model in extracting vascular structures, the Dice coefficient, the sensitivity of the target segmentation model, and its specificity. The Dice coefficient describes the similarity between the vascular structures extracted by the target segmentation model and the vascular structures in the angiography sample image.

[0081] Please see Figure 4 , Figure 4This illustration shows a schematic diagram of an image of the choroidal vessels extracted by the segmentation model provided in an embodiment of this application. Figure 4 As can be seen, the segmentation model provided in this application embodiment can effectively segment choroidal vessel images from non-invasive fundus images with an accuracy close to that of indocyanine green angiography images.

[0082] In this embodiment of the application, a generative adversarial network is trained by using cross-modal paired images, thereby enabling the target segmentation model to synthesize choroidal vessel images similar to ICGA images from CFP images, thereby improving the accuracy and efficiency of non-invasive fundus image extraction of choroidal vessel images.

[0083] In one embodiment of this application, as Figure 5 As shown, another embodiment of the method for generating choroidal vessel images provided in this application may further include the following steps after S103:

[0084] S104: Quantize the segmented choroidal vessel image based on a preset quantization dimension.

[0085] In some embodiments, the aforementioned preset quantification dimensions may specifically include one or more quantification standard parameters from five categories: vascular caliber, vascular branching angle, vascular tortuosity, vascular density, and vascular complexity.

[0086] A category's preset quantification dimensions can include multiple quantification standard parameters. For example, vascular density can include area density, skeleton density, length, etc., vascular branching angle can include angle asymmetry coefficient, branching coefficient, boundary index deviation coefficient, etc., and vascular tortuosity can include turning angle, linear regression tortuosity, angle-based tortuosity, square curvature tortuosity, etc.

[0087] It should be noted that the aforementioned preset quantitative dimensions may also include other quantitative indicators.

[0088] In practical applications, after obtaining the segmented image of the choroidal vessels, the segmented image of the choroidal vessels can be quantified according to the above-mentioned preset quantification dimensions to obtain the corresponding quantification results. In subsequent practice, the quantification results of multiple preset dimensions can be used as biomarkers in the management of myopia or other retinal and choroidal diseases. That is, the choroidal vessel status of the test subject can be analyzed through the quantification results, thereby enabling a more comprehensive analysis and management of myopia in the test subject.

[0089] In one embodiment of this application, S104 may specifically be: using a quantization model to quantize the choroidal vessel segmentation image to obtain a quantization result corresponding to the choroidal vessel segmentation image.

[0090] The quantization model can be used to quantify the vascular structure in the segmented choroidal vessel image. Specifically, the segmented choroidal vessel image can be input into the quantization model, which will automatically extract the features of the vascular structure in the segmented choroidal vessel image and automatically output the quantization results corresponding to the preset quantization dimensions. That is, when the segmented choroidal vessel image is input into the quantization model, the values ​​of the above-mentioned preset quantization dimensions corresponding to the segmented choroidal vessel image can be automatically output, namely the values ​​corresponding to the vessel diameter, vessel branch angle, vessel tortuosity, vessel density, and vessel complexity.

[0091] In other embodiments, the quantization model can also generate images of vessel diameter, vessel branch angle, vessel tortuosity, vessel density, and vessel complexity corresponding to the segmented choroidal vessel images.

[0092] For example, the corresponding blood vessels in the choroidal vessel segmentation image can be color-coded according to different blood vessel diameters, branching angles, tortuosity, density, and complexity. For instance, the larger the blood vessel diameter, the darker the color; the smaller the blood vessel diameter, the lighter the color; the larger the branching angle, the darker the color; the smaller the branching angle, the lighter the color; the greater the tortuosity, the darker the color; the smaller the tortuosity, the lighter the color; areas with higher blood vessel density are marked with darker colors, and areas with lower blood vessel density are marked with lighter colors; areas with higher blood vessel complexity are marked with darker colors, and areas with lower blood vessel complexity are marked with lighter colors, etc., to generate corresponding images.

[0093] It is understood that the above marking method is only an example and not a limitation. When generating images corresponding to the above multiple quantization results, other marking methods can also be used, such as marking brightness, marking different shapes, etc. Of course, the above color marking method is also only an example and not a limitation.

[0094] For example, the blood vessel diameter image finally output by the above quantization model can be as follows: Figure 6 As shown, the blood vessel tortuosity image output by the quantization model can be as follows: Figure 7 As shown.

[0095] In practical applications, by quantifying the vascular structure in choroidal vascular images and generating images such as vascular diameter, vascular branch angle, vascular tortuosity, vascular density, and vascular complexity corresponding to the vascular structure, it is possible to obtain vascular structure information in the target vascular image more intuitively, which facilitates accurate analysis of the fundus condition of the detected object.

[0096] In one embodiment of this application, the method for generating the above-mentioned choroidal vessel image may further include the following steps:

[0097] Obtain user information of the test subjects and correct the quantification results based on the user information of the test subjects.

[0098] In the embodiments of this application, the distribution of choroidal vessels varies among different population groups. For example, people of different heights have different choroidal vessel density, vessel diameter (referring to the largest vessel diameter), and vessel branching angle. Taller individuals have significantly higher choroidal vessel density, vessel diameter (referring to the largest vessel diameter), and vessel branching angle. The choroidal vascular coefficient also differs between genders and between different races. Furthermore, the choroidal vascular coefficient, including choroidal vessel density, vessel diameter (referring to the largest vessel diameter), and vessel branching angle, changes with age. The choroidal vessel density, vessel diameter (referring to the largest vessel diameter), and choroidal branching angle also differ among people with different levels of education.

[0099] Therefore, to improve the accuracy of the quantification results, the quantification results of the choroidal vessel segmentation image obtained from the non-invasive fundus image of the test subject can be corrected based on the user information of the test subject. This user information may include, but is not limited to, the user's age, gender, ethnicity, education level, and height.

[0100] In some embodiments of this application, the above-mentioned correction process may be to correct the parameters of the quantization model, that is, to set different parameters for different user groups, so that the quantization model corresponding to different user groups can quantify the choroidal vessel segmentation image of that user group, so as to correct the impact of risk factors on the analysis results.

[0101] Based on this, the above-mentioned correction of the quantization results according to the user information of the detection object is specifically to obtain the corresponding quantization model according to the user information of the detection object, and to quantize the choroidal vessel segmentation image based on the quantization model corresponding to the user information. That is, the choroidal vessel segmentation image output by the target segmentation model is input into the quantization model corresponding to the user information for quantization, so as to obtain the quantization results corresponding to multiple preset quantization dimensions, so as to manage and analyze myopia of the detection object.

[0102] It should be noted that during the training of the quantization model, a quantization model can be trained for different user groups, so that in the application stage, the corresponding quantization model can be obtained based on the user information of the detection object.

[0103] For example, if quantization model A is a quantization model for a user group aged 26-35, of Caucasian race, with a high school education, female gender, and a height of 160-165cm, and quantization model B is a quantization model for a user group aged 36-45, of Caucasian race, with a bachelor's degree education, female gender, and a height of 150-159cm, and assuming the obtained user information of the detection object is: 28-year-old female, Caucasian, with a high school education and a height of 163cm, then quantization model A can be called to quantize the choroidal vessel segmentation image output by the target segmentation model.

[0104] In some embodiments, setting different quantitative models for different user groups requires a lot of time and resources. Therefore, it is possible to obtain the degree of influence of different user information (i.e., risk factors) on the quantitative results, thereby determining the correction parameters for different risk factors, and then correcting the quantitative results output by the quantitative model based on the correction parameters.

[0105] In this embodiment of the application, by analyzing the degree of influence of risk factors on the quantification results and correcting the quantification results based on the shared factors, the reliability of the quantification results can be improved.

[0106] See Figure 8 , Figure 8 This illustration shows a schematic diagram of a choroidal vessel image generation device 80 according to an embodiment of this application. Specifically, it may include a matching module 801, a training module 802, and a segmentation model 803, wherein:

[0107] The matching module 801 is used to acquire sample non-invasive fundus images and sample indocyanine green angiography images, and to match the sample non-invasive fundus images and sample indocyanine green angiography images;

[0108] The training module 802 is used to extract choroidal vessels from the sample indocyanine green angiography image, use them as training labels for the matching sample non-invasive fundus image, train the segmentation model, and determine the trained segmentation model as the target segmentation model.

[0109] The segmentation module 803 is used to input non-invasive fundus images into the target segmentation model to generate choroidal vessel segmentation images with the same accuracy as indocyanine green angiography images.

[0110] In one implementation, the above-mentioned choroidal vessel image generation device 80 further includes a quantization module.

[0111] The quantization module is used to quantize choroidal vessel segmentation images based on a preset quantization dimension.

[0112] In one implementation, the matching module 801 can be specifically used to: form a CFP-ICGA sample pair by combining a sample non-invasive fundus image and a sample indocyanine green angiography image of the same detection object; and perform image alignment on the CFP-ICGA samples.

[0113] In one implementation, the segmentation model includes a generator and a discriminator. The training module 802 is specifically used to input the sample non-invasive fundus image into the generator to extract the vascular features of the sample non-invasive fundus image, and to merge the extracted vascular features to output the corresponding pseudo choroidal vascular image.

[0114] The pseudo choroidal vessel image output by the generator is input into the discriminator, which adjusts the network parameters of the generator based on the training labels and the pseudo choroidal vessel image.

[0115] In one implementation, the quantization module is specifically used to input the choroidal vessel segmentation image into the quantization model, and the quantization model extracts the features of the vascular structure in the choroidal vessel segmentation image; the quantization model outputs the values ​​corresponding to the vessel diameter, vessel branch angle, vessel tortuosity, vessel density and vessel complexity based on the features of the vascular structure in the choroidal vessel segmentation image.

[0116] In one implementation, the above-mentioned choroidal vessel image generation device 80 further includes a correction module.

[0117] The aforementioned correction module is used to obtain user information of the detection object and correct the quantification results based on the user information of the detection object.

[0118] In one implementation, the aforementioned correction module is specifically used to: obtain the corresponding quantization model based on the user information of the detection object; and quantize the choroidal vessel segmentation image based on the quantization model corresponding to the user information.

[0119] The choroidal vessel image generation device provided in this application embodiment belongs to the same inventive concept as the choroidal vessel image generation method provided in this application embodiment. Therefore, the choroidal vessel image generation device provided in this application embodiment can also train the segmentation model through cross-modal sample data, that is, use the choroidal vessel image in the extracted indocyanine green angiography image as the training label, so that the target segmentation model after training can generate high-precision choroidal vessel images based on non-invasive fundus images with poor visibility, thereby improving the accuracy of choroidal vessel image recognition based on non-invasive fundus images.

[0120] It should be noted that the information interaction and execution process between the above-mentioned devices are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0121] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0122] Figure 9 This is a schematic diagram of the structure of a terminal device provided in another embodiment of this application. For example... Figure 9 As shown, the terminal device 90 provided in this embodiment includes: a processor 901, a memory 902, and a computer program 903 stored in the memory 902 and executable on the processor 901, such as an image segmentation program. When the processor 901 executes the computer program 903, it implements the steps in the various embodiments of the choroidal vessel image generation method described above, for example... Figure 1 S101 to S103 are shown. Alternatively, when the processor 901 executes the computer program 903, it implements the functions of each module / unit in the above-described terminal device embodiments, for example... Figure 8 The functions of units 801 to 803 are shown.

[0123] For example, the computer program 903 described above can be divided into one or more modules / units. One or more of these modules / units are stored in the memory 902 and executed by the processor 901 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 903 in the terminal device 9. For example, the computer program 903 can be divided into an acquisition unit, a determination unit, and a calculation unit. For the specific functions of each unit, please refer to [link to relevant documentation]. Figure 8 The relevant descriptions in the corresponding embodiments are not repeated here.

[0124] The aforementioned terminal device may include, but is not limited to, a processor 901 and a memory 902. Those skilled in the art will understand that... Figure 9 This is merely an example of terminal device 90 and does not constitute a limitation on terminal device 90. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device described above may also include input / output devices, network access devices, buses, etc.

[0125] The processor 901 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0126] The aforementioned memory 902 can be an internal storage unit of the terminal device 90, such as a hard disk or RAM of the terminal device 90. The aforementioned memory 902 can also be an external storage device of the terminal device 90, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 90. Furthermore, the aforementioned memory 902 can include both internal and external storage units of the terminal device 90. The aforementioned memory 902 is used to store the aforementioned computer program and other programs and data required by the terminal device. The aforementioned memory 902 can also be used to temporarily store data that has been output or will be output.

[0127] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, can implement the above-described choroidal vessel image generation method.

[0128] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the above-described choroidal vessel image generation method.

[0129] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the terminal device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A method of generating a choroidal vessel image, characterized by, The method comprises: obtaining sample non-invasive fundus images and sample indocyanine green angiography images, and matching the sample non-invasive fundus images and the sample indocyanine green angiography images; extracting choroidal blood vessels on the sample indocyanine green angiography images as training labels of the matched sample non-invasive fundus images, training a segmentation model, and determining the trained segmentation model as a target segmentation model; inputting a non-invasive fundus image into the target segmentation model to generate a choroidal blood vessel segmentation image with corresponding accuracy of an indocyanine green angiography image.

2. The method of generating a choroidal vessel image according to claim 1, wherein, After inputting a non-invasive fundus image into the target segmentation model to generate a choroidal blood vessel segmentation image with corresponding accuracy of an indocyanine green angiography image, the method further comprises: quantizing the choroidal blood vessel segmentation image based on a preset quantization dimension.

3. The method of generating a choroidal vessel image according to claim 1, wherein, The matching of the sample non-invasive fundus images and the sample indocyanine green angiography images comprises: performing feature point matching on blood vessel images corresponding to the sample non-invasive fundus images and the sample indocyanine green angiography images of the same detection object, and aligning to form a CFP-ICGA sample pair.

4. The method of generating a choroidal vessel image according to claim 2, wherein The segmentation model comprises a generator and a discriminator, and the extraction of the choroidal blood vessels on the sample indocyanine green angiography images as the training labels of the matched sample non-invasive fundus images, and the training of the segmentation model comprise: inputting the sample non-invasive fundus images into the generator to extract blood vessel features of the sample non-invasive fundus images, combining the extracted blood vessel features, and outputting a corresponding pseudo choroidal blood vessel image; inputting the pseudo choroidal blood vessel image output by the generator into the discriminator, and adjusting network parameters of the generator based on the training labels and the pseudo choroidal blood vessel image by the discriminator.

5. The method of generating a choroidal vessel image according to claim 2, wherein The quantization of the choroidal blood vessel segmentation image based on the preset quantization dimension comprises: inputting the choroidal blood vessel segmentation image into the quantization model, extracting features of blood vessel structures in the choroidal blood vessel segmentation image by the quantization model, and outputting numerical values corresponding to blood vessel diameters, blood vessel branch angles, blood vessel curvatures, blood vessel densities, and blood vessel complexities according to the features of the blood vessel structures in the choroidal blood vessel segmentation image by the quantization model. After the extraction of the blood vessel structures of the color fundus image by the target segmentation model and the generation of the target blood vessel image, the method further comprises:

6. The method of generating a choroidal vessel image according to any one of claims 2 to 5, wherein obtaining user information of the detection object, and correcting the quantization result according to the user information of the detection object. The obtaining of the user information of the detection object and the correction of the quantization result according to the user information of the detection object comprise:

7. The method of generating a choroidal vessel image according to claim 6, wherein, obtaining a corresponding quantization model according to the user information of the detection object; quantizing the choroidal blood vessel segmentation image based on the quantization model corresponding to the user information. The non-invasive fundus photography comprises any one of fundus color photography, ultra-wide-angle fundus photography, multi-spectral fundus photography, infrared or near-infrared fundus photography, and the indocyanine green angiography image further comprises ultra-wide-angle ICGA.

8. The method of generating a choroidal vessel image according to any one of claims 1 to 7, wherein, The method comprises:

9. A choroidal blood vessel image generation apparatus characterized by comprising: ​ The matching module is configured to acquire a sample non-invasive fundus image and a sample indocyanine green angiography image, and match the sample non-invasive fundus image and the sample indocyanine green angiography image. The training module is configured to extract choroidal blood vessels on the sample indocyanine green angiography image as a training label of the matched sample non-invasive fundus image, train a segmentation model, and determine a trained segmentation model as a target segmentation model. The segmentation module is configured to input a non-invasive fundus image into the target segmentation model to generate a choroidal blood vessel segmentation image with a corresponding accuracy of an indocyanine green angiography image.

10. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-8.

11. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: The computer program is executed by the processor to implement the method of any one of claims 1-8.

Images