Retinal structure extraction method, system and application for OCTA images

Abstract

The present invention discloses a retinal structure extraction method, system and application for OCTA images. The retinal structure extraction method for OCTA images includes: acquiring an OCTA image; extracting multi-scale fusion features corresponding to each sub-layer from multiple sub-layers of the OCTA image; extracting voting gate features from multiple different voting gate modules; detecting key points of the voting gate features to obtain key point features; and using the key point features and selectable partial voting gate features as feature extraction results of the OCTA image. The retinal structure extraction method provided by the present invention utilizes the rich sub-layer information of the OCTA image for comprehensive extraction, fully integrating the characteristics of different sub-layers, and is a new end-to-end multi-task learning framework that jointly learns feature images in OCTA images, providing an effective and accurate way for quantitative analysis of retinal vascular indicators.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, in particular to the field of eye optical coherence tomography vascular image recognition, and more particularly to a retinal structure extraction method, system and application for OCTA images. Background Art

[0002] Optical coherence tomography angiography (OCTA) is a fast, non-invasive imaging technique based on the optical coherence tomography (OCT) platform that can generate images containing functional information of the retinal vascular and microvascular systems. The quantitative information of retinal indices obtained from OCTA images plays a vital role in the quantitative research and clinical decision-making of eye diseases and neurodegenerative diseases.

[0003] For example, the eyes of patients with Alzheimer's disease (AD) show a significant decrease in retinal blood vessel density compared to healthy controls. In some retinal diseases, the size and connection density of the foveal avascular zone (FAZ) are also significantly different from those of healthy controls.

[0004] Based on the above reasons, automatically and accurately extracting the characteristic images of these retinal structures from OCTA is of great significance for the early diagnosis of retinal circulation-related diseases and the assessment of disease progression.

[0005] However, existing work on extracting retinal feature images from OCTA images only focuses on single-task learning, which means that if multiple structural indicators need to be quantified, multiple different networks need to be trained separately. In the prior art, a feasible solution is to use multi-task learning (MTL) to train a model that can perform multiple tasks simultaneously, rather than building a set of independent networks. However, current MTL work is specifically designed for natural images, which limits their applicability to OCTA images due to some challenges. Most importantly, the above-mentioned methods take a single two-dimensional image as input, which makes it difficult to fully utilize the rich sub-layer information of OCTA images. Summary of the invention

[0006] In view of the deficiencies in the prior art, the object of the present invention is to provide a retinal structure extraction method, system and application for OCTA images.

[0007] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention includes:

[0008] In a first aspect, the present invention provides a method for extracting retinal structure from OCTA images, comprising:

[0009] 1) acquiring an OCTA image, wherein the OCTA image includes a plurality of sub-layers;

[0010] 2) extracting a multi-scale fusion feature corresponding to each sub-layer from the multiple sub-layers of the OCTA image;

[0011] 3) extracting voting gate features from the plurality of multi-scale fusion features using a plurality of different voting gate modules;

[0012] 4) Detecting key points of the voting gate features to obtain key point features;

[0013] 5) The key point features and the optional partial voting gate features are used as feature extraction results of the OCTA image.

[0014] In a second aspect, the present invention further provides a retinal structure extraction system for OCTA images, comprising a multi-task feature extraction network based on a voting mechanism, wherein the multi-task feature extraction network comprises:

[0015] A feature extraction module, comprising a plurality of feature extractors, wherein the plurality of feature extractors are respectively used to extract multi-scale fusion features corresponding to each sub-layer from different input OCTA image sub-layers,

[0016] Multiple independent task-specific voting gate modules to adaptively select the most important features for each task; and

[0017] Key point detection module, including:

[0018] The first task unit based on heat map regression is used to locate key points.

[0019] The second task unit based on region classification is used to identify the type of key points.

[0020] In a third aspect, the present invention further provides a neural network training method, comprising:

[0021] Acquire a training image, and mark the training image to generate marking information;

[0022] Using the above retinal structure extraction method to perform feature extraction on the training image, to obtain a feature extraction result corresponding to the training image;

[0023] The neural network used in the retinal structure extraction method is trained according to the marking information and the feature extraction result.

[0024] In a fourth aspect, the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps of the above-mentioned retinal structure extraction method when executing the computer program.

[0025] In a fifth aspect, the present invention further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned retinal structure extraction method are implemented.

[0026] Based on the above technical solution, compared with the prior art, the beneficial effects of the present invention at least include:

[0027] An OCTA retinal structure extraction method provided by the present invention utilizes the rich sub-layer information of OCTA images for comprehensive extraction, and utilizes a voting mechanism to extract voting gate features from the extracted multi-scale fusion features and identify some key point features therein based on the voting gate features, and finally fuses the key point features and some voting gate features as the final feature extraction result. This technical solution fully integrates the characteristics of different sub-layers and is a new end-to-end multi-task learning framework that jointly learns the feature images in OCTA images, providing another effective and accurate way for the quantitative analysis of retinal vascular indicators.

[0028] The above description is only an overview of the technical solution of the present invention. In order to enable those skilled in the art to more clearly understand the technical means of the present application and implement them according to the contents of the specification, the following is a description of the preferred embodiments of the present invention in conjunction with detailed drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of a simplified process of a method for extracting retinal structure from OCTA images provided by a typical embodiment of the present invention;

[0030] Figure 2 It is a schematic diagram of an extraction process of a retinal structure extraction method for OCTA images provided by a typical implementation scheme provided in an embodiment of the present invention;

[0031] Figure 3 It is a flowchart of a key point detection step of the voting gate feature in a retinal structure extraction method for OCTA images provided by a typical implementation scheme provided by an embodiment of the present invention;

[0032] Figure 4 It is a sub-layer image and a feature extraction result image used by a retinal structure extraction method for OCTA images provided by a typical implementation scheme provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In view of the deficiencies in the prior art, the inventor of this case has proposed the technical solution of the present invention after long-term research and extensive practice. The technical solution, its implementation process and principle will be further explained as follows.

[0034] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0035] Moreover, relational terms such as “first” and “second” are merely used to distinguish one component or method step from another with the same name, but do not necessarily require or imply any such actual relationship or order between these components or method steps.

[0036] See also Figure 1 and Figure 2 The embodiment of the present invention provides a method for extracting retinal structure from OCTA images, comprising the following steps:

[0037] 1) Acquire an OCTA image, wherein the OCTA image includes multiple sub-layers.

[0038] 2) Extracting multi-scale fusion features corresponding to each sub-layer from the multiple sub-layers of the OCTA image.

[0039] 3) Using a plurality of different voting gate modules to extract voting gate features from the plurality of multi-scale fusion features.

[0040] 4) Detect key points of the voting gate features to obtain key point features.

[0041] 5) The key point features and the optional partial voting gate features are used as feature extraction results of the OCTA image.

[0042] In some embodiments, the sublayers in step 1) may include at least a superficial vascular complex layer, a deep vascular complex layer, and a panneural plexus retinal layer.

[0043] In some embodiments, the key points in step 4) may include at least vascular bifurcation points and vascular intersection points.

[0044] In some embodiments, the feature extraction result in step 5) may include at least a retinal vascular image, a foveal avascular zone image, and a retinal vascular bifurcation / intersection image.

[0045] The above-mentioned embodiment of the present invention proposes an end-to-end multi-task learning algorithm, which uses the superficial vascular complex (SVC), deep vascular complex (DVC) and pan-plexus retinal (IVC) images to jointly learn the retinal vessels (RV), foveal avascular zone (FAZ) and retinal vascular bifurcation / junction points (RVJ) in OCTA images, providing another effective way for the quantitative analysis of retinal structural indicators and the diagnosis of fundus diseases.

[0046] It should be noted that the OCTA retinal structure extraction method provided by the present invention only involves the process of performing image processing based on OCTA images and proposing feature results, and does not involve subsequent disease diagnosis and treatment methods, nor does it involve methods of invading the human body to obtain samples or data. The extraction results obtained based on the OCTA retinal structure extraction method provided by the present invention alone are completely incapable of diagnosing the disease, and a diagnosis must be made in combination with a doctor's clinical judgment. That is, the extraction results obtained by the present invention are completely intermediate information or reference information, and do not belong to direct disease diagnosis information. The technical solution provided by the present invention does not belong to a disease diagnosis and treatment method.

[0047] Continue to see Figure 2 In some embodiments, step 2) may specifically include the following steps:

[0048] The multiple sub-layers are input into the first convolutional layers of multiple feature extractors to obtain multiple first convolutional features corresponding to each of the sub-layers.

[0049] Input the plurality of the first convolutional features into the remaining convolutional layers of the plurality of feature extractors respectively to obtain the multi-scale fusion features corresponding to each of the sub-layers.

[0050] In some embodiments, the weight parameters of the multiple feature extractors may be the same.

[0051] In some embodiments, the feature extractors may all be composed of a ResNet-50 network, and the convolution size of the first convolutional layer may be 3×3.

[0052] Based on the above implementation scheme, as an example, in some typical application cases, the feature extraction module can be composed of three feature extractors ResNet-50, and the first layer 7×7 convolution layer is replaced by a 3×3 convolution with the same padding to ensure that the output size of the voting gate module is consistent with the size of the input image. In the present invention, except for the first convolution layer, the three feature extractors share weights to limit the number of learnable parameters. Due to the different inputs and the independence of the first layer, the three feature extractors can still extract different features despite the use of the strategy of sharing weights.

[0053] In some embodiments, step 3) may specifically include the following steps:

[0054] Multiple first convolutional features output by the first convolutional layer are concatenated to obtain a concatenated feature.

[0055] The stitching features are input into a plurality of independent voting gate modules corresponding to specific tasks to obtain voting gate information corresponding to the specific tasks; wherein the specific task is selected from one of a retinal vascular image task, a foveal avascular area image task, and a retinal vascular bifurcation / intersection image task.

[0056] The voting gate information is used to extract voting gate features from the multi-scale fusion features.

[0057] In some embodiments, the voting gate module may select features at multiple sub-layers and multiple spatial positions as the voting gate information according to its corresponding specific task, wherein the spatial position refers to the position of the selected feature on the feature image.

[0058] In some embodiments, the plurality of spatial locations may include at least a macular central region, a location of a vascular phase, and an edge of a blood vessel.

[0059] In some implementation schemes, the above-mentioned method of extracting voting gate features from the multi-scale fusion features using the voting gate information may be implemented by the following steps:

[0060] The voting gate information is multiplied by each of the multi-scale fusion features to obtain a multiplied feature, and a plurality of multiplied features corresponding to all the multi-scale fusion features are summed to obtain a comprehensive feature map corresponding to the specific task.

[0061] The voting gate feature is obtained using the comprehensive feature map.

[0062] In the technical solution provided in the embodiment of the present invention, the task branch preferably includes three parts, namely, the task branches for RV, FAZ and RVJ. The task branches of RV and FAZ can adopt the same structure, each consisting of two 3×3 convolution blocks. Since the task of RVJ is more complex, it is preferred to design a special task branch to improve the accuracy.

[0063] Based on the above implementation scheme, as an example, in some typical application cases, three independent task-specific voting gate modules (VGMs) are included. The module consists of multiple 3×3 convolutional layers with batch normalization (BN) and ReLU activation functions. The last convolutional layer uses the sigmoid operator to map the features into probabilistic forms, and there are 3 channels that can be used as weights for selecting features. The VGM of each task ∈ {RV, FAZ, RVJ} takes the concatenation of the first layer outputs of the three feature extractors as input, and the corresponding output {G task} is a learned voter, which can select features from two levels. The first is the features of different layers of OCTA, which is considered that the importance of the features obtained from the three enface images is different for each task. The second is the features of different spatial positions of the feature extractor. Since each task has its own focus on position information, FAZ segmentation focuses on the central area of ​​the macula, the position of the vascular phase is more critical for the bifurcation / intersection classification task, and the vascular segmentation also needs to pay more attention to the edge information of the blood vessel. After obtaining the voting gate {G for each task task}, the multi-scale fusion features of the three feature extractors {F i}(i∈{1,2,3}) respectively with {G task}(task∈{RV,FAZ,RVJ}) and perform sum operation to obtain the comprehensive feature map {Y task}. This process can be described as:

[0064]

[0065] in Represented as voting gate {G task}, represents element-wise multiplication. Then the task-specific feature map {Y task} is sent to the corresponding task to obtain the final result. Therefore, in this way, the most important features can be adaptively selected to achieve the best effect.

[0066] The present invention proposes for the first time a feature integration method based on voting, which can automatically select features from different levels according to task attributes, that is, features at different levels in the image and features at different spatial positions, thereby effectively improving the effect of feature extraction.

[0067] Regarding other improvements of the present invention, the embodiments of the present invention provide the following implementation schemes:

[0068] In some embodiments, step 4) may specifically include the following steps:

[0069] The voting gate feature is divided into a plurality of grids.

[0070] Heat map regression is used to obtain predicted position information of key points in the grid from the grid.

[0071] The predicted category information of the key points is obtained from the grid using a regional classification model; wherein the predicted category information is used to indicate that the key points in the grid belong to vascular intersections or vascular bifurcations or there are no key points in the grid.

[0072] The predicted position information is fused with the predicted category information to obtain the key point feature.

[0073] In some embodiments, the regional classification model can predict the type confidence information of the key points in the grid, and can judge whether the key points in the grid belong to the vascular intersection or the vascular bifurcation or there are no key points in the grid according to whether the type confidence information is greater than a preset threshold. Wherein, the type confidence information refers to data used to indicate the degree of confidence of the key points in the grid belonging to each type. For example, in one example, in a certain grid, the intersection confidence is 85%, the bifurcation confidence is 5%, and the preset threshold is 80%, then it can be determined that the key points in the grid are intersections. In another example, in a certain grid, the intersection confidence is 50%, the bifurcation confidence is 20%, and the preset threshold is 60%, then it can be determined that there are no key points in the grid. As a preferred solution, when both confidences are greater than the preset threshold, the key point type with the largest confidence can be selected as the final key point type.

[0074] There are two challenges in bifurcation / intersection detection and classification: 1) Unlike ordinary key point detection, the number of vascular landmarks in the image and their spatial distribution are unknown; 2) Nodes are small targets that contain only a small number of precise pixels, and it is difficult for bounding box-based target detection methods to achieve satisfactory performance in this task. In response to the above problems, the present invention proposes a two-branch task head that combines heat map regression and region classification to detect bifurcation / intersection points. Specifically, this complex task can be divided into two simple tasks: using heat map regression to achieve precise positioning of all key points (i.e. bifurcation / intersection points), and the region classification branch is responsible for identifying the type of key points. The process is as follows: Figure 3 shown.

[0075] Based on the above implementation scheme, as an example, in some typical application cases, the feature map used for key point detection First, it is fed into a convolutional block, which consists of two 3×3 traditional layers with BN and activation functions. The last convolutional layer with activation function outputs the location of all key points through a 1-channel heat map. The other branch also takes The S×S grid as input and output represents different areas of the image. For each grid area, the region classification branch will predict whether it contains bifurcations and intersections, or has no key points. Since there is no need to predict the coordinates of the bounding box, the model can focus on predicting the region type, which has better performance than the bounding box-based method. At the same time, by predicting the confidence score corresponding to each region, the confidence of the model in the region containing key points can be reflected, which can be used for the threshold selection of the final processing. The present invention sets each region unit to 8×8. For an image with an input of 304×304, the final prediction of this branch is a tensor of 38×38×4. Finally, the results of the two branches are combined to obtain the final prediction result.

[0076] The present invention proposes for the first time a module combining heat map regression and regional classification for bifurcation / intersection detection and classification, which can effectively solve the problem of accurate positioning and classification caused by the complexity of the microvascular system in OCTA images.

[0077] In addition, the above-mentioned retinal structure extraction method provided by the embodiment of the present invention was evaluated on three OCTA datasets. The experimental stage was verified using the currently public dataset ROSE and an independent private dataset. Compared with other multi-task methods, the algorithm proposed in the embodiment of the present invention has significantly reduced model complexity: for example, the size of the existing multi-task model UberNet is 41.37M, the size of MTAN is 44.79M, the size of MTI-Net is 94.29M, and the size of the model provided by the embodiment of the present invention is 34.73M. At the same time, in terms of the accuracy of RV and FAZ segmentation results, DICE and Accuracy are improved by 5% and 2% respectively compared with the most advanced methods. In terms of RVJ classification, F1-Score is improved by 4%. The experimental results show that the method provided by the embodiment of the present invention is superior to the current single-target learning method and multi-task learning method. In a specific application case, the IVC, SVC, and DVC sub-layer images used in the above-mentioned retinal structure extraction method and the retinal structure feature images extracted by this method: RV, RVJ, FAZ are as follows: Figure 4 shown.

[0078] The embodiment of the present invention further provides a retinal structure extraction system for OCTA images, including a multi-task feature extraction network based on a voting mechanism, wherein the multi-task feature extraction network includes:

[0079] The feature extraction module includes a plurality of feature extractors, wherein the plurality of feature extractors are respectively used to extract multi-scale fusion features corresponding to each sub-layer from different input OCTA image sub-layers.

[0080] Multiple independent task-specific voting gate modules are used to adaptively select the most important voting gate features for each task.

[0081] And the key point detection module includes: a first task unit based on heat map regression, which is used to realize the positioning of key points, and a second task unit based on area classification, which is used to identify the type of key points.

[0082] The present invention also provides a neural network training method, comprising the following steps:

[0083] Acquire a training image, and mark the training image to generate marking information;

[0084] Using the retinal structure extraction method in any of the above embodiments to perform feature extraction on the training image, to obtain a feature extraction result corresponding to the training image;

[0085] The neural network used in the retinal structure extraction method is trained according to the marking information and the feature extraction result.

[0086] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the retinal structure extraction method in any of the above-mentioned embodiments when executing the computer program.

[0087] An embodiment of the present invention further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the retinal structure extraction method in any of the above embodiments are implemented.

[0088] The embodiment of the present invention further provides a retinal structure extraction device for OCTA images, comprising:

[0089] The image acquisition module is used to acquire an OCTA image, wherein the OCTA image includes a plurality of sub-layers, wherein the sub-layers may include at least a superficial vascular complex layer, a deep vascular complex layer, and a full plexus retinal layer.

[0090] The feature extraction module is used to extract the multi-scale fusion feature corresponding to each sub-layer from the multiple sub-layers of the OCTA image.

[0091] A plurality of different voting gate modules are used to extract voting gate features from the plurality of multi-scale fusion features.

[0092] The key point detection module is used to detect key points of the voting gate features to obtain key point features. The key points may at least include blood vessel bifurcation points and blood vessel intersection points.

[0093] The extraction result module is used to use the key point features and the optional voting gate features as the feature extraction results of the OCTA image. The feature extraction results may at least include retinal vascular images, foveal avascular area images and retinal vascular bifurcation / intersection images.

[0094] In some embodiments, the feature extraction module specifically includes:

[0095] The first convolution unit is used to input the multiple sub-layers into the first convolution layers of multiple feature extractors to obtain multiple first convolution features corresponding to each of the sub-layers.

[0096] The residual convolution unit is used to input the plurality of the first convolution features into the residual convolution layers of the plurality of feature extractors respectively, so as to obtain the multi-scale fusion features corresponding to each of the sub-layers.

[0097] In some embodiments, the voting gate module specifically includes:

[0098] A feature concatenation unit is used to concatenate multiple first convolution features output by the first convolution layer to obtain a concatenated feature.

[0099] The voting gate unit is used to input the stitching features into multiple independent voting gate modules corresponding to specific tasks to obtain voting gate information corresponding to the specific tasks; wherein the specific tasks are selected from one of retinal vascular image tasks, foveal avascular area image tasks, and retinal vascular bifurcation / intersection image tasks. The voting gate module can select features at multiple sub-layers and multiple spatial positions as the voting gate information according to the specific tasks corresponding to the voting gate module. The multiple spatial positions can be selected to include at least the central area of ​​the macula, the position of the vascular phase, and the edge of the blood vessel.

[0100] A feature extraction unit is used to extract voting gate features from the multi-scale fusion features using the voting gate information. This step may specifically include: multiplying the voting gate information with each of the multi-scale fusion features to obtain a multiplied feature, and summing multiple multiplied features corresponding to all multi-scale fusion features to obtain a comprehensive feature map corresponding to the specific task; and obtaining voting gate features using the comprehensive feature map.

[0101] In some embodiments, the key point detection module may specifically include:

[0102] A grid division unit is used to divide the voting gate feature into a plurality of grids.

[0103] The heat map regression unit is used to obtain the predicted position information of the key points in the grid from the grid by using the heat map regression.

[0104] A category prediction unit is used to obtain predicted category information of the key point from the grid using a regional classification model; wherein the predicted category information is used to indicate that the key point in the grid belongs to a vascular intersection or a vascular bifurcation or there is no key point in the grid. The regional classification model can predict the category confidence information of the key point in the grid, and judge whether the key point in the grid belongs to a vascular intersection or a vascular bifurcation or there is no key point in the grid according to whether the category confidence information is greater than a preset threshold.

[0105] The information fusion unit is used to fuse the predicted position information with the predicted category information to obtain the key point feature.

[0106] It should be understood that the above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for extracting retinal structure from OCTA images, characterized in that: include: 1) acquiring an OCTA image, wherein the OCTA image includes a plurality of sublayers, wherein the sublayers include at least a superficial vascular complex layer, a deep vascular complex layer, and a pan-plexus retinal layer; 2) Extracting multi-scale fusion features corresponding to each sub-layer from the multiple sub-layers of the OCTA image, specifically including: Inputting the plurality of sub-layers into the first convolutional layers of the plurality of feature extractors to obtain a plurality of first convolutional features corresponding to each of the sub-layers; Inputting the plurality of the first convolutional features into the remaining convolutional layers of the plurality of feature extractors respectively, to obtain a multi-scale fusion feature corresponding to each of the sub-layers; Wherein, the weight parameters of the plurality of feature extractors are the same; 3) Using a plurality of different voting gate modules to extract voting gate features from the plurality of multi-scale fusion features, specifically including: Splicing multiple first convolutional features output by the first convolutional layer to obtain a spliced ​​feature; Inputting the splicing features into a plurality of independent voting gate modules corresponding to specific tasks to obtain voting gate information corresponding to the specific tasks; wherein the specific tasks are selected from one of retinal vascular image tasks, foveal avascular area image tasks, and retinal vascular bifurcation / intersection image tasks; Extracting voting gate features from the multi-scale fusion features using the voting gate information; The voting gate module selects features as the voting gate information at multiple sub-layers and multiple spatial positions according to the specific tasks corresponding to the voting gate module, and the multiple spatial positions at least include the central area of ​​the macula, the position of the vascular phase, and the edge of the blood vessel; 4) Detecting key points of the voting gate features, where the key points at least include a blood vessel bifurcation point and a blood vessel intersection point, and obtaining key point features, specifically including: Dividing the voting gate feature into a plurality of grids; Obtaining predicted position information of key points in the grid from the grid using heat map regression; Obtaining predicted category information of the key points from the grid using a regional classification model; wherein the predicted category information is used to indicate whether the key points in the grid belong to a vascular intersection or a vascular bifurcation or whether there are no key points in the grid; The predicted position information is merged with the predicted category information to obtain the key point feature; The regional classification model predicts the type confidence information of the key points in the grid, and determines whether the key points in the grid belong to a vascular intersection or a vascular bifurcation or there are no key points in the grid according to whether the type confidence information is greater than a preset threshold; 5) Using the key point features and optional partial voting gate features as feature extraction results of the OCTA image, the feature extraction results at least include a retinal vascular image, a foveal avascular area image, and a retinal vascular bifurcation / intersection image.

2. The retinal structure extraction method according to claim 1, characterized in that: The feature extractors are all composed of ResNet-50 networks.

3. The retinal structure extraction method according to claim 1, characterized in that: Extracting voting gate features from the multi-scale fusion features using the voting gate information specifically includes: Multiplying the voting gate information with each of the multi-scale fusion features to obtain a multiplied feature, and summing a plurality of multiplied features corresponding to all the multi-scale fusion features to obtain a comprehensive feature map corresponding to the specific task; Obtaining voting gate features using the comprehensive feature map; In the above process, the following formula is specifically used to obtain the comprehensive feature map: ； in, represents the comprehensive feature map, represents the voting gate information corresponding to the i-th channel, represents multi-scale fusion features, Represents element-wise multiplication.

4. A retinal structure extraction system for OCTA images, used to execute the retinal structure extraction method according to any one of claims 1 to 3, characterized in that: A multi-task feature extraction network based on a voting mechanism is included, wherein the multi-task feature extraction network includes: A feature extraction module, comprising a plurality of feature extractors, wherein the plurality of feature extractors are respectively used to extract multi-scale fusion features corresponding to each sub-layer from different input OCTA image sub-layers, Multiple independent task-specific voting gate modules to adaptively select the most important voting gate features for each task; and Key point detection module, including: The first task unit based on heat map regression is used to locate key points. The second task unit based on region classification is used to identify the type of key points.

5. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the retinal structure extraction method according to any one of claims 1 to 3 when executing the computer program.

Images