OCTA image neovascularization segmentation method, training method, equipment and medium

Abstract

The invention provides an OCTA image-based neovascularization segmentation method, training method, equipment and medium, and the method comprises the steps: obtaining an OCTA image to be recognized, and carrying out the choroidal neovascularization CNV segmentation of the OCTA image based on a segmentation model, and obtaining a CNV segmentation result. Namely, for the obtained OCTA image, the ResNeSt module in the segmentation model is utilized to carry out group convolution and attention separation feature learning, CNV features can be accurately identified from the OCTA image, and great convenience is brought to subsequent CNV quantization.

Description

Technical Field

[0001] The present invention relates to the technical field of ophthalmic image processing, and in particular to a new blood vessel segmentation method based on OCTA images, a segmentation model training method, a device, an electronic device and a computer-readable storage medium. Background Art

[0002] In the medical field, age-related macular degeneration (AMD) is a common fundus lesion in the elderly. AMD can be divided into early, middle and late stages. The late stage includes two types: wet (also called exudative or neovascular) and dry. Among them, wet AMD is characterized by the presence of choroidal neovascularization (CNV). When AMD develops to the late stage, vision may deteriorate rapidly. Therefore, accurately identifying CNV before vision deteriorates seriously and intervening in a timely manner are of great significance to maintaining vision.

[0003] In the related technology, suspected neovascular AMD can be identified through the following methods: Fluorescein angiography (FA) is the gold standard for diagnosis, optical coherence tomography angiography (OCTA) can display the blood flow of each layer of the retina in a non-invasive way, and the blood vessels in the OCTA image can also be detected based on the saliency map method.

[0004] However, among the above methods, FA requires invasive operations and some patients may have allergic reactions; and the identification of CNV in OCTA based on saliency maps is highly dependent on image quality. When the image quality is poor, it is difficult to distinguish CNV from noise, which may lead to misdiagnosis.

[0005] Therefore, how to improve the accuracy of identifying choroidal neovascularization (AMD) is a technical problem that needs to be solved. Summary of the invention

[0006] The present invention provides a method for segmenting new blood vessels based on OCTA images, a method for training a segmentation model, a device, an electronic device, and a computer-readable storage medium, so as to at least solve the problem that choroidal new blood vessels cannot be accurately identified in the related art. The technical solution of the present invention is as follows:

[0007] According to a first aspect of an embodiment of the present invention, a method for segmenting new blood vessels based on OCTA images is provided, comprising:

[0008] Acquiring an optical coherence tomography angiography (OCTA) image to be identified;

[0009] Choroidal neovascularization CNV segmentation is performed on the OCTA image based on the segmentation model to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result.

[0010] Optionally, before performing choroidal neovascularization CNV segmentation on the processed OCTA image based on the segmentation model, the method further includes: performing contrast enhancement processing on the OCTA image to obtain the processed OCTA image;

[0011] Optionally, performing choroidal neovascularization CNV segmentation on the OCTA image based on the segmentation model to obtain a CNV segmentation result includes:

[0012] Features of different sizes are learned by adding a spatial pyramid pooling SPP module behind each encoder in the segmentation model to obtain deep feature maps of the same size, so as to enhance the recognition of CNVs of different sizes by the segmentation model.

[0013] Optionally, the method further comprises: pre-training a segmentation model based on a ResNeSt module in the following manner:

[0014] Acquire a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image;

[0015] Inputting the plurality of sample OCTA images into an initial segmentation model, wherein the segmentation model includes a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module;

[0016] The ResNeSt module is used to extract CNV features on each sample OCTA image through group convolution and separation attention;

[0017] Performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain corresponding segmentation results;

[0018] Compare the segmentation result with the corresponding CNV annotation map, and calculate the loss value according to the comparison result;

[0019] Based on the loss value, the parameters of the segmentation model are updated through a back-propagation mechanism, and after multiple iterations, until the loss value reaches a minimum value or the segmentation model converges, a trained segmentation model is obtained.

[0020] Optionally, the extracted features on each sample OCTA image are subjected to spatial pyramid pooling, cascading and convolution processing to obtain corresponding segmentation results, including:

[0021] The extracted features are passed through the corresponding spatial pyramid pooling SPP module to learn context information of different sizes and obtain deep feature maps of the same size;

[0022] After cascading and convolving the deep feature maps of the same size, the input of the next encoder layer is obtained;

[0023] The features output by the encoder are input into the decoder for decoding to obtain the corresponding segmentation results.

[0024] According to a second aspect of an embodiment of the present invention, a segmentation model training method is provided, the method comprising:

[0025] Acquire a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image;

[0026] Inputting the plurality of sample OCTA images into a segmentation model, the segmentation model comprising a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module;

[0027] The ResNeSt module is used to extract features on each sample OCTA image through group convolution and separation attention;

[0028] Perform spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain the corresponding segmentation result;

[0029] Compare the segmentation result with the corresponding CNV annotation map, and calculate the loss value according to the comparison result;

[0030] Based on the loss value, the parameters of the segmentation model are updated through a back-propagation mechanism, and after multiple iterations, until the loss value reaches a minimum value or the segmentation model converges, a trained segmentation model is obtained.

[0031] Optionally, the extracted features on each sample OCTA image are subjected to spatial pyramid pooling, cascading and convolution processing to obtain corresponding segmentation results, including:

[0032] The extracted features are passed through the corresponding spatial pyramid pooling SPP module to learn context information of different sizes and obtain deep feature maps of the same size;

[0033] After cascading and convolving the deep feature maps of the same size, the input of the next encoder layer is obtained;

[0034] The features output by the encoder are input into the decoder for decoding to obtain the corresponding segmentation results.

[0035] According to a third aspect of an embodiment of the present invention, there is provided a device for segmenting new blood vessels based on OCTA images, comprising:

[0036] An acquisition module, used for acquiring an optical coherence tomography angiography OCTA image to be detected;

[0037] A segmentation module is used to perform choroidal neovascularization CNV segmentation on the OCTA image based on a segmentation model to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result.

[0038] Optionally, the device further includes: a processing module, configured to perform contrast enhancement processing on the OCTA image acquired by the acquisition module to obtain the processed OCTA image.

[0039] Optionally, the segmentation module is also used to learn context information of different sizes by adding a spatial pyramid pooling SPP module behind each encoder in the segmentation model to obtain deep feature maps of the same size, so as to enhance the segmentation model's recognition of CNVs of different sizes.

[0040] Optionally, the device further comprises: a training module for pre-training a segmentation model based on a ResNeSt module; wherein the training module comprises:

[0041] A training set acquisition module is used to acquire a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image;

[0042] An input module, used for inputting the plurality of sample OCTA images into an initial segmentation model, wherein the segmentation model comprises a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module;

[0043] An extraction module, for extracting features on each sample OCTA image by using the ResNeSt module through group convolution and separation attention;

[0044] A processing module, used for performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain a corresponding segmentation result;

[0045] A determination module, used to compare the segmentation result with the corresponding CNV annotation map, and calculate the loss value according to the comparison result;

[0046] The iterative module is used to update the parameters of the segmentation model through a back-propagation mechanism based on the loss value, and obtain a trained segmentation model after multiple iterations until the loss value reaches a minimum value or the segmentation model converges.

[0047] Optionally, the processing module includes:

[0048] The pooling module is used to learn context information of different sizes through the corresponding spatial pyramid pooling SPP module to obtain deep feature maps of the same size;

[0049] A feature processing module, used to perform concatenation and convolution on the deep feature maps of the same size to obtain the input of the next layer encoder;

[0050] The decoding module is used to input the features output by the encoder into the decoder for decoding to obtain the corresponding segmentation results.

[0051] According to a fourth aspect of an embodiment of the present invention, a segmentation model training device is provided, the device comprising:

[0052] A training set acquisition module is used to acquire a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image;

[0053] An input module, used for inputting the plurality of sample OCTA images into a segmentation model for training, wherein the segmentation model comprises a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module;

[0054] An extraction module, for extracting features on each sample OCTA image by using the ResNeSt module through group convolution and separation attention;

[0055] A processing module, used for performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain a corresponding segmentation result;

[0056] A determination module, used to compare the segmentation result with the corresponding CNV annotation map, and calculate the loss value according to the comparison result;

[0057] The iterative module is used to update the parameters of the segmentation model through a back-propagation mechanism based on the loss value, and obtain a trained segmentation model after multiple iterations until the loss value reaches a minimum value or the segmentation model converges.

[0058] Optionally, the processing module includes:

[0059] The pooling module is used to learn context information of different sizes through the corresponding spatial pyramid pooling SPP module to obtain deep feature maps of the same size;

[0060] A feature processing module, used to perform concatenation and convolution on the deep feature maps of the same size to obtain the input of the next layer encoder;

[0061] The decoding module is used to input the features output by the encoder into the decoder for decoding to obtain the corresponding segmentation results.

[0062] According to a fifth aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0063] processor;

[0064] a memory for storing instructions executable by the processor;

[0065] Wherein, the processor is configured to execute the instructions to implement the above-mentioned OCTA image-based new blood vessel segmentation method or the above-mentioned segmentation model training method.

[0066] According to the sixth and fourth aspects of the embodiments of the present invention, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above-mentioned OCTA image-based new blood vessel segmentation method or the above-mentioned segmentation model training method.

[0067] According to a seventh aspect of an embodiment of the present invention, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor of an electronic device, implements the above-mentioned OCTA image-based new blood vessel segmentation method or the above-mentioned segmentation model training method.

[0068] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects:

[0069] In the embodiment of the present invention, an optical coherence tomography angiography OCTA image to be identified is first obtained; and CNV segmentation is performed on the OCTA image based on the segmentation model to obtain a CNV segmentation result. In other words, the embodiment of the present invention uses the ResNeSt module in the segmentation model to perform group convolution and separate attention feature learning, which can accurately identify CNV features from the OCTA image, bringing great convenience to the subsequent CNV quantification.

[0070] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention, and do not constitute an improper limitation of the present invention. In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0072] Figure 1 This is a flow chart of a method for segmenting new blood vessels based on OCTA images provided by an embodiment of the present invention.

[0073] Figure 2 It is a schematic diagram of a segmentation model training provided by an embodiment of the present invention.

[0074] Figure 3A It is a schematic diagram of a residual neural network ResNeSt provided in an embodiment of the present invention.

[0075] Figure 3B A schematic diagram of the structure of Split-Attention provided by an embodiment of the present invention.

[0076] Figure 4 It is a schematic diagram of a spatial pyramid pooling provided by an embodiment of the present invention.

[0077] Figure 5 This is another flow chart of a method for segmenting new blood vessels based on OCTA images provided by an embodiment of the present invention.

[0078] Figure 6 It is a flow chart of the segmentation model training method provided by an embodiment of the present invention.

[0079] Figure 7The present invention provides a block diagram of a new blood vessel segmentation device based on OCTA images.

[0080] Figure 8 It is a block diagram of an electronic device provided by an embodiment of the present invention.

[0081] Fig. 9 The invention is a block diagram of a device for segmenting new blood vessels based on OCTA images provided by an embodiment of the invention. DETAILED DESCRIPTION

[0082] In order to enable ordinary persons in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0083] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0084] In recent years, research on computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence has made important progress. Artificial Intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline involving many types of technologies such as chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks, etc. Computer vision, as an important branch of artificial intelligence, specifically allows machines to recognize the world. Computer vision technology usually includes face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, robot navigation and positioning and other technologies. With the research and advancement of artificial intelligence technology, this technology has been applied in many fields, such as security, urban management, traffic management, building management, park management, facial access, facial attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone imaging, cloud services, smart homes, wearable devices, unmanned driving, automatic driving, smart medical care, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile Internet, live streaming, beauty, makeup, medical beauty, and smart temperature measurement.

[0085] Figure 1 FIG. 1 is a flow chart of a method for segmenting new blood vessels based on OCTA images provided by an embodiment of the present invention. Figure 1 As shown, the OCTA image-based new blood vessel segmentation method includes the following steps:

[0086] Step 101: Acquire an optical coherence tomography angiography (OCTA) image to be identified;

[0087] Step 102: Perform choroidal neovascularization CNV segmentation on the OCTA image based on the segmentation model to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result.

[0088] The OCTA image-based neovascularization segmentation method described in the present invention can be applied to terminals, servers, etc., without limitation herein. Its terminal implementation device can be a smart phone, a laptop, a tablet computer, a personal computer, and other electronic devices, without limitation herein.

[0089] Combine the following Figure 1 , the specific implementation steps of a method for segmenting new blood vessels based on OCTA images provided by an embodiment of the present invention are described in detail.

[0090] In step 101, an optical coherence tomography angiography (OCTA) image to be identified is acquired.

[0091] In this step, there are many ways to obtain the optical coherence tomography angiography (OCTA) image to be detected. The OCTA image to be detected is the original OCTA image. For example, the original OCTA image comes from a clinical medical data set, and all OCTA images are labeled with information from professional doctors through clinical diagnosis. For another example, the user's OCTA image can also be obtained through professional fundus photography equipment (such as fundus photography equipment in a hospital, etc.), etc. In actual applications, it is not limited to this.

[0092] Optionally, in this embodiment, the user's fundus may be photographed by artificial intelligence (AI) fundus image recognition technology or an artificial fundus screening platform to obtain an OCTA image of the user.

[0093] In this embodiment, the OCTA image to be detected may be one or more OCTA images, which is not limited in this embodiment.

[0094] It should be noted that OCTA is a new non-invasive fundus imaging technology that can identify retinal choroidal blood flow movement information with high resolution and image the retinal choroidal microvascular circulation in living tissue. It has unique advantages in the management and follow-up of normal retinal choroidal vascular changes and disease and the detection of treatment effects.

[0095] In step 102, choroidal neovascularization CNV segmentation is performed on the processed OCTA image based on the segmentation model to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result.

[0096] In this step, the OCTA image is input into a pre-trained segmentation model for segmentation to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result. That is, in this embodiment, the group convolution and separation attention mechanisms are integrated together, which greatly improves the segmentation performance of the model and performs well in many fields such as image classification, segmentation, and detection.

[0097] In an optional embodiment, a spatial pyramid pooling SPP module can be added behind each encoder in the segmentation model to learn context information of different sizes and obtain deep feature maps of the same size to enhance the segmentation model's recognition ability for CNVs of different sizes.

[0098] That is to say, the processed OCTA image is input into the segmentation model based on the ResNeSt module; the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module uses group convolution and separation attention to improve the feature expression ability. Further, a spatial pyramid pooling (SPP) module can also be used behind each encoder to learn context information (i.e., feature representation) of different sizes to enhance the recognition ability of the segmentation model for CNVs of different sizes. Finally, the last layer output of the decoder is used as the final predicted segmentation result of the segmentation model. In particular, when the segmentation model is trained, the training loss value is calculated based on the predicted segmentation result of the segmentation model and the CNV annotation in the OCTA image, and the loss value is optimized to achieve the minimum loss value, or when the segmentation model converges, it is the trained best segmentation model.

[0099] In an embodiment of the present invention, a segmentation model based on a ResNeSt module is proposed, which includes multiple encoders and corresponding multiple decoders, each of which is constructed by a ResNeSt module. The ResNeSt module improves feature expression capabilities by using group convolution and separation attention mechanisms, and the ResNeSt through deep learning can more effectively learn the key information required for the target task. In addition, the present invention proposes spatial pyramid pooling, which solves the limitation that traditional deep learning methods cannot process CNVs of different sizes, and greatly improves the recognition accuracy of the segmentation model.

[0100] Optionally, in this embodiment, the method may further include: pre-training a segmentation model based on a ResNeSt module, the specific implementation process of which is as follows: Figure 2 FIG. 1 is a schematic diagram of a segmentation model training provided by an embodiment of the present invention, wherein the training process includes:

[0101] 1) obtaining a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image;

[0102] 2) inputting the plurality of sample OCTA images into an initial segmentation model, wherein the segmentation model comprises a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module;

[0103] 3) Using the ResNeSt module to extract CNV features on each sample OCTA image through group convolution and separate attention;

[0104] 4) performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain the corresponding segmentation result;

[0105] 5) comparing the segmentation result with the corresponding CNV annotation map, and calculating the loss value according to the comparison result;

[0106] 6) Based on the loss value, the parameters of the segmentation model are updated through a back propagation mechanism, and after multiple iterations, until the loss value reaches a minimum value or the segmentation model converges, a trained segmentation model is obtained.

[0107] The method comprises: performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain a corresponding segmentation result, including: learning context information of different scales through the extracted features through a corresponding spatial pyramid pooling (SPP) module to obtain a deep feature map of the same size; and cascading and convolving the deep feature map of the same size to obtain the input of the next encoder layer, and then inputting the features output by the encoder into the decoder for decoding to obtain the corresponding segmentation result.

[0108] Among them, the spatial pyramid pooling SPP first extracts features from the input image, selects candidate boxes on the extracted feature map, and then uses SPP to extract a fixed-length representation of the feature map area corresponding to the candidate box.

[0109] That is to say, Figure 2 As shown, in this embodiment, in order to enhance the representation ability of the segmentation model, a residual neural network ResNeSt is used (its structural diagram is shown in Figure 3A and Figure 3B As shown, Figure 3A A schematic diagram of a ResNeStblock provided by an embodiment of the present invention, Figure 3B A schematic diagram of a structure of a split-attention (Split-Attention) provided in an embodiment of the present invention) is used as the basic backbone of the encoder and decoder in the segmentation model, such as Figure 3A As shown, the ResNeSt module improves the feature representation ability of the segmentation model through group convolution and separation attention module. That is, in this embodiment, since ResNeSt integrates the group convolution and separation attention mechanisms together, the feature expression ability of the segmentation model is greatly improved, and it performs well in many fields such as image classification, segmentation, and detection. Therefore, the segmentation model of this embodiment is a segmentation model based on the ResNeSt module, and the segmentation model includes an encoder and a decoder, and the encoder and the decoder are both constructed by ResNeStblock (ResNeSt module). The ResNeSt module uses group convolution and separation attention to improve the feature expression ability, thereby improving the segmentation performance of the segmentation model. The OCTA image to be identified is input into the ResNeSt module, and the expression ability of the segmentation feature of the segmentation model is improved by the group convolution and separation attention of the ResNeSt module. The ResNeSt module extracts deep features from each sample OCTA image through group convolution and separation attention, and uses the extracted deep features through the corresponding spatial pyramid pooling SPP module to learn context information of different sizes to obtain deep feature maps of the same size; and after cascading and convolving the deep feature maps of the same size, the input of the next layer encoder is obtained, and then the output of the encoder is input into the decoder to obtain the final segmentation result. Among them, the specific implementation process of the segmentation model based on the ResNeSt module is already a well-known technology for those skilled in the art, and will not be repeated here.

[0110] Please also refer to Figure 3A and Figure 3B ,like Figure 3A As shown in the figure, for an input feature map, it is first evenly divided into two feature maps and then fed into two cardinality groups. In each cardinality group, the feature map is further split into two feature maps and then fed into two parallel convolution groups, each of which contains a 1x1 and a 3x3 convolution layer. Then, the two split features are fed into the Split-Attention module to integrate the features, as shown in Figure 3B shown.

[0111] Specifically, Figure 3BIn the above example, the features of the two branches are first fused. Then a global pooling module is applied to obtain the average features, followed by two 1x1 convolutional layers and a softmax layer to obtain the attention coefficients. Then, each branch feature is reweighted and concatenated along the channel dimension to obtain the output features.

[0112] like Figure 3A As shown, through Figure 3B After obtaining the features of the two cardinality groups respectively, they are concatenated through coancat and passed through a 1x1 convolutional layer. In addition, this embodiment also introduces a skip connection to obtain the final output features of the ResNeSt block.

[0113] In addition, in order to improve the recognition ability of the segmentation model for the extracted CNVs of different sizes, the embodiment of the present invention proposes to use a spatial pyramid pooling (SPP) module behind each encoder to learn feature representations of different sizes to enhance the recognition ability of the model for CNVs of different sizes. Finally, the output probability of the last layer of the decoder is used as the final predicted segmentation result. That is, this embodiment further adopts a spatial pyramid pooling module to replace the pooling unit after each layer of encoder in the segmentation model, and its different sizes can include multiple sizes. This embodiment takes four pooling kernels of different sizes (2×2), (3×3), (6×6) and (9×9) as examples to extract feature maps of different sizes. In practical applications, it is not limited to this size, and it can also be 4×4, 2×2, 1×1, etc. Then, the spatial pyramid pooling method is used to perform feature recognition of different sizes on the pooling unit after each layer of encoder. Finally, the obtained feature map is upsampled (UP, upsample) using bilinear interpolation so that the size is the input size of the next layer of encoder. After multiple iterations, the final feature map is obtained after cascading and convolution. In this way, this embodiment can use different receptive fields to extract multi-scale information to improve the segmentation performance of the segmentation model.

[0114] In this embodiment, the loss function used by the segmentation model is as follows:

[0115]

[0116] Where L ce The cross entropy loss function is used to predict the category probability p i and the ground truth annotation y i Calculate the loss value, L dice It is the dice loss, which is used to alleviate the imbalance phenomenon and improve the model's recognition performance for the foreground (CNV).

[0117] Dice loss is widely used in medical image segmentation tasks to measure the similarity between two sets. The definition of Dice coefficient is as follows:

[0118] DiceCoefficient=(∣A∣+∣B∣) / 2∣A∩B∣

[0119] Dice loss is equal to 1-dice coefficient

[0120] Diceloss=1-DiceCoefficient

[0121] It should be noted that in the OCTA images to be identified, choroidal neovascularization CNVs may have different shapes and sizes. Traditional deep learning models such as U-net may miss some small CNV areas due to their limited receptive field and cannot take into account CNVs of different sizes. The different anatomical structures and contextual information of CNVs in OCTA images may be ignored by the receptive field of the traditional deep learning model U-net. Therefore, this embodiment proposes a spatial pyramid pooling module to adaptively learn multi-scale feature representations to enhance the segmentation performance of the model. The schematic diagram of the spatial pyramid pooling is as follows: Figure 4 As shown. In this embodiment, the feature map pooling (poling) is performed with four pooling kernels of different sizes (2×2), (3×3), (6×6) and (9×9) to extract feature maps of different sizes, but it is not limited to these sizes in practical applications. Then, the spatial pyramid pooling method is used to perform feature recognition of different sizes on the pooling units after each layer of encoder. Finally, the obtained feature map is upsampled (upsample) by bilinear interpolation, and the OCTA image is obtained by downsampling (downsample), and the upsampled feature map is restored to the size of the original OCTA image and then cascaded, and the final feature map is obtained after convolution (Conv1×1).

[0122] In the embodiment of the present invention, an optical coherence tomography angiography OCTA image to be identified is first obtained, and then the CNV features of choroidal neovascularization are identified in the OCTA image for segmentation to obtain a CNV segmentation result. That is, the embodiment of the present invention can accurately identify CNV features from the OCTA image by performing CNV segmentation of choroidal neovascularization on the acquired OCTA image, which greatly facilitates the subsequent CNV quantification.

[0123] See also Figure 5 , is another flow chart of a method for segmenting new blood vessels based on OCTA images provided by an embodiment of the present invention, the method comprising:

[0124] Step 501: Acquire an optical coherence tomography angiography (OCTA) image to be identified.

[0125] The specific implementation process of this step is detailed in step 101 and will not be repeated here.

[0126] Step 502: performing contrast enhancement processing on the OCTA image to obtain the processed OCTA image.

[0127] In this step, for the acquired OCTA image, in this embodiment, a series of image preprocessing can be performed first to increase the contrast of the OCTA image to facilitate the recognition of the segmentation model. In one embodiment, contrast limited adaptive histogram equalization (CLAHE) can be used to enhance the contrast of the OCTA image, wherein the contrast limited adaptive histogram equalization includes: dividing the entire OCTA image into 8*8 blocks, then calculating the cumulative distribution of each block, and limiting its contrast when calculating the cumulative distribution to prevent the histogram from being particularly steep. The purpose is to find all the values ​​in the histogram that are higher than a certain threshold a to form a set B, and then each element b in the set B is changed to the threshold a, wherein the part of the element b that is higher than the threshold a is summed to obtain c, and c is divided by 256 (the 256 refers to the grayscale level, that is, 0-255) to obtain d, and d is added to each grayscale level, thereby enhancing the contrast of the image. That is to say, CLAHE can be specifically divided into four steps: 1) image block, with blocks as units; 2) first calculate the histogram, then trim the histogram, and finally balance; 3) traverse the image blocks and perform bilinear interpolation between blocks; 4) perform image color filter mixing operation with the original image. Its specific implementation method is already a well-known technology for those skilled in the art. Of course, in practical applications, it is not limited to this method to enhance the contrast of OCTA images, and other methods can also be used to enhance the contrast of images, which is not limited in this embodiment.

[0128] Step 503: Perform choroidal neovascularization CNV segmentation on the OCTA image based on the segmentation model to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result.

[0129] The specific implementation process of this step is detailed in step 102 and will not be repeated here.

[0130] In an embodiment of the present invention, an optical coherence tomography angiography OCTA image to be identified is first acquired, and then the OCTA image is subjected to contrast enhancement processing to obtain the processed OCTA image; finally, the choroidal neovascularization CNV features are identified from the processed OCTA image for segmentation to obtain a CNV segmentation result. In other words, the embodiment of the present invention improves the contrast of the OCTA image by performing contrast enhancement processing on the acquired OCTA image, that is, optimizing the foreground area (i.e., CNV) and background in the OCTA image, so as to accurately identify the CNV features from the processed OCTA image, which greatly facilitates the subsequent CNV quantification.

[0131] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that this implementation disclosure is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0132] See also Figure 6 , is a flow chart of a segmentation model training method provided by an embodiment of the present invention, the method comprising:

[0133] Step 601: obtaining a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image;

[0134] Step 602: inputting the plurality of sample OCTA images into a segmentation model, wherein the segmentation model comprises a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module;

[0135] Step 603: Using the ResNeSt module to extract features on each sample OCTA image through group convolution and separate attention;

[0136] Step 604: performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain a corresponding segmentation result;

[0137] Step 605: Compare the segmentation result with the corresponding CNV annotation map, and calculate the loss value according to the comparison result;

[0138] Step 606: Based on the loss value, update the parameters of the segmentation model through a back-propagation mechanism, and after multiple iterations, until the loss value reaches a minimum value or the segmentation model converges, a trained segmentation model is obtained.

[0139] The method comprises: performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain a corresponding segmentation result, including: using the extracted features through a corresponding spatial pyramid pooling SPP module to learn context information of different sizes to obtain a deep feature map of the same size; performing cascading and convolution on the deep feature map of the same size to obtain an input of the next encoder layer; and inputting the features output by the encoder into a decoder for decoding to obtain a corresponding segmentation result.

[0140] In this embodiment, the specific implementation process of each step is detailed in the corresponding implementation process in the above embodiment and will not be repeated here.

[0141] A segmentation model training method provided by an embodiment of the present invention is a segmentation model based on ResNeSt, which adopts ResNeSt modules as the basic backbone of the encoder and decoder in the segmentation model, and extracts better feature representation through group convolution and separation attention mechanism, so that the segmentation model can more effectively learn the key information required for the target task, so as to improve the feature representation ability of the segmentation model.

[0142] Furthermore, the embodiment of the present invention also proposes a spatial pyramid pooling module, which solves the limitation that traditional deep learning methods cannot process CNVs of different sizes, thereby improving the recognition accuracy of the segmentation model.

[0143] Figure 7 is a block diagram of a new blood vessel segmentation device based on OCTA images provided by an embodiment of the present invention. Figure 5 The device includes: an acquisition module 701 and a segmentation module 702, wherein:

[0144] The acquisition module 701 is used to acquire an optical coherence tomography angiography OCTA image to be detected;

[0145] The segmentation module 702 is used to perform choroidal neovascularization CNV segmentation on the OCTA image based on the segmentation model to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result.

[0146] Optionally, in another embodiment, based on the above embodiment, the device includes: a processing module, wherein:

[0147] The processing module is used to perform contrast enhancement processing on the OCTA image acquired by the acquisition module to obtain the processed OCTA image;

[0148] The segmentation module is also used to input the processed OCTA image into a pre-trained segmentation model for segmentation to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result.

[0149] Optionally, in another embodiment, based on the above embodiment, the segmentation module 503 is also used to learn context information of different sizes through a spatial pyramid pooling SPP module added after each encoder in the segmentation model to obtain deep feature maps of the same size, so as to enhance the segmentation model's recognition of CNVs of different sizes.

[0150] Optionally, in another embodiment, based on the above embodiment, the device further includes: a training module for pre-training a segmentation model based on a ResNeSt module.

[0151] The training module includes: a training set acquisition module, an input module, an extraction module, a processing module, a determination module and an iteration module, wherein:

[0152] The training set acquisition module is used to acquire a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image;

[0153] The input module is used to input the plurality of sample OCTA images into an initial segmentation model, wherein the segmentation model includes a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module;

[0154] The extraction module is used to extract features on each sample OCTA image by using the ResNeSt module through group convolution and separation attention;

[0155] The processing module is used to perform spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain corresponding segmentation results;

[0156] The determination module is used to compare the segmentation result with the corresponding CNV annotation map, and calculate the loss value according to the comparison result;

[0157] The iterative module is used to update the parameters of the segmentation model through a back-propagation mechanism based on the loss value, and obtain a trained segmentation model after multiple iterations until the loss value reaches a minimum value or the segmentation model converges.

[0158] Optionally, the processing module includes: a pooling module, a feature processing module and a decoding module, wherein:

[0159] The pooling module is used to learn context information of different sizes by passing the extracted features through the corresponding spatial pyramid pooling SPP module to obtain deep feature maps of the same size;

[0160] The feature processing module is used to cascade and convolve the deep feature maps of the same size to obtain the input of the next layer encoder;

[0161] The decoding module is used to input the features output by the encoder into the decoder for decoding to obtain the corresponding segmentation results.

[0162] In an embodiment of the present invention, a pre-trained segmentation model is used to perform choroidal neovascularization CNV segmentation on the processed OCTA image to obtain a CNV segmentation result, wherein the segmentation model uses a residual neural network ResNeSt module to construct an encoder and a decoder, and improves the feature expression ability through group convolution and separation attention of the ResNeSt module, and learns context information of different sizes for the spatial pyramid pooling module after each layer of the encoder in the segmentation model to obtain deep feature maps of the same size, so as to enhance the model's recognition ability for CNVs of different sizes, and finally use the last layer output of the decoder as the final predicted segmentation result.

[0163] The embodiment of the present invention also provides a segmentation model training device, which includes: a training set acquisition module, an input module, an extraction module, a processing module, a determination module and an iteration module, wherein:

[0164] The training set acquisition module is used to acquire a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image;

[0165] The input module is used to input the plurality of sample OCTA images into a segmentation model, wherein the segmentation model includes a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module;

[0166] The extraction module is used to extract features on each sample OCTA image by using the ResNeSt module through group convolution and separation attention;

[0167] The processing module is used to perform spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain a corresponding segmentation result;

[0168] The determination module is used to compare the segmentation result with the corresponding CNV annotation map and calculate the loss value according to the comparison result;

[0169] The iterative module is used to update the parameters of the segmentation model through a back-propagation mechanism based on the loss value, and obtain a trained segmentation model after multiple iterations until the loss value reaches a minimum value or the segmentation model converges.

[0170] Optionally, the processing module includes: a pooling module, a feature processing module and a decoding module, wherein:

[0171] The pooling module is used to learn context information of different sizes by passing the extracted features through the corresponding spatial pyramid pooling SPP module to obtain deep feature maps of the same size;

[0172] The feature processing module is used to cascade and convolve the deep feature maps of the same size to obtain the input of the next layer encoder;

[0173] The decoding module is used to input the features output by the encoder into the decoder for decoding to obtain the corresponding segmentation results.

[0174] In an embodiment of the present invention, a segmentation model training device is proposed. The segmentation model training device is a segmentation model based on the ResNeSt module. A better feature representation is extracted through group convolution and separation attention mechanism. The key information required for the target task can be learned more effectively through the deep learning ResNeSt module. Further, through spatial pyramid pooling, the limitation that traditional deep learning methods cannot handle CNVs of different sizes is solved, end-to-end training and prediction are realized, and CNV features in the input OCTA images can be quickly and conveniently identified, greatly improving the recognition accuracy of the segmentation model.

[0175] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0176] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0177] Optionally, an embodiment of the present invention further provides an electronic device, including:

[0178] processor;

[0179] a memory for storing instructions executable by the processor;

[0180] Wherein, the processor is configured to execute the instructions to implement the above-mentioned OCTA image-based new blood vessel segmentation method or the above-mentioned segmentation model training method.

[0181] Optionally, an embodiment of the present invention further provides a computer-readable storage medium, which, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the OCTA image-based new blood vessel segmentation method or the segmentation model training method as described above.

[0182] Optionally, an embodiment of the present invention further provides a computer program product, including a computer program or instructions, which, when executed by a processor of an electronic device, implements the above-mentioned OCTA image-based new blood vessel segmentation method or the above-mentioned segmentation model training method.

[0183] See also Figure 8 , is a block diagram of an electronic device 800 provided in an embodiment of the present invention. For example, the electronic device 800 may be a mobile terminal or a server, and the electronic device is a mobile terminal as an example for explanation in the embodiment of the present invention. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0184] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0185] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0186] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0187] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0188] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0189] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0190] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0191] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0192] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0193] In an embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned OCTA image-based neovascularization segmentation method.

[0194] In an embodiment, a computer-readable storage medium is also provided, and when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device 800 can execute the above-mentioned method for segmenting new blood vessels based on OCTA images. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0195] In an embodiment, a computer program product is further provided, including a computer program or instructions. When the computer program or instructions are executed by the processor 820 of the electronic device 800, the electronic device 800 executes the above-mentioned OCTA image-based new blood vessel segmentation method.

[0196] See also Fig. 9 , is a block diagram of a device 900 for segmenting new blood vessels based on OCTA images provided by an embodiment of the present invention. For example, the device 900 may be provided as a server. Fig. 9 The device 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932 for storing instructions that can be executed by the processing component 922, such as an application. The application stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute instructions to perform the above-mentioned OCTA image-based neovascularization segmentation method.

[0197] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0198] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0199] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for segmenting new blood vessels based on OCTA images, characterized in that: include: Acquiring an optical coherence tomography angiography (OCTA) image to be identified; Choroidal neovascularization CNV segmentation is performed on the OCTA image based on the segmentation model to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result.

2. The method for segmenting new blood vessels based on OCTA images according to claim 1, characterized in that: Before performing choroidal neovascularization CNV segmentation on the processed OCTA image based on the segmentation model, the method further includes: The OCTA image is subjected to contrast enhancement processing to obtain the processed OCTA image.

3. The method for segmenting new blood vessels based on OCTA images according to claim 1 or 2, characterized in that: The performing choroidal neovascularization CNV segmentation on the OCTA image based on the segmentation model to obtain a CNV segmentation result also includes: A spatial pyramid pooling SPP module is added behind each encoder in the segmentation model to learn context information of different scales, and a deep feature map of the same size is obtained to enhance the recognition of CNVs of different sizes by the segmentation model.

4. The method for segmenting new blood vessels based on OCTA images according to claim 1 or 2, characterized in that: The method further comprises: pre-training the segmentation model by: Acquire a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image; Inputting the plurality of sample OCTA images into a segmentation model, the segmentation model comprising a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module; The ResNeSt module is used to extract features on each sample OCTA image through group convolution and separation attention; Performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain corresponding segmentation results; Comparing the segmentation result with the corresponding CNV annotation map, and calculating the loss value according to the comparison result; Based on the loss value, the parameters of the segmentation model are updated through a back-propagation mechanism, and after multiple iterations, until the loss value reaches a minimum value or the segmentation model converges, a trained segmentation model is obtained.

5. The method for segmenting new blood vessels based on OCTA images according to claim 4, characterized in that: The features extracted from each sample OCTA image are subjected to spatial pyramid pooling, cascading and convolution processing to obtain corresponding segmentation results, including: The extracted features are passed through the corresponding spatial pyramid pooling SPP module to learn context information of different sizes and obtain deep feature maps of the same size; After cascading and convolving the deep feature maps of the same size, the input of the next encoder layer is obtained; The features output by the encoder are input into the decoder for decoding to obtain the corresponding segmentation results.

6. A segmentation model training method, characterized in that: The method comprises: Acquire a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image; Inputting the plurality of sample OCTA images into a segmentation model, the segmentation model comprising a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module; The ResNeSt module is used to extract features on each sample OCTA image through group convolution and separation attention; Performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain corresponding segmentation results; Comparing the segmentation result with the corresponding CNV annotation map, and calculating the loss value according to the comparison result; Based on the loss value, the parameters of the segmentation model are updated through a back-propagation mechanism, and after multiple iterations, until the loss value reaches a minimum value or the segmentation model converges, a trained segmentation model is obtained.

7. The segmentation model training method according to claim 6, characterized in that: The extracted features on each sample OCTA image are subjected to spatial pyramid pooling, cascading and convolution processing to obtain corresponding segmentation results, including: The extracted features are passed through the corresponding spatial pyramid pooling SPP module to learn context information of different sizes and obtain deep feature maps of the same size; After cascading and convolving the deep feature maps of the same size, the input of the next encoder layer is obtained; The features output by the encoder are input into the decoder for decoding to obtain the corresponding segmentation results.

8. A new blood vessel segmentation device based on OCTA images, characterized in that: include: An acquisition module, used for acquiring an optical coherence tomography angiography OCTA image to be detected; A segmentation module is used to perform choroidal neovascularization CNV segmentation on the OCTA image based on a segmentation model to obtain a CNV segmentation result, wherein the segmentation model includes multiple encoders and corresponding multiple decoders, each encoder and decoder is constructed by a ResNeSt module, and the ResNeSt module performs feature learning through group convolution and separation attention, and the probability output by the last layer of the decoder is used as the predicted segmentation result.

9. A segmentation model training device, characterized in that: The device comprises: A training set acquisition module is used to acquire a training set, wherein the training set includes: a plurality of sample OCTA images, and a CNV annotation image corresponding to each sample OCTA image; An input module, used for inputting the plurality of sample OCTA images into a segmentation model for training, wherein the segmentation model comprises a plurality of encoders and a corresponding plurality of decoders, each encoder and decoder being constructed by a ResNeSt module; An extraction module, for extracting features on each sample OCTA image by using the ResNeSt module through group convolution and separation attention; A processing module, used for performing spatial pyramid pooling, cascading and convolution processing on the features extracted from each sample OCTA image to obtain a corresponding segmentation result; A determination module, used to compare the segmentation result with the corresponding CNV annotation map, and calculate the loss value according to the comparison result; The iterative module is used to update the parameters of the segmentation model through a back-propagation mechanism based on the loss value, and obtain a trained segmentation model after multiple iterations until the loss value reaches a minimum value or the segmentation model converges.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the OCTA image-based neovascularization segmentation method as described in any one of claims 1 to 5 or the segmentation model training method as described in claim 6 or 7.

11. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the OCTA image-based neovascularization segmentation method as described in any one of claims 1 to 5 or the segmentation model training method as described in claim 6 or 7.

Images