A full-automatic partition measurement method for retinal vascular density

Abstract

The present application relates to the technical field of computer image processing, in particular to a kind of retinal vessel density full-automatic partition measurement method;The present application uses the retinal color photo that ultra-wide-angle fundus camera is shot in multiple imaging modes, uses specific image processing method to combine it into three-dimensional matrix, uses improved convolutional neural network to identify three-dimensional matrix, realizes the full-automatic positioning of each area of retinal periphery, blood vessel identification, blood vessel density calculation;Meanwhile, it also makes up the vacancy of retinal vessel density full-automatic partition measurement based on ultra-wide-angle multi-imaging mode fundus camera, proposes the scheme of fine partitioning of retinal periphery according to field angle and an improved method of convolutional neural network fusing the features of multi-imaging mode image, to realize the full-automatic measurement of blood vessel density of each area of retinal periphery.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer image processing, in particular to a full-automatic partition measurement method for retinal blood vessel density. BACKGROUND

[0002] The ultra-wide-angle multi-imaging mode fundus camera is a technology for imaging the retina using different wavelengths of light. It has the advantages of non-contact, high resolution, short image acquisition time, and wide imaging range, and is widely used in clinical practice. According to the wavelength of the emitted laser during shooting, it has multiple imaging modes, such as red-free imaging (RF, wavelength 488nm) and near-infrared imaging (IR, wavelength 820nm). In clinical practice, doctors can use the ultra-wide-angle fundus camera to observe the retinal periphery. Different imaging modes are beneficial for doctors to diagnose different retinal diseases. Previous studies were limited by the imaging range of the instrument, and were often limited to observing the blood vessels in the central retina and calculating and analyzing the blood vessel density. Few studies have divided and calculated the blood vessel density in the peripheral retina. The ultra-wide-angle color imaging range can reach 100 degrees, and combined with image stitching technology, the imaging range can reach up to 240 degrees.

[0003] The technology of classifying objects on a two-dimensional image at the pixel level is called semantic segmentation. Semantic segmentation can identify retinal blood vessels in an image. The conventional convolutional neural network used for retinal blood vessel semantic segmentation is Unet. Unet only accepts single-channel grayscale images, so in previous image preprocessing methods, retinal images are often converted to grayscale images before being fed into the network for training and prediction. The conventional method of converting multi-channel color images to grayscale images loses the data of multiple channels before being fed into the convolutional neural network, which is not conducive to the convolutional neural network finding the true features of the retinal structure and cannot complete the blood vessel identification of multi-imaging mode retinal images.

[0004] In order to locate the peripheral area of the retina, it is necessary to identify the special structure on the retina. Due to the large differences in the retinal structure of different individuals, their morphology changes greatly in pathological conditions, so the usual computer image recognition technology cannot meet the needs of this task. Mask region convolutional neural network (Mask RCNN) is a convolutional neural network for instance segmentation. It has good generalization and high final accuracy. While detecting the retinal structure, it can also generate a mask (Mask) for fine pixel segmentation.

[0005] In medical research, in order to explore the relationship between the blood vessel density of different regions of the retina and some diseases, the use of multi-imaging mode fundus cameras to obtain the retinal morphology, and the observation of the need of the peripheral blood vessel density of different regions of the patient still exists. Design a kind of retinal blood vessel density full-automatic partition measurement method based on the super wide-angle multi-imaging mode fundus camera, which will greatly improve the efficiency in medical research and clinical work.

[0006] Therefore, a kind of retinal blood vessel density full-automatic partition measurement method based on the super wide-angle multi-imaging mode fundus camera is proposed. SUMMARY

[0007] The purpose of the present application is to provide a kind of retinal blood vessel density full-automatic partition measurement method to make up the vacancy of using super wide-angle multi-imaging mode fundus camera to study the retinal peripheral blood vessel density in current clinical and scientific research work.

[0008] In order to achieve the above purpose, the technical scheme of the present application is as follows:

[0009] A kind of retinal blood vessel density full-automatic partition measurement method, comprising:

[0010] the super wide-angle fundus camera is obtained by super wide-angle multi-mode retinal color photograph, the image of retinal color photograph is handled, the imaging range of retinal color photograph is identified, and the ratio of small field angle pixel length and large field angle pixel length is calculated out;

[0011] based on imaging mode, the images of multiple imaging modes are combined, and multiple three-dimensional matrices are generated, image processing method is used, multiple three-dimensional matrix splicing is carried out, and spliced three-dimensional matrix is generated, including: four image channels are obtained by shooting through three modes, four channels are superimposed, and two three-dimensional image matrices respectively having four image channels are combined, wherein, 0 channel of three-dimensional image matrix is MC confocal laser image G channel, 1 channel is B channel of MC confocal laser image, 2 channel is IR no red light imaging image, and 3 channel is RF near infrared imaging image channel, and two three-dimensional image matrices are three-dimensional image matrices with retina macula and retina optic disc as center respectively;Two three-dimensional image matrices are spliced by image registration and image fusion, and spliced three-dimensional matrix is generated;

[0012] adjust the parameters of Mask RCNN and the spliced three-dimensional matrix based on the requirements of Mask RCNN to the image;

[0013] the spliced three-dimensional matrix is taken as input, and the retinal structure is identified based on Mask RCNN trained by self-labeled retinal structure instance segmentation dataset, so as to position the peripheral region of retina;

[0014] The three-dimensional matrix after splicing is adjusted as input, and an improved convolutional neural network trained based on a self-labeled retinal blood vessel semantic segmentation dataset is used to identify the retinal structure and identify the retinal blood vessels, wherein the improved convolutional neural network is a convolutional layer added in front of a conventional Unet network, which is used to make the improved convolutional neural network fuse the identified target features under different imaging mode images, and generate a feature map to identify the retinal blood vessels in the retinal color image.

[0015] Based on the retinal peripheral area positioning result and the retinal blood vessel identification result, the blood vessel density of different areas is calculated.

[0016] Specifically, the present application relates to image processing of the retinal color image, identifying the imaging range of the retinal color image, and calculating the small field angle pixel length corresponding to the large field angle image, which includes: image splicing of the retinal color image, removing the black edge of the image, identifying the field angle when the image is taken, and generating the pixel length of different field angles in the image according to the field angle of the image.

[0017] Optionally, the imaging range of the retinal color image is identified, and the ratio of the small field angle pixel length to the large field angle pixel length is calculated, and the calculation formula is as follows:

[0018]

[0019] Wherein, alpha is the large field angle, beta is the small field angle, EF is the pixel length of the small field angle, and CC' is the pixel length of the large field angle.

[0020] Specifically, the present application relates to based on imaging mode, combining images of multiple imaging modes, and generating multiple three-dimensional matrices, using image processing method, splicing multiple three-dimensional matrices to generate three-dimensional matrix after splicing; adjusting the parameters of Mask RCNN and the requirements of Mask RCNN for image to adjust the three-dimensional matrix after splicing, including: according to the number of imaging modes, judging the channel number of the matrix generated after the image of multiple imaging modes is superimposed, putting the image of each imaging mode into different channels of the matrix to generate the three-dimensional matrix after splicing; according to the channel number of the image matching three-dimensional matrix, adjusting the channel number of the image input into Mask RCNN, image mean value and other parameters, and the requirements of Mask RCNN for image are that the length and width of image are both integral multiples of six times of 2.

[0021] Optionally, the imaging mode includes one or more of confocal laser color imaging, red-free light imaging and near-infrared imaging.

[0022] Optionally, the Mask RCNN trained based on the self-labeled retinal structure instance segmentation dataset identifies the retinal structure, and the retinal peripheral region is positioned, wherein the retinal peripheral region is divided into 8 small areas and 2 large areas for zoned calculation of retinal peripheral vascular density, including: 100 degrees to 60 degrees nasal side from the center of the optic disc, 100 degrees to 60 degrees upper side from the center of the optic disc, 100 degrees to 60 degrees temporal side from the center of the optic disc, 100 degrees to 60 degrees lower side from the center of the optic disc, 60 degrees to 30 degrees nasal side from the center of the optic disc, 60 degrees to 30 degrees upper side from the center of the optic disc, 60 degrees to 30 degrees temporal side from the center of the optic disc, 60 degrees to 30 degrees lower side from the center of the optic disc, 100 degrees to 60 degrees from the center of the macula and optic disc, and 60 degrees to 30 degrees from the center of the macula and optic disc.

[0023] Specifically, the application relates to adding a convolution layer in front of a conventional Unet network, and the added convolution layer has an input channel number equal to the channel number of the three-dimensional matrix, an output channel number of 1, a kernel size of 3*3, zero padding, and a step of 1.

[0024] Optionally, the retinal peripheral region positioning result and the retinal vascular identification result are used to calculate the vascular density of different regions, and the calculation formula is as follows:

[0025]

[0026] Wherein a is the sum of the background pixels in the region, b is the sum of the vascular pixel numbers in the region on the retinal vascular image, and d is the vascular density of the region.

[0027] Specifically, the region background pixel is the pixel number of the white part of the mask, and b is the sum of the vascular pixel numbers with a possibility greater than 50% in the region.

[0028] The application has the following beneficial effects:

[0029] (1) The retinal vascular density full-automatic zoned measurement method provided by the application adopts a full-automatic measurement process, does not need manual intervention, excludes human factor interference, and has strong repeatability of results.

[0030] (2) The application uses an improved convolutional neural network to identify the retinal structure, and uses a mask to position the geometric center, which is more accurate.

[0031] (3) The retinal vascular density full-automatic zoned measurement method provided by the application has strong compatibility, has a low requirement for the shape of an image, and can correctly identify an image spliced from multiple retinal color photos.

[0032] (4) The improved convolutional neural network provided by the application adds a convolutional layer in front of the conventional Unet network, so that the Unet network has the ability to extract image features of multiple imaging modes without greatly increasing the number of Unet network layers;

[0033] (5) The retinal blood vessel density full-automatic partition measurement method provided by the application can greatly improve the analysis accuracy of blood vessel density by fine partitioning of the retinal peripheral region. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 It is a retinal color photograph obtained by the ultra-wide-angle fundus camera in the embodiment of the application; wherein, figures (a)-(c) are retinal color photographs taken with the retina macula as the center, figures (d)-(f) are retinal color photographs taken with the retina optic papilla as the center, specifically, figure (a) is an MC confocal laser image taken with the retina macula as the center, figure (b) is an RF non-red light imaging image taken with the retina macula as the center, figure (c) is an IR near-infrared imaging image taken with the retina macula as the center, figure (d) is an MC confocal laser image taken with the retina optic papilla as the center, figure (e) is an RF non-red light imaging image taken with the retina optic papilla as the center, and figure (f) is an IR near-infrared imaging image taken with the retina optic papilla as the center;

[0035] Figure 2 It is a spliced image of the MC confocal laser color photograph after the image is processed by the ultra-wide-angle image processing module in the embodiment of the application;

[0036] Figure 3 It is a radius range data display figure recognized by the ultra-wide-angle image processing module in the embodiment of the application;

[0037] Figure 4 It is an image after the image is recognized by the Mask RCNN in the embodiment of the application, wherein, opticpapilla is the optic disc (i.e. the optic papilla) of the retina structure, macula lutea is the macula of the retina structure, and the three concentric circles from outside to inside represent the range of 100 degrees, 60 degrees and 30 degrees field of view angle in the image;

[0038] Figure 5Figure 1 is a schematic diagram of 8 small areas of the peripheral retina after image recognition by Mask RCNN in an embodiment of the present application; wherein 1-100 degrees to 60 degrees nasal side from the center of the optic disc, 2-100 degrees to 60 degrees superior side from the center of the optic disc, 3-100 degrees to 60 degrees temporal side from the center of the optic disc, 4-100 degrees to 60 degrees inferior side from the center of the optic disc, 5-60 degrees to 30 degrees nasal side from the center of the optic disc, 6-60 degrees to 30 degrees superior side from the center of the optic disc, 7-60 degrees to 30 degrees temporal side from the center of the optic disc, 8-60 degrees to 30 degrees inferior side from the center of the optic disc;

[0039] Figure 6 Figure 2 is a schematic diagram of 2 large areas of the peripheral retina in an embodiment of the present application; wherein 9-60 degrees to 30 degrees from the center of the macula and the optic disc, 10-100 degrees to 60 degrees from the center of the macula and the optic disc;

[0040] Figure 7 Figure 3 is a result image of improved convolutional neural network recognition in an embodiment of the present application, wherein figure (a) is a visual image of a three-dimensional matrix, figure (b) is a gray-scale image converted from a probability matrix, and figure (c) is a retinal blood vessel image;

[0041] Figure 8 Figure 4 is a display image of the partitioned statistical results of retinal blood vessel density in an embodiment of the present application;

[0042] Figure 9 Figure 5 is a flow chart of the full-automatic partitioned measurement method of retinal blood vessel density in an embodiment of the present application;

[0043] Figure 10 Figure 6 is a structural schematic diagram of the retina as a spherical surface in an embodiment of the present application. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0045] Referring to the drawings Figure 1 and the drawings Figure 9 A full-automatic partitioned measurement method of retinal blood vessel density, comprising:

[0046] The super-wide-angle multi-mode retinal color photograph is obtained by a super-wide-angle fundus camera, the retinal color photograph is image-processed, the imaging range of the retinal color photograph is recognized, and the ratio of the small field angle pixel length to the large field angle pixel length is calculated; based on the imaging mode, the images of multiple imaging modes are combined, and multiple three-dimensional matrices are generated, the image processing method is used to splice the multiple three-dimensional matrices to generate a spliced three-dimensional matrix; the parameters of Mask RCNN and the spliced three-dimensional matrix based on the requirements of Mask RCNN on the image are adjusted; the Mask RCNN trained based on the self-labeled retinal structure instance segmentation dataset is used to recognize the retinal structure, and the retinal peripheral region is positioned; the improved convolutional neural network trained based on the self-labeled retinal blood vessel semantic segmentation dataset is used to recognize the retinal structure, and the retinal blood vessel is recognized; based on the retinal peripheral region positioning result and the retinal blood vessel recognition result, the blood vessel density of different regions is calculated.

[0047] Further, the application relates to a full-automatic retinal blood vessel density partition measurement method.

[0048] Further, the application relates to the development environment: a deep learning platform is built and code of a process is written by using Python3.6, Tensorflow2.4.0, Keras2.4.3, Pytorch1.10, CUDA11.4, CPU: i7 10700K and a graphics card: NVIDIA GeForce RTX 3080Ti.

[0049] Further, the application relates to a super-wide-angle image processing module, which is used for splicing two retinal color photographs taken by using multiple imaging modes, reducing the image size, recognizing the imaging range of the retinal color photograph, and calculating the small field angle corresponding pixel length of the large field angle image by a formula.

[0050] Further, the application relates to a super-wide-angle image processing module processing image method, and the specific steps include:

[0051] In the embodiment, the retinal color photograph is read, the retinal color photograph is an image taken by a super-wide-angle multi-mode fundus camera at a 100-degree field angle and divided into MC, RF and IR according to the shooting mode: the first shooting is performed by an operator taking the retina macula as the center, referring to (a)-(c) in the accompanying drawings; Figure 1 The second shooting is performed by the operator taking the retina optic disc as the center, referring to (d)-(f) in the accompanying drawings. Figure 1

[0052] ​In the embodiment, the MC confocal laser image has 3 channels (RGB), but the values in the R channel are all null values, and the IR achromatic light imaging image and the RF near-infrared imaging image are both gray-scale images, only one channel; therefore, in the embodiment, the images taken by the three modes have a total of 4 image channels (discard the R channel of the MC confocal laser image), and the 4 channels are superimposed to combine into 2 three-dimensional image matrices each having 4 channels; wherein, the 0 channel of the three-dimensional image matrix is the G channel of the previous MC confocal laser image, the 1 channel is the B channel of the previous MC confocal laser image, the 2 channel is the previous IR achromatic light imaging image, and the 3 channel is the previous RF near-infrared imaging image channel; by image registration and image fusion, the two three-dimensional image matrices are spliced, and the visualization results of the 0 and 1 channels in the spliced three-dimensional matrix are shown in Figs. 8 and 9. Figure 2 and Figs. Figure 3 , Figs. Figure 2 are the visualization results of the 0 and 1 channels in the spliced three-dimensional matrix.

[0053] Further, the present application involves identifying and cropping the image edge, scaling the image size to retain more effective data; using threshold screening to screen out the black part (the value of the position in the 4 channels is 0) in the three-dimensional matrix, and the large mask is used to screen out the circular part with the retinal color from the square picture; through the mask, the computer calculates the pixel diameter of the circular part in the vertical direction of the image, and the pixel diameter is taken as the pixel diameter of the 100-degree field of view; in the embodiment, the diameter in the vertical direction after scaling is 790; according to the geometric relationship of the field of view, the image taken at a 60-degree field of view is projected into the image taken at a 100-degree field of view, and the ratio is calculated according to the formula and the 2 effective digits are retained as 0.65, that is, the pixel length of the 60-degree field of view in the 100-degree image is 513.5, which is rounded to 514; the image taken at a 30-degree field of view is projected into the image taken at a 100-degree field of view, and the ratio is calculated according to the formula and the 2 effective digits are retained as 0.34, that is, the pixel length of the 30-degree field of view in the 100-degree image is 268.6, which is rounded to 269.

[0054] In the embodiment, the present application involves identifying the imaging range of the retinal color, calculating the ratio of the pixel length of the small field of view to the pixel length of the large field of view, and the calculation formula is as follows:

[0055]

[0056] Wherein, α is the large field of view, β is the small field of view, EF is the pixel length of the small field of view, and CC' is the pixel length of the large field of view.

[0057] In the embodiment, the specific analysis process of the above steps is as follows:

[0058] Referring to Figs.Figure 10 The retina is regarded as a spherical surface, the circular arc CBC' is the retina, A is the center of the spherical surface, and the cross section is as shown in FIG. 1. Figure 10 The center of the curved retina is point B, the large field angle is α, the small field angle is β, the range of the retina photographed by the large field angle is the circular arc CBC', the range of the retina photographed by the small field angle is the circular arc DBD', and the curved retina is projected onto the plane to form an image because the photosensitive element is a plane. The circular arc DBD' is projected into the line segment DD', and the circular arc CBC' is projected into the line segment CC'. The imaging range of the large field angle more than that of the small field angle is the two parts of CD and C'D' in the figure. Therefore, the ratio of the imaging range of the small field angle image to that of the large field angle image in the figure is

[0059] In the embodiment, the imaging range of the small field angle image is the imaging range of the large field angle image in the figure. The formula derivation is as follows:

[0060] According to the sine theorem:

[0061]

[0062] AC = AD = AD' = AC', and the ratio of the imaging range of the small field angle image to that of the large field angle image in the figure is obtained after simplification:

[0063]

[0064] The ratio of the imaging range of the small field angle image to that of the large field angle image in the figure is obtained by bringing α and β into the formula.

[0065] Further, the present application relates to an imaging mode based on the combination of images of multiple imaging modes, and the generation of multiple three-dimensional matrices. An image processing method is used to splice the multiple three-dimensional matrices to generate a spliced three-dimensional matrix. The parameters of Mask RCNN and the requirements of Mask RCNN for images are adjusted to adjust the spliced three-dimensional matrix, including: according to the number of imaging modes, judging the channel number of the matrix generated after the image superposition of multiple imaging modes, putting the images of each imaging mode into different channels of the matrix, and generating a spliced three-dimensional matrix; according to the channel number of the image matching three-dimensional matrix, adjusting the channel number of the image transmitted into Mask RCNN, the image mean, and other parameters, and the requirements of Mask RCNN for images are that the length and width of the image are both integral multiples of 2 to the power of 6.

[0066] In the embodiment, the retinal structure recognition module of the application is used for labeling a plurality of data sets to train a Mask RCNN neural network to realize position recognition of a macula and an optic disc of a retina; and parameters such as a channel number of an image input by the Mask RCNN and an image mean value are adjusted, in the embodiment, the channel number of the image input by the Mask RCNN should be 4, and the image mean value of each channel is adjusted according to an actual situation of each channel, and in the embodiment, the image mean value is [116.8, 103.9, 126.4, 127.5].

[0067] Further, referring to the accompanying drawings Figure 4 - the accompanying drawings Figure 6 The application relates to sorting and analyzing a prediction result of the Mask RCNN, extracting a macula and an optic disc with the highest confidence; using the predicted mask, the average value of the horizontal and vertical coordinates of the mask pixels is rounded to an integer to serve as a geometric center of the macula and the optic disc; using the macula and the optic disc as the center, a 30-degree and a 60-degree field of view angle imaging range generated by the super-wide-angle image recognition processing module in the 100-degree image as a diameter, and using the geometric center of the macula and the optic disc mask as a center, eight small areas and two large areas are divided, which are as follows: 100 degrees to 60 degrees nasal side of the optic disc, 100 degrees to 60 degrees upper side of the optic disc, 100 degrees to 60 degrees temporal side of the optic disc, 100 degrees to 60 degrees lower side of the optic disc, 60 degrees to 30 degrees nasal side of the optic disc, 60 degrees to 30 degrees upper side of the optic disc, 60 degrees to 30 degrees temporal side of the optic disc, 60 degrees to 30 degrees lower side of the optic disc, 100 degrees to 60 degrees of the macula and the optic disc, and 60 degrees to 30 degrees of the macula and the optic disc.

[0068] In the embodiment, referring to the accompanying drawings Figure 7 The blood vessel recognition module of the application is used for recognizing a retinal structure by using an improved convolutional neural network trained by a self-labeled retinal blood vessel semantic segmentation data set, and for recognizing the retinal blood vessels; when the data set is made, a part of the blood vessels is labeled by an ophthalmologist using an MC image; the improved convolutional neural network is as follows: a convolutional layer is added to the first layer of the Unet, the input of the convolutional layer is 4 channels, the convolution is 1 channel, the convolution kernel size is 3x3, and zero padding is used to ensure that the image size is unchanged before and after the convolution.

[0069] In the embodiment, the Pytorch deep learning framework of the application is used, and the representation code of the layer is as follows:

[0070] self.layer0_down = torch.nn.Conv2d(in_channels=4, out_channels=1, kernel_size=(3, 3), stride=1, padding_mode="zeros", padding=1).

[0071] In this embodiment, after the convolutional neural network is trained to converge, the three-dimensional matrix after the above normalization processing (the pixel value of each position is divided by 255) is used to perform semantic segmentation to generate a probability prediction map.

[0072] In this embodiment, referring to the accompanying drawings Figure 8 , the blood vessel density partition statistics module is used to apply the mask of each region to the image respectively, and the blood vessel density is statistically analyzed in the region, and an image report is generated after the statistics is completed; the value range of each value in the probability prediction map is 0 to 1, the closer to 1, the greater the possibility that the pixel belongs to the blood vessel, and the closer to 0, the smaller the possibility that the pixel belongs to the blood vessel; the binarization processing is performed with 0.5 as the threshold, the brightness value of the pixel with a possibility greater than 50% as the blood vessel is set to 255, that is, white, to generate a retinal blood vessel image.

[0073] Further, based on the retinal peripheral region positioning result and the retinal blood vessel identification result, the blood vessel density of different regions is calculated, and the calculation formula is as follows:

[0074]

[0075] Wherein, a is the sum of the background pixels of the region, b is the sum of the number of blood vessel pixels in the region on the retinal blood vessel image, and d is the blood vessel density of the region.

[0076] Specifically, the background pixels of the region are the number of pixels of the white part of the mask, and b is the sum of the number of pixels with a possibility greater than 50% as blood vessel pixels in the region.

[0077] In summary, the application discloses a full-automatic retinal blood vessel density partition measurement method, which can realize full-automatic computer retinal peripheral blood vessel density partition measurement; the flow calculates the low visual field angle corresponding pixel length and ratio of the high visual field angle image through a formula, fuses the retinal images shot in multiple imaging modes to generate an image three-dimensional matrix, accurately identifies the geometric center of the retinal structure in the matrix through Mask RCNN, adds a convolution layer in the original Unet network, makes the Unet network have the ability to fuse the features of the retinal in different imaging modes without greatly increasing the network layers, and can more accurately identify the blood vessel pixels; the retinal periphery is qualitatively partitioned based on the concept of the visual field angle, and the quantitative measurement of the blood vessel density of each region of the retinal periphery is completed.

[0078] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the purpose and scope of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. A full-automatic partition measurement method for retinal vascular density, characterized in that, The application comprises the following steps: An ultra-wide-angle multi-mode retinal color photograph is obtained by an ultra-wide-angle fundus camera, the retinal color photograph is image-processed, the imaging range of the retinal color photograph is identified, and the ratio of the pixel length of a small field angle to the pixel length of a large field angle is calculated; Based on the imaging mode, the images of multiple imaging modes are combined, and multiple three-dimensional matrices are generated, and the multiple three-dimensional matrices are spliced by using an image processing method to generate a spliced three-dimensional matrix, including: four image channels are obtained by photographing in three modes, four channels are superimposed, and two three-dimensional image matrices each having four image channels are combined, wherein the 0 channel of the three-dimensional image matrix is an MC confocal laser image G channel, the 1 channel is an MC confocal laser image B channel, the 2 channel is an IR non-red light imaging image, and the 3 channel is an RF near-infrared imaging image channel, and the two three-dimensional image matrices are three-dimensional image matrices centered on the retinal macula and the retinal disc respectively; the two three-dimensional image matrices are spliced by image registration and image fusion to generate a spliced three-dimensional matrix; The parameters of Mask RCNN and the spliced three-dimensional matrix based on the requirements of Mask RCNN on images are adjusted; The spliced three-dimensional matrix is taken as input, and Mask RCNN trained based on a self-labeled retinal structure instance segmentation dataset is used to identify retinal structures and locate the retinal peripheral region; The spliced three-dimensional matrix is taken as input, and an improved convolutional neural network trained based on a self-labeled retinal blood vessel semantic segmentation dataset is used to identify retinal structures and identify retinal blood vessels in the retinal color photograph, wherein the improved convolutional neural network is a convolutional layer added in front of a conventional Unet network, which is used to make the improved convolutional neural network fuse the identified target features under different imaging mode images and generate a feature map to identify retinal blood vessels in the retinal color photograph; Based on the retinal peripheral region positioning result and the retinal blood vessel identification result, the blood vessel density of different regions is calculated.

2. The method of claim 1, wherein the method further comprises: The imaging range of the retinal color photograph is identified, and the ratio of the pixel length of a small field angle to the pixel length of a large field angle is calculated, and the calculation formula is as follows: Wherein, α is a large field angle, β is a small field angle, EF is the pixel length of a small field angle, and CC' is the pixel length of a large field angle.

3. The method of claim 1, wherein the method further comprises: The imaging mode includes one or more of confocal laser color photography, non-red light imaging, and near-infrared imaging.

4. The method of claim 1, wherein the method further comprises: The Mask RCNN trained based on the self-labeled retinal structure instance segmentation dataset identifies the retinal structure, and the retinal peripheral region is positioned, wherein the retinal peripheral region is divided into 8 small areas and 2 large areas for partition calculation of the retinal peripheral vascular density, including: 100 degrees to 60 degrees nasal side from the optic disc center, 100 degrees to 60 degrees upper side from the optic disc center, 100 degrees to 60 degrees temporal side from the optic disc center, 100 degrees to 60 degrees lower side from the optic disc center, 60 degrees to 30 degrees nasal side from the optic disc center, 60 degrees to 30 degrees upper side from the optic disc center, 60 degrees to 30 degrees temporal side from the optic disc center, 60 degrees to 30 degrees lower side from the optic disc center, 100 degrees to 60 degrees from the macula and optic disc center, 60 degrees to 30 degrees from the macula and optic disc center.

5. The method of claim 1, wherein the method further comprises: The vascular density of different regions is calculated based on the retinal peripheral region positioning result and the retinal vascular identification result, and the calculation formula is as follows: Wherein, a is the sum of the background pixels of the region, b is the sum of the vascular pixel number in the region on the retinal vascular image, and d is the vascular density of the region.

Images