A method and system for drusen image lesion segmentation integration

Abstract

The application discloses a kind of fundus sugar net image lesion segmentation integrated method and system, it is related to image processing field, wherein the method comprises: obtaining multi-angle fundus image set;Obtain extended multi-angle fundus image set;Obtain basic segmentation model;Obtain mobile terminal integrated segmentation model;Based on this, obtain model output result, model output result includes feature point segmentation result image set;Obtain feature point recognition network model;Get target sugar net fundus image to be processed;It is input into mobile terminal integrated segmentation model and feature point recognition network model, and obtains fundus sugar net segmentation image.The technical problem that the accuracy of prior art is not high for fundus sugar net image lesion segmentation, and then leading to the effect of fundus sugar net image lesion segmentation is poor is solved.The technical effects of improving the accuracy of fundus sugar net image lesion segmentation and accuracy, improving the effect and quality of fundus sugar net image lesion segmentation are achieved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to a method and system for segmenting and integrating lesions in fundus diabetic retinopathy images. Background Technology

[0002] Diabetic retinopathy refers to a series of lesions caused by pathological changes in the retinal capillaries, arterioles, and venules, as well as leakage or blockage of these microvascular tissues. Diabetic retinopathy can lead to varying degrees of vision loss, and even blindness. Retinal fundus imaging is an important imaging tool for observing and diagnosing diabetic retinopathy. However, the acquisition of fundus image data is inevitably affected by factors such as insufficient field of view of traditional optical lenses, high requirements for pupil and refractive media, electronic system noise, and unsatisfactory image acquisition environment. As a result, the acquired fundus image data still has limitations in detecting and analyzing the location and morphology of diabetic retinopathy lesions.

[0003] In the existing technology, there is a technical problem that the accuracy of segmentation of diabetic retinopathy lesions in fundus images is not high, which leads to poor segmentation results of diabetic retinopathy lesions in fundus images. Summary of the Invention

[0004] This application provides a method and system for segmenting and integrating diabetic retinopathy lesions in fundus images, which solves the technical problem that the accuracy of segmentation of diabetic retinopathy lesions in fundus images is not high in the prior art, thus leading to poor segmentation results.

[0005] In view of the above problems, this application provides a method and system for lesion segmentation and integration of fundus diabetic retinopathy images.

[0006] In a first aspect, this application provides a method for segmenting and integrating diabetic retinopathy (DR) lesions in fundus images, wherein the method is applied to a system for segmenting and integrating diabetic retinopathy (DR) lesions in fundus images, and the method includes: obtaining a multi-angle fundus image set through an ultra-wide-angle laser image acquisition device; and performing data enhancement on the multi-angle fundus image set based on the training data enhancement module to obtain an expanded multi-angle fundus image set;

[0007] The expanded multi-angle fundus image set is used as training sample data. The basic network training module uses a deep learning network structure to perform semantic segmentation training on the training sample data to obtain a basic segmentation model. The basic segmentation model is used as input information. The mobile terminal network integration module integrates and trains the input information and the training sample data to obtain a mobile terminal integrated segmentation model.

[0008] Based on the mobile-integrated segmentation model, the model output results are obtained, including a set of feature point segmentation result images. The feature point recognition correction module is used to train a recognition network on the feature point segmentation result image set to obtain a feature point recognition network model. The fundus image of the target diabetic retinopathy to be processed is obtained. The fundus image of the target diabetic retinopathy to be processed is used as input image information and input into the mobile-integrated segmentation model and the feature point recognition network model to obtain a segmented fundus image of diabetic retinopathy.

[0009] Secondly, this application also provides a system for segmenting and integrating diabetic retinopathy lesions in fundus images, wherein the system includes: an image acquisition module for acquiring a multi-angle fundus image set using an ultra-wide-angle laser image acquisition device; a training data augmentation module for performing data augmentation on the multi-angle fundus image set to obtain an expanded multi-angle fundus image set; a basic network training module for using the expanded multi-angle fundus image set as training sample data and performing semantic segmentation training on the training sample data using a deep learning network structure to obtain a basic segmentation model; and a mobile network integration module for using the basic segmentation model as input information and integrating it through a mobile network. An integration module integrates and trains the input information and the training sample data to obtain a mobile-end integrated segmentation model; an intermediate output module is used to obtain model output results based on the mobile-end integrated segmentation model, the model output results including a set of feature point segmentation result images; a feature point recognition and correction module is used to train a recognition network on the feature point segmentation result image set to obtain a feature point recognition network model; an information acquisition module is used to acquire the fundus image of the target diabetic retinopathy to be processed; and a processing module is used to input the fundus image of the target diabetic retinopathy to be processed as input image information into the mobile-end integrated segmentation model and the feature point recognition network model to obtain a segmented fundus image of diabetic retinopathy.

[0010] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0011] A multi-angle fundus image set is obtained through an ultra-wide-angle laser image acquisition device; the data is augmented using a training data augmentation module to obtain an expanded multi-angle fundus image set; the expanded multi-angle fundus image set is used as training sample data, and semantic segmentation is trained using a deep learning network structure through the basic network training module to obtain a basic segmentation model; the basic segmentation model is used as input information, and the input information and the training sample data are integrated and trained through the mobile terminal network integration module to obtain a mobile terminal integrated segmentation model; based on this, the model output result is obtained, which includes a feature point segmentation result image set; the feature point recognition correction module is used to train a recognition network on the feature point segmentation result image set to obtain a feature point recognition network model; the fundus image of the target diabetic retinopathy to be processed is acquired; the fundus image of the target diabetic retinopathy to be processed is used as input image information and input into the mobile terminal integrated segmentation model and the feature point recognition network model to obtain a fundus diabetic retinopathy segmentation image. This study aims to improve the accuracy and precision of lesion segmentation in diabetic retinopathy (DR) fundus images, thereby enhancing the effectiveness and quality of lesion segmentation. Furthermore, it designs an optimized method for lesion segmentation in DR fundus images, enabling fine segmentation of multiple lesions in DR fundus images on mobile devices. This method also achieves the technical effect of efficiently analyzing the location and contour information of multiple lesions in DR fundus images on mobile devices. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a method for segmenting and integrating lesions in fundus diabetic retinopathy images according to this application;

[0013] Figure 2 This is a schematic diagram illustrating the process of stitching together a first standard angle fundus image set and a second standard angle fundus image set to obtain a multi-angle fundus image set in a method for segmenting and integrating fundus diabetic reticulum images according to this application.

[0014] Figure 3 This is a flowchart illustrating the process of obtaining an expanded multi-angle fundus image set in a method for segmenting and integrating fundus diabetic retinopathy images according to this application;

[0015] Figure 4 This is a flowchart illustrating the process of obtaining predicted semantic segmentation results in a method for merging lesion segmentation in fundus diabetic retinopathy images according to this application.

[0016] Figure 5 This is a schematic diagram of the structure of a system for segmenting and integrating diabetic retinopathy images of the fundus according to this application.

[0017] Figure labeling: Image acquisition module 11, training data augmentation module 12, basic network training module 13, mobile network integration module 14, intermediate output module 15, feature point recognition and correction module 16, information acquisition module 17, processing module 18. Detailed Implementation

[0018] This application provides a method and system for integrated segmentation of lesions in diabetic retinopathy (DR) images, solving the technical problem of low accuracy in lesion segmentation in existing technologies, which leads to poor segmentation results. It improves the accuracy and precision of lesion segmentation, enhancing the overall effect and quality. Furthermore, it designs an optimized method for lesion segmentation, enabling fine segmentation of multiple lesions in DR images on a mobile device, and efficiently analyzing the location and contour information of multiple lesions in DR images on a mobile device.

[0019] Example 1

[0020] Please see the appendix Figure 1 This application provides a method for segmenting and integrating lesions in fundus diabetic retinopathy images. The method is applied to a system for segmenting and integrating lesions in fundus diabetic retinopathy images, and specifically includes the following steps:

[0021] Step S100: Obtain a set of multi-angle fundus images using an ultra-wide-angle laser image acquisition device;

[0022] Furthermore, step S100 of this application also includes:

[0023] Step S110: The ultra-wide-angle laser image acquisition device includes a first wavelength laser and a second wavelength laser;

[0024] Step S120: Obtain a set of fundus images at a first angle based on the first wavelength laser and the second wavelength laser;

[0025] Step S130: Obtain a set of second-angle fundus images with different angle acquisition areas;

[0026] Step S140: Denoise the first angle fundus image set and the second angle fundus image set to obtain a first standard angle fundus image set and a second standard angle fundus image set;

[0027] Step S150: Perform image stitching on the first standard angle fundus image set and the second standard angle fundus image set to obtain the multi-angle fundus image set.

[0028] Specifically, lasers of different wavelengths have different tissue penetration capabilities, and multicolor photographs can display the tissue characteristics of fundus lesions at different levels in the image. The ultra-wide-angle laser image acquisition device used in this application has a first wavelength laser and a second wavelength laser. The first wavelength laser and the second wavelength laser are used to acquire images of the acquisition angle acquisition area to obtain a first-angle fundus image set; then, the first wavelength laser and the second wavelength laser are used to acquire images of the acquisition angle acquisition area at different angles to obtain a second-angle fundus image set. Consequently, the obtained first-angle and second-angle fundus image sets are inevitably affected by the ultra-wide-angle laser image acquisition device and external environmental noise interference during image acquisition and transmission, resulting in high noise and reduced image accuracy. Preferably, this application employs noise reduction processing to reduce the noise of the first-angle and second-angle fundus image sets, obtaining a first standard-angle fundus image set and a second standard-angle fundus image set, and then stitching these images together to obtain the multi-angle fundus image set.

[0029] The angle acquisition area refers to the region where images of the retina and fundus of various types of diabetic retinopathy lesions, such as hemorrhage, microaneurysms, hard exudates, soft exudates, neovascular fibrous membranes, and microvascular abnormalities, are acquired using an ultra-wide-angle laser image acquisition device. The multi-angle fundus image set is a multicolor photographic data information composed of a first standard angle fundus image set and a second standard angle fundus image set (i.e., laser images of two wavelengths). It has strong penetrating power, is not easily absorbed / reflected by refractive media, and facilitates disease differential diagnosis.

[0030] For example, the ultra-wide-angle laser image acquisition device can be an ultra-wide-angle laser confocal fundus camera (Micro-Clear CRO). Compared with traditional optical fundus cameras, the ultra-wide-angle laser confocal fundus camera has advantages such as adaptability to a wider range of patients, no need for pupil dilation, no need for a dark room, stronger penetration, and faster detection speed. The first wavelength laser can be a 532nm wavelength laser. The second wavelength laser can be a 785nm wavelength laser. The ultra-wide-angle laser confocal fundus camera also has a multi-angle wide-angle lens optical design. When the angle acquisition area is the macula, the ultra-wide-angle laser confocal fundus camera uses a 15° lens to acquire images of the macula, obtaining a first-angle fundus image set; the ultra-wide-angle laser confocal fundus camera then uses a 30° lens to acquire images of the macula, obtaining a second-angle fundus image set.

[0031] The technology achieves the effect of obtaining a set of fundus images from the first angle and a set of fundus images from the second angle using an ultra-wide-angle laser image acquisition device, and then performing noise reduction and image stitching to obtain a set of multi-angle fundus images with high accuracy, thereby improving the detection rate of lesions and avoiding the omission of lesions.

[0032] Further details are attached. Figure 2 As shown, step S150 of this application further includes:

[0033] Step S151: Select a reference image based on the first standard angle fundus image set and the second standard angle fundus image set to obtain a fundus reference image;

[0034] Step S152: Construct a reference image coordinate system based on the fundus reference image;

[0035] Step S153: Based on the aforementioned fundus reference image, obtain a set of fundus images to be stitched together;

[0036] Step S154: Map the set of fundus images to be stitched together in the reference image coordinate system to determine the image transformation matrix;

[0037] Step S155: Based on the image transformation matrix, the set of fundus images to be stitched is transformed and stitched to the fundus reference image to obtain the multi-angle fundus image set.

[0038] Specifically, when stitching together fundus images from a first standard angle set and a second standard angle set, firstly, a reference image is selected from the first and second standard angle fundus image sets to determine the fundus reference image, and a reference image coordinate system and a set of fundus images to be stitched are obtained based on it; then, the set of fundus images to be stitched is mapped to the reference image coordinate system to determine the image transformation matrix, and the matrix is ​​used to transform and stitch the set of fundus images to be stitched together with the fundus reference image to obtain the multi-angle fundus image set.

[0039] The fundus reference image is any image from the first set of standard-angle fundus images and the second set of standard-angle fundus images. The reference image coordinate system is a coordinate system with the fundus reference image as its origin. The fundus image set to be stitched is the image data information after removing the fundus reference image from the first set of standard-angle fundus images and the second set of standard-angle fundus images. The image transformation matrix is ​​data information used to represent the mapping relationship between the fundus image set to be stitched and the reference image coordinate system. Simultaneously, the image transformation matrix can also represent the specific position coordinate information of the fundus image set to be stitched in the reference image coordinate system. The multi-angle fundus image set is obtained by performing rotation, translation, and other transformation and stitching operations on the fundus image set to be stitched using the image transformation matrix, resulting in wide-angle, full-range fundus image data information after stitching the fundus image set to be stitched with the fundus reference image. This achieves the technical effect of using the image transformation matrix for relatively accurate image stitching, obtaining a multi-angle fundus image set with high accuracy and reliability, and providing data support for subsequently obtaining an expanded multi-angle fundus image set.

[0040] Step S200: Perform data augmentation on the multi-angle fundus image set based on the training data augmentation module to obtain an expanded multi-angle fundus image set;

[0041] Further details are attached. Figure 3 As shown, step S200 of this application further includes:

[0042] Step S210: Obtain image enhancement type information, which includes rotation, horizontal flip, vertical flip, and scaling;

[0043] Step S220: Determine the fundus image change coefficient based on the image enhancement type information;

[0044] Step S230: Based on the fundus image change coefficient, perform data amplification and change output on the multi-angle fundus image set to obtain the expanded multi-angle fundus image set.

[0045] Specifically, the training data augmentation module of the system integrating diabetic reticulum image lesion segmentation performs intelligent analysis on a multi-angle fundus image set to determine image enhancement type information and obtain fundus image variation coefficients based on it. Further, the training data augmentation module amplifies the multi-angle fundus image set using the fundus image variation coefficients to obtain the expanded multi-angle fundus image set. The image enhancement type information is data information used to characterize the data augmentation processing method required for the multi-angle fundus image set. The image enhancement type information includes rotation, horizontal flipping, vertical flipping, and scaling. The fundus image variation coefficients are data information used to characterize parameters such as the image enhancement type, specific image enhancement steps, and image enhancement degree corresponding to each image in the multi-angle fundus image set. The expanded multi-angle fundus image set is the data information obtained after amplifying each image in the multi-angle fundus image set using the fundus image variation coefficients. The technology achieves the goal of using the training data augmentation module to augment the multi-angle fundus image set, avoiding overfitting, and obtaining a larger expanded multi-angle fundus image set, thus laying the foundation for the subsequent basic network training module to use a deep learning network structure for semantic segmentation training.

[0046] Step S300: Using the expanded multi-angle fundus image set as training sample data, the training sample data is semantically segmented using a deep learning network structure through the basic network training module to obtain a basic segmentation model;

[0047] Further details are attached. Figure 4 As shown, step S300 of this application further includes:

[0048] Step S310: Divide the images in the expanded multi-angle fundus image set into a set of overlapping image blocks of a preset size;

[0049] Step S320: The deep learning network structure includes a network encoder and a network decoder;

[0050] Step S330: Input the set of overlapping image patches as input information into the network encoder to obtain multi-scale features of the image;

[0051] Step S340: Input the multi-scale features of the image into the network decoder to obtain the predicted semantic segmentation result.

[0052] Specifically, the basic network training module of the system integrating fundus diabetic reticulum image lesion segmentation automatically divides the images in the obtained expanded multi-angle fundus image set into a preset size to obtain an overlapping image block set. Then, the overlapping image block set is used as input information to the network encoder to obtain multi-scale image features, and these multi-scale features are then used as input information to the network decoder to obtain the predicted semantic segmentation result. Both the network encoder and the network decoder are included in the deep learning network structure. The preset size can be pre-set by the basic network training module or adaptively set according to actual conditions. The overlapping image block set is the image data information after dividing the images in the expanded multi-angle fundus image set into a preset size. For example, the expanded multi-angle fundus image set is an image of size H×W×3. The preset size is 7*7. After the basic network training module divides the expanded multi-angle fundus image set into a preset size, the resulting overlapping image block set is an expanded multi-angle fundus image set with a size of 7*7. Smaller image blocks are beneficial for predicting fundus images with dense lesions. The multi-scale features of the images include parameters such as the width and height of images in the overlapping image patch set, and the distance between adjacent images. The basic segmentation model is obtained by training an extended multi-angle fundus image set using a deep learning network structure through a basic network training module for semantic segmentation. This achieves the technical effect of obtaining a highly accurate basic segmentation model, providing data support for the subsequent determination of the mobile integrated segmentation model.

[0053] Furthermore, step S340 of this application also includes:

[0054] Step S341: The network decoder consists of an MLP layer;

[0055] Step S342: Unify the channel size of the multi-scale features of the image through the MLP layer;

[0056] Step S343: Perform feature sampling and feature concatenation on the multi-scale features of the image based on the channel size;

[0057] Step S344: The MLP layer is used to fuse concatenated features, and an MLP layer is added based on the concatenated features to obtain the predicted semantic segmentation result.

[0058] Specifically, to reduce computational load and improve efficiency, the network decoder consists of only one MLP layer. The MLP layer unifies the channel dimensions of the multi-scale features of the image, and then uses these channel dimensions to sample and concatenate the multi-scale features. Based on this, the MLP layer fuses the concatenated features, and another MLP layer is added to obtain the predicted semantic segmentation result. Here, MLP stands for Multi-Layer Perceptron, a generalization from the perceptron. Its characteristic is that it has multiple neuron layers, hence it is also called a deep neural network. The predictive ability of a neural network comes from its layered or multi-layered structure. A multi-layer perceptron refers to a neural network with at least three layers: an input layer, an intermediate layer, and an output layer. The concatenated features include overlapping features after feature sampling and concatenation of the multi-scale image features. The feature sampling can sample any feature from the multi-scale image features. The predicted semantic segmentation result is data information used to characterize the lesion feature segmentation result corresponding to the multi-scale image features. This achieves the technical effect of obtaining a highly accurate predicted semantic segmentation result, laying the foundation for obtaining a basic segmentation model.

[0059] Step S400: Using the basic segmentation model as input information, the mobile terminal network integration module integrates and trains the input information and the training sample data to obtain the mobile terminal integrated segmentation model;

[0060] Step S500: Based on the mobile terminal integrated segmentation model, obtain the model output result, which includes a set of feature point segmentation result images;

[0061] Specifically, the obtained basic segmentation model is used as input to the mobile network integration module, and integrated training is performed using the training sample data to obtain a mobile integrated segmentation model. This model is then used to obtain a set of feature point segmentation result images. The training sample data is an expanded multi-angle fundus image set. Compared to the basic network training module, the mobile network integration module has fewer parameters and a more lightweight network. The set of feature point segmentation result images includes image data information corresponding to lesion features. This achieves the technical effect of obtaining a mobile integrated segmentation model and determining the set of feature point segmentation result images based on it, providing data support for subsequently obtaining a feature point recognition network model.

[0062] Step S600: Train the recognition network on the feature point segmentation result image set through the feature point recognition and correction module to obtain the feature point recognition network model;

[0063] Furthermore, step S600 of this application also includes:

[0064] Step S610: Obtain a cascaded classifier based on the feature point recognition and correction module;

[0065] Step S620: The cascaded classifier selects DenseNet as the image classification network;

[0066] Step S630: Classify and correct the feature point segmentation result image set according to the classification image network to obtain the feature point segmentation image type;

[0067] Step S640: Train the recognition network based on the feature point segmentation image type to obtain the feature point recognition network model.

[0068] Specifically, small hemorrhages and microaneurysms in diabetic retinopathy fundus images are similar in appearance, making them difficult to distinguish. Therefore, to ensure accuracy, a cascaded classifier is connected to the feature point recognition and correction module, specifically for classifying and correcting the feature point segmentation result image set. The cascaded classifier uses DenseNet as the classification image network, directly connecting all layers while ensuring maximum information transfer between network layers, using a shortcut connection to pass input from one block to another. The feature point segmentation image type refers to the data information after classifying and correcting the lesion type in the feature point segmentation result image set. The feature point recognition network model is a neural network model trained on the feature point segmentation image type recognition network. This achieves the technical effect of obtaining a feature point recognition network model with high accuracy and reliability, thereby improving the accuracy of subsequently obtained diabetic retinopathy fundus segmentation images.

[0069] Step S700: Obtain the fundus image of the target diabetic retinopathy to be processed;

[0070] Step S800: The target diabetic retinopathy fundus image to be processed is used as input image information and input into the mobile terminal integrated segmentation model and the feature point recognition network model to obtain a diabetic retinopathy fundus segmentation image.

[0071] Specifically, the target diabetic retinopathy fundus image is used as input information and fed into a mobile-integrated segmentation model and feature point recognition network model. After complex and efficient calculations by the mobile-integrated segmentation model and feature point recognition network model, a segmented diabetic retinopathy fundus image is obtained. The target diabetic retinopathy fundus image is arbitrary diabetic retinopathy fundus image data information that uses the aforementioned integrated system for intelligent image segmentation and lesion analysis. The segmented diabetic retinopathy fundus image includes data information such as the lesion type and location corresponding to the target diabetic retinopathy fundus image. This achieves the technical effect of accurately identifying the target diabetic retinopathy fundus image using the mobile-integrated segmentation model and feature point recognition network model, obtaining a relatively accurate segmented diabetic retinopathy fundus image.

[0072] In summary, the method for segmenting and integrating lesions in fundus diabetic retinopathy images provided in this application has the following technical effects:

[0073] A multi-angle fundus image set is obtained through an ultra-wide-angle laser image acquisition device; the data is augmented using a training data augmentation module to obtain an expanded multi-angle fundus image set; the expanded multi-angle fundus image set is used as training sample data, and semantic segmentation is trained using a deep learning network structure through the basic network training module to obtain a basic segmentation model; the basic segmentation model is used as input information, and the input information and the training sample data are integrated and trained through the mobile terminal network integration module to obtain a mobile terminal integrated segmentation model; based on this, the model output result is obtained, which includes a feature point segmentation result image set; the feature point recognition correction module is used to train a recognition network on the feature point segmentation result image set to obtain a feature point recognition network model; the fundus image of the target diabetic retinopathy to be processed is acquired; the fundus image of the target diabetic retinopathy to be processed is used as input image information and input into the mobile terminal integrated segmentation model and the feature point recognition network model to obtain a fundus diabetic retinopathy segmentation image. This study aims to improve the accuracy and precision of lesion segmentation in diabetic retinopathy (DR) fundus images, thereby enhancing the effectiveness and quality of lesion segmentation. Furthermore, it designs an optimized method for lesion segmentation in DR fundus images, enabling fine segmentation of multiple lesions in DR fundus images on mobile devices. This method also achieves the technical effect of efficiently analyzing the location and contour information of multiple lesions in DR fundus images on mobile devices.

[0074] Example 2

[0075] Based on the method for segmenting and integrating diabetic retinopathy images in the foregoing embodiments, and using the same inventive concept, this invention also provides a system for segmenting and integrating diabetic retinopathy images. Please refer to the appendix. Figure 5 The system includes:

[0076] Image acquisition module 11, the image acquisition module 11 is used to obtain a set of multi-angle fundus images through an ultra-wide-angle laser image acquisition device, wherein the ultra-wide-angle laser image acquisition device includes a first wavelength laser and a second wavelength laser;

[0077] Training data augmentation module 12 is used to perform data augmentation on the multi-angle fundus image set to obtain an expanded multi-angle fundus image set.

[0078] The basic network training module 13 is used to use the expanded multi-angle fundus image set as training sample data, and to use a deep learning network structure to perform semantic segmentation training on the training sample data to obtain a basic segmentation model. The deep learning network structure includes a network encoder and a network decoder.

[0079] Mobile network integration module 14 is used to take the basic segmentation model as input information, and perform integrated training on the input information and the training sample data to obtain a mobile integrated segmentation model.

[0080] Intermediate output module 15 is used to obtain model output results based on the mobile terminal integrated segmentation model, the model output results including a set of feature point segmentation result images;

[0081] Feature point recognition and correction module 16, the feature point recognition and correction module 16 is used to train the recognition network on the feature point segmentation result image set to obtain a feature point recognition network model;

[0082] Information acquisition module 17, the information acquisition module is used to acquire the fundus image of the target diabetic retinopathy to be processed;

[0083] Processing module 18 is used to input the target diabetic retinopathy fundus image to be processed as input image information into the mobile terminal integrated segmentation model and the feature point recognition network model to obtain a diabetic retinopathy fundus segmentation image.

[0084] Furthermore, the image acquisition module also includes:

[0085] An image acquisition unit is used to obtain a set of fundus images at a first angle based on the first wavelength laser and the second wavelength laser;

[0086] Image acquisition unit, the image acquisition unit is used to acquire a set of second-angle fundus images with different angle acquisition areas;

[0087] A noise reduction processing unit is used to perform noise reduction processing on the first angle fundus image set and the second angle fundus image set to obtain a first standard angle fundus image set and a second standard angle fundus image set.

[0088] An image stitching unit is used to stitch together the first standard angle fundus image set and the second standard angle fundus image set to obtain the multi-angle fundus image set.

[0089] Furthermore, the image stitching unit also includes:

[0090] A reference image selection unit is used to select a reference image based on the first standard angle fundus image set and the second standard angle fundus image set to obtain a fundus reference image.

[0091] The construction unit is used to construct a reference image coordinate system based on the fundus reference image;

[0092] A stitching processing unit is used to obtain a set of fundus images to be stitched based on the fundus reference image;

[0093] The mapping unit is used to perform relational mapping on the set of fundus images to be stitched in the reference image coordinate system to determine the image transformation matrix;

[0094] A conversion and stitching unit is used to convert and stitch the set of fundus images to be stitched to the fundus reference image based on the image conversion matrix, thereby obtaining the multi-angle fundus image set.

[0095] Furthermore, the training data augmentation module also includes:

[0096] An enhancement type processing unit is used to acquire image enhancement type information, which includes rotation, horizontal flip, vertical flip, and scaling.

[0097] A variation coefficient unit is used to determine the variation coefficient of the fundus image based on the image enhancement type information;

[0098] A data augmentation unit is used to perform data augmentation and output on the multi-angle fundus image set based on the fundus image change coefficient, so as to obtain the expanded multi-angle fundus image set.

[0099] Furthermore, the basic network training module also includes:

[0100] An image segmentation unit is used to divide the images in the expanded multi-angle fundus image set into a set of overlapping image blocks of a preset size;

[0101] A scale feature unit is used to input the set of overlapping image patches as input information into the network encoder to obtain multi-scale features of the image;

[0102] A semantic segmentation unit is used to input the multi-scale features of the image into the network decoder to obtain a predicted semantic segmentation result, wherein the network decoder consists of an MLP layer.

[0103] Furthermore, the semantic segmentation unit also includes:

[0104] Channel size unit, the channel size unit being used to unify the channel size of the multi-scale features of the image through the MLP layer;

[0105] The feature processing unit is used to perform feature sampling and feature concatenation on the multi-scale features of the image based on the channel size;

[0106] The integrated processing unit is used to fuse concatenated features using the MLP layer and add an MLP layer based on the concatenated features to obtain the predicted semantic segmentation result.

[0107] Furthermore, the feature point recognition and correction module also includes:

[0108] A cascaded classification unit is used to obtain a cascaded classifier based on the feature point recognition and correction module.

[0109] The selection unit is used by the cascaded classifier to select DenseNet as the image classification network.

[0110] A classification correction unit is used to classify and correct the feature point segmentation result image set according to the classification image network to obtain the feature point segmentation image type.

[0111] A network training unit is used to train a recognition network based on the feature point segmentation image type to obtain the feature point recognition network model.

[0112] This specification and accompanying drawings are merely illustrative examples of this application. If any modifications and variations of this invention fall within the scope of this invention and its equivalents, this invention also intends to include such modifications and variations.

Claims

1. A method for segmenting and integrating lesions in fundus diabetic retinopathy images, characterized in that, The method is applied to a system for lesion segmentation and integration of fundus retinal diabetic reticulum images. The system includes a training data augmentation module, a basic network training module, a mobile network integration module, and a feature point recognition and correction module. The method includes: A first-angle fundus image set is obtained based on a first-wavelength laser and a second-wavelength laser; a second-angle fundus image set with different angle acquisition areas is obtained; the first-angle fundus image set and the second-angle fundus image set are denoised to obtain a first standard-angle fundus image set and a second standard-angle fundus image set; the first standard-angle fundus image set and the second standard-angle fundus image set are image-stitched to obtain a multi-angle fundus image set. Based on the training data augmentation module, the multi-angle fundus image set is augmented to obtain an expanded multi-angle fundus image set; The expanded multi-angle fundus image set is used as training sample data. The basic network training module uses a deep learning network structure to perform semantic segmentation training on the training sample data to obtain a basic segmentation model. Using the expanded multi-angle fundus image set as training sample data, semantic segmentation training is performed on the training sample data using a deep learning network structure through the basic network training module. This includes: dividing the images in the expanded multi-angle fundus image set into a set of overlapping image blocks of a preset size; the deep learning network structure includes a network encoder and a network decoder; inputting the set of overlapping image blocks as input information into the network encoder to obtain multi-scale image features; inputting the multi-scale image features into the network decoder, which consists of an MLP layer; unifying the channel size of the multi-scale image features through the MLP layer; performing feature sampling and feature concatenation on the multi-scale image features based on the channel size; fusing concatenated features using the MLP layer; and adding another MLP layer based on the concatenated features to obtain a predicted semantic segmentation result. The basic segmentation model is used as input information, and the mobile terminal network integration module integrates and trains the input information and the training sample data to obtain the mobile terminal integrated segmentation model. Based on the mobile-integrated segmentation model, the model output results are obtained, which include a set of feature point segmentation result images. The feature point recognition and correction module is used to train the recognition network on the feature point segmentation result image set to obtain a feature point recognition network model. Acquire fundus images of the target diabetic retinopathy; The target diabetic retinopathy fundus image is used as input image information and input into the mobile terminal integrated segmentation model and the feature point recognition network model to obtain a segmented image of diabetic retinopathy fundus.

2. The method as described in claim 1, characterized in that, Obtaining the multi-angle fundus image set includes: A reference image is obtained by selecting a reference image based on the first set of standard angle fundus images and the second set of standard angle fundus images; Based on the aforementioned fundus reference image, a reference image coordinate system is constructed; Based on the aforementioned fundus reference image, a set of fundus images to be stitched together is obtained; The set of fundus images to be stitched is mapped in relation to the reference image coordinate system to determine the image transformation matrix; Based on the image transformation matrix, the set of fundus images to be stitched is transformed and stitched to the fundus reference image to obtain the multi-angle fundus image set.

3. The method as described in claim 1, characterized in that, The obtained augmented multi-angle fundus image set includes: Obtain image enhancement type information, which includes rotation, horizontal flip, vertical flip, and scaling; Based on the image enhancement type information, determine the fundus image change coefficients; Based on the fundus image change coefficients, the data amplification and change output of the multi-angle fundus image set is performed to obtain the expanded multi-angle fundus image set.

4. The method as described in claim 1, characterized in that, The obtained feature point recognition network model includes: Based on the feature point recognition and correction module, a cascaded classifier is obtained; The cascaded classifier selects DenseNet as the image classification network. The feature point segmentation result image set is classified and corrected according to the classification image network to obtain the feature point segmentation image type; The feature point recognition network model is obtained by training a recognition network based on the image type segmented by the feature points.

5. A system for segmenting and integrating lesions in fundus diabetic retinopathy images, characterized in that, The system also includes: The image acquisition module is used to obtain a set of fundus images at a first angle based on a first wavelength laser and a second wavelength laser; acquire a set of fundus images at a second angle with different angle acquisition areas; perform noise reduction processing on the first angle fundus image set and the second angle fundus image set to obtain a set of fundus images at a first standard angle and a set of fundus images at a second standard angle; and perform image stitching on the first standard angle fundus image set and the second standard angle fundus image set to obtain a set of fundus images at multiple angles. A training data augmentation module is used to perform data augmentation on the multi-angle fundus image set to obtain an expanded multi-angle fundus image set. A basic network training module is used to use the expanded multi-angle fundus image set as training sample data, and to use a deep learning network structure to perform semantic segmentation training on the training sample data to obtain a basic segmentation model. Using the expanded multi-angle fundus image set as training sample data, semantic segmentation training is performed on the training sample data using a deep learning network structure through the basic network training module. This includes: dividing the images in the expanded multi-angle fundus image set into a set of overlapping image blocks of a preset size; the deep learning network structure includes a network encoder and a network decoder; inputting the set of overlapping image blocks as input information into the network encoder to obtain multi-scale image features; inputting the multi-scale image features into the network decoder, which consists of an MLP layer; unifying the channel size of the multi-scale image features through the MLP layer; performing feature sampling and feature concatenation on the multi-scale image features based on the channel size; fusing concatenated features using the MLP layer; and adding another MLP layer based on the concatenated features to obtain a predicted semantic segmentation result. A mobile network integration module is used to take the basic segmentation model as input information, and to integrate and train the input information and the training sample data to obtain a mobile integrated segmentation model. An intermediate output module is used to obtain model output results based on the mobile terminal integrated segmentation model. The model output results include a set of feature point segmentation result images. A feature point recognition and correction module is used to train a recognition network on the feature point segmentation result image set to obtain a feature point recognition network model. Information acquisition module, the information acquisition module is used to acquire fundus images of the target diabetic retinopathy to be processed; The processing module is used to input the target diabetic retinopathy fundus image as input image information into the mobile terminal integrated segmentation model and the feature point recognition network model to obtain a diabetic retinopathy fundus segmentation image.

Images