Intelligent fundus image analysis method and system

Abstract

The invention discloses an intelligent fundus image analysis method and system, and relates to intelligent medical treatment, image processing and artificial intelligence technologies, and the method comprises the steps: obtaining a fundus image and eye use behavior data of a target user; extracting eye using features in the eye using behavior data, and processing the fundus image by using rolling guide filtering to obtain an eye bottom noise image and an eye detail image; extracting bottom noise image features of the eye bottom noise image by adopting Resnet, and extracting detail image features of the eye detail image by adopting CNN; fusing the extracted background noise image features and the extracted detail image features; and according to the fusion features and the eye using features, performing classification by using a pre-trained classifier. According to the method, the eye fundus image is processed through rolling guide filtering, classification and recognition of the eye fundus artery and the eye fundus vein are achieved through the fused bottom noise image features and the detail image features, and the recognition effect of the eye fundus artery and the eye fundus vein is improved.

Description

Technical Field

[0001] The present application relates to the fields of smart medical treatment, image processing, and artificial intelligence technology, and in particular to an intelligent fundus image analysis method and system. Background Art

[0002] As the only internal vascular tissue in the human body that can be observed without trauma, retinal blood vessels have always attracted extensive attention from researchers. By analyzing retinal blood vessels, we can gain a deeper understanding of the structure of the retina.

[0003] The blood vessels of the retina are divided into arteries and veins. Identifying arteries and veins is not only the basis for distinguishing retinal arteriosclerosis, but also the basis for distinguishing chronic diseases and retinopathy of cardiovascular and cerebrovascular diseases (such as hypertensive retinopathy), and can reflect the structural damage of chronic diseases and cardiovascular and cerebrovascular diseases to the body. Therefore, identifying retinal arteries and veins and obtaining corresponding arteriovenous indicators is very important for the analysis of retinal diseases and systemic chronic diseases and cardiovascular and cerebrovascular diseases. However, the existing technology has low accuracy in identifying retinal arteries and veins. Summary of the invention

[0004] The embodiments of the present application provide an intelligent fundus image analysis method and system for processing fundus images through rolling guided filtering, and classifying and identifying fundus arteries and veins through fused background noise image features and detail image features, thereby improving the identification effect of fundus arteries and veins.

[0005] The present application embodiment provides an intelligent fundus image analysis method, including: Obtain fundus images and eye behavior data of target users; Extracting eye usage features from the eye usage behavior data, and processing the fundus image using a rolling guided filter to obtain an eye background noise image and an eye detail image; Resnet is used to extract the background noise image features of the eye background noise image, and CNN is used to extract the detail image features of the eye detail image; The extracted background noise image features and detail image features are fused; Classification is performed using a pre-trained classifier based on the fused features and the eye features.

[0006] Optionally, obtaining the target user's eye behavior data includes: Obtain target users’ medical records and eye description information; Extracting eye-use related data from the medical record data, and performing sentence expansion on the eye-use related data according to basic information of the corresponding user in the medical record data to obtain eye-use text; The eye usage behavior data is formed based on the combination of the eye usage text and the eye usage description information.

[0007] Optionally, extracting eye usage features from the eye usage behavior data includes: Perform word segmentation on the combined eye behavior data; Calculate the similarity between each segmented word obtained from the segmentation and the preset keyword; According to the calculated similarity higher than the preset threshold of the word frequency, and the corresponding keyword weight itself, the corresponding word is assigned a weight, wherein the higher the frequency of occurrence, the greater the corresponding keyword weight itself, and the greater the weight assigned to the corresponding word; Eye usage features are extracted based on each word in the combined eye usage behavior data whose similarity is higher than a preset threshold.

[0008] Optionally, using rolling guided filtering to process the fundus image to obtain an eye background noise image and an eye detail image includes: The fundus image is taken as input, and Gaussian filtering is used to remove small structures to filter the edge structure of the fundus image, where the Gaussian filter satisfies: ; in, For the output image, , Respectively represent the corresponding image pixel coordinates, is the standard deviation, Indicated in pixels The filter window centered at To input fundus image; Iteratively restore edge information in fundus images by using bilateral filter kernels: ; in, Indicates The result of the iterations, , For normalization, , They control the spatial range weight and the intensity difference range weight respectively.

[0009] Optionally, using rolling guided filtering to process the fundus image to obtain an eye background noise image and an eye detail image further includes: Using the least squares optimization model in orthogonal directions, the rolling guidance filter is optimized to meet the following requirements: ; in, Indicates RGF processing. , denote the input image and the output image respectively, Represents the gradient of the output fundus image in the x-axis and y-axis directions, Represents the gradient of the input fundus image in the x-axis and y-axis directions, represents the gradient direction, Take x and y, Represents the weight coefficient.

[0010] Optionally, the method further includes performing multi-scale decomposition on the input fundus image using an optimized rolling guided filter to obtain an eye background noise image and an eye detail image.

[0011] Optionally, fusing the extracted background noise image features and detail image features includes: By introducing the multi-scale channel attention module MSCAM, the extracted background noise image features and detail image features are convolved point by point to fuse the background noise image features and detail image features.

[0012] Optionally, performing classification using a pre-trained classifier according to the fusion feature and the eye feature includes: Segmentation is performed based on the fused image features, and the vascular information in each segmentation feature is identified; adding a label to the corresponding segmentation feature according to the eye feature and the identified blood vessel information; The marked segmentation features are classified using a pre-trained SVM classifier to obtain classification results at corresponding positions of the fundus image.

[0013] An embodiment of the present application also proposes an intelligent fundus image analysis system, including a processor and a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the processor, the steps of the intelligent fundus image analysis method as described above are implemented.

[0014] The method of the present application processes the fundus image based on the optimized rolling guided filter to obtain the eye background noise image and the eye detail image covering the edge details, and further realizes the classification and recognition of the fundus arteries and veins through the fusion of the background noise image features and the detail image features, thereby improving the recognition effect of the fundus arteries and veins.

[0015] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 The following is a schematic diagram of the basic process of the intelligent fundus image analysis method of this embodiment. DETAILED DESCRIPTION

[0017] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0018] The present application embodiment proposes an intelligent fundus image analysis method, such as Figure 1 As shown, the following steps are included: In step S101, fundus images and eye behavior data of a target user are acquired. In some embodiments, the eye behavior data may include the target user, the patient's medical records, communication information between the doctor and the user, and the like.

[0019] In step S102, the eye characteristics in the eye behavior data are extracted, and the fundus image is processed using a rolling guided filter to obtain an eye noise background image and an eye detail image. In some embodiments, the rolling guided filter can be improved based on least squares to refine the edge and detail information, thereby specifically highlighting the blood vessel direction, details and other information in the fundus image.

[0020] In step S103, Resnet is used to extract the background noise image features of the eye background noise image, and CNN is used to extract the detail image features of the eye detail image.

[0021] In step S104, the extracted background noise image features and detail image features are fused.

[0022] In step S105, classification is performed using a pre-trained classifier according to the fusion feature and the eye feature. In some embodiments, the classifier may use a binary classifier or a multi-classifier to distinguish the fundus arteries and veins in the fusion feature.

[0023] The method of the present application processes the fundus image based on the optimized rolling guided filter to obtain the eye background noise image and the eye detail image covering the edge details, and further realizes the classification and recognition of the fundus arteries and veins through the fusion of the background noise image features and the detail image features, thereby improving the recognition effect of the fundus arteries and veins.

[0024] In some embodiments, obtaining the target user's eye behavior data includes: Obtain the target user's medical record data and eye description information. In some examples, the medical record data may also include the user's physical examination data, etc. The eye description data may be the communication data between the user and the doctor, such as recordings, etc. The recordings can be converted into eye description text in specific applications.

[0025] The eye-use related data is extracted from the medical record data, and the eye-use related data is expanded according to the basic information of the corresponding user in the medical record data to obtain the eye-use text. In some embodiments, the eye-use text is obtained by sentence expansion according to the basic information of the corresponding user in the medical record data, such as name, gender, age, etc., and further according to, for example, the left and right eye conditions.

[0026] The eye behavior data is formed based on the eye text and the eye description information. The eye text and the converted description text are directly spliced ​​and combined to obtain the eye behavior data.

[0027] In some embodiments, extracting eye characteristics from the eye behavior data includes: Perform word segmentation on the combined eye behavior data.

[0028] The similarity between each word segmentation obtained by word segmentation and the preset keyword is calculated. In some embodiments, multiple bad eye behaviors can be pre-specified, or the bad behaviors can be graded, and the preset keywords under each grade can be determined. The keywords can be numerical values ​​(for example, 500 degrees of myopia) or word + numerical value (for example, looking at the mobile phone for 5 hours), or they can be separate word segmentation forms.

[0029] According to the frequency of occurrence of the segmented words whose calculated similarity is higher than the preset threshold, and the corresponding keyword's own weight, the corresponding segmented words are given weights, wherein the higher the frequency of occurrence, the greater the corresponding keyword's own weight, and the greater the weight given to the corresponding segmented words. In a specific example, the corresponding segmented word weights can be given according to the aforementioned segmented word classification and the frequency of occurrence of each segmented word. Specifically, a small proportion of the weight floating range can be set for any segmented word according to the frequency, thereby improving the pertinence and emphasis of the extracted features according to the frequency of occurrence of the segmented words and the weight of the classification.

[0030] Eye usage features are extracted based on each word in the combined eye usage behavior data whose similarity is higher than a preset threshold.

[0031] In some embodiments, processing the fundus image using rolling guided filtering to obtain an eye background noise image and an eye detail image includes: The fundus image is taken as input, and Gaussian filtering is used to remove small structures to filter the edge structure of the fundus image, where the Gaussian filter satisfies: ; in, For the output image, , Respectively represent the corresponding image pixel coordinates, is the standard deviation, Indicated in pixels The filter window centered at To input fundus image; Iteratively restore edge information in fundus images by using bilateral filter kernels: ; in, Indicates The result of the iterations, , For normalization, , are the control space range weight and intensity difference range weight respectively. Indicates The result of the iterations, in the first iteration, is the output image P after Gaussian filtering in the first step, is the original fundus image. In the iterative operation process, given The previous iteration , you can get the The result value of the iteration . In view of the fact that rolling-guided filtering is prone to produce gradient inversion and halo, which affect the identification of fundus arteriovenous boundaries, and such phenomena also affect further feature extraction, the embodiment of the present application further proposes to use a least squares optimization model for optimization. In some embodiments, using rolling-guided filtering to process the fundus image to obtain an eye background noise image and an eye detail image also includes: Using the least squares optimization model in orthogonal directions, the rolling guidance filter is optimized to meet the following requirements: ; in, Indicates RGF processing. , denote the input image and the output image respectively, Represents the gradient of the output fundus image in the x-axis and y-axis directions, Represents the gradient of the input fundus image in the x-axis and y-axis directions, represents the gradient direction, Take x and y, Refers to the gradient of the image in the x-axis and y-axis directions. Represents the weight coefficient. In the specific example, the least squares optimization model includes a data term and a regularization term, wherein the data term is used to ensure the similarity between the input fundus image and the output image and retain the arteriovenous edge details, and the regularization term is used to control the global properties of the output image. By optimizing the proportion of the data term and the regularization term, the arteriovenous edge details in the fundus image processed by the rolling-guided filter can be retained to the greatest extent, so as to retain more arteriovenous features after feature fusion.

[0032] In some embodiments, the input fundus image is further decomposed at multiple scales using an optimized rolling guided filter to obtain an eye noise floor image and an eye detail image. That is, the eye noise floor image and the eye detail image are obtained by processing the optimized rolling filter using the least squares optimization model in the orthogonal direction, so that the obtained eye detail image contains more arteriovenous detail features, thereby improving the recognition and classification effect of arteries and veins.

[0033] In some embodiments, fusing the extracted background noise image features and detail image features includes: using a multi-scale channel attention module MSCAM to perform point-by-point convolution on the extracted background noise image features and detail image features to fuse the background noise image features and detail image features. Using MSCAM is not in the backbone feature extraction network, and only point convolution is used to ensure the lightweight of the overall feature extraction fusion network structure.

[0034] In some embodiments, performing classification using a pre-trained classifier according to the fusion feature and the eye feature includes: Segmentation is performed based on the fused image features, and vascular information in each segmentation feature is identified, for example, the vascular information is determined by the number of vascular lines in each segmentation after detail preservation, and the position information of the eye where each segmentation is located is determined.

[0035] The corresponding segmentation features are marked according to the eye features and the identified blood vessel information. In the specific example, any segmentation is further marked according to the blood vessel information, position information and eye features.

[0036] The marked segmentation features are classified using a pre-trained SVM classifier to obtain the classification results of the corresponding positions of the fundus image. In the specific example, the SVM classifier is used to classify the segments belonging to arteries, veins, or both. The segments that contain both can be further subdivided and classified again to complete the classification and recognition.

[0037] The method of the present application processes the fundus image based on the optimized rolling guided filter to obtain the eye background noise image and the eye detail image covering the edge details, and further realizes the classification and recognition of the fundus arteries and veins through the fusion of the background noise image features and the detail image features, thereby improving the recognition effect of the fundus arteries and veins.

[0038] An embodiment of the present application also proposes an intelligent fundus image analysis system, including a processor and a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by the processor, the steps of the intelligent fundus image analysis method as described above are implemented.

[0039] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present disclosure with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. It is not limited to the examples described in this specification or during the practice of this application, and its examples will be interpreted as non-exclusive.

[0040] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description.

[0041] The above embodiments are merely exemplary embodiments of the present disclosure. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present disclosure, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. An intelligent fundus image analysis method, characterized in that: include: Obtain fundus images and eye behavior data of target users; Extracting eye usage features from the eye usage behavior data, and processing the fundus image using a rolling guided filter to obtain an eye background noise image and an eye detail image; Resnet is used to extract the background noise image features of the eye background noise image, and CNN is used to extract the detail image features of the eye detail image; The extracted background noise image features and detail image features are fused; Classification is performed using a pre-trained classifier based on the fused features and the eye features.

2. The intelligent fundus image analysis method according to claim 1, characterized in that: Obtaining target users’ eye behavior data includes: Obtain target users’ medical records and eye description information; Extracting eye-use related data from the medical record data, and performing sentence expansion on the eye-use related data according to basic information of the corresponding user in the medical record data to obtain eye-use text; The eye usage behavior data is formed based on the combination of the eye usage text and the eye usage description information.

3. The intelligent fundus image analysis method according to claim 2, characterized in that: Extracting eye usage features from the eye usage behavior data includes: Perform word segmentation on the combined eye behavior data; Calculate the similarity between each segmented word obtained from the segmentation and the preset keyword; According to the calculated frequency of occurrence of the segmented words with a similarity higher than the preset threshold, and the corresponding keyword's own weight, the corresponding segmented words are given a weight, wherein the higher the frequency of occurrence, the greater the corresponding keyword's own weight, and the greater the weight assigned to the corresponding segmented word; Eye usage features are extracted based on each word in the combined eye usage behavior data whose similarity is higher than a preset threshold.

4. The intelligent fundus image analysis method according to claim 3, characterized in that: Using rolling guided filtering to process the fundus image to obtain an eye background noise image and an eye detail image includes: The fundus image is taken as input, and Gaussian filtering is used to remove small structures to filter the edge structure of the fundus image, where the Gaussian filter satisfies: ； in, For the output image, , Respectively represent the corresponding image pixel coordinates, is the standard deviation, Indicated in pixels The filter window centered at To input fundus image; Iteratively restore edge information in fundus images by using bilateral filter kernels: ； in, Indicates The result of the iterations, , For normalization, , They control the spatial range weight and the intensity difference range weight respectively.

5. The intelligent fundus image analysis method according to claim 4, characterized in that: Using rolling guided filtering to process the fundus image to obtain an eye background noise image and an eye detail image also includes: Using the least squares optimization model in orthogonal directions, the rolling guidance filter is optimized to meet the following requirements: ； in, Indicates RGF processing. , denote the input image and the output image respectively, Represents the gradient of the output fundus image in the x-axis and y-axis directions, Represents the gradient of the input fundus image in the x-axis and y-axis directions, represents the gradient direction, Take x and y, Represents the weight coefficient.

6. The intelligent fundus image analysis method according to claim 5, characterized in that: The method also includes using an optimized rolling guided filter to perform multi-scale decomposition on the input fundus image to obtain an eye background noise image and an eye detail image.

7. The intelligent fundus image analysis method according to claim 5, characterized in that: The extracted background noise image features and detail image features are fused together including: By introducing the multi-scale channel attention module MSCAM, the extracted background noise image features and detail image features are convolved point by point to fuse the background noise image features and detail image features.

8. The intelligent fundus image analysis method according to claim 7, characterized in that: According to the fusion feature and the eye feature, performing classification using a pre-trained classifier includes: Segmentation is performed based on the fused image features, and the vascular information in each segmentation feature is identified; adding a label to the corresponding segmentation feature according to the eye feature and the identified blood vessel information; The marked segmentation features are classified using a pre-trained SVM classifier to obtain classification results at corresponding positions of the fundus image.

9. An intelligent fundus image analysis system, characterized in that: The method comprises a processor and a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, the steps of the intelligent fundus image analysis method according to any one of claims 1 to 8 are implemented.

Images