Image recognition-based grading method for KF rings in the cornea of Wilson's disease
The two-dimensional and three-dimensional features of the corneal K-F ring are extracted through image recognition technology, combined with machine learning models, and the inconsistency and subjectivity of traditional detection methods are solved, and more accurate K-F ring detection and grading are achieved, supporting personalized treatment plans. Formulate.
Patent Information
- Application Number
- CN202411240551.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-09-05
AI Technical Summary
Traditional corneal K-F ring detection methods cannot accurately reveal the internal conditions of copper deposition and are easily affected by light sources and subjective judgments, resulting in inconsistent detection results, making it difficult to assist in the evaluation of drug treatment effects and the formulation of personalized treatment plans.
Using an image recognition-based method, the two-dimensional and three-dimensional features of the ROI region of interest are extracted by acquiring corneal images, combined with naive Bayes classification and gradient enhancement decision tree model, K-F loop detection and grading are carried out to reduce subjective judgment and operation error.
The accuracy of corneal K-F ring detection and grading is improved, helping clinicians observe changes before and after treatment, understand the effects of different drugs, assist in formulating personalized treatment plans, reduce interference from external conditions, and improve clinical operation efficiency.
Smart Images

Figure CN118762006B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and more specifically to a method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition. Background Art
[0002] Hepatolenticular Degeneration (HLD), also known as Wilson's disease, is a copper metabolism disorder caused by mutations in the ATP7B gene. The disease causes abnormal accumulation of copper in the body, mainly affecting the liver and brain, causing liver damage, neurological symptoms and other systemic problems. A notable ocular feature of hepatolenticular degeneration is the corneal KF ring (Kayser-Fleischer ring), which is a ring-shaped pigment deposit formed by the deposition of copper in the cornea. Therefore, the detection and grading of the KF ring is of great significance for the diagnosis, disease monitoring and treatment effect evaluation of Wilson's disease.
[0003] The traditional method of detecting corneal KF ring mainly relies on slit lamp microscopy. Through this non-invasive ophthalmic examination method, pigment deposition on the surface of the cornea can be observed; however, slit lamp microscopy can only provide surface information of the cornea, and cannot reveal the specific situation of copper deposition in the internal layers of the cornea, which is not conducive to the early detection and intervention of KF ring lesions in patients with Wilson's disease; in addition, the results of slit lamp microscopy examination are easily affected by light source conditions, operator experience and subjective judgment, which will lead to inconsistency in KF ring detection and grading results; thus, it is impossible to help clinicians observe the changes in KF ring before and after drug treatment, and it is difficult to help clinicians understand the therapeutic effects of different drugs. Further, it is difficult to assist clinicians in formulating personalized treatment plans for different patients; therefore, how to provide a more objective and accurate KF ring detection and grading method has become a clinical problem that needs to be solved urgently. Summary of the invention
[0004] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a method for grading the KF ring of the hepatolenticular degeneration cornea based on image recognition.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] A method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition, the method comprising:
[0007] Acquire a first corneal image and a second corneal image of the eye of the subject to be captured, and after segmenting the ROI region of interest in the first corneal image and the second corneal image, extract the pigment deposition area in the ROI region of interest; the ROI region of interest includes a first ROI region of interest in the first corneal image and a second ROI region of interest in the second corneal image, and the pigment deposition area includes the pigment deposition area at the frontal angle and the lateral cross-sectional angle;
[0008] Extracting a two-dimensional feature X and a two-dimensional feature Y of a pigment deposition area in a first corneal image; inputting the two-dimensional feature X into a pre-built naive Bayes classification model for determining whether a corneal KF ring exists, so as to obtain a first recognition result; the two-dimensional feature X is a color feature, and the two-dimensional feature Y includes a morphological feature and a position feature; the first recognition result includes one of the presence or absence of a corneal KF ring;
[0009] When the first recognition result shows that there is a corneal KF ring, obtaining a three-dimensional feature of the ROI pigment deposition area in the second corneal image, wherein the three-dimensional feature is a thickness feature;
[0010] The two-dimensional feature Y and the three-dimensional feature are fused at the feature level to obtain a multimodal fusion feature, which is then input into a pre-built gradient boosting decision tree model for identifying the corneal KF ring level to obtain the corneal KF ring grading result.
[0011] Furthermore, segmenting the ROI region of interest in the first corneal image includes:
[0012] detecting a plurality of circular boundaries in the first corneal image using Hough transform;
[0013] Obtain the centroid coordinates of each circular boundary, obtain multiple centroid coordinates, extract the same centroid coordinates in the multiple centroid coordinates, and count the number of the same centroid coordinates;
[0014] Sort the number of identical centroid coordinates by numerical value from large to small, and mark the identical centroid coordinates corresponding to the first numerical value as shared centroid coordinates;
[0015] Extract the circular boundary of each common centroid coordinate, mark the circular boundary of each common centroid coordinate as a circular outline, and obtain the pixel perimeter of each circular outline;
[0016] Sort the pixel perimeter of each circular contour from small to large in numerical value, and mark the pupil contour with the circular contour corresponding to the first pixel perimeter sorted;
[0017] Extracting the centroid coordinates of the pupil contour, and calculating the radius distance from the centroid coordinates of the pupil contour to each circular contour to obtain multiple radius distances;
[0018] Sort the multiple radius distances in descending order, and mark the circular contour corresponding to the first radius distance as the corneal edge contour;
[0019] A pixel area formed by the pupil outline to the corneal edge outline is segmented from the first corneal image and used as the first ROI region of interest.
[0020] Furthermore, segmenting the ROI region of interest in the second corneal image includes:
[0021] Using an edge detection algorithm to extract boundary lines between different layers in the second corneal image, a plurality of boundary lines are obtained;
[0022] Mark the hierarchical region surrounded by every two dividing lines as a candidate region, and obtain multiple candidate regions;
[0023] Obtain the color histogram of the first ROI region of interest and the color histogram of each candidate region;
[0024] Marking a color histogram of the first ROI region of interest as a first color histogram, and marking a color histogram of the candidate region as a second color histogram;
[0025] Calculating the similarity between the first color histogram and the second color histogram to obtain multiple similarities;
[0026] The multiple similarities are sorted from large to small according to the numerical values, and the candidate region corresponding to the first similarity sort is segmented from the second corneal image and used as the second ROI region of interest.
[0027] Further, extracting the pigment deposition area in the ROI region of interest includes:
[0028] Extracting a first ROI region of interest, graying the first ROI region of interest, and obtaining a grayed image of the first ROI region of interest;
[0029] The K-means clustering algorithm is used to distinguish pixels of the grayscale first ROI region of interest image, and the pixel points are clustered to form an area as a to-be-matched area A, thereby obtaining a plurality of to-be-matched areas A;
[0030] Obtain the center value of each area A to be matched, and compare the center value with the preset grayscale value interval. If the center value belongs to the grayscale value interval, the corresponding area A to be matched is used as the pigment deposition area at the normal viewing angle; if the center value does not belong to the grayscale value interval, the corresponding area A to be matched is used as the non-pigment deposition area.
[0031] Furthermore, the step of extracting the pigment deposition area in the ROI region of interest further includes:
[0032] Extracting a second ROI region of interest, graying the second ROI region of interest, and obtaining a grayed second ROI region of interest image;
[0033] The K-means clustering algorithm is used to distinguish pixels of the grayscale second ROI region of interest image, and the pixel points are clustered to form an area as a to-be-matched area B, thereby obtaining a plurality of to-be-matched areas B;
[0034] Obtain the center value of each area B to be matched, and compare the center value with the preset grayscale value interval. If the center value belongs to the grayscale value interval, the corresponding area B to be matched is used as the pigment deposition area of the lateral cross-sectional angle; if the center value does not belong to the grayscale value interval, the corresponding area B to be matched is used as the non-pigment deposition area.
[0035] Furthermore, the color features include the average RGB value, saturation, brightness and color gradient of the front-angle pigmentation area; the morphological features include the number, shape and pixel area of the front-angle pigmentation area and the pixel area of the first ROI region of interest; the position feature is the relative position of the front-angle pigmentation area; wherein the relative position is the distance between the center of mass of the front-angle pigmentation area and the pupil contour.
[0036] Furthermore, the training logic of the naive Bayes classification model is as follows:
[0037] Acquire historical corneal KF ring recognition training data, and divide the historical corneal KF ring recognition training data into a corneal KF ring recognition training set and a corneal KF ring recognition test set; the historical corneal KF ring recognition training data has multiple color features and their corresponding classification labels;
[0038] Wherein, the classification labels include labels indicating the presence of corneal KF rings and labels indicating the absence of corneal KF rings;
[0039] A naive Bayes classification network is constructed, the color features in the corneal KF ring recognition training set are used as the input of the naive Bayes classification network, and the classification labels in the corneal KF ring recognition training set are used as the output of the naive Bayes classification network, and the naive Bayes classification network is trained to obtain an initial naive Bayes classification network;
[0040] The initial naive Bayes classification network is model verified using the corneal KF ring recognition test set, and an initial naive Bayes classification network with a preset first test accuracy is output as a naive Bayes classification model for determining whether a corneal KF ring exists.
[0041] Further, the thickness feature includes an average thickness of the pigment deposition area at each lateral cross-sectional angle, and an average thickness of the second ROI region of interest;
[0042] The training logic of the gradient boosting decision tree model is as follows:
[0043] Acquire historical corneal KF ring classification training data, and divide the historical corneal KF ring classification training data into a corneal KF ring classification training set and a corneal KF ring classification test set, wherein the historical corneal KF ring classification training data includes multi-modal fusion features and corresponding annotation labels;
[0044] The annotation labels include five corneal KF ring levels, namely level 0, level 1, level 2, level 3 and level 4;
[0045] Construct a GBDT network, use the multimodal fusion features in the corneal KF ring classification training set as the input of the GBDT network, and use the annotation labels in the corneal KF ring classification training set as the output of the GBDT network, train the GBDT network, and obtain an initial GBDT network;
[0046] The initial GBDT network was verified using the corneal KF ring grading test set, and an initial GBDT network with a value greater than or equal to the preset second test accuracy threshold was output as a gradient boosting decision tree model for identifying the corneal KF ring level.
[0047] An electronic device comprises a memory, a processor and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, any one of the above-mentioned methods for grading the corneal KF ring of Wilson's disease based on image recognition is implemented.
[0048] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed, the image recognition-based corneal KF ring grading method for Wilson's disease described in any one of the above is implemented.
[0049] Compared with the prior art, the present invention has the following beneficial effects:
[0050] The present application discloses a method for grading a corneal KF ring of hepatolenticular degeneration based on image recognition, comprising: extracting a pigment deposition area in the ROI region of interest after segmenting a first corneal image and a second corneal image; inputting a two-dimensional feature X into a pre-constructed naive Bayes classification model for determining whether a corneal KF ring exists, so as to obtain a first recognition result; when the first recognition result shows that a corneal KF ring exists, obtaining a three-dimensional feature of the ROI pigment deposition area in the second corneal image; performing feature-level fusion on the two-dimensional feature Y and the three-dimensional feature to obtain a multi-modal fusion feature, and inputting the multi-modal fusion feature into a pre-constructed gradient boosting decision tree model for identifying the corneal KF ring level. The corneal KF ring grading result is obtained; based on the above technical features, the present invention reduces subjective judgment and operation errors, which is conducive to ensuring the accuracy of corneal KF ring detection and grading. At the same time, it is conducive to helping clinicians observe the changes in KF rings before and after drug treatment, and is conducive to helping clinicians understand the therapeutic effects of different drugs. Further, it can assist clinicians in formulating personalized treatment plans for different patients; in addition, through the combination of multimodal images and the introduction of the consideration of the influence of light source on corneal KF ring detection, the present invention is conducive to avoiding large errors in the segmentation and grading process of corneal KF rings, reducing the interference of external conditions on the detection results, and improving clinical operation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 A flowchart of a method for grading the KF ring of the hepatolenticular degeneration cornea based on image recognition provided by the present invention;
[0052] Figure 2 A schematic diagram of the structure of an electronic device provided by the present invention;
[0053] Figure 3 A schematic diagram of the structure of a computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION
[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0055] Example 1: Please refer to Figure 1 As shown, this embodiment discloses a method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition, and the method includes:
[0056] S101: Acquire a first corneal image and a second corneal image of the eye of the subject to be captured, and after segmenting the ROI region of interest in the first corneal image and the second corneal image, extract the pigment deposition area in the ROI region of interest; the ROI region of interest includes a first ROI region of interest in the first corneal image and a second ROI region of interest in the second corneal image, and the pigment deposition area includes the pigment deposition area at the frontal angle and the lateral cross-sectional angle;
[0057] Wherein, the first corneal image is a slit lamp microscope image, and the second corneal image is an OCT image; wherein, the slit lamp microscope image is obtained by photographing the eye of the collection object using a slit lamp microscope or a slit lamp camera; and the OCT image is obtained by performing optical coherence tomography on the eye of the collection object;
[0058] It should be understood that in slit lamp microscope images, usually only the surface features of the cornea can be seen, which are two-dimensional, such as the edge shape of the cornea and surface color changes; while in OCT images, due to its tomographic characteristics, the internal structure of the cornea can be seen, and the information of these internal layers constitutes three-dimensional features; for example, the thickness changes of each structural layer;
[0059] It should be noted that the pigment deposition area refers to the pigment deposition part caused by Wilson's disease or other diseases. In addition to the corneal KF ring caused by Wilson's disease, the pigment deposition part may also be pigment deposition caused by other diseases, such as corneal senile ring caused by abnormal lipid metabolism;
[0060] It should also be understood that Wilson's disease is an inherited copper metabolism disorder that causes abnormal accumulation of copper in the body, especially in the liver and brain. This copper accumulation causes liver damage, neurological symptoms, and other systemic problems. The main pathological mechanism of Wilson's disease is due to ATP7B gene mutation, which causes the liver to be unable to excrete copper into the bile normally, causing copper to accumulate in the body and trigger a toxic reaction. The most significant manifestation of the disease is that the copper caused by it is deposited in the cornea from the blood to form a corneal KF ring. Therefore, by identifying and grading the corneal KF ring, it is helpful for auxiliary physicians to evaluate and understand the therapeutic effects of various copper removal therapies on hepatolenticular degeneration (HLD) by observing the changes in the corneal KF ring grading before and after long-term treatment with different drugs, which is conducive to assisting physicians in performing targeted and effective treatment for patients with hepatolenticular degeneration.
[0061] It should also be noted that: before segmenting the ROI region of interest in the first corneal image and the second corneal image, the first corneal image and the second corneal image will be preprocessed, and the preprocessing includes but is not limited to resolution unification, image enhancement, image denoising and image registration, etc.; the resolution unification is achieved based on image scaling technology, such as bilinear interpolation or cubic interpolation, etc.; the image enhancement and image denoising are achieved by histogram equalization, Gaussian filtering or median filtering, etc.; the image registration is used to align the slit lamp microscope image with the OCT image, and can be performed using a feature point-based method (such as SIFT, SURF) or a mutual information-based method;
[0062] It can be understood that, since the first corneal image and the second corneal image belong to different modalities, the first corneal image and the second corneal image can be unified in the same scale space through resolution unification and image registration, so as to facilitate subsequent feature fusion;
[0063] In implementation, segmenting the ROI region of interest in the first corneal image includes:
[0064] detecting a plurality of circular boundaries in the first corneal image using Hough transform;
[0065] It should be understood that: Hough circle transform is a type of Hough transform technology, which can identify circular structures in images and determine the center position and radius of the circular structure;
[0066] It should be noted that the dark brown ring at the junction of the iris and the sclera is the limbal ring, and the corneal KF ring generally occurs in the part from the inside of the limbal ring to the outside of the pupil contour. Therefore, the part is used as the ROI region of interest of the plane perspective (i.e., the first ROI region of interest). The existing technology for the ROI region of interest of the plane perspective is usually achieved by directly applying the Hough circle transform, which lacks consideration of the light source influencing factors (usually manifested in the light source dots mapped on the corneal surface). This results in the first ROI region of interest being inaccurate in the actual ROI region of interest acquisition process, which in turn leads to large errors in the subsequent segmentation and classification of the corneal KF ring.
[0067] Obtain the centroid coordinates of each circular boundary, obtain multiple centroid coordinates, extract the same centroid coordinates in the multiple centroid coordinates, and count the number of the same centroid coordinates;
[0068] For example, assume that there are multiple centroid coordinates, namely (3,2), (5,3), (3,2), (5,3), (3,2) and (2,1), among which (3,2) appears 3 times, indicating that 3 circular boundaries share this centroid, therefore, the centroid coordinate is regarded as the same centroid coordinate. Similarly, (5,3) appears 2 times, indicating that 2 circular boundaries share this centroid, therefore, the centroid coordinate is also regarded as the same centroid coordinate. (2,1) appears only once, indicating that no other circular boundary shares the centroid with the circular boundary corresponding to this centroid coordinate, therefore, (2,1) is not regarded as the same centroid coordinate.
[0069] Sort the number of identical centroid coordinates by numerical value from large to small, and mark the identical centroid coordinates corresponding to the first numerical value as shared centroid coordinates;
[0070] Extract the circular boundary of each common centroid coordinate, mark the circular boundary of each common centroid coordinate as a circular outline, and obtain the pixel perimeter of each circular outline;
[0071] Sort the pixel perimeter of each circular contour from small to large in numerical value, and mark the pupil contour with the circular contour corresponding to the first pixel perimeter sorted;
[0072] Extracting the centroid coordinates of the pupil contour, and calculating the radius distance from the centroid coordinates of the pupil contour to each circular contour to obtain multiple radius distances;
[0073] Sort the multiple radius distances in descending order, and mark the circular contour corresponding to the first radius distance as the corneal edge contour;
[0074] Segment a pixel area formed by the pupil outline to the corneal edge outline from the first corneal image and use it as a first ROI region of interest;
[0075] It can be understood that: this step obtains an accurate first ROI region of interest through multi-layer screening, which can effectively solve the problem caused by the influence of light source factors compared with the existing technology, thereby facilitating the improvement of the accuracy of the division of the first ROI region of interest, thereby avoiding errors in the subsequent segmentation and classification of the corneal KF ring;
[0076] In implementation, segmenting the ROI region of interest in the second corneal image includes:
[0077] Extracting boundary lines between different layers in the second corneal image using an edge detection algorithm to obtain a plurality of boundary lines; wherein the edge detection algorithm includes but is not limited to a Sobel operator and a Canny operator, etc.;
[0078] Mark the hierarchical region surrounded by every two dividing lines as a candidate region, and obtain multiple candidate regions;
[0079] Obtain the color histogram of the first ROI region of interest and the color histogram of each candidate region;
[0080] Marking a color histogram of the first ROI region of interest as a first color histogram, and marking a color histogram of the candidate region as a second color histogram;
[0081] Calculating the similarity between the first color histogram and the second color histogram to obtain multiple similarities;
[0082] Sort the multiple similarities from large to small according to the numerical values, segment the candidate region corresponding to the first similarity ranking from the second corneal image, and use it as the second ROI region of interest;
[0083] It can be understood that: on the basis of segmenting the first ROI region of interest, through histogram matching, it can be ensured that the obtained second ROI region of interest is consistent with the first ROI region of interest, so as to facilitate accurate detection of subsequent pigment deposition areas;
[0084] In a specific embodiment, extracting the pigment deposition area in the ROI region of interest includes:
[0085] Extracting a first ROI region of interest, graying the first ROI region of interest, and obtaining a grayed image of the first ROI region of interest;
[0086] The K-means clustering algorithm is used to distinguish pixels of the grayscale first ROI region of interest image, and the pixel points are clustered to form an area as a to-be-matched area A, thereby obtaining a plurality of to-be-matched areas A;
[0087] Obtain the center value of each to-be-matched area A, and compare the center value with a preset grayscale value interval. If the center value belongs to the grayscale value interval, the corresponding to-be-matched area A is regarded as a pigment deposition area at a normal viewing angle; if the center value does not belong to the grayscale value interval, the corresponding to-be-matched area A is regarded as a non-pigment deposition area;
[0088] It is worth knowing that in the K-means clustering algorithm, the center value is the average value of all data points (in this case, the pixels of the image) in each cluster category; it is a feature vector that represents the feature center of the cluster category;
[0089] In another specific embodiment, the step of extracting the pigment deposition area in the ROI region of interest further includes:
[0090] Extracting a second ROI region of interest, graying the second ROI region of interest, and obtaining a grayed second ROI region of interest image;
[0091] The K-means clustering algorithm is used to distinguish pixels of the grayscale second ROI region of interest image, and the pixel points are clustered to form an area as a to-be-matched area B, thereby obtaining a plurality of to-be-matched areas B;
[0092] Obtain the center value of each to-be-matched region B, and compare the center value with a preset grayscale value interval. If the center value belongs to the grayscale value interval, the corresponding to-be-matched region B is regarded as a pigment deposition region of the lateral cross-sectional angle; if the center value does not belong to the grayscale value interval, the corresponding to-be-matched region B is regarded as a non-pigment deposition region;
[0093] It should be noted that the preset grayscale value interval is obtained based on a preset standard corneal KF ring image, and the standard corneal KF ring image is a grayscale image. Further explanation is that the center value of the pigment deposition area and the center value of the non-pigment deposition area are marked in the standard corneal KF ring image.
[0094] S102: extracting a two-dimensional feature X and a two-dimensional feature Y of a pigment deposition area in the first corneal image; inputting the two-dimensional feature X into a pre-built naive Bayes classification model for determining whether a corneal KF ring exists, so as to obtain a first recognition result; the two-dimensional feature X is a color feature, and the two-dimensional feature Y includes a morphological feature and a position feature; the first recognition result includes one of the presence or absence of a corneal KF ring;
[0095] It should be understood that pigment deposition is not only present in corneal KF rings, but also in corneal senile rings where lipids are deposited, thus forming a pigment deposition portion similar to corneal KF rings, i.e., a pigment deposition area. Although corneal senile rings and KF rings differ in color, their positions in the image are roughly the same. If they cannot be accurately distinguished, this will bring difficulties or great trouble to the method of detecting KF rings using image analysis, thereby leading to misdiagnosis of hepatolenticular degeneration (HLD) or incorrect grading of KF rings.
[0096] The color features include the average RGB value, saturation, brightness and color gradient of the emmetropia pigmentation area; the morphological features include the number, shape and pixel area of the emmetropia pigmentation area and the pixel area of the first ROI region of interest; the position features are the relative positions of the emmetropia pigmentation areas; the relative positions are the distance between the centroid of the emmetropia pigmentation area and the pupil contour (i.e., the distance between the centroid of the emmetropia pigmentation area and the centroid of the pupil contour);
[0097] Specifically, the training logic of the naive Bayes classification model is as follows:
[0098] Acquire historical corneal KF ring recognition training data, and divide the historical corneal KF ring recognition training data into a corneal KF ring recognition training set and a corneal KF ring recognition test set; the historical corneal KF ring recognition training data has multiple color features and their corresponding classification labels;
[0099] Wherein, the classification labels include labels indicating the presence of corneal KF rings and labels indicating the absence of corneal KF rings;
[0100] It should be noted that: the color features in the historical corneal KF ring recognition training data are collected by technicians using various extraction tools, including but not limited to NumPy and OpenCV; the classification labels in the historical corneal KF ring recognition training data are manually annotated by technicians or data annotators;
[0101] A naive Bayes classification network is constructed, the color features in the corneal KF ring recognition training set are used as the input of the naive Bayes classification network, and the classification labels in the corneal KF ring recognition training set are used as the output of the naive Bayes classification network, and the naive Bayes classification network is trained to obtain an initial naive Bayes classification network;
[0102] The initial naive Bayes classification network is model verified using the corneal KF ring recognition test set, and an initial naive Bayes classification network with a preset first test accuracy is output as a naive Bayes classification model for determining whether a corneal KF ring exists.
[0103] S103: when the first recognition result shows that there is a corneal KF ring, obtaining a three-dimensional feature of the ROI pigment deposition area in the second corneal image, wherein the three-dimensional feature is a thickness feature;
[0104] Wherein, the thickness feature includes the average thickness of the pigment deposition area at each lateral cross-sectional angle and the average thickness of the second ROI region of interest;
[0105] It should be noted that: the pigment deposition area at the lateral cross-sectional angle includes a left boundary and a right boundary, and the average thickness of the pigment deposition area at the lateral cross-sectional angle is obtained by scanning the pigment deposition area column by column, finding the pixel positions of the upper boundary and the lower boundary in each pixel column, and then calculating the pixel distance between the left boundary and the right boundary of each pixel column to obtain the thickness (the specific calculation formula is: thickness = |left boundary position−right boundary position|), and finally calculating the average thickness in all pixel columns to obtain the average thickness of the pigment deposition area at the lateral cross-sectional angle (average thickness = ∑(thickness of each pixel column) / number of pixel columns); similarly, the logic for obtaining the average thickness of the second ROI region of interest is the same, which will not be repeated here.
[0106] S104: performing feature-level fusion on the two-dimensional feature Y and the three-dimensional feature to obtain a multi-modal fusion feature, and inputting the multi-modal fusion feature into a pre-built gradient boosting decision tree model for identifying the corneal KF ring level to obtain a corneal KF ring grading result; wherein the feature-level fusion can be directly achieved by feature splicing;
[0107] The Gradient Boosting Decision Trees (GBDT) model is a powerful machine learning algorithm that is widely used in regression and classification problems. It gradually trains a series of decision tree models, and each new tree corrects the errors of the previous tree, thereby continuously improving the performance of the overall model.
[0108] Boosting is an ensemble learning technique that builds a powerful model by combining multiple weak learners (usually simple models). The core idea of boosting is to gradually improve the model performance through a series of iterations, with each new learner correcting the errors of the previous learner.
[0109] Gradient Boosting, "gradient" refers to the use of gradient descent to minimize the loss function; in each iteration, the model calculates the gradient of the current model's loss function with respect to the target value, and then fits a new decision tree on the training data, the goal of which is to minimize the gradient; then, by combining the predictions of the new tree with the previous model, the performance of the model is gradually improved;
[0110] Specifically, the training logic of the gradient boosting decision tree model is as follows:
[0111] Acquire historical corneal KF ring classification training data, and divide the historical corneal KF ring classification training data into a corneal KF ring classification training set and a corneal KF ring classification test set, wherein the historical corneal KF ring classification training data includes multi-modal fusion features and corresponding annotation labels;
[0112] The label includes five corneal KF ring levels, namely level 0, level 1, level 2, level 3 and level 4; see Table 1 below for details:
[0113] Table 1: Corneal KF ring grading data table
[0114] It should be noted that: the multimodal fusion features in the historical corneal KF ring grading training data are obtained by technicians based on experimental data analysis and processing, and the analysis and processing are described in the above-mentioned related introduction; the annotation labels in the historical corneal KF ring grading training data are obtained by technicians or data annotators according to the above-mentioned Table 1 through manual annotation; wherein, the volume ratio is the volume ratio of the pigment deposition area to the ROI area of interest, and its specific calculation formula is: ; Where: is the volume ratio, is the pixel area of the i-th pigment deposition area, is the average thickness of the i-th pigment deposition area, is the pixel area of the ROI region of interest, is the average thickness of the ROI region of interest;
[0115] It should be understood that: since the pigment deposition area at the front view angle is obtained from the slit lamp microscope image, and the pigment deposition area at the lateral cross-sectional angle is obtained from the OCT image, and the slit lamp microscope image shows the front view angle image, and the OCT image shows the measured cross-sectional image, therefore, whenever there is a pigment deposition area at the front view angle, there is a corresponding pigment deposition area at the lateral cross-sectional angle, therefore, the above-mentioned i-th pigment deposition area refers to a data pair, that is, a data pair including a pair of front view angle images and a measured cross-sectional image;
[0116] Construct a GBDT network, use the multimodal fusion features in the corneal KF ring classification training set as the input of the GBDT network, and use the annotation labels in the corneal KF ring classification training set as the output of the GBDT network, train the GBDT network, and obtain an initial GBDT network;
[0117] The initial GBDT network is verified using the corneal KF ring classification test set, and an initial GBDT network with a value greater than or equal to a preset second test accuracy threshold is output as a gradient boosting decision tree model for identifying the corneal KF ring level;
[0118] By combining multimodal images and taking advantage of the advantages of slit lamp microscope images and OCT images, the corneal KF ring is feature extracted and fused, and the two-dimensional features of the corneal surface and the three-dimensional structural information of the cornea are obtained simultaneously. The present invention is conducive to ensuring the accuracy of the detection and grading of the corneal KF ring, reducing subjective judgment and operation errors, helping clinicians observe the changes in the KF ring before and after drug treatment, and helping clinicians understand the therapeutic effects of different drugs. Furthermore, it is conducive to assisting clinicians in formulating personalized treatment plans for different patients.
[0119] Example 2: Please refer to Figure 2 As shown, this embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, any one of the image recognition-based corneal KF ring grading methods for Wilson's disease provided by the above methods is implemented.
[0120] Since the electronic device introduced in this embodiment is an electronic device used to implement the image recognition-based hepatolenticular degeneration corneal KF ring grading method in the embodiment of the present application, based on the image recognition-based hepatolenticular degeneration corneal KF ring grading method introduced in the embodiment of the present application, a person skilled in the art can understand the specific implementation of the electronic device in this embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not described in detail here. As long as a person skilled in the art implements the electronic device used in the image recognition-based hepatolenticular degeneration corneal KF ring grading method in the embodiment of the present application, it belongs to the scope of protection of the present application.
[0121] Example 3: Please refer to Figure 3 As shown, this embodiment discloses a computer-readable storage medium, including a memory, a processor, and a computer program stored in the memory and running on the processor, and when the processor executes the computer program, any one of the image recognition-based corneal KF ring grading methods for Wilson's disease provided by the above methods is implemented.
[0122] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters, weights and thresholds in the formula are set by technicians in this field according to actual conditions.
[0123] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired network or a wireless network. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD) or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.
[0124] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0125] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0126] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0127] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0128] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0129] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
[0130] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition, characterized in that: The method comprises: Acquire a first corneal image and a second corneal image of the eye of the subject to be captured, and after segmenting the ROI region of interest in the first corneal image and the second corneal image, extract the pigment deposition area in the ROI region of interest; the ROI region of interest includes a first ROI region of interest in the first corneal image and a second ROI region of interest in the second corneal image, and the pigment deposition area includes the pigment deposition area at the frontal angle and the lateral cross-sectional angle; Extracting a two-dimensional feature X and a two-dimensional feature Y of a pigment deposition area in a first corneal image; inputting the two-dimensional feature X into a pre-built naive Bayes classification model for determining whether a corneal KF ring exists, so as to obtain a first recognition result; the two-dimensional feature X is a color feature, and the two-dimensional feature Y includes a morphological feature and a position feature; the first recognition result includes one of the presence or absence of a corneal KF ring; When the first recognition result shows that there is a corneal KF ring, obtaining a three-dimensional feature of the ROI pigment deposition area in the second corneal image, wherein the three-dimensional feature is a thickness feature; The two-dimensional feature Y and the three-dimensional feature are fused at the feature level to obtain a multimodal fusion feature, which is then input into a pre-built gradient boosting decision tree model for identifying the corneal KF ring level to obtain the corneal KF ring grading result.
2. The method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition according to claim 1, characterized in that: The step of segmenting the ROI region of interest in the first corneal image includes: detecting a plurality of circular boundaries in the first corneal image using Hough transform; Obtain the centroid coordinates of each circular boundary, obtain multiple centroid coordinates, extract the same centroid coordinates in the multiple centroid coordinates, and count the number of the same centroid coordinates; Sort the number of identical centroid coordinates by numerical value from large to small, and mark the identical centroid coordinates corresponding to the first numerical value as shared centroid coordinates; Extract the circular boundary of each common centroid coordinate, mark the circular boundary of each common centroid coordinate as a circular outline, and obtain the pixel perimeter of each circular outline; Sort the pixel perimeter of each circular contour from small to large in numerical value, and mark the pupil contour with the circular contour corresponding to the first pixel perimeter sorted; Extracting the centroid coordinates of the pupil contour, and calculating the radius distance from the centroid coordinates of the pupil contour to each circular contour to obtain multiple radius distances; Sort the multiple radius distances in descending order, and mark the circular contour corresponding to the first radius distance as the corneal edge contour; A pixel area formed by the pupil outline to the corneal edge outline is segmented from the first corneal image and used as the first ROI region of interest.
3. The method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition according to claim 2, characterized in that: The step of segmenting the ROI region of interest in the second corneal image includes: Using an edge detection algorithm to extract boundary lines between different layers in the second corneal image, a plurality of boundary lines are obtained; Mark the hierarchical region surrounded by every two dividing lines as a candidate region, and obtain multiple candidate regions; Obtain the color histogram of the first ROI region of interest and the color histogram of each candidate region; Marking a color histogram of the first ROI region of interest as a first color histogram, and marking a color histogram of the candidate region as a second color histogram; Calculating the similarity between the first color histogram and the second color histogram to obtain multiple similarities; The multiple similarities are sorted from large to small according to the numerical values, and the candidate region corresponding to the first similarity sort is segmented from the second corneal image and used as the second ROI region of interest.
4. The method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition according to claim 3, characterized in that: The step of extracting the pigment deposition area in the ROI region of interest comprises: Extracting a first ROI region of interest, graying the first ROI region of interest, and obtaining a grayed image of the first ROI region of interest; The K-means clustering algorithm is used to distinguish pixels of the grayscale first ROI region of interest image, and the pixel points are clustered to form an area as a to-be-matched area A, thereby obtaining a plurality of to-be-matched areas A; Obtain the center value of each area A to be matched, and compare the center value with the preset grayscale value interval. If the center value belongs to the grayscale value interval, the corresponding area A to be matched is used as the pigment deposition area at the normal viewing angle; if the center value does not belong to the grayscale value interval, the corresponding area A to be matched is used as the non-pigment deposition area.
5. The method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition according to claim 4, characterized in that: The step of extracting the pigment deposition area in the ROI region of interest also includes: Extracting a second ROI region of interest, graying the second ROI region of interest, and obtaining a grayed image of the second ROI region of interest; The K-means clustering algorithm is used to distinguish pixels of the grayscale second ROI region of interest image, and the pixel points are clustered to form an area as a to-be-matched area B, thereby obtaining a plurality of to-be-matched areas B; Obtain the center value of each area B to be matched, and compare the center value with the preset grayscale value interval. If the center value belongs to the grayscale value interval, the corresponding area B to be matched is used as the pigment deposition area of the lateral cross-sectional angle; if the center value does not belong to the grayscale value interval, the corresponding area B to be matched is used as the non-pigment deposition area.
6. The method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition according to claim 5, characterized in that: The color features include the average RGB value, saturation, brightness and color gradient of the normal-angle pigmentation area; the morphological features include the number, shape and pixel area of the normal-angle pigmentation area and the pixel area of the first ROI region of interest; the position feature is the relative position of the normal-angle pigmentation area; wherein the relative position is the distance between the center of mass of the normal-angle pigmentation area and the pupil contour.
7. The method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition according to claim 6, characterized in that: The training logic of the Naive Bayes classification model is as follows: Acquire historical corneal KF ring recognition training data, and divide the historical corneal KF ring recognition training data into a corneal KF ring recognition training set and a corneal KF ring recognition test set; the historical corneal KF ring recognition training data has multiple color features and their corresponding classification labels; Wherein, the classification labels include labels indicating the presence of corneal KF rings and labels indicating the absence of corneal KF rings; A naive Bayes classification network is constructed, the color features in the corneal KF ring recognition training set are used as the input of the naive Bayes classification network, and the classification labels in the corneal KF ring recognition training set are used as the output of the naive Bayes classification network, and the naive Bayes classification network is trained to obtain an initial naive Bayes classification network; The initial naive Bayes classification network is model verified using the corneal KF ring recognition test set, and an initial naive Bayes classification network with a preset first test accuracy is output as a naive Bayes classification model for determining whether a corneal KF ring exists.
8. The method for grading the KF ring of hepatolenticular degeneration cornea based on image recognition according to claim 7, characterized in that: The thickness feature includes the average thickness of the pigment deposition area at each lateral cross-sectional angle and the average thickness of the second ROI region of interest; The training logic of the gradient boosting decision tree model is as follows: Acquire historical corneal KF ring classification training data, and divide the historical corneal KF ring classification training data into a corneal KF ring classification training set and a corneal KF ring classification test set, wherein the historical corneal KF ring classification training data includes multi-modal fusion features and corresponding annotation labels; The annotation labels include five corneal KF ring levels, namely level 0, level 1, level 2, level 3 and level 4; Construct a GBDT network, use the multimodal fusion features in the corneal KF ring classification training set as the input of the GBDT network, and use the annotation labels in the corneal KF ring classification training set as the output of the GBDT network, train the GBDT network, and obtain an initial GBDT network; The initial GBDT network was verified using the corneal KF ring grading test set, and an initial GBDT network with a value greater than or equal to the preset second test accuracy threshold was output as a gradient boosting decision tree model for identifying the corneal KF ring level.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the image recognition-based corneal KF ring grading method for Wilson's disease described in any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the image recognition-based corneal KF ring grading method for Wilson's disease described in any one of claims 1 to 8 is implemented.
