A method applicable to measuring multiple indicators of the ciliary muscle and scleral structure of the human eye

Abstract

The present invention belongs to the technical field of computer image processing, and specifically discloses a method applicable to measuring multiple indexes of the ciliary muscle and scleral structure of the human eye, including obtaining a grayscale image of the ciliary body part; preprocessing the grayscale image; identifying and performing instance segmentation on the ciliary muscle to obtain a ciliary muscle mask; performing morphological processing on the obtained ciliary muscle mask, including screening out the contour with the largest area in the ciliary muscle mask to calculate the ciliary muscle area, ciliary muscle perimeter, ciliary muscle length, ciliary muscle thickness, scleral thickness, and maximum ciliary muscle thickness; measuring relevant angles of the ciliary muscle, and the relevant angles include the anterior angle of the ciliary muscle, the caudal angle of the ciliary muscle, the inferior angle of the ciliary muscle, TCA, ICA, and the anterior chamber angle. This technical solution can complete the automatic measurement of relevant parameters of the ciliary muscle and sclera, improve the efficiency in medical research and clinical work, without manual intervention, eliminate the interference of human factors, and the results have strong repeatability.

Description

Technical Field

[0001] The invention belongs to the technical field of computer image processing, and in particular relates to a method suitable for measuring multiple indexes of ciliary muscle and sclera structure of human eyes. Background Art

[0002] Accurate measurement of biological parameters related to the ciliary muscle of the human eye is an important part of ophthalmic research. The traditional method uses ultrasonic biomicroscopy (UBM) to obtain grayscale images of the ciliary body of the subject and manually measure various biological indicators of the ciliary muscle on the image. This method is greatly affected by the subjective influence of the measurer and has low repeatability. The biological parameters of the ciliary muscle of the human eye include: the circumference, area, thickness, length, and angle of the ciliary muscle. Due to the limitations of image resolution and the discontinuity of pixels, image-based angle estimation is a major difficulty in computer image processing problems.

[0003] The ciliary muscle can be identified by obtaining the ciliary muscle mask through the Mask Region Convolutional Neural Network (Mask RCNN), and then various computer vision methods are applied to perform image morphological processing to measure ciliary muscle related indicators. In order to fully automatically measure the angle indicators related to the ciliary muscle, it is necessary to find the key points near the ciliary body. Key point detection technology refers to the computer automatically identifying the key parts of the detected object from the image, and has been widely used in the fields of human posture estimation and behavior recognition. There are two major methods of key point detection technology, which are regression method and heat map method. The regression method is to flatten the feature map after the backbone feature extraction network, and use the fully connected layer to regress the coordinate values ​​of the key points. The heat map method is to let the last layer of the convolutional neural network generate a Gaussian heat map, and then find the index of the maximum position of the heat map as the position of the key point. Both methods have their own advantages and disadvantages. The regression method has better robustness, but it flattens the feature map and loses the spatial features of the key points. The heat map method focuses more on local spatial features, and the network converges better. Using the DSNT (Differentiable Spatial to Numerical Transform) layer to calculate the expectation of the heat map can also make the heat map method have high accuracy, but the heat map method is not good at detecting key points of rigid objects. Angle estimation pays more attention to the vertices of the angle and the slopes of the two rays, that is, the angle between the two vectors, rather than the coordinate values ​​of the three vertices. Using traditional methods to generate heat maps of three key points, or using the coordinate values ​​of the three key points as the output of the neural network, does not meet the purpose of the angle measurement task.

[0004] Therefore, there is an urgent need to design a method that can fully automatically measure ciliary muscle-related parameters to improve the efficiency in medical research and clinical work. Summary of the invention

[0005] The object of the present invention is to provide a method applicable to measuring multiple indexes of the ciliary muscle and scleral structure of the human eye, so as to solve the problems that the regression method currently adopted will cause the loss of the spatial characteristics of key points and the heat map method currently adopted is not good at detecting key points of rigid objects.

[0006] To achieve the above object, the technical solution of the present invention is: a method applicable to measuring multiple indexes of the ciliary muscle and scleral structure of the human eye, including

[0007] Obtaining a grayscale image of the ciliary body part;

[0008] Preprocessing the grayscale image using a deep learning platform;

[0009] Identifying the ciliary muscle using a deep learning platform and performing instance segmentation to obtain a ciliary muscle mask;

[0010] Performing morphological processing on the obtained ciliary muscle mask using a deep learning platform; including screening out the contour with the largest area in the ciliary muscle mask and calculating the ciliary muscle area, ciliary muscle perimeter, ciliary muscle length, ciliary muscle thickness, scleral thickness, and maximum ciliary muscle thickness according to the contour with the largest area;

[0011] Measuring relevant angles of the ciliary muscle using a deep learning platform, and the relevant angles include the anterior angle of the ciliary muscle, the tail angle of the ciliary muscle, the inferior angle of the ciliary muscle, TCA, ICA, and the anterior chamber angle.

[0012] Furthermore, the preprocessing includes the recognition of the grayscale image scale and the cropping of the grayscale image; the method for recognizing the scale is: screening out the scale of the image from the grayscale image using color, setting the high threshold and low threshold of the scale color, and generating a mask corresponding to the scale color; calculating the pixel length between the maximum values of two scales, and this pixel length is the scale; cropping the grayscale image according to the signal distribution of the grayscale image to generate an interested rectangular area.

[0013] Furthermore, the deep learning platform includes a ciliary muscle recognition module, and the establishment method of the ciliary muscle recognition module is as follows:

[0014] Scanning the ciliary bodies of both eyes of n subjects, scanning four orientations of each eye, including the ciliary bodies at 0°, 90°, 180°, and 270°, to generate 4n grayscale images;

[0015] Using software to label and train the ciliary muscle masks of 4n grayscale images respectively to obtain weights and predict the optimal ciliary muscle mask.

[0016] Furthermore, the deep learning platform includes a ciliary muscle morphological processing module, and the processing process of the ciliary muscle morphological processing module is as follows:

[0017] Extract the optimal ciliary muscle mask predicted;

[0018] Use a function to find the contours in the ciliary muscle mask and filter out the contour with the largest occupied area;

[0019] Calculate the ciliary muscle area: Use a function to calculate the area of this contour, and divide this area by (scale factor^2) to obtain the ciliary muscle area based on the scale factor;

[0020] Calculate the ciliary muscle perimeter: Use a function to calculate the perimeter of this contour, and divide this perimeter by the scale factor to obtain the ciliary muscle perimeter based on the scale factor;

[0021] Calculate the ciliary muscle length: Use an algorithm to find the two points with the farthest distance on this contour, and divide the distance between the two points by the scale factor to obtain the ciliary muscle length based on the scale factor; The straight line where the two points are located is the major axis of the ciliary muscle;

[0022] Calculate the ciliary muscle thickness and scleral thickness: According to the major axis of the ciliary muscle, divide the ciliary muscle contour into the upper surface and the lower surface of the ciliary muscle; Calculate the ciliary muscle thickness and scleral thickness at different positions; The calculation is performed in the normal mode;

[0023] Calculate the maximum ciliary muscle thickness: Calculate the maximum distance between the perpendicular line of the tangent of the upper surface of the ciliary muscle and the lower surface of the ciliary muscle.

[0024] Furthermore, rotate the grayscale image with the midpoint of the major axis of the ciliary muscle as the origin so that the major axis of the ciliary muscle is in the horizontal direction of the grayscale image.

[0025] Furthermore, the deep learning platform includes a convolutional neural network, and the convolutional neural network is used to predict relevant angles. The convolutional neural network includes several convolutional layers. A grayscale image with an input tensor size of (1, 512, 512) is input, and a 3-channel heat map with an output tensor size of (3, 512, 512) is output; The deep learning platform uses the converged weights to predict the angles: Analyze the generated heat map, find the index of the maximum value of each heat map, and use this index as the position of the vertex and edge points to calculate the angles.

[0026] Furthermore, use the grayscale images of n subjects as the input of the convolutional neural network, label the grayscale images of n subjects, and label the dataset of the ciliary muscle related angles as the output of the convolutional neural network, and train the convolutional neural network according to the input and output.

[0027] Furthermore, the convolutional neural network uses a special method to output a 3-channel image of relevant angles. The special method includes a Gaussian heat map generation method for vertex coordinates and an edge point vector heat map generation method:

[0028] An angle contains a vertex coordinate and two edge point coordinates. The true vertex coordinate is (X t , Y t ). The coordinate of the true edge point G1 is (X g1 , Y g1 ), and the coordinate of the true edge point G2 is (X g2 , Y g2 ); Two true edge point unit vectors are generated from the true edge points and the true vertex, namely and . The included angle between these two true edge point unit vectors is the true angle of the angle;

[0029] The method for generating the Gaussian heatmap of vertex T is as follows: First, generate a tensor with a size of (1, 512, 512), and then calculate the value of each position in the tensor according to the two-dimensional Gaussian distribution. The two-dimensional Gaussian distribution formula is:

[0030]

[0031] In this formula, the coordinate of a certain point on the heatmap is (X 顶 , Y 顶 ), e is the natural constant, σ is the variance, and the variance can be adjusted according to the actual situation;

[0032] The method for generating the heatmap of the true edge point unit vector is:

[0033] First, generate a tensor with a size of (2, 512, 512), and two channels are used to place the heatmaps of two different edge point vectors respectively; Calculate the true unit vectors and , and its formula is:

[0034]

[0035] In this formula, i takes 1 or 2;

[0036] Suppose the coordinate of a certain point on the heatmap is (X 边 , Y 边 ), then the unit vector from the true vertex coordinate to this point is:

[0037]

[0038] The cosine value of the included angle between the unit vector of this point and the true edge point unit vector can be calculated as:

[0039]

[0040] The value range of this value is [-1, 1]. First, use the Relu function, and then use a high-order function to map this value to generate a gradient.

[0041] Furthermore, the basic composition structure of the convolutional neural network follows the encode-decode pattern, and the basic composition unit is Conv3. Conv3 includes three convolutions. The first convolution uses a 3×3 convolution kernel + BN + Relu, the second uses a 5×5 convolution kernel + BN + Relu, and the third uses a 3×3 convolution kernel + BN + Relu. Each convolution uses padding to ensure that the size of the feature map remains unchanged before and after convolution.

[0042] Furthermore, a grayscale image obtained using UBM is used.

[0043] The beneficial effects of this technical solution are as follows: ① This technical solution can complete the full-automatic measurement of the relevant parameters of the ciliary muscle and the sclera, greatly improving the efficiency in medical research and clinical work. ② There is no need for manual intervention, eliminating the interference of human factors, and the results are highly repeatable. ③ The vector-based heat map generation method can improve the accuracy of angle recognition by the neural network. ④ The convolution unit uses three convolutions instead of two. The kernel size of the first convolution is 3×3, the second is 5×5, and the third is 3×3, which can significantly increase the receptive field of the neural network. ④ Since the size of the input tensor is equal to the size of the output tensor, the loss of coordinate accuracy caused by the heat map size is avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flowchart of a method for measuring multiple indicators of the ciliary muscle and scleral structure of the human eye according to the present invention;

[0045] Figure 2 is a grayscale image of the ciliary body obtained by UBM;

[0046] Figure 3 is a ciliary muscle mask predicted by Mask RCNN;

[0047] Figure 4 is a schematic diagram of the anterior chamber angle, TCA, and ICA;

[0048] Figure 5 is a schematic diagram of SCT1-3, CMTM, CMT1-3, the anterior angle of the ciliary muscle, the tail angle of the ciliary muscle, and the inferior angle of the ciliary muscle;

[0049] Figure 6 is an analysis diagram during the heat map generation process;

[0050] Figure 7 is an annotation diagram and a 3-channel heat map generated according to the heat map generation method in the training set;

[0051] Figure 8 is an annotation diagram of the anterior chamber angle and a 3-channel heat map predicted by the convolutional neural network;

[0052] Figure 9It is a diagram of an encode - decode convolutional neural network. Specific implementation manner

[0053] The following is a further detailed description through specific implementation manners:

[0054] The following will combine the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0055] The embodiment is basically as shown in the appendix Figure 1 shown: A method applicable to measuring multiple indicators of the ciliary muscle and scleral structure of the human eye includes the following steps:

[0056] S1: Use Python3.6, TensorFlow2.4.0, Keras2.4.3, PyTorch1.10, CUDA11.4, CPU: i7 10700K, graphics card: NVIDIA GeForce RTX 3090 to build a deep learning platform and write the code for the process.

[0057] S2: Obtain the grayscale image of the ciliary body part through UBM, as Figure 2 shown.

[0058] S3: Preprocess the obtained grayscale image using a deep learning platform; the preprocessing includes the recognition of the scale of the grayscale image and the cropping of the grayscale image; the method for scale recognition is as follows: Use color to screen out the scale ruler of the image from the grayscale image, set the high threshold and low threshold of the scale color, and generate a mask corresponding to the scale color; calculate the pixel length between the maximum values of two scales, and this pixel length is the scale; recognize the scale rulers on the left and bottom of the image, and crop the grayscale image according to the signal distribution of the grayscale image to generate a region of interest (ROI) rectangle to reduce the size of the dataset and increase the training efficiency. In this example, the scale is blue (actually blue, but the color display of the attached drawing scale in the patent drawing is changed), set the high threshold of blue as [250, 120, 0] (BGR) and the low threshold as [255, 130, 5] (BGR), and use functions such as "cv2.inRange" to generate a mask of the scale. Given that the actual length corresponding to the distance between two large scales is 1 mm, then screen out the large scales and calculate the pixel length between the two large scales, and this pixel length is the scale. In this example, the scale is 65.57 pixels per mm. For the cropping of the image, the upper half of the image is black, and the signal-free upper half can be removed through the signal distribution of the image. Finally, the size is changed to (1, 512, 512) through methods such as surrounding zero padding (padding) and resizing to facilitate input into the convolutional neural network.

[0059] S4: Use a deep learning platform to identify the ciliary muscle and perform instance segmentation to obtain a ciliary muscle mask; the deep learning platform includes a ciliary muscle recognition module, and the establishment method of the ciliary muscle recognition module is as follows:

[0060] Ophthalmologists perform UBM scans on the ciliary bodies of the binocular eyes of a number of (such as 67) subjects. Each eye is scanned in four directions, including the ciliary bodies at 0°, 90°, 180°, and 270°, generating 536 grayscale images;

[0061] Use the labelme software to label the ciliary muscle masks of 536 grayscale images respectively, and send them into MaskRCNN for training to obtain weights and predict the best ciliary muscle mask. Use the trained weights to perform instance segmentation of the ciliary muscle, and the ciliary muscle mask obtained by the instance segmentation is as Figure 3 shown.

[0062] S5: Use a deep learning platform to perform morphological processing on the obtained ciliary muscle mask; including screening out the contour with the largest area in the ciliary muscle mask and calculating the ciliary muscle area, ciliary muscle perimeter, ciliary muscle length, ciliary muscle thickness, scleral thickness, and maximum ciliary muscle thickness based on the contour with the largest area; the deep learning platform includes a ciliary muscle morphological processing module, and the processing process of the ciliary muscle morphological processing module is as follows:

[0063] Extract the best ciliary muscle mask predicted by Mask RCNN;

[0064] Use functions such as "cv2.findContours" to find the contours in the ciliary muscle mask and filter out the contour with the largest area; the purpose of this step is that the mode of generating the mask by Mask RCNN is semantic segmentation with pixel-by-pixel prediction, which may cause the mask not to form a complete ciliary muscle. Therefore, only the largest contour in the predicted mask is taken;

[0065] Calculate the ciliary muscle area (CMA): Use functions such as "cv2.contourArea" to calculate the area of this contour. The unit of this area is square pixels. Divide this area by (scale factor ^ 2) to obtain the ciliary muscle area based on the scale factor, which is also the area in square millimeters;

[0066] Calculate the ciliary muscle perimeter (CMP): Use functions such as "cv2.arcLength" to calculate the perimeter of this contour. Divide the perimeter by the scale factor to obtain the ciliary muscle perimeter based on the scale factor, which is also the ciliary muscle perimeter in millimeters;

[0067] Calculate the ciliary muscle length (CML): Use an algorithm to find the two points farthest apart on this contour. Divide the distance between the two points by the scale factor to obtain the ciliary muscle length based on the scale factor; the straight line where the two points are located is the major axis of the ciliary muscle; Rotate the grayscale image with the midpoint of the major axis of the ciliary muscle as the origin so that the major axis of the ciliary muscle is in the horizontal direction of the grayscale image;

[0068] Calculate the ciliary muscle thickness and scleral thickness: According to the major axis of the ciliary muscle, divide the ciliary muscle contour into the upper surface and the lower surface of the ciliary muscle; Calculate the ciliary muscle thickness and scleral thickness at different positions; As Figure 5 shown, the ciliary muscle thickness includes the thickness at 1 mm of the ciliary muscle (CMT1), the thickness at 2 mm of the ciliary muscle (CMT2), and the thickness at 3 mm of the ciliary muscle (CMT3). If the length of the ciliary muscle of the subject is less than 3 mm, then CMT3 does not exist; The thickness at a mm of the ciliary muscle is defined as the thickness at a mm from the scleral spur in the major axis direction of the ciliary muscle; The scleral thickness (SCT) includes the scleral thickness at 1 mm (SCT1), the scleral thickness at 2 mm (SCT2), and the scleral thickness at 3 mm (SCT3), that is, the scleral thickness at 1 mm, 2 mm, and 3 mm from the scleral spur; There are obvious edge effects on the upper surface of the sclera, which can be identified by methods such as convolution and contour extraction; The calculation adopts the normal mode, that is, the lengths of the normal line segments at a certain distance from the scleral spur on the upper surface of the ciliary muscle are calculated for the above 6 indicators, rather than the vertical distance;

[0069] Calculating the maximum ciliary muscle thickness (CMTM): Calculate the maximum distance between the perpendicular line of the tangent on the upper surface of the ciliary muscle and the lower surface of the ciliary muscle. Traverse the distances from each point on the upper surface of the ciliary muscle to the lower surface of the ciliary muscle in the vertical direction to find the position with the maximum distance, and calculate the length of the normal line segment from this position to the lower surface of the ciliary muscle, which is recorded as the maximum ciliary muscle thickness.

[0070] S6: Use a deep learning platform to measure the relevant angles of the ciliary muscle. The relevant angles include the anterior angle of the ciliary muscle, the caudal angle of the ciliary muscle, the inferior angle of the ciliary muscle, TCA, ICA, and the anterior chamber angle, as Figure 4 , 5 shown. The deep learning platform includes a convolutional neural network (encode - decode mode). The convolutional neural network is used to predict the relevant angles. The convolutional neural network includes several convolutional layers and follows the encode - decode mode. The input tensor size is a grayscale image of (1, 512, 512), and the output tensor size is a 3 - channel heat map of (3, 512, 512). The deep learning platform uses the converged weights of the convolutional neural network for angle prediction: Analyze the generated heat map, use the argmax method to find the index of the maximum value of each heat map, and use this index as the position of the vertex and edge points for angle calculation.

[0071] The convolutional neural network uses a special method to output a 3 - channel image of the relevant angles. The special method includes the Gaussian heat map generation method for vertex coordinates and the heat map generation method for edge point vectors. As Figure 6 shown, taking the anterior chamber angle as an example to elaborate on the heat map generation method. For the vertex of the anterior chamber angle, the heat map is generated using the conventional method, that is, two - dimensional Gaussian distribution. For the two edge points of the anterior chamber angle, the heat map is generated using the vector method.

[0072] First, define the symbols: An angle contains a vertex and two edge points. The true vertex coordinates are (X t , Y t ), the coordinates of the true edge point G1 are (X g1 , Y g1 ), and the coordinates of the true edge point G2 are (X g2 , Y g2 ); Two true edge - point unit vectors are generated from the true edge points and the true vertex, that is, and The included angle between these two true edge - point unit vectors is the true angle of this angle;

[0073] Combined with Appendix Figure 6 and 7 , the Gaussian heat map generation method for the vertex T is as follows: First, generate a tensor of size (1, 512, 512), and then calculate the value of each position in the tensor according to the two - dimensional Gaussian distribution. The two - dimensional Gaussian distribution formula is:

[0074]

[0075] In this formula, the real vertex coordinates are (X t , Y t ), the coordinates of a point on the heatmap are (X 顶 , Y 顶 ), e is the natural constant, σ is the variance, and the variance can be adjusted according to the actual situation; Figure 7 The image in the lower left corner is the heatmap of vertex T.

[0076] The method for generating the heatmap of the real edge point unit vector is as follows:

[0077] First, generate a tensor of size (2, 512, 512). Two channels are used to place the heatmaps of two different edge point vectors respectively; calculate the real unit vector and The formula is:

[0078]

[0079] In this formula, i takes 1 or 2;

[0080] Let the coordinates of a point on the heatmap be (X 边 , Y 边 ), then the unit vector from the real vertex coordinates to this point is:

[0081]

[0082] The cosine value of the angle between the unit vector of this point and the real edge point unit vector can be calculated as:

[0083]

[0084] The value range of this value is [-1, 1]. First, use the Relu function, and then use a high-order function, such as f(x) = x n (n > 2), to map this value to generate a gradient. The purpose of this step is that the derivative of the cosine function changes too little near 1, so using a high-order function to generate a gradient is more conducive to the convergence of the neural network. Use this value as the value at this place in the heatmap matrix. In addition, if this point is too close or too far from the real vertex coordinates, the value of this point is set to 0. The CUDA kernel function used here is:

[0085] @cuda.jit

[0086] def Calheatmap(heatmap, pointbase, v1base):

[0087] """

[0088] param: heatmap: Heatmap matrix, with shape (512, 512)

[0089] pointbase: Coordinates (Xt, Yt) of the vertex, a list, e.g., [185.0, 357.0]

[0090] v1base: A unit vector of a real edge point, a list, e.g., [0.92787744, -0.37288532]

[0091] """

[0092] # Allocate threads for Cuda cores

[0093] rows, cols = heatmap.shape

[0094] ty = (cuda.blockIdx.x * cuda.blockDim.x + cuda.threadIdx.x) / / rows

[0095] tx = (cuda.blockIdx.x * cuda.blockDim.x + cuda.threadIdx.x) - ty * rows

[0096] if ty < rows and tx < cols:

[0097] Xgo = np.float64(tx)

[0098] Ygo = np.float64(ty)

[0099] Xt = np.float64(pointbase[0])

[0100] Yt = np.float64(pointbase[1])

[0101] # If the distance of this point from the vertex is too large or too small, set this point to 0

[0102] If ((Xgo - Xt) ** 2 + (Ygo - Yt) ** 2) ** 0.5 < 20 or ((Xgo - Xt) ** 2 + (Ygo - Yt) ** 2) ** 0.5 > 200:

[0103] heatmap[ty, tx] = 0

[0104] return

[0105] # Calculate vector Un

[0106] Unbase0 = (Xgo - Xt) / ((Xgo - Xt) ** 2 + (Ygo - Yt) ** 2) ** 0.5

[0107] Unbase1 = (Ygo - Yt) / ((Xgo - Xt) ** 2 + (Ygo - Yt) ** 2) ** 0.5

[0108] # Calculate the cosine value of the angle between the unit vector of this point and the unit vector of the true edge point

[0109] cos_u1baseANDv1base = np.float64(v1base[0]) * Unbase0 + np.float64(v1base[1]) * Unbase1

[0110] # Code expression of the Relu function

[0111] if cos_u1baseANDv1base < 0:

[0112] deal1 = 0

[0113] else:

[0114] deal1 = cos_u1baseANDv1base

[0115] # High-order function, here F(x) = x ** 21 is used

[0116] deal2 = deal1 ** 21

[0117] # Assign the calculation result to this position on the heatmap

[0118] heatmap[ty, tx] = deal2

[0119] Using this method, a heatmap can be generated based on the principle of unit vectors. The values on the heatmap represent the directions that are more concerned about in angle prediction, rather than coordinates. The value of each point on the heatmap represents the angle value between the unit vector from the vertex coordinates to this position and the true unit vector. Attached Figure 7 The image in the upper right corner is the heatmap of edge point G1, and the image in the lower right corner is the heatmap of edge point G2. These 3 heatmaps (the heatmap of vertex T, the heatmap of edge point G1, the heatmap of edge point G2) are combined into a tensor of (3, 512, 512) as the true value (Ground Truth) output by the convolutional neural network for the training of the convolutional neural network.

[0120] The initial training method of the convolutional neural network is as follows: The grayscale images of several subjects (tensor size (1, 512, 512)) are used as the input of the convolutional neural network. The grayscale images of several subjects are labeled, and the dataset of the angles related to the ciliary muscle is labeled as the output of the convolutional neural network. The output tensor size is (3, 512, 512). The convolutional neural network is trained according to the input and output.

[0121] The architecture of the convolutional neural network is shown in the appendix Figure 9 This convolutional neural network is modified on the basis of the Unet structure. The basic composition structure of the convolutional neural network follows the encode-decode mode. The basic composition unit is Conv3. Conv3 includes 3 convolutions. The first time uses a 3×3 convolution kernel + BN (normalization) + Relu (activation), the second time uses a 5×5 convolution kernel + BN + Relu, and the third time uses a 3×3 convolution kernel + BN + Relu. Each convolution uses padding to ensure that the size of the feature map remains unchanged before and after convolution. The loss function used is MSE, the optimizer is Adam, and the learning rate is 0.005. The network converges after training for 1000 epochs.

[0122] In this example, the 3-channel heat map result of the convolutional neural network predicting the anterior chamber angle is shown in the appendix Figure 8 The upper left is the labeled map, the upper right is the heat map of edge point 1, the lower left is the heat map of the vertex, and the lower right is the heat map of edge point 2. According to the 3 heat maps, the argmax method is used to find the positions of the vertex and edge points.

[0123] S7: Integrate the data to generate a report. In this example, the measurement results of the ciliary muscle are as follows:

[0124]

[0125]

[0126] This solution identifies the image scale ruler through image preprocessing to calculate the scale, uses the ciliary muscle recognition module and the ciliary muscle morphological processing module to complete the measurement of parameters such as the perimeter and area of the ciliary muscle, and completes the fully automatic measurement of the angles related to the ciliary muscle through the convolutional neural network and the vector-based heat map generation method.

[0127] It should be noted that, in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0128] The above are only embodiments of the present invention. Specific structures and common knowledge such as characteristics that are well known in the art are not described in detail herein. Those of ordinary skill in the art know all the common general technical knowledge in the technical field to which the invention pertains before the filing date or the priority date, are able to learn all the prior art in this field, and have the ability to apply conventional experimental means before this date. Those of ordinary skill in the art can, under the inspiration given in this application, complete and implement this solution in combination with their own capabilities. Some typical well-known structures or well-known methods should not become an obstacle for those of ordinary skill in the art to implement this application. It should be pointed out that, for those skilled in the art, without departing from the structure of the present invention, several modifications and improvements can still be made, and these should also be regarded as the protection scope of the present invention, which will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope claimed in this application should be based on the content of its claims, and the specific implementation manners described in the specification can be used to interpret the content of the claims.

Claims

1. A method applicable to measuring multiple indicators of the ciliary muscle and scleral structure of the human eye, characterized in that: It includes obtaining a grayscale image of the ciliary body part; preprocessing the grayscale image using a deep learning platform; the preprocessing includes the recognition of the grayscale image scale and the cropping of the grayscale image; the method for recognizing the scale is: screening out the scale of the image from the grayscale image by color, setting the high threshold and low threshold of the scale color, and generating a mask corresponding to the scale color; calculating the pixel length between the maximum values of two scales, and this pixel length is the scale; cropping the grayscale image according to the signal distribution of the grayscale image to generate an interested rectangular area; the deep learning platform includes a ciliary muscle recognition module, which recognizes the ciliary muscle and performs instance segmentation to obtain a ciliary muscle mask; The ciliary muscle recognition module is established as follows: scanning the ciliary bodies of the eyes of n subjects, scanning four orientations for each eye, including the ciliary bodies at 0°, 90°, 180°, and 270°, generating 4n grayscale images; using software to label the ciliary muscle masks of the 4n grayscale images respectively, and training to obtain weights and predict the best ciliary muscle mask; the deep learning platform includes a ciliary muscle morphology processing module, and the processing process of the ciliary muscle morphology processing module is as follows: extracting the predicted best ciliary muscle mask; using a function to find the contour in the ciliary muscle mask and screening out the contour with the largest occupied area; calculating the ciliary muscle area: using a function to calculate the area of this contour, and dividing this area by (scale^2) to obtain the ciliary muscle area based on the scale; calculating the ciliary muscle perimeter: using a function to calculate the perimeter of this contour, and dividing this perimeter by the scale to obtain the ciliary muscle perimeter based on the scale; calculating the ciliary muscle length: using an algorithm to find the two points farthest apart on this contour, and dividing the distance between the two points by the scale to obtain the ciliary muscle length based on the scale; the straight line where the two points are located is the major axis of the ciliary muscle; calculating the ciliary muscle thickness and scleral thickness: according to the major axis of the ciliary muscle, dividing the ciliary muscle contour into the upper surface and the lower surface of the ciliary muscle; calculating the ciliary muscle thickness and scleral thickness at different positions; calculating the maximum ciliary muscle thickness: calculating the maximum distance between the perpendicular line of the tangent of the upper surface of the ciliary muscle and the lower surface of the ciliary muscle; measuring the relevant angles of the ciliary muscle using a deep learning platform, the relevant angles include the anterior angle of the ciliary muscle, the caudal angle of the ciliary muscle, the inferior angle of the ciliary muscle, TCA, ICA, and the anterior chamber angle; the deep learning platform includes a convolutional neural network, and the convolutional neural network is used to predict the relevant angles, the convolutional neural network includes several convolutional layers, inputting a grayscale image with a tensor size of (1, 512, 512), and outputting a 3-channel heat map with a tensor size of (3, 512, 512); the deep learning platform uses the converged weights to predict the angles: parsing the generated heat map, finding the index of the maximum value of each heat map, and using this index as the position of the vertex and the edge point to calculate the angles.

2. The method applicable to measuring multiple indicators of the ciliary muscle and scleral structure of the human eye according to claim 1, characterized in that: Rotate the grayscale image with the midpoint of the long axis of the ciliary muscle as the origin, so that the long axis of the ciliary muscle is in the horizontal direction of the grayscale image.

3. A method for measuring multiple indexes of the ciliary muscle and scleral structure of the human eye according to claim 1, characterized in that: The grayscale images of n subjects are used as the input of the convolutional neural network. The grayscale images of n subjects are labeled, and the data set of the angles related to the ciliary muscle is labeled as the output of the convolutional neural network. The convolutional neural network is trained according to the input and output.

4. A method for measuring multiple indexes of the ciliary muscle and scleral structure of the human eye according to claim 1, characterized in that: The convolutional neural network uses a special method to output a 3-channel image of the relevant angle. The special method includes a Gaussian heat map generation method for vertex coordinates and a heat map generation method for edge point vectors: An angle contains a vertex coordinate and two edge point coordinates. The true vertex coordinate is (X t , Y t ). The coordinate of the true edge point G1 is (X g1 , Y g1 ). The coordinate of the true edge point G2 is (X g2 , Y g2 ); Two true edge point unit vectors are generated from the true edge points and the true vertex, namely and The included angle between these two true edge point unit vectors is the true angle of the angle; The Gaussian heat map generation method for vertex T is: first generate a tensor with a size of (1, 512, 512), and then calculate the value of each position in the tensor according to the two-dimensional Gaussian distribution. The two-dimensional Gaussian distribution formula is: In this formula, the coordinates of a certain point on the heat map are (X 顶 , Y 顶 ), e is the natural constant, σ is the variance, and the variance can be adjusted according to the actual situation; The heat map generation method for the unit vector of the real edge point is: First, generate a tensor of size (2, 512, 512). Two channels are used to separately place the heatmaps of two different edge point vectors; calculate the true unit vector and The formula is as follows: In this formula, i takes 1 or 2; Let the coordinates of a point on the heat map be (X 边 , Y 边 ). Then the unit vector from the true vertex coordinates to this point is: The cosine value of the angle between the unit vector of this point and the unit vector of the real edge point can be calculated as: The value range of this value is [-1, 1]. First, use the Relu function, and then use a high-order function to map this value to generate a gradient.

5. A method for measuring multiple indexes of the ciliary muscle and scleral structure of the human eye according to claim 1, characterized in that: The basic composition structure of the convolutional neural network follows the encode-decode mode, and the basic composition unit is Conv3. Conv3 includes 3 convolutions. The first time uses a 3×3 convolution kernel + BN + Relu, the second time uses a 5×5 convolution kernel + BN + Relu, and the third time uses a 3×3 convolution kernel + BN + Relu. Each convolution uses padding to ensure that the size of the feature map remains unchanged before and after convolution.

6. A method for measuring multiple indexes of the ciliary muscle and scleral structure of the human eye according to claim 1, characterized in that: The grayscale image obtained by UBM.

Images