Scleral spur positioning method, apparatus, device, and storage medium

Abstract

The application relates to a scleral spur positioning method, device, equipment and storage medium. According to an eye image to be recognized and a scleral spur positioning model, a first heat map corresponding to the image to be recognized is determined, so that target position information of the scleral spur in the image to be recognized is determined according to the first heat map. The scleral spur positioning model comprises a convolution model, the convolution model comprises at least one convolution module, the convolution module comprises a depth separable convolution layer and a spatial separable convolution layer, the scleral spur positioning model is obtained by training an initial scleral spur positioning model according to an eye image sample, and the eye image sample comprises a scleral spur. In the application, the spatial separable convolution layer can reduce the parameter quantity, and in combination with the depth separable convolution layer, the scleral spur positioning model can achieve high accuracy with less parameter quantity, the scleral spur positioning time is reduced while the scleral spur positioning accuracy is ensured, the scleral spur positioning has real-time performance, and therefore the efficiency of the scleral spur positioning is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for scleral protrusion localization. Background Technology

[0002] The anterior chamber angle, the angle formed between the inner surface of the cornea and the anterior surface of the iris, is an important component of the aqueous humor circulation. Clinically, the degree of opening of the anterior chamber angle reflects the risk of angle closure; the smaller the angle opening, the greater the risk of angle closure or primary angle-closure glaucoma. Therefore, quantitative assessment of the anterior chamber angle opening is an essential part of the diagnosis and treatment process for anyone diagnosed with or suspected of having primary angle-closure glaucoma, and scleral spur localization is a crucial foundation for achieving this quantitative assessment.

[0003] In related technologies, scleral spur localization based on deep learning can achieve automatic assessment of the anterior chamber angle opening. However, existing deep learning network models are complex, resulting in time-consuming scleral spur localization. Summary of the Invention

[0004] Therefore, it is necessary to provide a scleral spur localization method, apparatus, device, and storage medium that can reduce the localization time and improve the localization efficiency of the scleral spur in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for locating the scleral protuberance, the method comprising:

[0006] Based on the image of the eye to be identified and the scleral spur localization model, a first heatmap corresponding to the image to be identified is determined; the scleral spur localization model includes a convolutional model, the convolutional model includes at least one convolutional module, the convolutional module includes a depth-separable convolutional layer and a spatially separable convolutional layer, the scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, the eye image samples including the scleral spur;

[0007] The target location information of the scleral protuberance in the image to be identified is determined based on the first heat map.

[0008] In one embodiment, determining the target location information of the scleral protuberance in the image to be identified based on the first thermal image includes:

[0009] Determine the target pixels in the first heatmap; the target pixels include pixels that are greater than or equal to a preset pixel threshold;

[0010] The target location information is determined based on the coordinates of the target pixel and the corresponding pixel value.

[0011] In one embodiment, the scleral protuberance localization model further includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a deconvolutional model;

[0012] The convolutional model is positioned between the first convolutional layer and the second convolutional layer, and the deconvolutional model is positioned between the second convolutional layer and the third convolutional layer.

[0013] In one embodiment, the convolution module further includes a fourth convolutional layer, a connection layer, a fifth convolutional layer, and a pooling layer;

[0014] The depth-separable convolutional layer is connected to the fourth convolutional layer, and both the fourth convolutional layer and the spatially separable convolutional layer are connected to the connecting layer. The fifth convolutional layer is disposed between the connecting layer and the pooling layer.

[0015] In one embodiment, the spatially separable convolutional layer includes a dilated convolutional layer;

[0016] Both the fourth convolutional layer and the dilated convolutional layer are connected to the connecting layer.

[0017] In one embodiment, the method further includes:

[0018] Obtain the eye image sample;

[0019] A second heatmap is determined based on the positional information of the scleral protrusion and other pixels in the eye image sample; the other pixels include all pixels in the eye image sample except those corresponding to the scleral protrusion.

[0020] Based on the eye image samples and the second heatmap, the initial scleral spur localization model is trained to obtain the scleral spur localization model.

[0021] In one embodiment, training the initial scleral spur localization model based on the eye image sample and the second thermal image to obtain the scleral spur localization model includes:

[0022] A third thermogram is determined based on the eye image samples and the initial scleral protuberance localization model;

[0023] Based on the difference between the second heatmap and the third heatmap, the initial scleral protuberance localization model is trained to obtain the scleral protuberance localization model.

[0024] Secondly, this application also provides a scleral spur positioning device, the device comprising:

[0025] The first determining module is used to determine a first heatmap corresponding to the image to be identified based on the image to be identified of the eye and the scleral spur localization model; the scleral spur localization model includes a convolutional model, the convolutional model includes at least one convolutional module, the convolutional module includes a depth-separable convolutional layer and a spatially separable convolutional layer, the scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, the eye image samples include the scleral spur;

[0026] The second determining module is used to determine the target location information of the scleral protrusion in the image to be identified based on the first heat map.

[0027] Thirdly, this application also provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0028] Based on the image of the eye to be identified and the scleral spur localization model, a first heatmap corresponding to the image to be identified is determined; the scleral spur localization model includes a convolutional model, the convolutional model includes at least one convolutional module, the convolutional module includes a depth-separable convolutional layer and a spatially separable convolutional layer, the scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, the eye image samples including the scleral spur;

[0029] The target location information of the scleral protuberance in the image to be identified is determined based on the first heat map.

[0030] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0031] Based on the image of the eye to be identified and the scleral spur localization model, a first heatmap corresponding to the image to be identified is determined; the scleral spur localization model includes a convolutional model, the convolutional model includes at least one convolutional module, the convolutional module includes a depth-separable convolutional layer and a spatially separable convolutional layer, the scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, the eye image samples including the scleral spur;

[0032] The target location information of the scleral protuberance in the image to be identified is determined based on the first heat map.

[0033] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, performs the following steps:

[0034] Based on the image of the eye to be identified and the scleral spur localization model, a first heatmap corresponding to the image to be identified is determined; the scleral spur localization model includes a convolutional model, the convolutional model includes at least one convolutional module, the convolutional module includes a depth-separable convolutional layer and a spatially separable convolutional layer, the scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, the eye image samples including the scleral spur;

[0035] The target location information of the scleral protuberance in the image to be identified is determined based on the first heat map.

[0036] The aforementioned scleral spur localization method, apparatus, device, and storage medium determine a first heatmap corresponding to the image to be identified based on the image of the eye to be identified and the scleral spur localization model, thereby determining the target location information of the scleral spur in the image to be identified based on the first heatmap. The scleral spur localization model includes a convolutional model, which includes at least one convolutional module. Each convolutional module includes a depthwise separable convolutional layer and a spatially separable convolutional layer. The scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, where the eye image samples include the scleral spur. In this embodiment, the spatially separable convolutional layer can reduce the number of parameters. Combined with the depthwise separable convolutional layer, the scleral spur localization model can achieve similar accuracy to traditional convolutional models with fewer parameters. While ensuring accurate scleral spur localization, it reduces the localization time, making scleral spur localization real-time and thus improving localization efficiency. Attached Figure Description

[0037] Figure 1 This is a schematic diagram illustrating the location information of the scleral spur in one embodiment;

[0038] Figure 2 This is a schematic diagram of the scleral spur localization network structure in one embodiment;

[0039] Figure 3 This is a schematic diagram of the network structure for scleral protuberance localization in another embodiment;

[0040] Figure 4 This is an application environment diagram of the scleral protuberance localization method in one embodiment;

[0041] Figure 5 This is a flowchart illustrating a scleral protuberance localization method in one embodiment;

[0042] Figure 6 This is a schematic diagram of the image to be identified in one embodiment;

[0043] Figure 7 This is a schematic diagram of an image to be identified, which includes scleral spur location information, in one embodiment.

[0044] Figure 8This is a schematic diagram of the process for determining the target location information of the scleral ridge in an image to be identified in one embodiment;

[0045] Figure 9 This is a schematic diagram of the framework of a scleral protuberance localization model in one embodiment;

[0046] Figure 10 This is a schematic diagram of the framework of a convolution module in one embodiment;

[0047] Figure 11 This is a flowchart illustrating the process of determining the scleral protuberance localization model in one embodiment;

[0048] Figure 12 This is a schematic diagram of the second heatmap in one embodiment;

[0049] Figure 13 This is a flowchart illustrating the process of determining the scleral protuberance localization model in another embodiment;

[0050] Figure 14 This is a structural block diagram of a scleral protuberance positioning device in one embodiment;

[0051] Figure 15 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] The anterior chamber angle, the angle formed between the inner surface of the cornea and the anterior surface of the iris, is an important component of the aqueous humor circulation. Clinically, the degree of opening of the anterior chamber angle reflects the risk of angle closure; the smaller the angle opening, the greater the risk of angle closure or primary angle-closure glaucoma. Therefore, quantitative assessment of the anterior chamber angle opening is an essential part of the diagnosis and treatment process for anyone diagnosed with or suspected of having primary angle-closure glaucoma. Scleral spur localization is a crucial foundation for this quantitative assessment, thus accurate localization facilitates anterior chamber angle assessment and early screening for angle-closure glaucoma.

[0054] Ophthalmic ultrasound biomicroscopy (UBM) is widely used for assessing the anterior chamber angle due to its excellent resolution and penetration. For example... Figure 1As shown, in UBM images, the intersection of the boundary between the scleral and ciliary body tissues and the extended line of the inner surface of the cornea is the location of the scleral spur. The scleral spur is a key anatomical structure for quantitative assessment of the anterior chamber angle opening, and its localization is a challenge for automated UBM image analysis systems. Because some UBM images have low signal-to-noise ratios and blurred edges, the judgment of key anatomical points such as the scleral spur heavily relies on the physician's professional knowledge and clinical experience. Furthermore, the subjective differences in judgment criteria among different physicians can affect the accuracy of scleral spur localization, thus impacting the assessment of the anterior chamber angle opening.

[0055] In recent years, deep learning-based artificial intelligence has developed rapidly in the field of medical image processing. Among related technologies, deep learning-based scleral spur localization helps in the automatic measurement of anterior segment parameters and the automatic assessment of anterior chamber angle opening. For example... Figure 2 As shown, this method uses a deep learning algorithm to automatically locate the scleral spur in Optical Coherence Tomography (OCT) images of the eye. First, ophthalmologists manually label the location of the scleral spur. Then, the ResNet18 network model is trained using the coordinates of the scleral spur as learning labels to directly regress the coordinates of the scleral spur. Alternatively, as... Figure 3 As shown, ophthalmologists first manually annotate the location of the scleral spur. Then, a Gaussian heatmap is generated based on the coordinates annotated by the ophthalmologists. The Gaussian heatmap is used as a label to train a semantic segmentation network model composed of EfficientNet and Unet network models. The output of the network model is a heatmap, and then the location of the scleral spur is extracted from the heatmap.

[0056] However, many current neural network models are large in scale and have a large number of parameters, resulting in long processing times for scleral spur localization. Therefore, this application proposes a method, apparatus, device, and storage medium for scleral spur localization.

[0057] The scleral protuberance localization method provided in this application embodiment can be applied to, for example... Figure 4 The application environment shown includes a computer device, which may be a server, and its internal structure diagram may be as follows. Figure 4As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data related to scleral spur localization. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a scleral spur localization method. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0058] In one embodiment, such as Figure 5 As shown, a method for scleral protuberance localization is provided, which can be applied to... Figure 4 Taking a computer device as an example, the explanation includes the following steps:

[0059] S501, Based on the image to be identified of the eye and the scleral spur localization model, determine the first heatmap corresponding to the image to be identified; the scleral spur localization model includes a convolutional model, the convolutional model includes at least one convolutional module, the convolutional module includes a depth-separable convolutional layer and a spatially separable convolutional layer, the scleral spur localization model is obtained by training the initial scleral spur localization model based on eye image samples, the eye image samples include the scleral spur.

[0060] Optionally, the image to be identified for the eye can be a UBM image or an optical coherence tomography (OCT) image.

[0061] In this embodiment, as Figure 6 As shown, Figure 6 In one embodiment, the image to be identified for the eye is input into a scleral spur localization model. The image is then processed sequentially through a convolutional model and a deconvolutional model to obtain a first heatmap with the same size as the image to be identified. Optionally, the scleral spur localization model is a model trained based on Gaussian heatmaps corresponding to eye image samples. Inputting the image to be identified for the eye into the scleral spur localization model yields a first heatmap that is a Gaussian-like heatmap similar to the Gaussian heatmap.

[0062] Optionally, the scleral ridge localization model can be a localization model based on a convolutional model, or it can be a localization model composed of a convolutional model, a deconvolutional model, and other convolutional layers, pooling layers, etc. The convolutional model can include at least one convolutional module, and the deconvolutional model includes deconvolutional modules corresponding to the convolutional modules. For example, if there are 5 convolutional modules, there will also be 5 corresponding deconvolutional modules. Other convolutional layers, pooling layers, etc., can be located before the convolutional model, between the convolutional and deconvolutional models, or after the deconvolutional model.

[0063] Optionally, depthwise separable convolutional layers can be channel-wise convolutional layers, where each channel is convolved by only one kernel, and the number of feature maps obtained by the channel-wise convolutional layer is the same as the number of input channels; they can also be pointwise convolutional layers. Spatially separable convolutional layers divide the convolutional kernel into two smaller kernels, for example, dividing a 3x3 convolutional kernel into 3x1 and 1x3 kernels. Depthwise and spatially separable convolutional layers have fewer parameters and can perform real-time scleral spur localization.

[0064] S502, determine the target location information of the scleral protrusion in the image to be identified based on the first thermal image.

[0065] In this embodiment, the target location information of the scleral spur can be extracted from the first heat map using either the maximum likelihood estimation algorithm or a Bayesian estimation algorithm. Figure 7 The image shown is a schematic diagram of marking the extracted target location information of the scleral spur on the image of the eye to be identified. The "+" in the figure represents the target location information of the scleral spur.

[0066] In one possible implementation, the coordinates of the point with the largest pixel value in the first heatmap can be directly used as the target location information of the scleral protuberance.

[0067] In the aforementioned scleral spur localization method, a first heatmap corresponding to the image to be identified is determined based on the image of the eye to be identified and the scleral spur localization model. The target location information of the scleral spur in the image to be identified is then determined based on the first heatmap. The scleral spur localization model includes a convolutional model, which includes at least one convolutional module. Each convolutional module includes a depthwise separable convolutional layer and a spatially separable convolutional layer. The scleral spur localization model is obtained by training an initial scleral spur localization model using eye image samples, which include the scleral spur. In this embodiment, the spatially separable convolutional layer reduces the number of parameters. Combined with the depthwise separable convolutional layer, the scleral spur localization model can achieve similar accuracy to traditional convolutional models with fewer parameters. This ensures accurate scleral spur localization while reducing localization time, making scleral spur localization real-time and thus improving localization efficiency.

[0068] Figure 8 This is a schematic diagram of the process for determining the target location information of the scleral ridge in an image to be identified in one embodiment, such as... Figure 8 As shown, this application embodiment relates to a possible implementation of how to determine the target location information of the scleral protuberance in an image to be identified based on a first thermal image, including the following steps:

[0069] S801, determine the target pixel in the first heat map; the target pixel includes pixels that are greater than or equal to a preset pixel threshold.

[0070] In this embodiment, the maximum pixel value in the first heatmap can be determined first, half of the maximum pixel value can be taken as a preset pixel threshold, and pixels in the first heatmap that are greater than half of the maximum pixel value can be taken as target pixels.

[0071] In one possible implementation, the pixel value corresponding to any pixel in the first heatmap can be used as a preset pixel threshold, or the pixel value of each pixel can be transformed to obtain the preset pixel threshold.

[0072] S802 determines the target position information based on the coordinates of the target pixel and the corresponding pixel value.

[0073] In this embodiment, when determining the x-coordinate of the target location based on the coordinates of the target pixel and its corresponding pixel value, the x-coordinate of each pixel in the target pixel set is multiplied by its corresponding pixel value to obtain first product results. These first product results are then summed to obtain a first summation result. Simultaneously, all pixels in the target pixel set are summed to obtain a second summation result. The x-coordinate of the target location is determined based on the ratio of the first summation result to the second summation result.

[0074] Similarly, for the y-coordinate of the target location, the y-coordinate of each pixel in the target pixel set is multiplied by its corresponding pixel value to obtain the second product results. These second product results are then summed to obtain the third sum. The y-coordinate of the target location is determined based on the ratio of the third sum to the second sum.

[0075] The location of the target can be determined using the following formula:

[0076]

[0077]

[0078] Where C is the set of target pixels, p i This represents the pixel value of the i-th pixel in the target pixel set, i.e., the pixel value corresponding to the target pixel. (xi y i (x, y) represents the coordinates of the i-th pixel in the target pixel set, and (x, y) represents the target position information of the scleral protuberance.

[0079] In this embodiment, by determining the target pixel in the first heat map, the target location information is determined based on the coordinates of the target pixel and the corresponding pixel value. The method is simple and can quickly determine the location information of the scleral spur, thereby improving the efficiency of scleral spur localization.

[0080] In one embodiment, the scleral protuberance localization model further includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a deconvolutional model; the convolutional model is disposed between the first and second convolutional layers, and the deconvolutional model is disposed between the second and third convolutional layers.

[0081] In this embodiment, as Figure 9 As shown, the scleral protuberance localization model also includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a deconvolutional model. The convolutional model is positioned between the first and second convolutional layers, and the deconvolutional model is positioned between the second and third convolutional layers. The convolutional model can include four convolutional modules, and the deconvolutional model includes four deconvolutional upsampling layers. The first convolutional layer is... Figure 9 The first convolutional layer is a 3x3 convolutional layer located between the input layer and the convolutional module. The second convolutional layer is a 3x3 convolutional layer located between the convolutional module and the deconvolutional upsampling layer. The third convolutional layer is a 3x3 convolutional layer and a 1x1 convolutional layer located between the deconvolutional upsampling layer and the output layer.

[0082] Optionally, the first, second, and third convolutional layers can all be multiple convolutional layers; there can be multiple convolutional modules and one deconvolutional module, as long as the number of convolutional modules and deconvolutional modules is the same.

[0083] In this embodiment, the convolutional model is set between the first and second convolutional layers, and the deconvolutional model is set between the second and third convolutional layers. The scleral protuberance localization model does not use convolutional operations similar to those in semantic segmentation networks after each deconvolutional module. Instead, it uses the deconvolutional model to continuously perform deconvolutional upsampling, which greatly reduces the number of parameters.

[0084] In one embodiment, the convolutional module further includes a fourth convolutional layer, a connection layer, a fifth convolutional layer, and a pooling layer; the depthwise separable convolutional layer is connected to the fourth convolutional layer, both the fourth convolutional layer and the spatially separable convolutional layer are connected to the connection layer, and the fifth convolutional layer is disposed between the connection layer and the pooling layer.

[0085] In this embodiment, as Figure 10As shown, the first part of the convolutional module consists of two channel-wise convolutional layers from two depthwise separable convolutions and a fourth convolutional layer (i.e., a 1x1 convolutional layer) connected in series. The second part consists of four spatially separable convolutional layers connected in series. The features obtained from the first and second parts are concatenated and merged, and then one convolutional layer and one pooling layer are connected in series. The depthwise separable convolutional layers are connected to the fourth convolutional layer (i.e., the 1x1 convolutional layer). The fourth convolutional layer and the dilated convolutional layer are connected to the connection layer. The fifth convolutional layer (i.e., the 1x1 convolutional layer) is placed between the connection layer (concat layer) and the pooling layer.

[0086] Furthermore, the spatially separable convolutional layer includes a dilated convolutional layer; both the fourth convolutional layer and the dilated convolutional layer are connected to the connecting layer.

[0087] The above Figure 10 In this model, the spatially separable convolutional layers include dilated convolutional layers. Both the fourth and second dilated convolutional layers are connected to connecting layers. All four spatially separable convolutional layers are set as dilated convolutional layers, or the first spatially separable convolutional layer is set as a dilated convolutional layer; alternatively, the first two spatially separable convolutional layers are set as dilated convolutional layers, etc. Optionally, the dilation rate of the dilated convolutional layers is 2, 3, etc.

[0088] For example, in combination with the above Figure 9 The kernel size in the first convolutional layer is 3×3, the stride is 1, and the number of channels is 32. The input image size of the first convolutional layer is 576×768, and the output feature size is 576×768×32.

[0089] The kernel size for channel-wise convolution is 3×3 with a stride of 1. The kernel size for the fourth convolutional layer is 1×1 with a stride of 1. The kernel sizes for spatially separable convolutions are 1×3, 3×1, 1×3, and 3×1, respectively. The dilatancy of the last two spatially separable convolutional layers (i.e., dilated convolutional layers) is set to 2. The kernel size for the fifth convolutional layer is 1×1 with a stride of 1, and the pooling kernel for the pooling layer is 2×2 with a stride of 2.

[0090] Since the size of the feature map is halved after each convolutional module, the number of channels in the second, third, fourth, and fifth layers are 64, 64, 128, and 256, respectively. The size of the output feature map of the second layer is 576×768×64, and the size of the output feature map of the fifth layer is 36×48×256.

[0091] The sixth layer consists of a second convolutional layer with a kernel size of 3×3, a stride of 1, and 256 channels. Therefore, the feature map output by the sixth layer has a size of 36×48×256.

[0092] Layers 7, 8, 9, and 10 are all deconvolution upsampling layers, with output channels of 256, 128, 64, and 64, respectively. The kernel size of the deconvolution upsampling layer is 3×3, and the stride is 2. The feature map size doubles after each deconvolution upsampling layer. Therefore, the feature map size output by layer 10 is 576×768×64.

[0093] The eleventh layer is the third convolutional layer, which consists of a concatenated convolutional layer with 32 channels and a 3×3 kernel size, and a convolutional layer with 1 channel, a 1×1 kernel size, and a sigmoid activation function. Therefore, the output size of the eleventh layer is 576×768×1.

[0094] In this embodiment, the convolution module further includes a fourth convolutional layer, a connection layer, a fifth convolutional layer, and a pooling layer. The convolution module combines depthwise separable convolution and spatially separable convolution, integrating the features extracted from both parts using the connection layer. This ensures the accuracy of feature extraction, thereby improving the accuracy of scleral punctum localization. Furthermore, the spatially separable convolutional layer includes a dilated convolutional layer, which can increase the receptive field while improving the feature space resolution, thus enhancing the accuracy of the scleral punctum localization model. Moreover, the scleral punctum localization model provided in this embodiment eliminates the skip connections between the encoding and decoding modules, significantly reducing the number of parameters.

[0095] Figure 11 This is a flowchart illustrating the process of determining the scleral ridge localization model in one embodiment, as shown below. Figure 11 As shown, it includes the following steps:

[0096] S1101, Obtain eye image samples.

[0097] In this embodiment, UBM images of the anterior chamber angle of patients from multiple clinical hospitals can be acquired using a 50MHz ophthalmic ultrasound biomicroscope. A total of 4000 UBM images from no fewer than 1000 patients are collected, and these acquired UBM images are directly used as ocular image samples. Preprocessing operations can also be performed on the anterior chamber angle UBM images to obtain ocular image samples. For example, if the acquired UBM image size is 576×1024, the image size is cropped to 576×768 to desensitize the images; images with incomplete anterior chamber angles or images not intended for anterior chamber angle observation are excluded.

[0098] Optionally, the image size can be cropped to other sizes such as 576×576 or 288×384.

[0099] S1102, determine the second heatmap based on the position information of the scleral protrusion and other pixels in the eye image sample; other pixels include pixels in the eye image sample other than those corresponding to the scleral protrusion.

[0100] In this embodiment, the positions of the scleral spurs in the eye image samples are labeled to obtain the positional information of the scleral spurs. To ensure the accuracy of the labeling of the scleral spur positions in the UBM images, each UBM image is labeled by an ophthalmologist with many years of clinical experience, and the average value of the labeled coordinates is taken as the labeling result. The labeling is then reviewed and corrected by another ophthalmologist with even higher seniority.

[0101] like Figure 12 As shown, based on the positional information of the scleral spur and other pixels in the eye image sample, a second heatmap with the same size as the eye image sample is determined. This second heatmap is a Gaussian heatmap, a smooth probability distribution that better represents the position of the scleral spur. Centered on the labeled scleral spur location, a Gaussian distribution function is applied to that location to generate a Gaussian heatmap centered on the scleral spur. The Gaussian distribution function at the scleral spur location achieves its maximum Gaussian value; the value decreases with distance from the scleral spur. Specifically, this can be determined using the following formula:

[0102]

[0103] In the above formula, δ is a hyperparameter that controls the radius of the pixel region that plays a role in localization in the second heatmap, (u0, v0) is the position information of the scleral protrusion in the eye image sample, and (u, v) is the position information of other pixels in the eye image sample.

[0104] S1103, based on eye image samples and the second thermal image, train the initial scleral spur localization model to obtain the scleral spur localization model.

[0105] In this embodiment, an eye image sample and a second heatmap can be simultaneously input into an initial scleral spur localization model. A third heatmap is obtained based on the initial scleral spur localization model and the eye image sample. The difference between the third heatmap and the second heatmap is determined using the initial scleral spur localization model. The initial scleral spur localization model is then trained based on the difference value to obtain the scleral spur localization model.

[0106] In one possible implementation, an eye image sample is first input into an initial scleral spur localization model to obtain a third heatmap. The difference between the third heatmap and the second heatmap is calculated, and then the difference is input into the initial scleral spur localization model. The initial scleral spur localization model is trained until the difference meets a preset difference value, thus obtaining the scleral spur localization model.

[0107] In this embodiment, an eye image sample is acquired, and a second heatmap is determined based on the position information of the scleral spur and other pixels in the eye image sample. Then, an initial scleral spur localization model is trained based on the eye image sample and the second heatmap to obtain the scleral spur localization model. This not only improves the prediction speed of the scleral spur localization model, but also has scleral spur localization accuracy close to that of ophthalmologists, thereby improving the robustness of the scleral spur localization model.

[0108] Figure 13 This is a flowchart illustrating the process of determining the scleral process localization model in another embodiment, as shown below. Figure 13 As shown, this application embodiment relates to a possible implementation of how to train an initial scleral spur localization model based on eye image samples and a second thermal image to obtain the scleral spur localization model, including the following steps:

[0109] S1301, determine the third thermogram based on eye image samples and the initial scleral protrusion localization model.

[0110] S1302, Based on the difference between the second and third heatmaps, train the initial scleral protuberance localization model to obtain the scleral protuberance localization model.

[0111] In this embodiment, an eye image sample and a second heatmap are input into an initial scleral spur localization model. A third heatmap is obtained based on the eye image sample and the initial scleral spur localization model. The size of the third heatmap is the same as that of the eye image sample. The initial learning rate is set to 0.001. The mean squared error (MSE) loss function is used to calculate the difference between the second and third heatmaps. The Adam optimizer is used to update the parameters of the initial scleral spur localization model. When the difference between the second and third heatmaps is small enough, or meets the preset difference, the update of the parameters of the initial scleral spur localization model stops, and the scleral spur localization model is obtained.

[0112] Alternatively, cross-entropy loss function, 0-1 loss function, log loss function, etc. can also be used;

[0113] Optionally, the optimizer can also be a gradient descent optimizer, a momentum optimizer, or an adaptive learning rate optimizer. For example, gradient descent can include gradient descent, stochastic gradient descent, mini-batch gradient descent, etc.; momentum optimizer can include Newton acceleration gradient momentum optimization, momentum stochastic gradient descent, etc.; adaptive learning rate optimizers, in addition to the Adam optimizer mentioned above, can also include RMSProp, adaptive gradient, etc.

[0114] Optionally, the initial learning rate can also be set to other values, such as 0.0001, 0.01, etc.

[0115] Furthermore, to prevent overfitting that may occur during the initial training of the scleral protrusion localization model, image enhancement processing is performed on the eye image samples during training, including data normalization, random rotation, random translation, random reversal, and random adjustment of brightness and contrast, without affecting the morphology of the anterior chamber angle in the eye image samples.

[0116] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0117] Based on the same inventive concept, this application also provides a scleral spur positioning device for implementing the above-described scleral spur positioning method. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the scleral spur positioning device provided below can be found in the limitations of the scleral spur positioning method described above, and will not be repeated here.

[0118] In one embodiment, such as Figure 14 As shown, a scleral protuberance positioning device is provided, comprising: a first determining module 11 and a second determining module 12, wherein:

[0119] The first determining module is used to determine the first heatmap corresponding to the image to be identified based on the image to be identified of the eye and the scleral spur localization model. The scleral spur localization model includes a convolutional model, which includes at least one convolutional module. The convolutional module includes a depth-separable convolutional layer and a spatially separable convolutional layer. The scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, and the eye image samples include the scleral spur.

[0120] The second determining module is used to determine the target location information of the scleral protrusion in the image to be identified based on the first heat map.

[0121] In one embodiment, the second determining module includes:

[0122] The first determining unit is used to determine the target pixel in the first heatmap; the target pixel includes pixels that are greater than or equal to a preset pixel threshold.

[0123] The second determining unit is used to determine the target position information based on the coordinates of the target pixel and the corresponding pixel value.

[0124] In one embodiment, the scleral protuberance localization model further includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a deconvolutional model;

[0125] The convolutional model is set between the first and second convolutional layers, and the deconvolutional model is set between the second and third convolutional layers.

[0126] In one embodiment, the convolutional module further includes a fourth convolutional layer, a connection layer, a fifth convolutional layer, and a pooling layer.

[0127] The depth-separable convolutional layer is connected to the fourth convolutional layer, the fourth convolutional layer and the dilated convolutional layer are connected to the connecting layer, and the fifth convolutional layer is placed between the connecting layer and the pooling layer.

[0128] In one embodiment, the spatially separable convolutional layer includes a dilated convolutional layer; both the fourth convolutional layer and the dilated convolutional layer are connected to the connecting layer.

[0129] In one embodiment, the scleral protuberance positioning device further includes:

[0130] The acquisition module is used to acquire eye image samples;

[0131] The third determining module is used to determine the second heatmap based on the position information of the scleral protrusion and the position information of other pixels in the eye image sample; other pixels include pixels in the eye image sample other than those corresponding to the scleral protrusion.

[0132] The training module is used to train an initial scleral spur localization model based on eye image samples and a second heatmap, thereby obtaining the scleral spur localization model.

[0133] In one embodiment, the training module includes:

[0134] The third determining unit is used to determine the third thermogram based on the eye image sample and the initial scleral protrusion localization model;

[0135] The training unit is used to train an initial scleral protuberance localization model based on the difference between the second and third heatmaps, thereby obtaining the scleral protuberance localization model.

[0136] Each module in the aforementioned scleral protuberance positioning device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0137] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 15 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a scleral protuberance localization method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0138] Those skilled in the art will understand that Figure 15 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0139] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0140] Based on the image of the eye to be identified and the scleral spur localization model, the first heatmap corresponding to the image to be identified is determined. The scleral spur localization model includes a convolutional model and a deconvolutional model. The convolutional model includes at least one convolutional module, and the deconvolutional model includes a deconvolutional module corresponding to the convolutional module. The convolutional module includes a depth-separable convolutional layer and a spatially separable convolutional layer. The scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, and the eye image samples include the scleral spur.

[0141] The target location information of the scleral protuberance in the image to be identified is determined based on the first heat map.

[0142] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0143] Determine the target pixels in the first heatmap; the target pixels include pixels that are greater than or equal to a preset pixel threshold.

[0144] The target location information is determined based on the coordinates of the target pixel and the corresponding pixel value.

[0145] In one embodiment, the scleral protuberance localization model further includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a deconvolutional model;

[0146] The convolutional model is set between the first and second convolutional layers, and the deconvolutional model is set between the second and third convolutional layers.

[0147] In one embodiment, the convolution module further includes a fourth convolutional layer, a connection layer, a fifth convolutional layer, and a pooling layer;

[0148] The depthwise separable convolutional layer is connected to the fourth convolutional layer. Both the fourth convolutional layer and the spatially separable convolutional layer are connected to the connection layer. The fifth convolutional layer is placed between the connection layer and the pooling layer.

[0149] In one embodiment, the spatially separable convolutional layer includes a dilated convolutional layer; both the fourth convolutional layer and the dilated convolutional layer are connected to the connecting layer.

[0150] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0151] Obtain eye image samples;

[0152] The second heatmap is determined based on the location information of the scleral spur and other pixels in the eye image sample; other pixels include pixels in the eye image sample other than those corresponding to the scleral spur.

[0153] Based on eye image samples and a second thermal image, an initial scleral spur localization model is trained to obtain the scleral spur localization model.

[0154] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0155] The third thermogram was determined based on eye image samples and the initial scleral protuberance localization model;

[0156] Based on the difference between the second and third heatmaps, an initial scleral protuberance localization model is trained to obtain the scleral protuberance localization model.

[0157] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0158] Based on the image to be identified of the eye and the scleral spur localization model, the first heatmap corresponding to the image to be identified is determined; the scleral spur localization model includes a convolution model, the convolution model includes at least one convolution module, the convolution module includes a depth-separable convolution layer and a spatially separable convolution layer, the scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, the eye image samples include the scleral spur.

[0159] The target location information of the scleral protuberance in the image to be identified is determined based on the first heat map.

[0160] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0161] Determine the target pixels in the first heatmap; the target pixels include pixels that are greater than or equal to a preset pixel threshold.

[0162] The target location information is determined based on the coordinates of the target pixel and the corresponding pixel value.

[0163] In one embodiment, the scleral protuberance localization model further includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a deconvolutional model;

[0164] The convolutional model is set between the first and second convolutional layers, and the deconvolutional model is set between the second and third convolutional layers.

[0165] In one embodiment, the convolution module further includes a fourth convolutional layer, a connection layer, a fifth convolutional layer, and a pooling layer;

[0166] The depthwise separable convolutional layer is connected to the fourth convolutional layer. Both the fourth convolutional layer and the spatially separable convolutional layer are connected to the connection layer. The fifth convolutional layer is placed between the connection layer and the pooling layer.

[0167] In one embodiment, the spatially separable convolutional layer includes a dilated convolutional layer; both the fourth convolutional layer and the dilated convolutional layer are connected to the connecting layer.

[0168] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0169] Obtain eye image samples;

[0170] The second heatmap is determined based on the location information of the scleral spur and other pixels in the eye image sample; other pixels include pixels in the eye image sample other than those corresponding to the scleral spur.

[0171] Based on eye image samples and a second thermal image, an initial scleral spur localization model is trained to obtain the scleral spur localization model.

[0172] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0173] The third thermogram was determined based on eye image samples and the initial scleral protuberance localization model;

[0174] Based on the difference between the second and third heatmaps, an initial scleral protuberance localization model is trained to obtain the scleral protuberance localization model.

[0175] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0176] Based on the image to be identified of the eye and the scleral spur localization model, the first heatmap corresponding to the image to be identified is determined; the scleral spur localization model includes a convolution model, the convolution model includes at least one convolution module, the convolution module includes a depth-separable convolution layer and a spatially separable convolution layer, the scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, the eye image samples include the scleral spur.

[0177] The target location information of the scleral protuberance in the image to be identified is determined based on the first heat map.

[0178] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0179] Determine the target pixels in the first heatmap; the target pixels include pixels that are greater than or equal to a preset pixel threshold.

[0180] The target location information is determined based on the coordinates of the target pixel and the corresponding pixel value.

[0181] In one embodiment, the scleral protuberance localization model further includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a deconvolutional model;

[0182] The convolutional model is set between the first and second convolutional layers, and the deconvolutional model is set between the second and third convolutional layers.

[0183] In one embodiment, the convolution module further includes a fourth convolutional layer, a connection layer, a fifth convolutional layer, and a pooling layer;

[0184] The depthwise separable convolutional layer is connected to the fourth convolutional layer. Both the fourth convolutional layer and the spatially separable convolutional layer are connected to the connection layer. The fifth convolutional layer is placed between the connection layer and the pooling layer.

[0185] In one embodiment, the spatially separable convolutional layer includes a dilated convolutional layer; both the fourth convolutional layer and the dilated convolutional layer are connected to the connecting layer.

[0186] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0187] Obtain eye image samples;

[0188] The second heatmap is determined based on the location information of the scleral spur and other pixels in the eye image sample; other pixels include pixels in the eye image sample other than those corresponding to the scleral spur.

[0189] Based on eye image samples and a second thermal image, an initial scleral spur localization model is trained to obtain the scleral spur localization model.

[0190] In one embodiment, when the computer program is executed by a processor, it further performs the following steps:

[0191] The third thermogram was determined based on eye image samples and the initial scleral protuberance localization model;

[0192] Based on the difference between the second and third heatmaps, an initial scleral protuberance localization model is trained to obtain the scleral protuberance localization model.

[0193] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0194] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0195] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0196] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for locating the scleral protuberance, characterized in that, The method includes: Based on the image of the eye to be identified and the scleral spur localization model, a first heatmap corresponding to the image to be identified is determined. The scleral spur localization model includes a convolutional model, a first convolutional layer, a second convolutional layer, a third convolutional layer, and a deconvolutional model. The convolutional model is positioned between the first and second convolutional layers, and the deconvolutional model is positioned between the second and third convolutional layers. The convolutional model includes a fourth convolutional layer, a connection layer, a fifth convolutional layer, a pooling layer, and at least one convolutional module. The convolutional module includes depth-dependent... The system includes a separable convolutional layer and a spatially separable convolutional layer, the spatially separable convolutional layer including a dilated convolutional layer; a depthwise separable convolutional layer connected to a fourth convolutional layer, both the fourth convolutional layer and the dilated convolutional layer being connected to a connecting layer; a fifth convolutional layer positioned between the connecting layer and the pooling layer; a deconvolutional model including the same number of deconvolutional upsampling layers as the convolutional module; and a scleral spur localization model trained on an initial scleral spur localization model based on eye image samples, the eye image samples including the scleral spur. The target location information of the scleral protuberance in the image to be identified is determined based on the first heat map.

2. The method according to claim 1, characterized in that, The step of determining the target location information of the scleral protuberance in the image to be identified based on the first heat map includes: Determine the target pixels in the first heatmap; the target pixels include pixels that are greater than or equal to a preset pixel threshold; The target location information is determined based on the coordinates of the target pixel and the corresponding pixel value.

3. The method according to claim 1, characterized in that, The method further includes: Obtain the eye image sample; A second heatmap is determined based on the positional information of the scleral protrusion and other pixels in the eye image sample; the other pixels include all pixels in the eye image sample except those corresponding to the scleral protrusion. Based on the eye image samples and the second heatmap, the initial scleral spur localization model is trained to obtain the scleral spur localization model.

4. The method according to claim 3, characterized in that, The step of training the initial scleral spur localization model based on the eye image samples and the second thermal image to obtain the scleral spur localization model includes: A third thermogram is determined based on the eye image samples and the initial scleral protuberance localization model; Based on the difference between the second heatmap and the third heatmap, the initial scleral protuberance localization model is trained to obtain the scleral protuberance localization model.

5. A scleral protuberance positioning device, characterized in that, The device includes: A first determining module is used to determine a first heatmap corresponding to the image to be identified based on the image of the eye to be identified and the scleral spur localization model. The scleral spur localization model includes a convolutional model, a first convolutional layer, a second convolutional layer, a third convolutional layer, and a deconvolutional model. The convolutional model is disposed between the first convolutional layer and the second convolutional layer, and the deconvolutional model is disposed between the second convolutional layer and the third convolutional layer. The convolutional model includes a fourth convolutional layer, a connection layer, a fifth convolutional layer, a pooling layer, and at least one convolutional module. The convolutional module includes... The system includes depthwise separable convolutional layers and spatially separable convolutional layers, the spatially separable convolutional layers including dilated convolutional layers; the depthwise separable convolutional layers are connected to the fourth convolutional layer, both the fourth convolutional layer and the dilated convolutional layer are connected to the connecting layer, the fifth convolutional layer is disposed between the connecting layer and the pooling layer, the deconvolutional model includes the same number of deconvolutional upsampling layers as the convolutional module; the scleral spur localization model is obtained by training an initial scleral spur localization model based on eye image samples, the eye image samples including the scleral spur; The second determining module is used to determine the target location information of the scleral protrusion in the image to be identified based on the first heat map.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images